Data Processing Inequality

A very intuitive yet powerful inequality in information theory is the data processing inequality. Lemma: If random variable , and form a Markov chain , then . The great thing about the inequality is that unlike some results in information theory, it works for both discrete and continuous random variables. (Actually it works even for…

Some Results for Hermitian Matrix

Lemma 1: All eigenvalues of a Hermitian matrix are real. Proof: Let be Hermitian and and be an eigenvalue and the corresponding eigenvector of . We have . Thus we have as is real in general. Lemma 2: Two eigenvectors of a Hermitian matrix are orthogonal to each other if their eigenvalues are different. Proof:…

Schur Complement and Positive Definite Matrix

For a matrix , we call the Schur complement of in . Note that naturally appear in block matrix inversion. Note that when is symmetric and is positive definite, is positive definite if and only if is also positive definite. The proof is rather straightforward. Consider the function . Let’s try to minimize w.r.t. ,…

Block Matrix Inversion

Here are some formula for matrix inversion. Lemma 1: For a block matrix , , where is basically the Schur’s complement of block . Proof: Let , gives us From the four equations, we have And similarly from , we have Together, they show the first inequality. Also note that and thus . Substituting into…

Screen capture of a selected window with ffmpeg in Ubuntu

After some trial and error and reading some other posts, I finally manage to capture screencast of a selected window with ffmpeg under 10.04. I am using the native ffmpeg from ubuntu and so recompiling is not needed. The command I’m using is as below. ffmpeg -f alsa -ac 2 -i pulse -f x11grab -r…

Code Geass Season 1

What can I say? I just can’t stop watching and and basically used up my whole day to finish the first season. Thinking back, it didn’t incorporate exactly lots of new elements. After all, the studio is Sunrise. It has been making war animes for more than 30 years. It just did everything they used…

America and Socialism

One thing I found it amazing is how average americans detest and even are terrified by the word socialism. Sometimes I heard people will say chineses are “communists” and “socialists”. It is interesting that America is actually a much more socialistic country than China. It is true that the chinese government is dictatorship. But it…

Another look of Steins Gate

I finally finished Steins Gate. Honestly, I was a little bit disappointed. It is still good, even great. But I don’t think it is a classic as some may suggest. And the quality (both visual and story-wise) definitely declines about mid way until it came back up again near the end. I guess it is…