Skip to content

Why covariance matrix is positive semi definite?

30/08/2011

Today it is a simple question for me. But yesterday, I was bothered by it.

Problem: Let x be a random vector with mean as \mu and

var(x)=E[(x-\mu)(x-\mu)^T]

Now prove var(x) is positive semi definite.

Proof: Let v be an arbitrary vector (not random vector). Then

v^TE[(x-\mu)(x-\mu)^T]v=E[(v^T(x-\mu))^2]\ge 0

Q.E.D.

Highlight:

  • The matrix (x-\mu)(x-\mu)^T is p.s.d and of rank-1. But we can’t simply say E[(x-\mu)(x-\mu)^T] is p.s.d, and of course E[(x-\mu)(x-\mu)^T] is not of rank-1.
  • Need not to use the definition of expectation to prove, but need use the definition of positive definite matrices.
  • For scalar cases, it is easy to see because E[(x-\mu)^2]\ge 0

Further question: when does E[(v^T(x-\mu))^2]= 0?? when the elements of x are independent? linear or statistically independent? So if x is independent, we have the covariance positive definite instead of positive semi-definite?

Advertisements
No comments yet

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: