Why covariance matrix is positive semi definite?
Today it is a simple question for me. But yesterday, I was bothered by it.
Problem: Let x be a random vector with mean as and
Now prove is positive semi definite.
Proof: Let be an arbitrary vector (not random vector). Then
- The matrix is p.s.d and of rank-1. But we can’t simply say is p.s.d, and of course is not of rank-1.
- Need not to use the definition of expectation to prove, but need use the definition of positive definite matrices.
- For scalar cases, it is easy to see because
Further question: when does ?? when the elements of are independent? linear or statistically independent? So if x is independent, we have the covariance positive definite instead of positive semi-definite?