Edit: 2015-10-05

Useful fact: if $A\ge B$, then $\sigma_{\min}(A)\ge \sigma_{\min}(B)$ and $\sigma_{\max}(A)\ge \sigma_{\max}(B)$

Proof: For the eigenvector x1 that associated with the smallest eigenvalue of A

$\sigma_{min}(A)=x_1^TAx_1\ge x_1^TBx_1\ge \sigma_{min}(B)$

For the eigenvector yn that associated with the largest eigenvalue of B

$\sigma_{max}(B)=y_n^T B y_n\le y_n^TAy_n\le \sigma_{max}(A)$

Remark: it is not correct that $\sigma_{min}(A)\ge \sigma_{max}(B)$. Counterexamples can be easily found if you consider the geometric interpretation of the ellipsoids x^T*A*x=1 and x^T*B*x=1.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Edit: 2015-09-23

Fact 1.
If $x^TAx=1$, then it defines an ellipsoid.
If $x^TAx<1$, then the vector x is inside the ellipsoid.
If $x^TAx>1$, then x is outside the ellipsoid.

Proof: suppose $x^T Ax<1$. Then there exists $k^2=1/(x^TAx)$ such that $(kx)^TA(kx)=1$. Therefore, $kx$ is on the surface of the ellipsoid and since $k>1$, we know $x$ is inside the ellipsoid.

Fact 2. For any PD matrices A and B, if A>B (i.e., A-B>0), then $x^TAx=1$ is contained inside of $x^TBx=1$.

Remark: The more positive A is, the smaller the ellipsoid it is. Roughly speaking, that is because the length of each semi-axis is the inverse of the eigenvalue.
Remark: note A and B may have different eigenvectors. So if you plot the two ellipsoids, they are not parallel.
Remark: you cannot naively say that the singular values of A are larger than those of B even if A>B. This can be intuitively seen by plotting the two ellipsoid. But we can say the largest (smallest) eigenvalue of A is larger than the largest (smallest) one of B. This can be seen from the plot and also proved. We cannot say the smallest eigenvalue of A is larger than then maximum eigenvalue.

Proof: If $x^TAx=1$, then x is on the surface of the ellipsoid of A. Then $x^TBx=x^TAx-x^T(A-B)x. Hence x is inside of the ellipsoid of B. Hence any point on the ellipsoid of A is inside the ellipsoid of B.

=====================================================

$x^T A x=1$ represents an ellipsoid.

1. The axis of the ellipsoid is along the eigenvectors of A.
2. The length of the semi-axis is $1/\sqrt{\lambda_i}$.
3. The volume of the ellipsoid is $\frac{4}{3}\pi \frac{1}{\sqrt{\lambda_1 \lambda_2 \lambda_3}}$.

Here is a detailed memo.