Editing Covariance matrix (section)

==Which matrices are covariance matrices?==
From basic property 4. above, let <math>\mathbf{b}</math> be a <math>(p \times 1)</math> real-valued vector, then
<math display="block">\operatorname{var}(\mathbf{b}^\mathsf{T}\mathbf{X}) = \mathbf{b}^\mathsf{T} \operatorname{var}(\mathbf{X}) \mathbf{b},\,</math>
which must always be nonnegative, since it is the [[variance#Properties|variance]] of a real-valued random variable, so a covariance matrix is always a [[positive-semidefinite matrix]].

The above argument can be expanded as follows:<math display="block">
\begin{align}
& w^\mathsf{T} \operatorname{E} \left[(\mathbf{X} - \operatorname{E}[\mathbf{X}]) (\mathbf{X} - \operatorname{E}[\mathbf{X}])^\mathsf{T}\right] w
= \operatorname{E} \left[w^\mathsf{T}(\mathbf{X} - \operatorname{E}[\mathbf{X}]) (\mathbf{X} - \operatorname{E}[\mathbf{X}])^\mathsf{T}w\right] \\
&= \operatorname{E} \big[\big( w^\mathsf{T}(\mathbf{X} - \operatorname{E}[\mathbf{X}]) \big)^2 \big] \geq 0,
\end{align}
</math>where the last inequality follows from the observation that <math>w^\mathsf{T}(\mathbf{X} - \operatorname{E}[\mathbf{X}])</math> is a scalar.

Conversely, every symmetric positive semi-definite matrix is a covariance matrix. To see this, suppose <math>M</math> is a <math>p \times p</math> symmetric positive-semidefinite matrix. From the finite-dimensional case of the [[spectral theorem]], it follows that <math>M</math> has a nonnegative symmetric [[Square root of a matrix|square root]], which can be denoted by '''M'''<sup>1/2</sup>. Let <math>\mathbf{X}</math> be any <math>p \times 1</math> column vector-valued random variable whose covariance matrix is the <math>p \times p</math> identity matrix. Then
<math display="block">\operatorname{var}(\mathbf{M}^{1/2} \mathbf{X}) = \mathbf{M}^{1/2} \, \operatorname{var}(\mathbf{X}) \, \mathbf{M}^{1/2} = \mathbf{M}.</math>