Editing Covariance matrix (section)

===Basic properties===
For <math>\operatorname{K}_{\mathbf{X}\mathbf{X}}=\operatorname{var}(\mathbf{X}) = \operatorname{E} \left[ \left( \mathbf{X} - \operatorname{E}[\mathbf{X}] \right) \left( \mathbf{X} - \operatorname{E}[\mathbf{X}] \right)^\mathsf{T} \right]</math> and <math> \boldsymbol{\mu}_\mathbf{X} = \operatorname{E}[\textbf{X}]</math>, where <math>\mathbf{X} = (X_1,\ldots,X_n)^\mathsf{T}</math> is an <math>n</math>-dimensional random variable, the following basic properties apply:<ref name=taboga>{{cite web |last1=Taboga |first1=Marco |url=http://www.statlect.com/varian2.htm |title=Lectures on probability theory and mathematical statistics |year=2010}}</ref>
# <math> \operatorname{K}_{\mathbf{X}\mathbf{X}} = \operatorname{E}(\mathbf{X X^\mathsf{T}}) - \boldsymbol{\mu}_\mathbf{X}\boldsymbol{\mu}_\mathbf{X}^\mathsf{T} </math>
# <math> \operatorname{K}_{\mathbf{X}\mathbf{X}} \,</math> is [[Positive-semidefinite matrix|positive-semidefinite]], i.e. <math>\mathbf{a}^T \operatorname{K}_{\mathbf{X}\mathbf{X}} \mathbf{a} \ge 0 \quad \text{for all } \mathbf{a} \in \mathbb{R}^n</math>
{{Hidden|
  title = ''Proof'' |
  content = 
<!------------------------------------------------------------------------------------->
Indeed, from the property 4 it follows that under linear transformation of random variable <math>\mathbf{X}</math> with covariation matrix <math>\mathbf{\Sigma_{X}} = \mathrm{cov}(\mathbf{X})</math> by linear operator <math>\mathbf{A}</math> s.a. <math>\mathbf{Y} = \mathbf{A}\mathbf{X}</math>, the covariation matrix is tranformed as
: <math>\mathbf{\Sigma_{Y}} = \mathrm{cov}\left(\mathbf{Y}\right) = \mathbf{A\, \Sigma_{X}\,A}^{\top}</math>.
As according to the property 3 matrix <math>\mathbf{\Sigma_{X}}</math> is symmetric, it can be diagonalized by a linear orthogonal transformation, i.e. there exists such orthogonal matrix <math>\mathbf{A}</math> (meanwhile <math>\mathbf{A}^{\top} = \mathbf{A}^{-1}</math>), that
: <math>\mathbf{A\, \Sigma_{X}\,A}^{\top} = \mathbf{A\, \Sigma_{X}\,A}^{-1} = \mbox{diag}(\sigma_1,\ldots,\sigma_n),</math>
and <math>\sigma_1,\ldots,\sigma_n</math> are eigenvalues of <math>\mathbf{\Sigma_{X}}</math>. But this means that this matrix is a covariation matrix for a random variable <math>\mathbf{Y} = \mathbf{A}\mathbf{X}</math>, and the main diagonal of <math>\mathbf{\Sigma_{Y}} = \mathrm{cov}\left(\mathbf{Y}\right)</math> consists of variances of elements of <math>\mathbf{Y}</math> vector. As variance is always non-negative, we conclude that <math>\sigma_i \geq 0</math> for any <math>i</math>. But this means that matrix <math>\mathbf{\Sigma_{X}}</math> is positive-semidefinite.
<!------------------------------------------------------------------------------------->
}}
# <math> \operatorname{K}_{\mathbf{X}\mathbf{X}} \,</math> is [[symmetric matrix|symmetric]], i.e. <math>\operatorname{K}_{\mathbf{X}\mathbf{X}}^\mathsf{T} = \operatorname{K}_{\mathbf{X}\mathbf{X}}</math>
# For any constant (i.e. non-random) <math>m \times n</math> matrix <math>\mathbf{A}</math> and constant <math>m \times 1</math> vector <math>\mathbf{a}</math>, one has <math> \operatorname{var}(\mathbf{A X} + \mathbf{a}) = \mathbf{A}\, \operatorname{var}(\mathbf{X})\, \mathbf{A}^\mathsf{T} </math>
# If <math>\mathbf{Y}</math> is another random vector with the same dimension as <math>\mathbf{X}</math>, then <math>\operatorname{var}(\mathbf{X} + \mathbf{Y}) = \operatorname{var}(\mathbf{X}) + \operatorname{cov}(\mathbf{X},\mathbf{Y}) + \operatorname{cov}(\mathbf{Y}, \mathbf{X}) + \operatorname{var}(\mathbf{Y}) </math> where <math>\operatorname{cov}(\mathbf{X}, \mathbf{Y})</math> is the [[cross-covariance matrix]] of <math>\mathbf{X}</math> and <math>\mathbf{Y}</math>.