Editing Covariance matrix (section)

=== Block matrices ===

The joint mean <math>\boldsymbol\mu</math> and [[cross-covariance matrix|joint covariance matrix]] <math>\boldsymbol\Sigma</math> of <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> can be written in block form
<math display="block">
\boldsymbol\mu
=
\begin{bmatrix}
 \boldsymbol{\mu}_X \\
 \boldsymbol{\mu}_Y
\end{bmatrix}, \qquad
\boldsymbol\Sigma
=
\begin{bmatrix}
 \operatorname{K}_\mathbf{XX} & \operatorname{K}_\mathbf{XY} \\
 \operatorname{K}_\mathbf{YX} & \operatorname{K}_\mathbf{YY}
\end{bmatrix}
</math>
where <math> \operatorname{K}_\mathbf{XX} = \operatorname{var}(\mathbf{X}) </math>, <math> \operatorname{K}_\mathbf{YY} = \operatorname{var}(\mathbf{Y}) </math> and <math> \operatorname{K}_\mathbf{XY} = \operatorname{K}^\mathsf{T}_\mathbf{YX} = \operatorname{cov}(\mathbf{X}, \mathbf{Y}) </math>.

<math> \operatorname{K}_\mathbf{XX} </math> and <math> \operatorname{K}_\mathbf{YY} </math> can be identified as the variance matrices of the [[marginal distribution]]s for <math> \mathbf{X} </math> and <math> \mathbf{Y} </math> respectively.

If <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> are [[Multivariate normal distribution|jointly normally distributed]],
<math display="block">
\mathbf{X}, \mathbf{Y} \sim\ \mathcal{N}(\boldsymbol\mu, \operatorname{\boldsymbol\Sigma}),
</math>
then the [[conditional distribution]] for <math>\mathbf{Y}</math> given <math>\mathbf{X}</math> is given by<ref name=eaton>{{cite book|last=Eaton|first=Morris L.|title=Multivariate Statistics: a Vector Space Approach|year=1983|publisher=John Wiley and Sons|isbn=0-471-02776-6|pages=116–117}}</ref>
<math display="block">
\mathbf{Y} \mid \mathbf{X} \sim\ \mathcal{N}(\boldsymbol{\mu}_\mathbf{Y|X}, \operatorname{K}_\mathbf{Y|X}),
</math>
defined by [[conditional mean]]
<math display="block">
\boldsymbol{\mu}_{\mathbf{Y}|\mathbf{X}}
=
\boldsymbol{\mu}_\mathbf{Y} + \operatorname{K}_\mathbf{YX} \operatorname{K}_\mathbf{XX}^{-1}
\left(
 \mathbf{X} - \boldsymbol{\mu}_\mathbf{X}
\right)
</math>
and [[conditional variance]]
<math display="block">
  \operatorname{K}_\mathbf{Y|X}
  = \operatorname{K}_\mathbf{YY} - \operatorname{K}_\mathbf{YX} \operatorname{K}_\mathbf{XX}^{-1} \operatorname{K}_\mathbf{XY}.
</math>

The matrix <math> \operatorname{K}_\mathbf{YX} \operatorname{K}_\mathbf{XX}^{-1} </math> is known as the matrix of [[regression analysis|regression]] coefficients, while in linear algebra <math> \operatorname{K}_\mathbf{Y|X} </math> is the [[Schur complement]] of <math> \operatorname{K}_\mathbf{XX} </math> in <math> \boldsymbol\Sigma </math>.

The matrix of regression coefficients may often be given in transpose form, <math> \operatorname{K}_\mathbf{XX}^{-1} \operatorname{K}_\mathbf{XY} </math>, suitable for post-multiplying a row vector of explanatory variables <math> \mathbf{X}^\mathsf{T} </math> rather than pre-multiplying a column vector <math> \mathbf{X} </math>. In this form they correspond to the coefficients obtained by inverting the matrix of the [[normal equations]] of [[ordinary least squares]] (OLS).