Editing Multivariate random variable (section)

===Expectation of a quadratic form===
One can take the expectation of a [[Quadratic form (statistics)|quadratic form]] in the random vector <math>\mathbf{X}</math> as follows:<ref name=Kendrick>{{cite book |last=Kendrick |first=David |title=Stochastic Control for Economic Models |publisher=McGraw-Hill |year=1981 |isbn=0-07-033962-7 }}</ref>{{rp|p.170–171}}

:<math>\operatorname{E}[\mathbf{X}^{T}A\mathbf{X}] = \operatorname{E}[\mathbf{X}]^{T}A\operatorname{E}[\mathbf{X}] + \operatorname{tr}(A K_{\mathbf{X}\mathbf{X}}),</math>

where <math>K_{\mathbf{X}\mathbf{X}}</math> is the covariance matrix of <math>\mathbf{X}</math> and <math>\operatorname{tr}</math> refers to the [[Trace (linear algebra)|trace]] of a matrix — that is, to the sum of the elements on its main diagonal (from upper left to lower right).  Since the quadratic form is a scalar, so is its expectation.

'''Proof''': Let <math>\mathbf{z}</math> be an <math>m \times 1</math> random vector with <math>\operatorname{E}[\mathbf{z}] = \mu</math> and <math>\operatorname{Cov}[\mathbf{z}]= V</math> and let <math>A</math> be an <math>m \times m</math> non-stochastic matrix.

Then based on the formula for the covariance, if we denote <math>\mathbf{z}^T = \mathbf{X}</math> and <math>\mathbf{z}^T A^T = \mathbf{Y}</math>, we see that:

:<math>\operatorname{Cov}[\mathbf{X},\mathbf{Y}] = \operatorname{E}[\mathbf{X}\mathbf{Y}^T]-\operatorname{E}[\mathbf{X}]\operatorname{E}[\mathbf{Y}]^T </math>

Hence

:<math>\begin{align}
\operatorname{E}[XY^T]   &= \operatorname{Cov}[X,Y]+\operatorname{E}[X]\operatorname{E}[Y]^T \\
\operatorname{E}[z^T Az] &= \operatorname{Cov}[z^T,z^T A^T] + \operatorname{E}[z^T]\operatorname{E}[z^T A^T ]^T  \\
&=\operatorname{Cov}[z^T , z^T A^T] + \mu^T (\mu^T A^T)^T \\
&=\operatorname{Cov}[z^T , z^T A^T] + \mu^T A \mu ,
\end{align}</math>

which leaves us to show that

:<math>\operatorname{Cov}[z^T , z^T A^T ]=\operatorname{tr}(AV).</math>

This is true based on the fact that one can [[Trace (linear algebra)#Properties|cyclically permute matrices when taking a trace]] without changing the end result (e.g.: <math>\operatorname{tr}(AB) = \operatorname{tr}(BA)</math>).

We see [[Covariance#Definition|that]]

:<math>\begin{align}
\operatorname{Cov}[z^T,z^T A^T] &= \operatorname{E} \left[\left(z^T - E(z^T) \right)\left(z^T A^T - E\left(z^T A^T \right) \right)^T \right] \\
&= \operatorname{E} \left[ (z^T - \mu^T) (z^T A^T - \mu^T A^T )^T \right]\\
&= \operatorname{E} \left[ (z - \mu)^T (Az - A\mu) \right].
\end{align}</math>

And since

:<math>\left( {z - \mu } \right)^T \left( {Az - A\mu } \right)</math>

is a [[scalar (mathematics)|scalar]], then

:<math>(z - \mu)^T ( Az - A\mu)= \operatorname{tr}\left( {(z - \mu )^T (Az - A\mu )} \right) = \operatorname{tr} \left((z - \mu )^T A(z - \mu ) \right)</math>

trivially. Using the permutation we get:

:<math>\operatorname{tr}\left( {(z - \mu )^T A(z - \mu )} \right) = \operatorname{tr}\left( {A(z - \mu )(z - \mu )^T} \right),</math>

and by plugging this into the original formula we get:

:<math>\begin{align}
\operatorname{Cov} \left[ {z^T,z^T A^T} \right] &= E\left[ {\left( {z - \mu } \right)^T (Az - A\mu)} \right] \\
&= E \left[ \operatorname{tr}\left( A(z - \mu )(z - \mu )^T \right) \right] \\
&= \operatorname{tr} \left( {A \cdot \operatorname{E} \left((z - \mu )(z - \mu )^T \right) } \right) \\
&= \operatorname{tr} (A V).
\end{align}</math>