Editing Principal component analysis (section)

== Derivation using the covariance method ==

Let '''X''' be a ''d''-dimensional random vector expressed as column vector. Without loss of generality, assume '''X''' has zero mean.

We want to find <math>(\ast)</math> a {{math|''d'' × ''d''}} [[orthonormal basis|orthonormal transformation matrix]] '''P''' so that '''PX''' has a diagonal covariance matrix (that is, '''PX''' is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming <math>P</math> were unitary yields:

:<math>\begin{align}
\operatorname{cov}(PX) &= \operatorname{E}[PX~(PX)^{*}]\\
 &= \operatorname{E}[PX~X^{*}P^{*}]\\
 &= P\operatorname{E}[XX^{*}]P^{*}\\
 &= P\operatorname{cov}(X)P^{-1}\\
\end{align}</math>

Hence <math>(\ast)</math> holds if and only if <math>\operatorname{cov}(X)</math> were diagonalisable by <math>P</math>.

This is very constructive, as cov('''X''') is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.