Editing Canonical correlation (section)

===Derivation===
Let <math>\Sigma _{XY}</math> be the [[cross-covariance matrix]] for any pair of (vector-shaped) random variables <math>X</math> and <math>Y</math>. The target function to maximize is

:<math>
\rho = \frac{a^T \Sigma _{XY} b}{\sqrt{a^T \Sigma _{XX} a} \sqrt{b^T \Sigma _{YY} b}}.
</math>

The first step is to define a [[change of basis]] and define

:<math>
c = \Sigma _{XX} ^{1/2} a,
</math>

:<math>
d = \Sigma _{YY} ^{1/2} b,
</math>
where <math>\Sigma_{XX}^{1/2}</math> and <math>\Sigma_{YY}^{1/2}</math> can be obtained from the eigen-decomposition (or by [[Square root of a matrix#By diagonalization|diagonalization]]):

:<math>
\Sigma _{XX} ^{1/2} = V_X D_X^{1/2} V_X^\top,\qquad V_X D_X V_X^\top = \Sigma_{XX},
</math>
and
:<math>
\Sigma _{YY} ^{1/2} = V_Y D_Y^{1/2} V_Y^\top,\qquad V_Y D_Y V_Y^\top = \Sigma_{YY}.
</math>

Thus
:<math>
\rho = \frac{c^T \Sigma _{XX} ^{-1/2} \Sigma _{XY} \Sigma _{YY} ^{-1/2} d}{\sqrt{c^Tc} \sqrt{d^Td}}.
</math>

By the [[Cauchy–Schwarz inequality]], ...can someone check the this, particularly the term to the right of "(d) leq"?
:<math>
\left(c^T \Sigma _{XX} ^{-1/2} \Sigma _{XY} \Sigma _{YY} ^{-1/2}  \right)  (d)  \leq \left(c^T \Sigma _{XX} ^{-1/2} \Sigma _{XY} \Sigma _{YY} ^{-1/2} \Sigma _{YY} ^{-1/2} \Sigma _{YX} \Sigma _{XX} ^{-1/2} c \right)^{1/2} \left(d^T d \right)^{1/2},
</math>

:<math>
\rho \leq \frac{\left(c^T \Sigma _{XX}^{-1/2} \Sigma _{XY} \Sigma _{YY}^{-1} \Sigma _{YX} \Sigma_{XX}^{-1/2} c \right)^{1/2}}{\left(c^T c \right)^{1/2}}.
</math>

There is equality if the vectors <math>d</math> and <math>\Sigma_{YY}^{-1/2} \Sigma_{YX} \Sigma_{XX}^{-1/2} c</math> are collinear. In addition, the maximum of correlation is attained if <math>c</math> is the [[eigenvector]] with the maximum eigenvalue for the matrix <math>\Sigma_{XX}^{-1/2} \Sigma_{XY} \Sigma_{YY}^{-1} \Sigma_{YX} \Sigma_{XX}^{-1/2}</math> (see [[Rayleigh quotient]]). The subsequent pairs are found by using [[eigenvalues]] of decreasing magnitudes. Orthogonality is guaranteed by the symmetry of the correlation matrices.

Another way of viewing this computation is that <math>c</math> and <math>d</math> are the left and right [[Singular value decomposition|singular vectors]] of the correlation matrix of X and Y corresponding to the highest singular value.