Editing Principal component analysis (section)

=== Covariances ===
'''X'''<sup>T</sup>'''X''' itself can be recognized as proportional to the empirical sample [[covariance matrix]] of the dataset '''X<sup>T</sup>'''.<ref name="Jolliffe2002"/>{{rp|30–31}}

The sample covariance ''Q'' between two of the different principal components over the dataset is given by:

:<math>\begin{align}
Q(\mathrm{PC}_{(j)}, \mathrm{PC}_{(k)})
& \propto (\mathbf{X}\mathbf{w}_{(j)})^\mathsf{T} (\mathbf{X}\mathbf{w}_{(k)}) \\
& = \mathbf{w}_{(j)}^\mathsf{T} \mathbf{X}^\mathsf{T} \mathbf{X} \mathbf{w}_{(k)} \\
& = \mathbf{w}_{(j)}^\mathsf{T} \lambda_{(k)} \mathbf{w}_{(k)} \\
& = \lambda_{(k)} \mathbf{w}_{(j)}^\mathsf{T} \mathbf{w}_{(k)}
\end{align}</math>

where the eigenvalue property of '''w'''<sub>(''k'')</sub> has been used to move from line 2 to line 3. However eigenvectors '''w'''<sub>(''j'')</sub> and '''w'''<sub>(''k'')</sub> corresponding to eigenvalues of a symmetric matrix are orthogonal (if the eigenvalues are different), or can be orthogonalised (if the vectors happen to share an equal repeated value). The product in the final line is therefore zero; there is no sample covariance between different principal components over the dataset.

Another way to characterise the principal components transformation is therefore as the transformation to coordinates which diagonalise the empirical sample covariance matrix.

In matrix form, the empirical covariance matrix for the original variables can be written
:<math>\mathbf{Q} \propto \mathbf{X}^\mathsf{T} \mathbf{X} = \mathbf{W} \mathbf{\Lambda} \mathbf{W}^\mathsf{T}</math>

The empirical covariance matrix between the principal components becomes
:<math>\mathbf{W}^\mathsf{T} \mathbf{Q} \mathbf{W}
\propto \mathbf{W}^\mathsf{T} \mathbf{W} \, \mathbf{\Lambda} \, \mathbf{W}^\mathsf{T} \mathbf{W}
= \mathbf{\Lambda} </math>

where '''Λ''' is the diagonal matrix of eigenvalues ''λ''<sub>(''k'')</sub> of '''X'''<sup>T</sup>'''X'''. ''λ''<sub>(''k'')</sub> is equal to the sum of the squares over the dataset associated with each component ''k'', that is, ''λ''<sub>(''k'')</sub> = Σ<sub>''i''</sub> ''t''<sub>''k''</sub><sup>2</sup><sub>(''i'')</sub> = Σ<sub>''i''</sub> ('''x'''<sub>(''i'')</sub> ⋅ '''w'''<sub>(''k'')</sub>)<sup>2</sup>.