Editing Principal component analysis (section)

=== Properties ===

Some properties of PCA include:<ref name="Jolliffe2002"/>{{page needed|date=November 2020}}

:<big>'''''Property 1'':'''</big> For any integer ''q'', 1 ≤ ''q'' ≤ ''p'', consider the orthogonal [[linear transformation]]
::<math>y =\mathbf{B'}x</math>
:where <math>y</math> is a ''q-element'' vector and <math>\mathbf{B'}</math> is a (''q'' × ''p'') matrix, and let <math>\mathbf{{\Sigma}}_y = \mathbf{B'}\mathbf{\Sigma}\mathbf{B}</math> be the [[variance]]-[[covariance]] matrix for <math>y</math>. Then the trace of <math>\mathbf{\Sigma}_y</math>, denoted <math>\operatorname{tr} (\mathbf{\Sigma}_y)</math>, is maximized by taking <math>\mathbf{B} = \mathbf{A}_q</math>, where <math>\mathbf{A}_q</math> consists of the first ''q'' columns of <math>\mathbf{A}</math> <math>(\mathbf{B'}</math> is the transpose of <math>\mathbf{B})</math>. (<math>\mathbf{A}</math> is not defined here)

:<big>'''''Property 2'':'''</big> Consider again the [[orthonormal transformation]]
::<math>y = \mathbf{B'}x</math>
:with <math>x, \mathbf{B}, \mathbf{A}</math> and <math>\mathbf{\Sigma}_y</math> defined as before. Then <math>\operatorname{tr}(\mathbf{\Sigma}_y)</math> is minimized by taking <math>\mathbf{B} = \mathbf{A}_q^*,</math> where <math>\mathbf{A}_q^*</math> consists of the last ''q'' columns of <math>\mathbf{A}</math>.

The statistical implication of this property is that the last few PCs are not simply unstructured left-overs after removing the important PCs. Because these last PCs have variances as small as possible they are useful in their own right. They can help to detect unsuspected near-constant linear relationships between the elements of {{mvar|x}}, and they may also be useful in [[regression analysis|regression]], in selecting a subset of variables from {{mvar|x}}, and in outlier detection.

:<big>'''''Property 3'':'''</big> (Spectral decomposition of {{math|'''Σ'''}})
::<math>\mathbf{{\Sigma}} = \lambda_1\alpha_1\alpha_1' + \cdots + \lambda_p\alpha_p\alpha_p'</math>

Before we look at its usage, we first look at [[diagonal]] elements,
:<math>\operatorname{Var}(x_j) = \sum_{k=1}^P \lambda_k\alpha_{kj}^2</math>
Then, perhaps the main statistical implication of the result is that not only can we decompose the combined variances of all the elements of {{mvar|x}} into decreasing contributions due to each PC, but we can also decompose the whole [[covariance matrix]] into contributions <math>\lambda_k\alpha_k\alpha_k'</math> from each PC. Although not strictly decreasing, the elements of <math>\lambda_k\alpha_k\alpha_k'</math> will tend to become smaller as <math>k</math> increases, as <math>\lambda_k\alpha_k\alpha_k'</math> is nonincreasing for increasing <math>k</math>, whereas the elements of <math>\alpha_k</math> tend to stay about the same size because of the normalization constraints: <math>\alpha_{k}'\alpha_{k}=1, k=1, \dots, p</math>.