Editing Factor analysis (section)

===Definition===
The model attempts to explain a set of <math>p</math> observations in each of <math>n</math> individuals with a set of <math>k</math> ''common factors'' (<math>f_{i,j}</math>) where there are fewer factors per unit than observations per unit (<math>k<p</math>). Each individual has <math>k</math> of their own common factors, and these are related to the observations via the factor ''loading matrix'' (<math>L  \in \mathbb{R}^{p \times k}</math>), for a single observation, according to

: <math>x_{i,m} - \mu_{i} = l_{i,1} f_{1,m} + \dots + l_{i,k} f_{k,m} + \varepsilon_{i,m} </math>

where
* <math>x_{i,m}</math> is the value of the <math>i</math>th observation of the <math>m</math>th individual,
* <math>\mu_i</math> is the observation mean for the <math>i</math>th observation,
* <math>l_{i,j}</math> is the loading for the <math>i</math>th observation of the <math>j</math>th factor,
* <math>f_{j,m}</math> is the value of the <math>j</math>th factor of the <math>m</math>th individual, and
* <math>\varepsilon_{i,m} </math> is the <math>(i,m)</math>th ''unobserved stochastic error term'' with mean zero and finite variance.

In matrix notation

: <math>X - \Mu = L F + \varepsilon</math>

where observation matrix <math>X \in \mathbb{R}^{p \times n}</math>, loading matrix <math>L \in \mathbb{R}^{p \times k}</math>, factor matrix <math>F \in \mathbb{R}^{k \times n}</math>, error term matrix <math>\varepsilon \in \mathbb{R}^{p \times n}</math> and mean matrix <math>\Mu \in \mathbb{R}^{p \times n}</math> whereby the <math>(i,m)</math>th element is simply <math>\Mu_{i,m}=\mu_i</math>.

Also we will impose the following assumptions on <math>F</math>:

# <math>F</math> and <math>\varepsilon</math> are independent.
# <math>\mathrm{E}(F) = 0</math>; where <math>\mathrm E</math>  is [[Multivariate random variable#Expected value|Expectation]]
# <math>\mathrm{Cov}(F)=I</math> where <math>\mathrm{Cov}</math>  is the [[covariance matrix]], to make sure that the factors are uncorrelated, and <math>I</math> is the [[identity matrix]].

Suppose <math>\mathrm{Cov}(X - \Mu)=\Sigma</math>. Then

: <math>\Sigma=\mathrm{Cov}(X - \Mu)=\mathrm{Cov}(LF + \varepsilon),\,</math>

and therefore, from conditions 1 and 2 imposed on <math>F</math> above, <math>E[LF]=LE[F]=0</math> and <math>Cov(LF+\epsilon)=Cov(LF)+Cov(\epsilon)</math>, giving

: <math>\Sigma = L \mathrm{Cov}(F) L^T + \mathrm{Cov}(\varepsilon),\,</math>

or, setting <math>\Psi:=\mathrm{Cov}(\varepsilon)</math>,

: <math>\Sigma = LL^T + \Psi.\,</math>

For any [[orthogonal matrix]] <math>Q</math>, if we set <math>L^\prime=\ LQ</math> and <math>F^\prime=Q^T F</math>, the criteria for being factors and factor loadings still hold. Hence a set of factors and factor loadings is unique only up to an [[orthogonal transformation]].