Editing Covariance matrix (section)

==Applications==
The covariance matrix is a useful tool in many different areas. From it a [[transformation matrix]] can be derived, called a [[whitening transformation]], that allows one to completely decorrelate the data <ref>{{Cite journal |last1=Kessy |first1=Agnan |last2=Strimmer |first2=Korbinian |last3=Lewin |first3=Alex |year = 2018 |title=Optimal Whitening and Decorrelation |url=https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1277159 |journal=The American Statistician | publisher = Taylor & Francis |volume=72 |issue=4 |pages=309–314 |doi=10.1080/00031305.2016.1277159 | arxiv = 1512.00809}} </ref> or, from a different point of view, to find an optimal basis for representing the data in a compact way{{citation needed|date=February 2012}} (see [[Rayleigh quotient]] for a formal proof and additional properties of covariance matrices).
This is called [[principal component analysis]] (PCA) and the [[Karhunen–Loève transform]] (KL-transform).

The covariance matrix plays a key role in [[financial economics]], especially in [[Modern portfolio theory|portfolio theory]] and its [[mutual fund separation theorem]] and in the [[capital asset pricing model]]. The matrix of covariances among various assets' returns is used to determine, under certain assumptions, the relative amounts of different assets that investors should (in a [[Normative economics|normative analysis]]) or are predicted to (in a [[Positive economics|positive analysis]]) choose to hold in a context of [[Diversification (finance)|diversification]].

===Use in optimization===
The [[evolution strategy]], a particular family of Randomized Search Heuristics, fundamentally relies on a covariance matrix in its mechanism. The characteristic mutation operator draws the update step from a multivariate normal distribution using an evolving covariance matrix. There is a formal proof that the [[evolution strategy]]'s covariance matrix adapts to the inverse of the [[Hessian matrix]] of the search landscape, [[up to]] a scalar factor and small random fluctuations (proven for a single-parent strategy and a static model, as the population size increases, relying on the quadratic approximation).<ref>{{cite journal | doi = 10.1016/j.tcs.2019.09.002 | first = O.M. | last = Shir | author2 = A. Yehudayoff | title = On the covariance-Hessian relation in evolution strategies | journal = Theoretical Computer Science | volume = 801 | pages = 157–174 | publisher = Elsevier | year = 2020 | doi-access = free | arxiv = 1806.03674 }}</ref>
Intuitively, this result is supported by the rationale that the optimal covariance distribution can offer mutation steps whose equidensity probability contours match the level sets of the landscape, and so they maximize the progress rate.

===Covariance mapping===
In '''covariance mapping''' the values of the <math> \operatorname{cov}(\mathbf{X}, \mathbf{Y}) </math> or <math> \operatorname{pcov}(\mathbf{X}, \mathbf{Y} \mid \mathbf{I}) </math> matrix are plotted as a 2-dimensional map. When vectors <math> \mathbf{X} </math> and <math> \mathbf{Y} </math> are discrete [[random function]]s, the map shows statistical relations between different regions of the random functions. Statistically independent regions of the functions show up on the map as zero-level flatland, while positive or negative correlations show up, respectively, as hills or valleys.

In practice the column vectors <math> \mathbf{X}, \mathbf{Y} </math>, and <math> \mathbf{I} </math> are acquired experimentally as rows of <math> n </math> samples, e.g.
<math display="block"> \left[\mathbf{X}_1, \mathbf{X}_2, \dots, \mathbf{X}_n\right] =
\begin{bmatrix}
  X_1(t_1) & X_2(t_1) & \cdots & X_n(t_1) \\ \\
  X_1(t_2) & X_2(t_2) & \cdots & X_n(t_2) \\ \\
  \vdots   &   \vdots & \ddots & \vdots   \\ \\
  X_1(t_m) & X_2(t_m) & \cdots & X_n(t_m)
\end{bmatrix} ,
</math>
where <math> X_j(t_i) </math> is the ''i''-th discrete value in sample ''j'' of the random function <math> X(t) </math>. The expected values needed in the covariance formula are estimated using the [[sample mean]], e.g.
<math display="block"> \langle \mathbf{X} \rangle = \frac{1}{n} \sum_{j=1}^{n} \mathbf{X}_j </math>
and the covariance matrix is estimated by the [[sample covariance]] matrix
<math display="block">
  \operatorname{cov}(\mathbf{X},\mathbf{Y})
  \approx \langle \mathbf{XY^\mathsf{T}} \rangle
  - \langle \mathbf{X} \rangle \langle \mathbf{Y}^\mathsf{T} \rangle ,
</math>
where the angular brackets denote sample averaging as before except that the [[Bessel's correction]] should be made to avoid [[bias of an estimator|bias]]. Using this estimation the partial covariance matrix can be calculated as
<math display="block">
  \operatorname{pcov}(\mathbf{X},\mathbf{Y} \mid \mathbf{I})
  = \operatorname{cov}(\mathbf{X},\mathbf{Y})
  - \operatorname{cov}(\mathbf{X},\mathbf{I})
  \left (    \operatorname{cov}(\mathbf{I},\mathbf{I})
  \backslash \operatorname{cov}(\mathbf{I},\mathbf{Y}) \right ),
</math>
where the backslash denotes the [[Division (mathematics)#Left and right division|left matrix division]] operator, which bypasses the requirement to invert a matrix and is available in some computational packages such as [[Matlab]].<ref name="LJF16">L J Frasinski "Covariance mapping techniques" ''J. Phys. B: At. Mol. Opt. Phys.'' '''49''' 152004 (2016), {{doi|10.1088/0953-4075/49/15/152004}}</ref>

[[Image:Stages of partial covariance mapping.png|thumb|600px|'''Figure 1: Construction of a partial covariance map of N<sub>2</sub> molecules undergoing Coulomb explosion induced by a free-electron laser.'''<ref name="OK13"/> Panels '''a''' and '''b''' map the two terms of the covariance matrix, which is shown in panel '''c'''. Panel '''d''' maps common-mode correlations via intensity fluctuations of the laser. Panel '''e''' maps the partial covariance matrix that is corrected for the intensity fluctuations. Panel '''f''' shows that 10% overcorrection improves the map and makes ion-ion correlations clearly visible. Owing to momentum conservation these correlations appear as lines approximately perpendicular to the autocorrelation line (and to the periodic modulations which are caused by detector ringing).]]
Fig. 1 illustrates how a partial covariance map is constructed on an example of an experiment performed at the [[DESY#FLASH|FLASH]] [[free-electron laser]] in Hamburg.<ref name="OK13">O Kornilov, M Eckstein, M Rosenblatt, C P Schulz, K Motomura, A Rouzée, J Klei, L Foucar, M Siano, A Lübcke, F. Schapper, P Johnsson, D M P Holland, T Schlatholter, T Marchenko, S Düsterer, K Ueda, M J J Vrakking and L J Frasinski "Coulomb explosion of diatomic molecules in intense XUV fields mapped by partial covariance" ''J. Phys. B: At. Mol. Opt. Phys.'' '''46''' 164028 (2013), {{doi|10.1088/0953-4075/46/16/164028}}</ref> The random function <math> X(t) </math> is the [[Time-of-flight_mass_spectrometry|time-of-flight]] spectrum of ions from a [[Coulomb explosion]] of nitrogen molecules multiply ionised by a laser pulse. Since only a few hundreds of molecules are ionised at each laser pulse, the single-shot spectra are highly fluctuating. However, collecting typically <math> m=10^4 </math> such spectra, <math> \mathbf{X}_j(t) </math>, and averaging them over <math> j </math> produces a smooth spectrum <math> \langle \mathbf{X}(t) \rangle </math>, which is shown in red at the bottom of Fig. 1. The average spectrum <math> \langle \mathbf{X} \rangle </math> reveals several nitrogen ions in a form of peaks broadened by their kinetic energy, but to find the correlations between the ionisation stages and the ion momenta requires calculating a covariance map.

In the example of Fig. 1 spectra <math> \mathbf{X}_j(t) </math> and <math> \mathbf{Y}_j(t) </math> are the same, except that the range of the time-of-flight <math> t </math> differs. Panel '''a''' shows <math> \langle \mathbf{XY^\mathsf{T}} \rangle </math>, panel '''b''' shows <math> \langle \mathbf{X} \rangle \langle \mathbf{Y}^\mathsf{T} \rangle </math> and panel '''c''' shows their difference, which is <math> \operatorname{cov}(\mathbf{X},\mathbf{Y}) </math> (note a change in the colour scale). Unfortunately, this map is overwhelmed by uninteresting, common-mode correlations induced by laser intensity fluctuating from shot to shot. To suppress such correlations the laser intensity <math> I_j </math> is recorded at every shot, put into <math> \mathbf{I} </math> and <math> \operatorname{pcov}(\mathbf{X},\mathbf{Y} \mid \mathbf{I}) </math> is calculated as panels '''d''' and '''e''' show. The suppression of the uninteresting correlations is, however, imperfect because there are other sources of common-mode fluctuations than the laser intensity and in principle all these sources should be monitored in vector <math> \mathbf{I} </math>. Yet in practice it is often sufficient to overcompensate the partial covariance correction as panel '''f''' shows, where interesting correlations of ion momenta are now clearly visible as straight lines centred on ionisation stages of atomic nitrogen.

===Two-dimensional infrared spectroscopy===
Two-dimensional infrared spectroscopy employs [[two-dimensional correlation analysis|correlation analysis]] to obtain 2D spectra of the [[condensed matter physics|condensed phase]]. There are two versions of this analysis: [[Two-dimensional correlation analysis#Calculation of the synchronous spectrum|synchronous]] and [[Two-dimensional correlation analysis#Calculation of the asynchronous spectrum|asynchronous]]. Mathematically, the former is expressed in terms of the sample covariance matrix and the technique is equivalent to covariance mapping.<ref>{{cite journal |first=I. |last=Noda |title=Generalized two-dimensional correlation method applicable to infrared, Raman, and other types of spectroscopy |journal=Appl. Spectrosc. |volume=47 |issue= 9|pages=1329–36 |year=1993 |doi=10.1366/0003702934067694 |bibcode=1993ApSpe..47.1329N }}</ref>