Editing Cholesky decomposition

{{Short description|Matrix decomposition method}}
{{CS1 config|mode=cs1}}
In [[linear algebra]], the '''Cholesky decomposition''' or '''Cholesky factorization''' (pronounced {{IPAc-en|ʃ|ə|ˈ|l|ɛ|s|k|i}} {{respell|shə|LES|kee}}) is a [[matrix decomposition|decomposition]] of a [[Hermitian matrix|Hermitian]], [[positive-definite matrix]] into the product of a [[lower triangular matrix]] and its [[conjugate transpose]], which is useful for efficient numerical solutions, e.g., [[Monte Carlo simulation]]s. It was discovered by [[André-Louis Cholesky]] for real matrices, and posthumously published in 1924.<ref name="Bulletin">{{Cite journal|last=Benoit|date=1924|title=Note sur une méthode de résolution des équations normales provenant de l'application de la méthode des moindres carrés à un système d'équations linéaires en nombre inférieur à celui des inconnues (Procédé du Commandant Cholesky)|journal=[[Bulletin Géodésique]]|language=fr|volume=2|pages=66–67| doi=10.1007/BF03031308}}</ref>
When it is applicable, the Cholesky decomposition is roughly twice as efficient as the [[LU decomposition]] for solving [[System of linear equations|systems of linear equations]].<ref name="NR">{{cite book|last=Press|first=William H.|author2=Saul A. Teukolsky|author3=William T. Vetterling|author4=Brian P. Flannery|title=Numerical Recipes in C: The Art of Scientific Computing|edition=second|publisher=Cambridge University England EPress|year=1992|page=[https://archive.org/details/numericalrecipes0865unse/page/994 994]|url=https://archive.org/details/numericalrecipes0865unse/page/994|isbn=0-521-43108-5|access-date=2009-01-28}}</ref>

== Statement ==
The Cholesky decomposition of a [[Hermitian matrix|Hermitian]] [[positive-definite matrix]] {{math|'''A'''}}, is a decomposition of the form

<math display=block>\mathbf{A} = \mathbf{L L}^{*},</math>

where {{math|'''L'''}} is a [[lower triangular matrix]] with real and positive diagonal entries, and {{math|'''L'''}}* denotes the [[conjugate transpose]] of {{math|'''L'''}}. Every Hermitian positive-definite matrix (and thus also every real-valued symmetric positive-definite matrix) has a unique Cholesky decomposition.<ref>{{harvtxt|Golub|Van Loan|1996|p=143}}, {{harvtxt|Horn|Johnson|1985|p=407}}, {{harvtxt|Trefethen|Bau|1997|p=174}}.</ref>

The converse holds trivially: if {{math|'''A'''}} can be written as {{math|'''LL'''*}} for some invertible {{math|'''L'''}}, lower triangular or otherwise, then {{math|'''A'''}} is Hermitian and positive definite.

When {{math|'''A'''}} is a real matrix (hence symmetric positive-definite), the factorization may be written
<math display=block>\mathbf{A} = \mathbf{L L}^\mathsf{T},</math>
where {{math|'''L'''}} is a real lower triangular matrix with positive diagonal entries.<ref>{{harvtxt|Horn|Johnson|1985|p=407}}.</ref><ref>{{Cite web|url=https://mathoverflow.net/questions/125960/diagonalizing-a-complex-symmetric-matrix|title=matrices - Diagonalizing a Complex Symmetric Matrix|website=MathOverflow|access-date=2020-01-25}}</ref><ref>{{Cite journal|last1=Schabauer|first1=Hannes|last2=Pacher|first2=Christoph|last3=Sunderland|first3=Andrew G.|last4=Gansterer|first4=Wilfried N.|date=2010-05-01|title=Toward a parallel solver for generalized complex symmetric eigenvalue problems|journal=Procedia Computer Science|series=ICCS 2010|language=en|volume=1|issue=1|pages=437–445|doi=10.1016/j.procs.2010.04.047|issn=1877-0509|doi-access=free}}</ref>

=== Positive semidefinite matrices ===
If a Hermitian matrix {{math|'''A'''}} is only positive semidefinite, instead of positive definite, then it still has a decomposition of the form {{math|1='''A''' = '''LL'''*}} where the diagonal entries of {{math|'''L'''}} are allowed to be zero.<ref>{{harvtxt|Golub|Van Loan|1996|p=147}}.</ref>
The decomposition need not be unique, for example:
<math display=block>\begin{bmatrix}0 & 0 \\0 & 1\end{bmatrix} = \mathbf L \mathbf L^*, \quad \quad \mathbf L=\begin{bmatrix}0 & 0\\ \cos \theta & \sin\theta\end{bmatrix},</math>
for any {{mvar|θ}}. However, if the rank of {{math|'''A'''}} is {{mvar|r}}, then there is a unique lower triangular {{math|'''L'''}} with exactly {{mvar|r}} positive diagonal elements and {{math|''n'' − ''r''}} columns containing all zeroes.<ref>
{{Cite book |last=Gentle |first=James E. |date=1998 |title=Numerical Linear Algebra for Applications in Statistics |isbn=978-1-4612-0623-1 |publisher=Springer |language=en |page= 94}}</ref>

Alternatively, the decomposition can be made unique when a pivoting choice is fixed. Formally, if {{math|'''A'''}} is an {{math|''n'' × ''n''}} positive semidefinite matrix of rank {{mvar|r}}, then there is at least one permutation matrix {{math|'''P'''}} such that {{math|'''P A P'''<sup>T</sup>}} has a unique decomposition of the form {{math|1='''P A P'''<sup>T</sup> = '''L L'''<sup>*</sup>}} with
<math display=inline> \mathbf L =  \begin{bmatrix} \mathbf L_1 & 0 \\ \mathbf L_2 & 0\end{bmatrix} </math>,
where {{math|'''L'''<sub>1</sub>}} is an {{math|''r'' × ''r''}} lower triangular matrix with positive diagonal.<ref>{{Cite book |last=Higham |first=Nicholas J. |chapter-url=http://eprints.maths.manchester.ac.uk/1193/ |title=Reliable Numerical Computation |publisher=Oxford University Press |year=1990 |isbn=978-0-19-853564-5 |editor-last=Cox |editor-first=M. G. |location=Oxford, UK |pages=161–185 |language=en |editor-last2=Hammarling |editor-first2=S. J. |chapter=Analysis of the Cholesky Decomposition of a Semi-definite Matrix}}</ref>

== LDL decomposition ==

A closely related variant of the classical Cholesky decomposition is the LDL decomposition,

<math display=block>\mathbf{A} = \mathbf{L D L}^*,</math>

where {{math|'''L'''}} is a [[Unitriangular matrix|lower unit triangular (unitriangular)]] matrix, and {{math|'''D'''}} is a [[diagonal matrix|diagonal]] matrix. That is, the diagonal elements of {{math|'''L'''}} are required to be 1 at the cost of introducing an additional diagonal matrix {{math|'''D'''}} in the decomposition. The main advantage is that the LDL decomposition can be computed and used with essentially the same algorithms, but avoids extracting square roots.<ref name="kri">{{cite conference|last=Krishnamoorthy|first=Aravindh|author2=Menon, Deepak|contribution=Matrix Inversion Using Cholesky Decomposition|title=2013 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA)|pages=70–72|publisher=IEEE|arxiv=1111.4144|url=https://ieeexplore.ieee.org/document/6710599}}</ref>

For this reason, the LDL decomposition is often called the ''square-root-free Cholesky'' decomposition. For real matrices, the factorization has the form {{math|1='''A''' = '''LDL'''<sup>T</sup>}} and is often referred to as '''{{math|LDLT}} decomposition''' (or {{math|'''LDL<sup>T</sup>'''}} decomposition, or '''LDL′''').  It is reminiscent of the [[eigendecomposition of a matrix#Real symmetric matrices|eigendecomposition of real symmetric matrices]], {{math|1='''A''' = '''QΛQ'''<sup>T</sup>}}, but is quite different in practice because {{math|'''Λ'''}} and {{math|'''D'''}} are not [[similar matrices]].

The LDL decomposition is related to the classical Cholesky decomposition of the form {{math|'''LL'''*}} as follows:

<math display=block>\mathbf{A} = \mathbf{L D L}^* = \mathbf L \mathbf D^{1/2} \left(\mathbf D^{1/2} \right)^* \mathbf L^* =
\mathbf L \mathbf D^{1/2} \left(\mathbf L \mathbf D^{1/2}\right)^*.</math>

Conversely, given the classical Cholesky decomposition <math display=inline>\mathbf A = \mathbf C \mathbf C^*</math> of a positive definite matrix, if {{math|'''S'''}} is a diagonal matrix that contains the main diagonal of <math display=inline>\mathbf C</math>, then {{math|'''A'''}} can be decomposed as <math display=inline>\mathbf L \mathbf D \mathbf L^*</math> where
<math display=block> \mathbf L  = \mathbf C \mathbf S^{-1} </math> (this rescales each column to make diagonal elements 1),
<math display="block"> \mathbf D  = \mathbf S\mathbf S^*. </math>

If {{math|'''A'''}} is positive definite then the diagonal elements of {{math|'''D'''}} are all positive.
For positive semidefinite {{math|'''A'''}}, an  <math display=inline>\mathbf L \mathbf D \mathbf L^*</math> decomposition exists where the number of non-zero elements on the diagonal {{math|'''D'''}} is exactly the rank of {{math|'''A'''}}.<ref>{{Cite thesis |last=So |first=Anthony Man-Cho |title=A Semidefinite Programming Approach to the Graph Realization Problem: Theory, Applications and Extensions |date=2007 |url=http://www.se.cuhk.edu.hk/~manchoso/papers/thesis.pdf |language=en |type=PhD| at=Theorem 2.2.6}}</ref>
Some indefinite matrices for which no Cholesky decomposition exists have an LDL decomposition with negative entries in {{math|'''D'''}}: it suffices that the first {{math|''n'' − 1}} [[Minor (linear algebra)#Other applications|leading principal minors]] of {{math|'''A'''}} are non-singular.<ref>{{harvtxt|Golub|Van Loan|1996|loc=Theorem 4.1.3}}</ref>

== Example ==

Here is the Cholesky decomposition of a symmetric real matrix:

<math display=block>\begin{align}
  \begin{pmatrix}
      4 &  12 & -16 \\
     12 &  37 & -43 \\
    -16 & -43 &  98 \\
  \end{pmatrix}
=
  \begin{pmatrix}
     2 &  0 &  0 \\
     6 &  1 &  0 \\
    -8 &  5 &  3 \\
  \end{pmatrix}
  \begin{pmatrix}
     2 &  6 & -8 \\
     0 &  1 &  5 \\
     0 &  0 &  3 \\
  \end{pmatrix}.
\end{align}</math>

And here is its LDL<sup>T</sup> decomposition:

<math display=block>\begin{align}
  \begin{pmatrix}
      4 &  12 & -16 \\
     12 &  37 & -43 \\
    -16 & -43 &  98 \\
  \end{pmatrix}
& =
  \begin{pmatrix}
     1 &  0 &  0 \\
     3 &  1 &  0 \\
    -4 &  5 &  1 \\
  \end{pmatrix}
  \begin{pmatrix}
     4 &  0 &  0 \\
     0 &  1 &  0 \\
     0 &  0 &  9 \\
  \end{pmatrix}
  \begin{pmatrix}
     1 &  3 & -4 \\
     0 &  1 &  5 \\
     0 &  0 &  1 \\
  \end{pmatrix}.
\end{align}</math>

== Geometric interpretation ==
{{See also|Whitening transformation}}
[[File:Cholesky decomposition with two ellipses.svg|thumb|The ellipse is a linear image of the unit circle. The two vectors <math display=inline>v_1, v_2</math> are conjugate axes of the ellipse chosen such that <math display=inline>v_1</math> is parallel to the first axis and <math display=inline>v_2</math> is within the plane spanned by the first two axes.]]
The Cholesky decomposition is equivalent to a particular choice of [[Conjugate diameters|conjugate axes]] of an [[ellipsoid]].<ref>Pope, Stephen B. "[https://tcg.mae.cornell.edu/pubs/Pope_FDA_08.pdf Algorithms for ellipsoids.]" Cornell University Report No. FDA (2008): 08-01.</ref> In detail, let the ellipsoid be defined as <math display=inline>y^TAy = 1</math>, then by definition, a set of vectors <math display=inline>v_1, ..., v_n</math> are conjugate axes of the ellipsoid iff <math display=inline>v_i^T A v_j = \delta_{ij}</math>. Then, the ellipsoid is precisely<math display="block">\left\{ \sum_i x_i v_i : x^T x = 1 \right\} = f(\mathbb S^n)</math>where <math display=inline>f</math> maps the basis vector <math display=inline>e_i \mapsto v_i</math>, and <math display=inline>\mathbb S^n</math> is the unit sphere in n dimensions. That is, the ellipsoid is a linear image of the unit sphere.

Define the matrix <math display=inline>V := [v_1 | v_2 | \cdots | v_n]</math>, then <math display=inline>v_i^T A v_j = \delta_{ij}</math> is equivalent to <math display=inline>V^TAV = I</math>. Different choices of the conjugate axes correspond to different decompositions.

The Cholesky decomposition corresponds to choosing <math display=inline>v_1</math> to be parallel to the first axis, <math display=inline>v_2</math> to be within the plane spanned by the first two axes, and so on. This makes <math display=inline>V</math> an upper-triangular matrix. Then, there is <math display=inline>A = LL^T</math>, where <math display=inline>L = (V^{-1})^T</math> is lower-triangular.

Similarly, [[principal component analysis]] corresponds to choosing <math display=inline>v_1, ..., v_n</math> to be perpendicular. Then, let <math display=inline>\lambda = 1/\|v_i\|^2</math> and <math display=inline>\Sigma = \mathrm{diag}(\lambda_1, ..., \lambda_n)</math>,  and there is <math display=inline>V = U\Sigma^{-1/2}</math> where <math display=inline>U</math> is an orthogonal matrix. This then yields <math display=inline>A = U\Sigma U^T</math>.

== Applications ==
===Numerical solution of system of linear equations===
The Cholesky decomposition is mainly used for the numerical solution of [[system of linear equations|linear equations]] <math display=inline>\mathbf{Ax} = \mathbf{b}</math>. If {{math|'''A'''}} is symmetric and positive definite, then  <math display=inline>\mathbf{Ax} = \mathbf{b}</math> can be solved by first computing the Cholesky decomposition  <math display=inline>\mathbf{A} = \mathbf{LL}^\mathrm{*}</math>, then solving <math display=inline>\mathbf{Ly} = \mathbf{b}</math> for {{math|'''y'''}} by [[forward substitution]], and finally solving <math display=inline>\mathbf{L^*x} = \mathbf{y}</math> for {{math|'''x'''}} by [[back substitution]].

An alternative way to eliminate taking square roots in the <math display=inline>\mathbf{LL}^\mathrm{*}</math> decomposition is to compute the LDL decomposition <math display=inline>\mathbf{A} = \mathbf{LDL}^\mathrm{*}</math>, then solving <math display=inline>\mathbf{Ly} = \mathbf{b}</math> for {{math|'''y'''}}, and finally solving <math display=inline>\mathbf{DL}^\mathrm{*}\mathbf{x} = \mathbf{y}</math>.

For linear systems that can be put into symmetric form, the Cholesky decomposition (or its LDL variant) is the method of choice, for superior efficiency and numerical stability. Compared to the [[LU decomposition]], it is roughly twice as efficient.<ref name="NR"/>

===Linear least squares===
In [[linear least squares (mathematics)|linear least squares]] problem one seeks a solution {{math|1='''x'''}} of an over-determined system {{math|1='''Ax''' = '''l'''}}, such that quadratic norm of the residual vector {{math|1='''Ax-l'''}} is minimum. This may be accomplished by solving by Cholesky decomposition
normal equations <math>\mathbf{Nx}=\mathbf{A}^\mathsf{T}\mathbf{l}</math>, where <math>\mathbf{N}=\mathbf{A}^\mathsf{T}\mathbf{A}</math> is symmetric positive definite. Symmetric equation matrix
may also come from an energy functional, which must be positive from physical considerations; this happens frequently in the numerical solution of [[partial differential equation]]s.

Such method is economic and works well in many applications, however it fails for near singular
{{math|1='''N'''}}. This is best illustrated in pathological case of square <math>\mathbf{A}</math>,
where determinant of {{math|1='''N'''}} is square of that of the original system {{math|1='''Ax''' = '''l'''}}.
Then it is best to apply SVD or QR decomposition. Givens QR has the advantage that similarly
to normal equations there is no need to keep the whole matrix {{math|1='''A'''}} as it is
possible to update Cholesky factor with consecutive rows of {{math|1='''A'''}}.

===Non-linear optimization===
[[Non-linear least squares]] are a particular case of nonlinear optimization. Let <math display=inline>\mathbf{f}(\mathbf{x})=\mathbf{l}</math> be an over-determined system of equations with a non-linear function <math>\mathbf{f}</math> returning vector results. The aim is to minimize square norm of residuals <math display=inline>\mathbf{v}=\mathbf{f}(\mathbf{x})-\mathbf{l}</math>. An approximate [[Newton's method]] solution is obtained by expanding <math>\mathbf{f}</math> into curtailed Taylor series 
<math>\bf f(x_{\rm 0}+\delta x)\approx f(x_{\rm 0})+(\partial f/\partial x)\delta x</math> yielding
linear least squares problem for <math>\bf\delta x</math>
: <math>{\bf(\partial f/\partial x)\delta x=l-f(x_{\rm 0})=v,\;\;\min_{\delta x}=\|v\|^2}.</math>
Of course because of neglect of higher Taylor terms such solution is only approximate, if it ever exists. Now one could update expansion point to <math>\bf x_{\rm n+1}=x_{\rm n}+\delta x</math>
and repeat the whole procedure, hoping that (i) iterations converge to a solution
and (ii) that the solution is the one needed. Unfortunately neither is guaranteed and
must be verified.

[[Non-linear least squares]] may be also applied to the linear least squares problem by setting  
<math>\bf x_{\rm 0}=0</math> and <math>\bf f(x_{\rm 0})=Ax</math>. This may be useful if Cholesky decomposition yields an inaccurate inverse <math>\bf R^{\rm -1}</math> for the triangle matrix where
<math>\bf R^{\rm T}R=N</math>, because of rounding errors. Such a procedure is called a ''differential correction'' of the solution. As long as iterations converge, by virtue of the [[Banach fixed-point theorem]] they yield the solution with a precision that is only limited by the precision of the calculated residuals <math>\bf v=Ax-l</math>. The precision is independent rounding errors in <math>\bf R^{\rm -1}</math>. Poor <math>\bf R^{\rm -1}</math> may restrict region of initial <math>\bf x_{\rm 0}</math>
yielding convergence or altogether preventing it. Usually convergence is slower e.g. linear
so that <math>\bf\|\delta x_{\rm n+1}\|\approx\|=\alpha\delta x_{\rm n}\|</math> where constant
<math>\alpha<1</math>. Such slow convergence may be sped by ''Aitken <math>\delta^2</math>'' method.
If calculation of <math>\bf R^{\rm -1}</math> is very costly, it is possible to use it from previous 
iterations as long as convergence is maintained. Such Cholesky procedure may work even for Hilbert matrices, 
notoriously difficult to invert.<ref>{{cite journal
 | last1 = Schwarzenberg-Czerny | first1 = A.
 | journal = Astronomy and Astrophysics Supplement
 | pages = 405–410
 | title = On matrix factorization and efficient least squares solution
 | volume = 110
 | year = 1995| bibcode = 1995A&AS..110..405S
 }}</ref> 

Non-linear multi-variate functions may be minimized over their parameters using variants of [[Newton's method]] called ''quasi-Newton'' methods.  At iteration k, the search steps in a direction <math display=inline> p_k </math> defined by solving <math display=inline> B_k p_k  =  -g_k </math> for <math display=inline> p_k </math>, where <math display=inline> p_k </math> is the step direction, <math display=inline> g_k </math> is the [[gradient]], and <math display=inline> B_k </math> is an approximation to the [[Hessian matrix]] formed by repeating rank-1 updates at each iteration.  Two well-known update formulas are called [[Davidon–Fletcher–Powell]] (DFP) and [[BFGS method|Broyden–Fletcher–Goldfarb–Shanno]] (BFGS). Loss of the positive-definite condition through round-off error is avoided if rather than updating an approximation to the inverse of the Hessian, one updates the Cholesky decomposition of an approximation of the Hessian matrix itself.<ref>{{Cite book |last=Arora |first=Jasbir Singh |url=https://books.google.com/books?id=9FbwVe577xwC&pg=PA327 |title=Introduction to Optimum Design |date=2004-06-02 |publisher=Elsevier |isbn=978-0-08-047025-2 |language=en}}</ref>

===Monte Carlo simulation===
The Cholesky decomposition is commonly used in the [[Monte Carlo method]] for simulating systems with multiple correlated variables. The [[covariance matrix]] is decomposed to give the lower-triangular {{math|'''L'''}}. Applying this to a vector of uncorrelated observations in a sample {{math|'''u'''}} produces a sample vector '''Lu''' with the covariance properties of the system being modeled.<ref name="Matlab documentation">[http://www.mathworks.com/help/techdoc/ref/randn.html Matlab randn documentation]. mathworks.com.</ref>

The following simplified example shows the economy one gets from the Cholesky decomposition:  suppose the goal is to generate two correlated normal variables <math display=inline>x_1</math> and <math display=inline>x_2</math> with given correlation coefficient <math display=inline>\rho</math>. To accomplish that, it is necessary to first generate two uncorrelated Gaussian random variables <math display=inline>z_1</math> and <math display=inline>z_2</math> (for example, via a [[Box–Muller transform]]).  Given the  required correlation coefficient <math display=inline>\rho</math>, the correlated normal variables can be obtained via the transformations <math display=inline>x_1 = z_1</math> and <math display="inline">x_2 =  \rho z_1 + \sqrt{1 - \rho^2} z_2</math>.

===Kalman filters===
[[Unscented Kalman filter]]s commonly use the Cholesky decomposition to choose a set of so-called sigma points.  The Kalman filter tracks the average state of a system as a vector {{math|'''x'''}} of length {{mvar|N}} and covariance as an {{math|''N'' × ''N''}} matrix {{math|'''P'''}}.  The matrix {{math|'''P'''}} is always positive semi-definite and can be decomposed into '''LL'''<sup>T</sup>.  The columns of {{math|'''L'''}} can be added and subtracted from the mean {{math|'''x'''}} to form a set of {{math|2''N''}} vectors called ''sigma points''.  These sigma points completely capture the mean and covariance of the system state.

===Matrix inversion===
The explicit [[inverse matrix|inverse]] of a Hermitian matrix can be computed by Cholesky decomposition, in a manner similar to solving linear systems, using <math display=inline>n^3</math> operations (<math display=inline>\tfrac{1}{2} n^3</math> multiplications).<ref name="kri"/> The entire inversion can even be efficiently performed in-place.

A non-Hermitian matrix {{math|'''B'''}} can also be inverted using the following identity, where '''BB'''* will always be Hermitian:

<math display=block>\mathbf{B}^{-1} = \mathbf{B}^* (\mathbf{B B}^*)^{-1}.</math>

== Computation ==
There are various methods for calculating the Cholesky decomposition. The computational complexity of commonly used algorithms is {{math|''O''(''n''<sup>3</sup>)}} in general.{{Citation needed|date=June 2011}} The algorithms described below all involve about {{math|(1/3)''n''<sup>3</sup>}} [[FLOP]]s ({{math|''n''<sup>3</sup>/6}} multiplications and the same number of additions) for real flavors and {{math|(4/3)''n''<sup>3</sup>}} [[FLOP]]s for complex flavors,<ref>?potrf Intel® Math Kernel Library [https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/lapack-routines/lapack-linear-equation-routines/lapack-linear-equation-computational-routines/matrix-factorization-lapack-computational-routines/potrf.html#potrf]</ref> where {{mvar|n}} is the size of the matrix {{math|'''A'''}}. Hence, they have half the cost of the [[LU decomposition]], which uses {{math|2''n''<sup>3</sup>/3}} FLOPs (see Trefethen and Bau 1997).

Which of the algorithms below is faster depends on the details of the implementation. Generally, the first algorithm will be slightly slower because it accesses the data in a less regular manner.
The Cholesky decomposition was shown to be numerically stable without need for pivoting.<ref>{{cite journal
 | last1 = Turing | first1 = A. M.
 | journal = Quart. J. Mech. Appl. Math.
 | pages = 287–308
 | title = Rounding-off errors in matrix processes
 | volume = 1
 | year = 1948| doi = 10.1093/qjmam/1.1.287
 }}</ref>

=== The Cholesky algorithm ===
The '''Cholesky algorithm''', used to calculate the decomposition matrix {{math|'''L'''}}, is a modified version of [[Gaussian elimination]].

The recursive algorithm starts with {{math|1=''i'' := 1}} and
:{{math|1='''A'''<sup>(1)</sup> := '''A'''}}.

At step {{mvar|i}}, the matrix {{math|'''A'''<sup>(''i'')</sup>}} has the following form:
<math display=block>\mathbf{A}^{(i)}=
\begin{pmatrix}
\mathbf{I}_{i-1} & 0              & 0 \\
0                & a_{i,i}        & \mathbf{b}_{i}^{*} \\
0                & \mathbf{b}_{i} & \mathbf{B}^{(i)}
\end{pmatrix},
</math>
where {{math|'''I'''<sub>''i''−1</sub>}} denotes the [[identity matrix]] of dimension {{math|''i'' − 1}}.

If the matrix {{math|'''L'''<sub>''i''</sub>}} is defined by
<math display=block>\mathbf{L}_{i}:=
\begin{pmatrix}
\mathbf{I}_{i-1} & 0                                  & 0 \\
0                & \sqrt{a_{i,i}}           & 0 \\
0                & \frac{1}{\sqrt{a_{i,i}}} \mathbf{b}_{i} & \mathbf{I}_{n-i}
\end{pmatrix},
</math>
(note that {{math|''a''<sub>''i,i''</sub> > 0}} since  {{math|'''A'''<sup>(''i'')</sup>}} is positive definite),
then {{math|'''A'''<sup>(''i'')</sup>}} can be written as
<math display=block>\mathbf{A}^{(i)} = \mathbf{L}_{i} \mathbf{A}^{(i+1)} \mathbf{L}_{i}^{*}</math>
where
<math display=block>\mathbf{A}^{(i+1)}=
\begin{pmatrix}
\mathbf{I}_{i-1} & 0 & 0 \\
0                & 1 & 0 \\
0                & 0 & \mathbf{B}^{(i)} - \frac{1}{a_{i,i}} \mathbf{b}_{i} \mathbf{b}_{i}^{*}
\end{pmatrix}.</math>
Note that {{math|'''b'''<sub>''i''</sub> '''b'''<sub>''i''</sub>*}} is an [[outer product]], therefore this algorithm is called the ''outer-product version'' in (Golub & Van Loan).

This is repeated for {{mvar|i}} from 1 to {{mvar|n}}. After {{mvar|n}} steps, {{math|1='''A'''<sup>(''n''+1)</sup> = '''I'''}} is obtained, and hence, the lower triangular matrix {{mvar|L}} sought for is calculated as

<math display=block>\mathbf{L} := \mathbf{L}_{1} \mathbf{L}_{2} \dots \mathbf{L}_{n}.</math>

=== The Cholesky–Banachiewicz and Cholesky–Crout algorithms ===
[[File:Chol.gif|thumb|Access pattern (white) and writing pattern (yellow) for the in-place Cholesky—Banachiewicz algorithm on a 5×5 matrix]]
If the equation
<math display=block>\begin{align}
\mathbf{A} = \mathbf{LL}^T & =
\begin{pmatrix}   L_{11} & 0 & 0 \\
   L_{21} & L_{22} & 0 \\
   L_{31} & L_{32} & L_{33}\\
\end{pmatrix}
\begin{pmatrix}   L_{11} & L_{21} & L_{31} \\
   0 & L_{22} & L_{32} \\
   0 & 0 & L_{33}
\end{pmatrix} \\[8pt]
& =
\begin{pmatrix}   L_{11}^2 &   &(\text{symmetric})   \\
   L_{21}L_{11} & L_{21}^2 + L_{22}^2& \\
   L_{31}L_{11} & L_{31}L_{21}+L_{32}L_{22} & L_{31}^2 + L_{32}^2+L_{33}^2
\end{pmatrix},
\end{align}</math>

is written out, the following is obtained:

<math display=block>\begin{align}
\mathbf{L} = 
\begin{pmatrix} \sqrt{A_{11}} &  0 & 0  \\
A_{21}/L_{11} & \sqrt{A_{22} - L_{21}^2} & 0 \\
A_{31}/L_{11} &  \left( A_{32} - L_{31}L_{21} \right) /L_{22}  &\sqrt{A_{33}- L_{31}^2 - L_{32}^2}
\end{pmatrix}
\end{align}</math>

and therefore the following formulas for the entries of {{math|'''L'''}}:

<math display=block> L_{j,j} = (\pm)\sqrt{ A_{j,j} - \sum_{k=1}^{j-1} L_{j,k}^2 }, </math>
<math display=block> L_{i,j} = \frac{1}{L_{j,j}} \left( A_{i,j} - \sum_{k=1}^{j-1} L_{i,k} L_{j,k} \right) \quad \text{for } i>j. </math>

For complex and real matrices, inconsequential arbitrary sign changes of diagonal and associated off-diagonal elements are allowed. The expression under the [[square root]] is always positive if {{math|'''A'''}} is real and positive-definite. 

For complex Hermitian matrix, the following formula applies:

<math display=block> L_{j,j} = \sqrt{ A_{j,j} - \sum_{k=1}^{j-1} L_{j,k}^*L_{j,k} }, </math>
<math display=block> L_{i,j} = \frac{1}{L_{j,j}} \left( A_{i,j} - \sum_{k=1}^{j-1} L_{j,k}^* L_{i,k} \right) \quad \text{for } i>j. </math>

So it now is possible to compute the {{math|(''i'', ''j'')}} entry if the entries to the left and above are known. The computation is usually arranged in either of the following orders:
* The '''Cholesky–Banachiewicz algorithm''' starts from the upper left corner of the matrix {{mvar|L}} and proceeds to calculate the matrix row by row. 
<syntaxhighlight lang="C">
for (i = 0; i < dimensionSize; i++) {
    for (j = 0; j <= i; j++) {
        float sum = 0;
        for (k = 0; k < j; k++)
            sum += L[i][k] * L[j][k];

        if (i == j)
            L[i][j] = sqrt(A[i][i] - sum);
        else
            L[i][j] = (1.0 / L[j][j] * (A[i][j] - sum));
    }
}
</syntaxhighlight>
The above algorithm can be succinctly expressed as combining a [[dot product]] and [[matrix multiplication]] in vectorized programming languages such as [[Fortran]] as the following,
<syntaxhighlight lang="Fortran">
do i = 1, size(A,1)
    L(i,i) = sqrt(A(i,i) - dot_product(L(i,1:i-1), L(i,1:i-1)))
    L(i+1:,i) = (A(i+1:,i) - matmul(conjg(L(i,1:i-1)), L(i+1:,1:i-1))) / L(i,i)
end do
</syntaxhighlight>
where <code>conjg</code> refers to complex conjugate of the elements.

* The '''Cholesky–Crout algorithm''' starts from the upper left corner of the matrix {{mvar|L}} and proceeds to calculate the matrix column by column. <syntaxhighlight lang="C">
for (j = 0; j < dimensionSize; j++) {
    float sum = 0;
    for (k = 0; k < j; k++) {
        sum += L[j][k] * L[j][k];
    }
    L[j][j] = sqrt(A[j][j] - sum);

    for (i = j + 1; i < dimensionSize; i++) {
        sum = 0;
        for (k = 0; k < j; k++) {
            sum += L[i][k] * L[j][k];
        }
        L[i][j] = (1.0 / L[j][j] * (A[i][j] - sum));
    }
}
</syntaxhighlight>
The above algorithm can be succinctly expressed as combining a [[dot product]] and [[matrix multiplication]] in vectorized programming languages such as [[Fortran]] as the following,
<syntaxhighlight lang="Fortran">
do i = 1, size(A,1)
    L(i,i) = sqrt(A(i,i) - dot_product(L(1:i-1,i), L(1:i-1,i)))
    L(i,i+1:) = (A(i,i+1:) - matmul(conjg(L(1:i-1,i)), L(1:i-1,i+1:))) / L(i,i)
end do
</syntaxhighlight>
where <code>conjg</code> refers to complex conjugate of the elements.

Either pattern of access allows the entire computation to be performed in-place if desired.

=== Stability of the computation ===
Suppose that there is a desire to solve a [[condition number|well-conditioned]] system of linear equations. If the LU decomposition is used, then the algorithm is unstable unless some sort of pivoting strategy is used. In the latter case, the error depends on the so-called growth factor of the matrix, which is usually (but not always) small.

Now, suppose that the Cholesky decomposition is applicable. As mentioned above, the algorithm will be twice as fast. Furthermore, no [[Pivot element|pivoting]] is necessary, and the error will always be small. Specifically, if {{math|1='''Ax''' = '''b'''}}, and {{math|'''y'''}} denotes the computed solution, then {{math|'''y'''}} solves the perturbed system ({{math|1='''A''' + '''E''')'''y''' = '''b'''}}, where
<math display=block> \|\mathbf{E}\|_2 \le c_n \varepsilon \|\mathbf{A}\|_2. </math>
Here ||·||<sub>2</sub> is the [[matrix norm|matrix 2-norm]], ''c<sub>n</sub>'' is a small constant depending on {{mvar|n}}, and {{mvar|ε}} denotes the [[unit round-off]].

One concern with the Cholesky decomposition to be aware of is the use of square roots. If the matrix being factorized is positive definite as required, the numbers under the square roots are always positive ''in exact arithmetic''. Unfortunately, the numbers can become negative because of [[round-off error]]s, in which case the algorithm cannot continue. However, this can only happen if the matrix is very ill-conditioned. One way to address this is to add a diagonal correction matrix to the matrix being decomposed in an attempt to promote the positive-definiteness.<ref>{{cite journal
 | last1 = Fang | first1 = Haw-ren
 | last2 = O'Leary | first2 = Dianne P. | author2-link = Dianne P. O'Leary
 | doi = 10.1007/s10107-007-0177-6
 | issue = 2
 | journal = Mathematical Programming
 | mr = 2411401
 | pages = 319–349
 | title = Modified Cholesky algorithms: a catalog with new approaches
 | url = https://www.cs.umd.edu/~oleary/tr/tr4807.pdf
 | volume = 115
 | year = 2008| hdl = 1903/3674
 }}</ref> While this might lessen the accuracy of the decomposition, it can be very favorable for other reasons; for example, when performing [[Newton's method in optimization]], adding a diagonal matrix can improve stability when far from the optimum.

=== LDL decomposition ===
An alternative form, eliminating the need to take square roots when {{math|'''A'''}} is symmetric, is the symmetric indefinite factorization<ref>{{cite book |first=D. |last=Watkins |year=1991 |title=Fundamentals of Matrix Computations |url=https://archive.org/details/fundamentalsofma0000watk |url-access=registration |location=New York |publisher=Wiley |page=[https://archive.org/details/fundamentalsofma0000watk/page/84 84] |isbn=0-471-61414-9 }}</ref>
<math display=block>
\begin{align}
\mathbf{A} = \mathbf{LDL}^\mathrm{T} & =
\begin{pmatrix}   1 & 0 & 0 \\
   L_{21} & 1 & 0 \\
   L_{31} & L_{32} & 1\\
\end{pmatrix}
\begin{pmatrix}   D_1 & 0 & 0 \\
   0 & D_2 & 0 \\
   0 & 0 & D_3\\
\end{pmatrix}
\begin{pmatrix}   1 & L_{21} & L_{31} \\
   0 & 1 & L_{32} \\
   0 & 0 & 1\\
\end{pmatrix} \\[8pt]
& = \begin{pmatrix}   D_1 &   &(\mathrm{symmetric})   \\
   L_{21}D_1 & L_{21}^2D_1 + D_2& \\
   L_{31}D_1 & L_{31}L_{21}D_{1}+L_{32}D_2 & L_{31}^2D_1 + L_{32}^2D_2+D_3.
\end{pmatrix}.
\end{align}
</math>

The following recursive relations apply for the entries of {{math|'''D'''}} and {{math|'''L'''}}:
<math display=block> D_j = A_{jj} - \sum_{k=1}^{j-1} L_{jk}^2 D_k, </math>
<math display=block> L_{ij} = \frac{1}{D_j} \left( A_{ij} - \sum_{k=1}^{j-1} L_{ik} L_{jk} D_k \right) \quad \text{for } i>j. </math>

This works as long as the generated diagonal elements in {{math|'''D'''}} stay non-zero. The decomposition is then unique. {{math|'''D'''}} and {{math|'''L'''}} are real if {{math|'''A'''}} is real.

For complex Hermitian matrix {{math|'''A'''}}, the following formula applies:

<math display=block> D_{j} = A_{jj} - \sum_{k=1}^{j-1} L_{jk}L_{jk}^* D_k, </math>
<math display=block> L_{ij} = \frac{1}{D_j} \left( A_{ij} - \sum_{k=1}^{j-1} L_{ik} L_{jk}^* D_k \right) \quad \text{for } i>j. </math>

Again, the pattern of access allows the entire computation to be performed in-place if desired.

===Block variant===
When used on indefinite matrices, the '''LDL'''* factorization is known to be unstable without careful pivoting;<ref>{{cite book|last=Nocedal|first=Jorge|title=Numerical Optimization|year=2000|publisher=Springer}}</ref> specifically, the elements of the factorization can grow arbitrarily. A possible improvement is to perform the factorization on block sub-matrices, commonly 2 × 2:<ref>{{cite journal
 | last = Fang | first = Haw-Ren
 | doi = 10.1093/imanum/drp053
 | issue = 2
 | journal = IMA Journal of Numerical Analysis
 | mr = 2813183
 | pages = 528–555
 | title = Stability analysis of block <math>LDL^T</math> factorization for symmetric indefinite matrices
 | volume = 31
 | year = 2011}}</ref>

<math display=block>\begin{align}
\mathbf{A} = \mathbf{LDL}^\mathrm{T} & =
\begin{pmatrix}
 \mathbf I & 0 & 0 \\
 \mathbf L_{21} & \mathbf I & 0 \\
 \mathbf L_{31} & \mathbf L_{32} & \mathbf I\\
\end{pmatrix}
\begin{pmatrix}
 \mathbf D_1 & 0 & 0 \\
 0 & \mathbf D_2 & 0 \\
 0 & 0 & \mathbf D_3\\
\end{pmatrix}
\begin{pmatrix}
 \mathbf I & \mathbf L_{21}^\mathrm T & \mathbf L_{31}^\mathrm T \\
 0 & \mathbf I & \mathbf L_{32}^\mathrm T \\
 0 & 0 & \mathbf I\\
\end{pmatrix} \\[8pt]
& = \begin{pmatrix}
 \mathbf D_1 &   &(\mathrm{symmetric})   \\
 \mathbf L_{21} \mathbf D_1 & \mathbf L_{21} \mathbf D_1 \mathbf L_{21}^\mathrm T + \mathbf D_2& \\
 \mathbf L_{31} \mathbf D_1 & \mathbf  L_{31} \mathbf D_{1} \mathbf L_{21}^\mathrm T + \mathbf  L_{32} \mathbf D_2 & \mathbf L_{31} \mathbf D_1 \mathbf L_{31}^\mathrm T + \mathbf L_{32} \mathbf D_2 \mathbf L_{32}^\mathrm T + \mathbf D_3
\end{pmatrix},
\end{align}
</math>

where every element in the matrices above is a square submatrix. From this, these analogous recursive relations follow:

<math display=block>\mathbf D_j = \mathbf A_{jj} - \sum_{k=1}^{j-1} \mathbf L_{jk} \mathbf D_k \mathbf L_{jk}^\mathrm T,</math>
<math display=block>\mathbf L_{ij} = \left(\mathbf A_{ij} - \sum_{k=1}^{j-1} \mathbf L_{ik} \mathbf D_k \mathbf L_{jk}^\mathrm T\right) \mathbf D_j^{-1}.</math>

This involves matrix products and explicit inversion, thus limiting the practical block size.

===Updating the decomposition===
A task that often arises in practice is that one needs to update a Cholesky decomposition. In more details, one has already computed the Cholesky decomposition <math display=inline>\mathbf{A} = \mathbf{L}\mathbf{L}^*</math> of some matrix <math display=inline>\mathbf{A}</math>, then one changes the matrix <math display=inline>\mathbf{A}</math> in some way into another matrix, say <math display=inline> \tilde{\mathbf{A}} </math>, and one wants to compute the Cholesky decomposition of the updated matrix: <math display=inline> \tilde{\mathbf{A}} = \tilde{\mathbf{L}} \tilde{\mathbf{L}}^* </math>. The question is now whether one can use the Cholesky decomposition of <math display=inline>\mathbf{A}</math> that was computed before to compute the Cholesky decomposition of <math display=inline> \tilde{\mathbf{A}} </math>.

==== Rank-one update ====
The specific case, where the updated matrix <math display=inline> \tilde{\mathbf{A}} </math> is related to the matrix <math display=inline>\mathbf{A}</math> by <math display=inline> \tilde{\mathbf{A}} = \mathbf{A} + \mathbf{x} \mathbf{x}^* </math>, is known as a ''rank-one update''.

Here is a function<ref>Based on: {{cite book|last=Stewart|first=G. W.|title=Basic decompositions|year=1998|publisher=Soc. for Industrial and Applied Mathematics|location=Philadelphia|isbn=0-89871-414-1}}</ref> written in [[Matlab]] syntax that realizes a rank-one update:
<syntaxhighlight lang="matlab">
function [L] = cholupdate(L, x)
    n = length(x);
    for k = 1:n
        r = sqrt(L(k, k)^2 + x(k)^2);
        c = r / L(k, k);
        s = x(k) / L(k, k);
        L(k, k) = r;
        if k < n
            L((k+1):n, k) = (L((k+1):n, k) + s * x((k+1):n)) / c;
            x((k+1):n) = c * x((k+1):n) - s * L((k+1):n, k);
        end
    end
end
</syntaxhighlight>
A ''rank-n update'' is one where for a matrix <math display=inline>\mathbf{M}</math> one updates the decomposition such that <math display=inline> \tilde{\mathbf{A}} = \mathbf{A} + \mathbf{M} \mathbf{M}^* </math>.  This can be achieved by successively performing rank-one updates for each of the columns of <math display=inline>\mathbf{M}</math>.

==== Rank-one downdate ====
A ''rank-one downdate'' is similar to a rank-one update, except that the addition is replaced by subtraction: <math display=inline> \tilde{\mathbf{A}} = \mathbf{A} - \mathbf{x} \mathbf{x}^* </math>. This only works if the new matrix <math display=inline> \tilde{\mathbf{A}} </math> is still positive definite.

The code for the rank-one update shown above can easily be adapted to do a rank-one downdate: one merely needs to replace the two additions in the assignment to <code>r</code> and <code>L((k+1):n, k)</code> by subtractions.

==== Adding and removing rows and columns ====
If a symmetric and positive definite matrix <math display=inline> \mathbf A </math> is represented in block form as

<math display=block>
\mathbf{A} = 
\begin{pmatrix}
 \mathbf A_{11} & \mathbf A_{13} \\
 \mathbf A_{13}^{\mathrm{T}} & \mathbf A_{33} \\
\end{pmatrix}
</math>

and its upper Cholesky factor
<math display=block>
\mathbf{L} = 
\begin{pmatrix}
 \mathbf L_{11} &  \mathbf L_{13} \\
  0      & \mathbf L_{33} \\
\end{pmatrix},
</math>

then for a new matrix <math display=inline> \tilde{\mathbf{A}} </math>, which is the same as <math display=inline> \mathbf A </math> but with the insertion of new rows and columns,
<math display=block>\begin{align}
\tilde{\mathbf{A}} &= 
\begin{pmatrix}
 \mathbf A_{11} & \mathbf A_{12} & \mathbf A_{13} \\
 \mathbf A_{12}^{\mathrm{T}} & \mathbf A_{22} & \mathbf A_{23} \\
 \mathbf A_{13}^{\mathrm{T}} & \mathbf A_{23}^{\mathrm{T}} &  \mathbf A_{33} \\
\end{pmatrix}
\end{align}
</math>

Now there is an interest in finding the Cholesky factorization of <math display=inline> \tilde{\mathbf{A}} </math>, which can be called <math display=inline> \tilde{\mathbf S} </math>, without directly computing the entire decomposition. 
<math display=block>\begin{align}
\tilde{\mathbf{S}} &= 
\begin{pmatrix}
 \mathbf S_{11} & \mathbf S_{12} & \mathbf S_{13} \\
 0 & \mathbf S_{22} & \mathbf S_{23} \\
 0 & 0 &  \mathbf S_{33} \\
\end{pmatrix}.
\end{align}
</math>

Writing <math display=inline> \mathbf A \setminus \mathbf{b}</math> for the solution of <math display=inline> \mathbf A \mathbf x = \mathbf b</math>, which can be found easily for triangular matrices, and <math display=inline> \text{chol} (\mathbf M)</math> for the Cholesky decomposition of <math display=inline> \mathbf M </math>, the following relations can be found:
<math display=block>\begin{align}
\mathbf S_{11} &= \mathbf L_{11}, \\
\mathbf S_{12} &= \mathbf L_{11}^{\mathrm{T}} \setminus \mathbf A_{12}, \\
\mathbf S_{13} &= \mathbf L_{13}, \\
\mathbf S_{22} &= \mathrm{chol} \left(\mathbf A_{22}  - \mathbf S_{12}^{\mathrm{T}} \mathbf S_{12}\right), \\
\mathbf S_{23} &= \mathbf S_{22}^{\mathrm{T}} \setminus \left(\mathbf A_{23} - \mathbf S_{12}^{\mathrm{T}} \mathbf S_{13}\right), \\
\mathbf S_{33} &= \mathrm{chol} \left(\mathbf L_{33}^{\mathrm{T}} \mathbf L_{33} - \mathbf S_{23}^{\mathrm{T}} \mathbf S_{23}\right).
\end{align}
</math>

These formulas may be used to determine the Cholesky factor after the insertion of rows or columns in any position, if the row and column dimensions are appropriately set (including to zero). The inverse problem, 

<math display=block>\begin{align}
\tilde{\mathbf{A}} &= 
\begin{pmatrix}
 \mathbf A_{11} & \mathbf A_{12} & \mathbf A_{13} \\
 \mathbf A_{12}^{\mathrm{T}} & \mathbf A_{22} & \mathbf A_{23} \\
 \mathbf A_{13}^{\mathrm{T}} & \mathbf A_{23}^{\mathrm{T}} &  \mathbf A_{33} \\
\end{pmatrix}
\end{align}
</math>
with known Cholesky decomposition 
<math display=block>\begin{align}
\tilde{\mathbf{S}} &= 
\begin{pmatrix}
 \mathbf S_{11} & \mathbf S_{12} & \mathbf S_{13} \\
 0 & \mathbf S_{22} & \mathbf S_{23} \\
 0 & 0 &  \mathbf S_{33} \\
\end{pmatrix}
\end{align}
</math>

and the desire to determine the Cholesky factor
<math display=block>\begin{align}
\mathbf{L} &= 
\begin{pmatrix}
 \mathbf L_{11} &  \mathbf L_{13} \\
  0      & \mathbf L_{33} \\
\end{pmatrix}
\end{align}
</math>

of the matrix <math display=inline> \mathbf A </math> with rows and columns removed,
<math display=block>\begin{align}
\mathbf{A} &= 
\begin{pmatrix}
 \mathbf A_{11} & \mathbf A_{13} \\
 \mathbf A_{13}^{\mathrm{T}} & \mathbf A_{33} \\
\end{pmatrix},
\end{align}
</math>

yields the following rules:
<math display=block>\begin{align}
\mathbf L_{11} &= \mathbf S_{11}, \\
\mathbf L_{13} &= \mathbf S_{13}, \\
\mathbf L_{33} &= \mathrm{chol} \left(\mathbf S_{33}^{\mathrm{T}} \mathbf S_{33} + \mathbf S_{23}^{\mathrm{T}} \mathbf S_{23}\right).
\end{align}
</math>

Notice that the equations above that involve finding the Cholesky decomposition of a new matrix are all of the form <math display=inline> \tilde{\mathbf{A}} = \mathbf{A} \pm \mathbf{x} \mathbf{x}^* </math>, which allows them to be efficiently calculated using the update and downdate procedures detailed in the previous section.<ref>Osborne, M. (2010), Appendix B.</ref>

== Proof for positive semi-definite matrices ==

=== Proof by limiting argument ===
The above algorithms show that every positive definite matrix <math display=inline> \mathbf{A} </math> has a Cholesky decomposition. This result can be extended to the positive semi-definite case by a limiting argument. The argument is not fully constructive, i.e., it gives no explicit numerical algorithms for computing Cholesky factors.

If <math display=inline> \mathbf{A} </math> is an <math display=inline> n \times n </math> [[Positive-definite matrix|positive semi-definite matrix]], then the sequence <math display="inline"> \left(\mathbf{A}_k\right)_k := \left(\mathbf{A} + \frac{1}{k} \mathbf{I}_n\right)_k </math> consists of [[Positive-definite matrix|positive definite matrices]]. (This is an immediate consequence of, for example, the spectral mapping theorem for the polynomial functional calculus.) Also,
<math display=block> 
\mathbf{A}_k \rightarrow \mathbf{A}
\quad \text{for} \quad
k \rightarrow \infty
</math> 
in [[operator norm]]. From the positive definite case, each <math display=inline> \mathbf{A}_k </math> has Cholesky decomposition <math display=inline> \mathbf{A}_k = \mathbf{L}_k\mathbf{L}_k^* </math>. By property of the operator norm,

<math display=block>\| \mathbf{L}_k \|^2 \leq \| \mathbf{L}_k \mathbf{L}_k^* \| = \| \mathbf{A}_k \| \,.</math>

The <math display=inline>\leq</math> holds because <math display=inline>M_n(\mathbb{C})</math> equipped with the operator norm is a C* algebra. So <math display=inline> \left(\mathbf{L}_k \right)_k</math> is a bounded set in the [[Banach space]] of operators, therefore [[relatively compact]] (because the underlying vector space is finite-dimensional). 
Consequently, it has a convergent subsequence, also denoted by <math display=inline> \left( \mathbf{L}_k \right)_k</math>, with limit <math display=inline> \mathbf{L}</math>. 
It can be easily checked that this <math display=inline> \mathbf{L}</math> has the desired properties, i.e. <math display=inline> \mathbf{A} = \mathbf{L}\mathbf{L}^* </math>, and <math display=inline> \mathbf{L}</math> is lower triangular with non-negative diagonal entries: for all <math display=inline> x</math> and <math display=inline> y</math>,

<math display=block> 
\langle \mathbf{A} x, y \rangle 
= \left\langle \lim \mathbf{A}_k x, y \right\rangle 
= \langle \lim \mathbf{L}_k \mathbf{L}_k^* x, y \rangle 
= \langle \mathbf{L} \mathbf{L}^*x, y \rangle \,. 
</math>

Therefore, <math display=inline> \mathbf{A} = \mathbf{L}\mathbf{L}^* </math>. 
Because the underlying vector space is finite-dimensional, all topologies on the space of operators are equivalent. 
So <math display=inline> \left( \mathbf{L}_k \right)_k</math> tends to <math display=inline> \mathbf{L}</math> in norm means <math display=inline> \left( \mathbf{L}_k \right)_k</math> tends to <math display=inline> \mathbf{L}</math> entrywise. 
This in turn implies that, since each <math display=inline> \mathbf{L}_k</math> is lower triangular with non-negative diagonal entries, <math display=inline> \mathbf{L}</math> is also.

=== Proof by QR decomposition ===

Let <math display=inline>\mathbf{A}</math> be a [[Positive-definite matrix|positive semi-definite]] Hermitian matrix. Then it can be written as a product of its [[Square root of a matrix|square root matrix]], <math display=inline>\mathbf{A} = \mathbf{B} \mathbf{B}^*</math>. Now [[QR decomposition]] can be applied to <math display=inline>\mathbf{B}^*</math>, resulting in <math display=inline>\mathbf{B}^* = \mathbf{Q}\mathbf{R}</math>
, where <math display=inline>\mathbf{Q}</math> is unitary and <math display=inline>\mathbf{R}</math> is upper triangular. Inserting the decomposition into the original equality yields <math display=inline>A = \mathbf{B} \mathbf{B}^* = (\mathbf{QR})^*\mathbf{QR} = \mathbf{R}^*\mathbf{Q}^*\mathbf{QR} = \mathbf{R}^*\mathbf{R}</math>. Setting <math display=inline>\mathbf{L} = \mathbf{R}^*</math> completes the proof.

== Generalization ==
The Cholesky factorization can be generalized {{Citation needed|date=October 2016}} to (not necessarily finite) matrices with operator entries. Let <math display=inline>\{\mathcal{H}_n \}</math> be a sequence of [[Hilbert spaces]]. Consider the operator matrix

<math display=block>
\mathbf{A} =
\begin{bmatrix}
\mathbf{A}_{11}   & \mathbf{A}_{12}   & \mathbf{A}_{13} & \; \\
\mathbf{A}_{12}^* & \mathbf{A}_{22}   & \mathbf{A}_{23} & \; \\
\mathbf{A} _{13}^* & \mathbf{A}_{23}^* & \mathbf{A}_{33} & \; \\
\;       & \;       & \;     & \ddots
\end{bmatrix}
</math>

acting on the direct sum

<math display=block>\mathcal{H} = \bigoplus_n \mathcal{H}_n,</math>

where each

<math display=block>\mathbf{A}_{ij} : \mathcal{H}_j \rightarrow \mathcal{H} _i</math>

is a [[bounded operator]]. If {{math|'''A'''}} is positive (semidefinite) in the sense that for all finite {{mvar|k}} and for any

<math display=block>h \in \bigoplus_{n = 1}^k \mathcal{H}_k ,</math>

there is <math display=inline>\langle h, \mathbf{A} h\rangle \ge 0</math>, then there exists a lower triangular operator matrix {{math|'''L'''}} such that {{math|1='''A''' = {{math|'''LL'''*}}}}. One can also take the diagonal entries of {{math|'''L'''}} to be positive.

== Implementations in programming libraries ==
* [[C programming language]]: the [[GNU Scientific Library]] provides several implementations of Cholesky decomposition.
* [[Maxima (software)|Maxima]] computer algebra system: function <code>cholesky</code> computes Cholesky decomposition.
* [[GNU Octave]] numerical computations system provides several functions to calculate, update, and apply a Cholesky decomposition.
* The [[LAPACK]] library provides a high performance implementation of the Cholesky decomposition that can be accessed from [[Fortran]], [[C (programming language)|C]] and most languages.
* In [[Python (programming language)|Python]], the function <code>cholesky</code> from the <code>numpy.linalg</code> module performs Cholesky decomposition.
* In [[Matlab]], the <code>chol</code> function gives the Cholesky decomposition. Note that <code>chol</code> uses the upper triangular factor of the input matrix by default, i.e. it computes <math display=inline>A = R^* R</math> where <math display=inline>R</math> is upper triangular. A flag can be passed to use the lower triangular factor instead.
* In [[R (programming language)|R]], the <code>chol</code> function gives the Cholesky decomposition.
* In [[Julia (programming language)|Julia]], the <code>cholesky</code> function from the <code>LinearAlgebra</code> standard library gives the Cholesky decomposition.
* In [[Mathematica]], the function "<code>CholeskyDecomposition</code>" can be applied to a matrix.
* In [[C++]], multiple linear algebra libraries support this decomposition:
** The [[Armadillo (C++ library)]] supplies the command <code>chol</code> to perform Cholesky decomposition. 
** The [[Eigen (C++ library)|Eigen library]] supplies Cholesky factorizations for both sparse and dense matrices.
** In the [[ROOT]] package, the <code>TDecompChol</code> class is available. 

* In [[Analytica (software)|Analytica]], the function <code>Decompose</code> gives the Cholesky decomposition.
* The [https://commons.apache.org/proper/commons-math/commons-math-docs/apidocs/org/apache/commons/math4/legacy/linear/CholeskyDecomposition.html Apache Commons Math library has an implementation] which can be used in Java, Scala and any other JVM language.

==See also==
* [[Cycle rank]]
* [[Incomplete Cholesky factorization]]
* [[Matrix decomposition]]
* [[Minimum degree algorithm]]
* [[Square root of a matrix]]
* [[Sylvester's law of inertia]]
* [[Symbolic Cholesky decomposition]]

==Notes==
{{Reflist}}

==References==
* {{Cite conference
 | last1 = Dereniowski
 | first1 = Dariusz
 | last2 = Kubale
 | first2 = Marek
 | contribution = Cholesky Factorization of Matrices in Parallel and Ranking of Graphs
 | doi = 10.1007/978-3-540-24669-5_127
 | pages = 985–992
 | publisher = Springer-Verlag
 | series = Lecture Notes on Computer Science
 | title = 5th International Conference on Parallel Processing and Applied Mathematics
 | url = http://www.eti.pg.gda.pl/katedry/kams/wwwkams/pdf/Cholesky_fmprg.pdf
 | volume = 3019
 | year = 2004
 | isbn = 978-3-540-21946-0
 | url-status = dead
 | archive-url = https://web.archive.org/web/20110716060800/http://www.eti.pg.gda.pl/katedry/kams/wwwkams/pdf/Cholesky_fmprg.pdf
 | archive-date = 2011-07-16
 }}
* {{Cite book| first1=Gene H. | last1=Golub | author1-link=Gene H. Golub | first2=Charles F. | last2=Van Loan | author2-link=Charles F. Van Loan | year=1996 | title=Matrix Computations | edition=3rd | publisher=Johns Hopkins | place=Baltimore | isbn=978-0-8018-5414-9}}
* {{Cite book| first1=Roger A. | last1=Horn | first2=Charles R. | last2=Johnson | year=1985 | title=Matrix Analysis | publisher=Cambridge University Press | isbn=0-521-38632-2 }}
* S. J. Julier and J. K. Uhlmann. "[https://web.archive.org/web/20190224070338/http://pdfs.semanticscholar.org/523a/865ffabb50d10f85d141963d40528e952760.pdf A General Method for Approximating Nonlinear Transformations of ProbabilityDistributions]".
* S. J. Julier and J. K. Uhlmann, "[http://kom.aau.dk/~tba/ESIF/julier97new.pdf A new extension of the Kalman filter to nonlinear systems]", in Proc. AeroSense: 11th Int. Symp. Aerospace/Defence Sensing, Simulation and Controls, 1997, pp.&nbsp;182–193.
* {{Cite book| last1=Trefethen | first1=Lloyd N. | author1-link=Lloyd N. Trefethen | last2=Bau | first2=David | title=Numerical linear algebra | publisher=Society for Industrial and Applied Mathematics | location=Philadelphia | isbn=978-0-89871-361-9 | year=1997}}
* {{cite thesis |last= Osborne|first= Michael|date= 2010|title= Bayesian Gaussian Processes for Sequential Prediction, Optimisation and Quadrature|type= thesis|publisher= University of Oxford|url= http://www.robots.ox.ac.uk/~mosb/public/pdf/2160/full_thesis.pdf }}
* Ruschel, João Paulo Tarasconi, Bachelor degree "[https://www.lume.ufrgs.br/bitstream/handle/10183/151001/001009773.pdf Parallel Implementations of the Cholesky Decomposition on CPUs and GPUs]" Universidade Federal Do Rio Grande Do Sul, Instituto De Informatica, 2016, pp.&nbsp;29-30.

== External links ==

===History of science===
* ''Sur la résolution numérique des systèmes d'équations linéaires'', Cholesky's 1910 manuscript, online and analyzed on [http://bibnum.education.fr/mathematiques/algebre/sur-la-resolution-numerique-des-systemes-d-equations-lineaires BibNum] {{in lang|fr|en}} <small>[for English, click 'A télécharger']</small>

===Information===
* {{springer|title=Cholesky factorization|id=p/c120160}}
* [https://web.archive.org/web/20060518112024/http://rkb.home.cern.ch/rkb/AN16pp/node33.html#SECTION000330000000000000000 Cholesky Decomposition], The Data Analysis BriefBook
* [http://www.math-linux.com/spip.php?article43 Cholesky Decomposition] on www.math-linux.com
* [http://sciencemeanderthal.wordpress.com/2012/06/28/cholesky-decomposition-of-variance-covariance-matrices-in-the-classic-twin-study/ Cholesky Decomposition Made Simple] on Science Meanderthal

===Computer code===
* [http://netlib.org/lapack/ LAPACK] is a collection of FORTRAN subroutines for solving dense linear algebra problems (DPOTRF, DPOTRF2, [http://www.netlib.org/utk/papers/factor/node9.html details] [http://www.netlib.org/utk/papers/factor/node13.html performance])
* [http://www.alglib.net/ ALGLIB] includes a partial port of the LAPACK to C++, C#, Delphi, Visual Basic, etc. (spdmatrixcholesky, hpdmatrixcholesky)
* [http://www.cs.utexas.edu/users/flame/ libflame] is a C library with LAPACK functionality.
* [http://www.cs.utexas.edu/users/flame/Movies.html#Chol Notes and video on high-performance implementation of Cholesky factorization] at The University of Texas at Austin.
* [http://upcommons.upc.edu/pfc/handle/2099.1/10988/ Cholesky : TBB + Threads + SSE] is a book explaining the implementation of the CF with TBB, threads and SSE (in Spanish).
* [http://ceres-solver.org/ library "Ceres Solver" ] by Google.
* [https://web.archive.org/web/20120807190828/http://infohost.nmt.edu/~borchers/ldlt.html LDL decomposition] routines in Matlab.
* [http://arma.sourceforge.net/download.html Armadillo] is a C++ linear algebra package
* [http://rosettacode.org/wiki/Rosetta_Code Rosetta Code] is a programming chrestomathy site. [https://rosettacode.org/wiki/Cholesky_decomposition on page topic].
* [https://algowiki-project.org/en/Open_Encyclopedia_of_Parallel_Algorithmic_Features AlgoWiki] is an open encyclopedia of algorithms’ properties and features of their implementations [https://algowiki-project.org/en/Cholesky_decomposition on page topic]
* [https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html Intel® oneAPI Math Kernel Library] Intel-Optimized Math Library for Numerical Computing [https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/lapack-routines/lapack-linear-equation-routines/lapack-linear-equation-computational-routines/matrix-factorization-lapack-computational-routines/potrf.html#potrf ?potrf], [https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/lapack-routines/lapack-linear-equation-routines/lapack-linear-equation-computational-routines/solving-systems-of-linear-equations-lapack-computational-routines/potrs.html#potrs ?potrs]
===Use of the matrix in simulation===
* [http://www.columbia.edu/~mh2078/MonteCarlo/MCS_Generate_RVars.pdf Generating Correlated Random Variables and Stochastic Processes], Martin Haugh, [[Columbia University]]

===Online calculators===
* [https://web.archive.org/web/20081212221215/http://www.bluebit.gr/matrix-calculator/  Online Matrix Calculator] Performs Cholesky decomposition of matrices online.

{{Numerical linear algebra}}

[[Category:Operator theory]]
[[Category:Matrix decompositions]]
[[Category:Numerical linear algebra]]
[[Category:Articles with example MATLAB/Octave code]]
<!-- Dummy edit -->