Editing Singular value decomposition (section)

==Reduced SVDs==
[[File:Reduced Singular Value Decompositions.svg|thumb|Visualization of Reduced SVD variants. From top to bottom: 1: Full SVD,

2: Thin SVD (remove columns of {{math|'''U'''}} not corresponding to rows of {{math|'''V'''<sup>*</sup>}}),

3: Compact SVD (remove vanishing singular values and corresponding columns/rows in {{math|'''U'''}} and {{math|'''V'''<sup>*</sup>}}),

4: Truncated SVD (keep only largest t singular values and corresponding columns/rows in {{math|'''U'''}} and {{math|'''V'''<sup>*</sup>}})]]

In applications it is quite unusual for the full SVD, including a full unitary decomposition of the null-space of the matrix, to be required.  Instead, it is often sufficient (as well as faster, and more economical for storage) to compute a reduced version of the SVD.  The following can be distinguished for an {{tmath|m \times n}} matrix {{tmath|\mathbf M}} of rank {{tmath|r}}:

===Thin SVD===
The thin, or economy-sized, SVD of a matrix {{tmath|\mathbf M}} is given by<ref>
{{cite book
| chapter-url = http://www.netlib.org/utk/people/JackDongarra/etemplates/node43.html
| contribution = Decompositions
| contributor-last = Demmel
| contributor-first = James
| last1 = Bai
| first1 = Zhaojun
| last2 = Demmel
| first2 = James
| last3 = Dongarra
| first3 = Jack J.
| last4 = Ruhe
| first4 = Axel
| last5 = van der Vorst
| first5 = Henk A.
| title = Templates for the Solution of Algebraic Eigenvalue Problems
| doi = 10.1137/1.9780898719581
| url = https://www.cs.ucdavis.edu/~bai/ET/contents.html
| year = 2000
| isbn = 978-0-89871-471-5
| publisher = Society for Industrial and Applied Mathematics
}}
</ref>

<math display=block>
\mathbf{M} = \mathbf{U}_k \mathbf \Sigma_k \mathbf{V}^*_k,
</math>

where <math>k = \min(m, n),</math> the matrices {{tmath|\mathbf U_k}} and {{tmath|\mathbf V_k}} contain only the first {{tmath|k}} columns of {{tmath|\mathbf U}} and {{tmath|\mathbf V,}} and {{tmath|\mathbf \Sigma_k}} contains only the first {{tmath|k}} singular values from {{tmath|\mathbf \Sigma.}} The matrix {{tmath|\mathbf U_k}} is thus {{tmath|m \times k,}} {{tmath|\mathbf \Sigma_k}} is {{tmath|k \times k}} diagonal, and {{tmath|\mathbf V_k^*}} is {{tmath|k \times n.}}

The thin SVD uses significantly less space and computation time if {{tmath|k \ll \max(m, n).}} The first stage in its calculation will usually be a [[QR decomposition]] of {{tmath|\mathbf M,}} which can make for a significantly quicker calculation in this case.

===Compact SVD===

The compact SVD of a matrix {{tmath|\mathbf M}} is given by

<math display=block>
\mathbf{M} = \mathbf U_r \mathbf \Sigma_r \mathbf V_r^*.
</math>

Only the {{tmath|r}} column vectors of {{tmath|\mathbf U}} and {{tmath|r}} row vectors of {{tmath|\mathbf V^*}} corresponding to the non-zero singular values {{tmath|\mathbf \Sigma_r}} are calculated. The remaining vectors of {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} are not calculated. This is quicker and more economical than the thin SVD if {{tmath|r \ll \min(m,n).}} The matrix {{tmath|\mathbf U_r}} is thus {{tmath|m \times r,}} {{tmath|\mathbf \Sigma_r}} is {{tmath|r \times r}} diagonal, and {{tmath|\mathbf V_r^*}} is {{tmath|r \times n.}}

===Truncated SVD===
In many applications the number {{tmath|r}} of the non-zero singular values is large making even the Compact SVD impractical to compute. In such cases, the smallest singular values may need to be truncated to compute only {{tmath|t \ll r}} non-zero singular values. The truncated SVD is no longer an exact decomposition of the original matrix {{tmath|\mathbf M,}} but rather provides the optimal [[#Low-rank matrix approximation|low-rank matrix approximation]] {{tmath|\tilde{\mathbf M} }} by any matrix of a fixed rank {{tmath|t}}

<math display=block>
\tilde{\mathbf{M}} = \mathbf{U}_t \mathbf \Sigma_t \mathbf{V}_t^*,
</math>

where matrix {{tmath|\mathbf U_t}} is {{tmath|m \times t,}} {{tmath|\mathbf \Sigma_t}} is {{tmath|t \times t}} diagonal, and {{tmath|\mathbf V_t^*}} is {{tmath|t \times n.}} Only the {{tmath|t}} column vectors of {{tmath|\mathbf U}} and {{tmath|t}} row vectors of {{tmath|\mathbf V^*}} corresponding to the {{tmath|t}} largest singular values {{tmath|\mathbf \Sigma_t}} are calculated. This can be much quicker and more economical than the compact SVD if {{tmath|t \ll r,}} but requires a completely different toolset of numerical solvers.

In applications that require an approximation to the [[Moore–Penrose inverse]] of the matrix {{tmath|\mathbf M,}} the smallest singular values of {{tmath|\mathbf M}} are of interest, which are more challenging to compute compared to the largest ones.

Truncated SVD is employed in [[latent semantic indexing]].<ref>{{cite journal | last1 = Chicco | first1 = D | last2 = Masseroli | first2 = M | year = 2015 | title = Software suite for gene and protein annotation prediction and similarity search | journal = IEEE/ACM Transactions on Computational Biology and Bioinformatics | volume = 12 | issue = 4 | pages = 837–843 | doi=10.1109/TCBB.2014.2382127 | pmid = 26357324 | hdl = 11311/959408 | s2cid = 14714823 | url = https://doi.org/10.1109/TCBB.2014.2382127 | hdl-access = free }}
</ref>