Editing Singular value decomposition (section)

== Intuitive interpretations ==
[[File:Singular value decomposition.gif|thumb|right|280px|Animated illustration of the SVD of a 2D, real [[Shear mapping|shearing matrix]] {{math|'''M'''}}. First, we see the [[unit disc]] in blue together with the two [[standard basis|canonical unit vectors]]. We then see the actions of {{math|'''M'''}}, which distorts the disk to an [[ellipse]]. The SVD decomposes {{math|'''M'''}} into three simple transformations: an initial [[Rotation matrix|rotation]] {{math|'''V'''<sup>⁎</sup>}}, a [[Scaling matrix|scaling]] <math>\mathbf{\Sigma}</math> along the coordinate axes, and a final rotation {{math|'''U'''}}. The lengths {{math|''σ''<sub>1</sub>}} and {{math|''σ''<sub>2</sub>}} of the [[Ellipse#Elements of an ellipse|semi-axes]] of the ellipse are the [[singular value]]s of {{math|'''M'''}}, namely {{math|Σ<sub>1,1</sub>}} and {{math|Σ<sub>2,2</sub>}}.]]
[[File:Singular_value_decomposition_visualisation.svg|thumb|Visualization of the matrix multiplications in singular value decomposition]]

=== Rotation, coordinate scaling, and reflection ===
In the special case when {{tmath|\mathbf M}} is an {{tmath|m \times m}} real [[square matrix]], the matrices {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} can be chosen to be real {{tmath|m \times m}} matrices too.  In that case, "unitary" is the same as "[[orthogonal matrix|orthogonal]]".  Then, interpreting both unitary matrices as well as the diagonal matrix, summarized here as {{tmath|\mathbf A,}} as a [[linear transformation]] {{tmath| \mathbf x \mapsto \mathbf{Ax} }} of the space {{tmath|\mathbf R_m,}} the matrices {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} represent [[rotation (geometry)|rotations]] or [[reflection (geometry)|reflection]] of the space, while {{tmath|\mathbf \Sigma}} represents the [[scaling matrix|scaling]] of each coordinate {{tmath|\mathbf x_i}} by the factor {{tmath|\sigma_i.}}  Thus the SVD decomposition breaks down any linear transformation of {{tmath|\mathbf R^m}} into a [[function composition|composition]] of three geometrical [[transformation (geometry)|transformations]]: a rotation or reflection {{nobr|({{tmath|\mathbf V^*}}),}} followed by a coordinate-by-coordinate [[scaling (geometry)|scaling]] {{nobr|({{tmath|\mathbf \Sigma}}),}} followed by another rotation or reflection {{nobr|({{tmath|\mathbf U}}).}}

In particular, if {{tmath|\mathbf M}} has a positive determinant, then {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} can be chosen to be both rotations with reflections, or both rotations without reflections.{{Citation needed|date=September 2022}} If the determinant is negative, exactly one of them will have a reflection. If the determinant is zero, each can be independently chosen to be of either type.

If the matrix {{tmath|\mathbf M}} is real but not square, namely {{tmath|m\times n}} with {{tmath|m \neq n,}} it can be interpreted as a linear transformation from {{tmath|\mathbf R^n}} to  {{tmath|\mathbf R^ m.}}  Then {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} can be chosen to be rotations/reflections of {{tmath|\mathbf R^m}} and {{tmath|\mathbf R^n,}} respectively; and {{tmath|\mathbf \Sigma,}} besides scaling the first {{tmath|\min\{m,n\} }} coordinates, also extends the vector with zeros, i.e. removes trailing coordinates, so as to turn {{tmath|\mathbf R^n}} into {{tmath|\mathbf R^m.}}

=== Singular values as semiaxes of an ellipse or ellipsoid ===
As shown in the figure, the [[singular values]] can be interpreted as the magnitude of the semiaxes of an [[ellipse]] in 2D. This concept can be generalized to {{tmath|n}}-dimensional [[Euclidean space]], with the singular values of any {{tmath|n \times n}} [[square matrix]] being viewed as the magnitude of the semiaxis of an {{tmath|n}}-dimensional [[ellipsoid]]. Similarly, the singular values of any {{tmath|m \times n}} matrix can be viewed as the magnitude of the semiaxis of an {{tmath|n}}-dimensional [[ellipsoid]] in {{tmath|m}}-dimensional space, for example as an ellipse in a (tilted) 2D plane in a 3D space. Singular values encode magnitude of the semiaxis, while singular vectors encode direction.  See [[#Geometric meaning|below]] for further details.

=== The columns of {{math|U}} and {{math|V}} are orthonormal bases ===
Since {{tmath|\mathbf U}} and {{tmath|\mathbf V^*}} are unitary, the columns of each of them form a set of [[orthonormal vectors]], which can be regarded as [[basis vectors]]. The matrix {{tmath|\mathbf M}} maps the basis vector {{tmath|\mathbf V_i}} to the stretched unit vector {{tmath|\sigma_i \mathbf U_i.}}  By the definition of a unitary matrix, the same is true for their conjugate transposes {{tmath|\mathbf U^*}} and {{tmath|\mathbf V,}} except the geometric interpretation of the singular values as stretches is lost. In short, the columns of {{tmath|\mathbf U,}} {{tmath|\mathbf U^*,}} {{tmath|\mathbf V,}} and {{tmath|\mathbf V^*}} are [[Orthonormal basis|orthonormal bases]]. When {{tmath|\mathbf M}} is a [[Definite matrix|positive-semidefinite]] [[Hermitian matrix]], {{tmath|\mathbf U}} and {{tmath|\mathbf V}} are both equal to the unitary matrix used to diagonalize  {{tmath|\mathbf M.}} However, when {{tmath|\mathbf M}} is not positive-semidefinite and Hermitian but still [[diagonalizable]], its [[eigendecomposition]] and singular value decomposition are distinct.

=== Relation to the four fundamental subspaces ===
* The first {{tmath|r}} columns of {{tmath|\mathbf U}} are a basis of the [[column space]] of {{tmath|\mathbf M}}.
* The last {{tmath|m-r}} columns of {{tmath|\mathbf U}} are a basis of the [[null space]] of {{tmath|\mathbf M^*}}.
* The first {{tmath|r}} columns of {{tmath|\mathbf V}} are a basis of the column space of {{tmath|\mathbf M^*}} (the [[row space]] of {{tmath|\mathbf M}} in the real case).
* The last {{tmath|n-r}} columns of {{tmath|\mathbf V}} are a basis of the null space of {{tmath|\mathbf M}}.

=== Geometric meaning ===
Because {{tmath|\mathbf U}} and {{tmath|\mathbf V}} are unitary, we know that the columns {{tmath|\mathbf U_1, \ldots, \mathbf U_m}} of {{tmath|\mathbf U}} yield an [[orthonormal basis]] of {{tmath|K^m}} and the columns {{tmath|\mathbf V_1, \ldots, \mathbf V_n}} of {{tmath|\mathbf V}} yield an orthonormal basis of {{tmath|K^n}} (with respect to the standard [[scalar product]]s on these spaces).

The [[linear transformation]]

<math display=block>
T : \left\{\begin{aligned}
  K^n &\to K^m \\
  x &\mapsto \mathbf{M}x \end{aligned}\right.
</math>

has a particularly simple description with respect to these orthonormal bases: we have

<math display=block>
T(\mathbf{V}_i) = \sigma_i \mathbf{U}_i, \qquad i
= 1, \ldots, \min(m, n),
</math>

where {{tmath|\sigma_i}} is the {{tmath|i}}-th diagonal entry of {{tmath|\mathbf \Sigma,}} and {{tmath|T(\mathbf V_i) {{=}} 0}} for {{tmath|i > \min(m,n).}}

The geometric content of the SVD theorem can thus be summarized as follows: for every linear map {{tmath|T : K^n \to K^m }} one can find orthonormal bases of {{tmath|K^n}} and {{tmath|K^m}} such that {{tmath|T}} maps the {{tmath|i}}-th basis vector of {{tmath|K^n}} to a non-negative multiple of the {{tmath|i}}-th basis vector of {{tmath|K^m,}} and sends the leftover basis vectors to zero. With respect to these bases, the map {{tmath|T}} is therefore represented by a diagonal matrix with non-negative real diagonal entries.

To get a more visual flavor of singular values and SVD factorization – at least when working on real vector spaces – consider the sphere {{tmath|S}} of radius one in {{tmath|\mathbf R^n.}} The linear map {{tmath|T}} maps this sphere onto an [[ellipsoid]] in {{tmath|\mathbf R^m.}} Non-zero singular values are simply the lengths of the [[Semi-minor axis|semi-axes]] of this ellipsoid. Especially when {{tmath|n {{=}} m,}} and all the singular values are distinct and non-zero, the SVD of the linear map {{tmath|T}} can be easily analyzed as a succession of three consecutive moves: consider the ellipsoid {{tmath|T(S)}} and specifically its axes; then consider the directions in {{tmath|\mathbf R^n}} sent by {{tmath|T}} onto these axes. These directions happen to be mutually orthogonal. Apply first an isometry {{tmath|\mathbf V^*}} sending these directions to the coordinate axes of {{tmath|\mathbf R^n.}} On a second move, apply an [[endomorphism]] {{tmath|\mathbf D}} diagonalized along the coordinate axes and stretching or shrinking in each direction, using the semi-axes lengths of {{tmath|T(S)}} as stretching coefficients. The composition {{tmath|\mathbf D \circ \mathbf V^*}} then sends the unit-sphere onto an ellipsoid isometric to {{tmath|T(S).}} To define the third and last move, apply an isometry {{tmath|\mathbf U}} to this ellipsoid to obtain {{tmath|T(S).}} As can be easily checked, the composition {{tmath|\mathbf U \circ \mathbf D \circ \mathbf V^*}} coincides with {{tmath|T.}}