Editing Matrix norm (section)

=="Entry-wise" matrix norms==
These norms treat an <math> m \times n </math> matrix as a vector of size <math> m \cdot n </math>, and use one of the familiar vector norms. For example, using the ''p''-norm for vectors, {{nowrap|''p'' ≥ 1}}, we get:

:<math>\| A \|_{p,p} = \| \mathrm{vec}(A) \|_p = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{ij}|^p \right)^{1/p}</math>

This is a different norm from the induced ''p''-norm (see above) and the Schatten ''p''-norm (see below), but the notation is the same.

The special case ''p'' = 2 is the Frobenius norm, and ''p'' = &infin; yields the maximum norm.

==={{math|''L''<sub>2,1</sub>}} and {{math|''L<sub>p,q</sub>''}} norms===

Let <math>(a_1, \ldots, a_n) </math> be the dimension {{mvar|m}} columns of matrix <math>A</math>. From the original definition, the matrix <math> A </math> presents {{mvar|n}} data points in an {{mvar|m}}-dimensional space. The <math>L_{2,1}</math> norm<ref>{{cite conference | last1=Ding | first1=Chris | last2=Zhou | first2=Ding | last3=He | first3=Xiaofeng | last4=Zha | first4=Hongyuan  |date = June 2006 | title = R1-PCA: Rotational invariant L1-norm principal component analysis for robust subspace factorization | conference = 23rd International Conference on Machine Learning | series=ICML '06 | isbn = 1-59593-383-2 | place = Pittsburgh, PA | pages=281–288 | doi=10.1145/1143844.1143880 | publisher=[[Association for Computing Machinery]] }}</ref> is the sum of the Euclidean norms of the columns of the matrix:

:<math>\| A \|_{2,1} = \sum_{j=1}^n \| a_{j} \|_2 = \sum_{j=1}^n \left( \sum_{i=1}^m |a_{ij}|^2 \right)^{1/2}</math>

The <math>L_{2,1}</math> norm as an error function is more robust, since the error for each data point (a column) is not squared. It is used in [[robust data analysis]] and [[sparse coding]].

For {{nowrap|''p'', ''q'' ≥ 1}}, the <math>L_{2,1}</math> norm can be generalized to the <math>L_{p,q}</math> norm as follows:

:<math>\| A \|_{p,q} =  \left(\sum_{j=1}^n \left( \sum_{i=1}^m |a_{ij}|^p \right)^{\frac{q}{p}}\right)^{\frac{1}{q}}.</math>

===Frobenius norm===
{{Main|Hilbert–Schmidt operator}}
{{See also|Frobenius inner product}}

When {{nowrap|1=''p'' = ''q'' = 2}} for the <math>L_{p,q}</math> norm, it is called the '''Frobenius norm''' or the '''Hilbert–Schmidt norm''', though the latter term is used more frequently in the context of operators on (possibly infinite-dimensional) [[Hilbert space]]. This norm can be defined in various ways:

:<math>\|A\|_\text{F} = \sqrt{\sum_{i}^m\sum_{j}^n |a_{ij}|^2} = \sqrt{\operatorname{trace}\left(A^* A\right)} = \sqrt{\sum_{i=1}^{\min\{m, n\}} \sigma_i^2(A)},</math>

where the [[trace (matrix)|trace]] is the sum of diagonal entries, and <math>\sigma_i(A)</math> are the [[singular value]]s of <math>A</math>. The second equality is proven by explicit computation of <math>\mathrm{trace}(A^*A)</math>. The third equality is proven by [[singular value decomposition]] of <math>A</math>, and the fact that the trace is invariant under circular shifts.

The Frobenius norm is an extension of the Euclidean norm to <math>K^{n \times n}</math> and comes from the [[Frobenius inner product]] on the space of all matrices.

The Frobenius norm is sub-multiplicative and is very useful for [[numerical linear algebra]]. The sub-multiplicativity of Frobenius norm can be proved using the [[Cauchy–Schwarz inequality]]. In fact, it is more than sub-multiplicative, as <math display="block">\|AB\|_F \leq\|A\|_{op}\|B\|_F</math>where the operator norm <math>\|\cdot\|_{op} \leq \|\cdot\|_{F}</math>.

Frobenius norm is often easier to compute than induced norms, and has the useful property of being invariant under [[rotation matrix|rotations]] (and [[Unitary operator|unitary]] operations in general). That is, <math>\|A\|_\text{F} = \|AU\|_\text{F} = \|UA\|_\text{F}</math> for any unitary matrix <math>U</math>. This property follows from the cyclic nature of the trace (<math>\operatorname{trace}(XYZ) =\operatorname{trace}(YZX) = \operatorname{trace}(ZXY)</math>):

:<math>\|AU\|_\text{F}^2 = \operatorname{trace}\left( (AU)^{*}A U \right)
  = \operatorname{trace}\left( U^{*} A^{*}A U \right)
  = \operatorname{trace}\left( UU^{*} A^{*}A \right)
  = \operatorname{trace}\left( A^{*} A \right)
  = \|A\|_\text{F}^2,</math>

and analogously:

:<math>\|UA\|_\text{F}^2 = \operatorname{trace}\left( (UA)^{*}UA \right)
  = \operatorname{trace}\left( A^{*} U^{*} UA  \right)
  = \operatorname{trace}\left( A^{*}A \right)
  = \|A\|_\text{F}^2,</math>

where we have used the unitary nature of <math>U</math> (that is, <math>U^* U = U U^* = \mathbf{I}</math>).

It also satisfies

:<math>\|A^* A\|_\text{F} = \|AA^*\|_\text{F} \leq \|A\|_\text{F}^2</math>
and 
:<math>\|A + B\|_\text{F}^2 = \|A\|_\text{F}^2 + \|B\|_\text{F}^2 + 2 \operatorname{Re} \left( \langle A, B \rangle_\text{F} \right),</math>

where <math>\langle A, B \rangle_\text{F}</math> is the [[Frobenius inner product]], and Re is the real part of a complex number (irrelevant for real matrices)

===Max norm===
The '''max norm''' is the elementwise norm in the limit as {{nowrap|1=''p'' = ''q''}} goes to infinity:

:<math> \|A\|_{\max} = \max_{i, j} |a_{ij}|. </math>

This norm is not [[#Definition|sub-multiplicative]]; but modifying the right-hand side to <math>\sqrt{m n} \max_{i, j} \vert a_{i j} \vert</math> makes it so.

Note that in some literature (such as [[Communication complexity]]), an alternative definition of max-norm, also called the <math>\gamma_2</math>-norm, refers to the factorization norm:

:<math> \gamma_2(A) = \min_{U,V: A = UV^T} \| U \|_{2,\infty} \| V \|_{2,\infty} =  \min_{U,V: A = UV^T} \max_{i,j} \| U_{i,:} \|_2 \| V_{j,:} \|_2 </math>