Editing Matrix norm (section)

==Matrix norms induced by vector norms==
{{Main|Operator norm}}
Suppose a [[vector norm]] <math>\|\cdot\|_{\alpha}</math> on <math>K^n</math> and a vector norm <math>\|\cdot\|_{\beta}</math> on <math>K^m</math> are given. Any <math>m \times n</math> matrix {{mvar|A}} induces a linear operator from <math>K^n</math> to <math>K^m</math> with respect to the standard basis, and one defines the corresponding ''induced norm'' or ''[[operator norm]]'' or ''subordinate norm'' on the space <math>K^{m \times n}</math> of all <math>m \times n</math> matrices as follows:
<math display="block">
\|A\|_{\alpha, \beta} = \sup\{ \|Ax\|_\beta : x \in K^n \text{ such that } \|x\|_\alpha \leq 1 \}
</math>
where <math> \sup </math> denotes the [[Infimum and supremum|supremum]]. This norm measures how much the mapping induced by <math>A</math> can stretch vectors.
Depending on the vector norms <math>\|\cdot\|_{\alpha}</math>, <math>\|\cdot\|_{\beta}</math> used, notation other than <math>\|\cdot\|_{\alpha,\beta}</math> can be used for the operator norm.

===Matrix norms induced by vector ''p''-norms===
If the [[Vector norm#p-norm|''p''-norm for vectors]] (<math>1 \leq p \leq \infty</math>) is used for both spaces <math>K^n</math> and <math>K^m,</math> then the corresponding operator norm is:<ref name=":1" />
<math display="block">
\|A\|_p = \sup \{ \|Ax\|_p : x \in K^n \text{ such that } \|x\|_p \leq 1 \}.
</math>
These induced norms are different from the [[#"Entry-wise" matrix norms|"entry-wise"]] ''p''-norms and the [[Schatten norm|Schatten ''p''-norms]] for matrices treated below, which are also usually denoted by <math> \|A\|_p .</math>

Geometrically speaking, one can imagine a ''p''-norm unit ball <math>V_{p, n} = \{x\in K^n : \|x\|_p \le 1 \}</math> in <math>K^n</math>, then apply the linear map <math>A</math> to the ball. It would end up becoming a distorted convex shape <math>AV_{p, n} \subset K^m</math>, and <math> \|A\|_p </math> measures the longest "radius" of the distorted convex shape. In other words, we must take a ''p''-norm unit ball <math>V_{p, m}</math> in <math>K^m</math>, then multiply it by at least <math> \|A\|_p </math>, in order for it to be large enough to contain <math>AV_{p, n}</math>.

==== ''p'' = 1 or ∞ ====

When <math>\ p = 1\ ,</math> or <math>\ p = \infty\ ,</math> we have simple formulas.
:<math display="block"> \|A\|_1 = \max_{1 \leq j \leq n} \sum_{i=1}^m \left| a_{ij} \right|\ ,</math>
which is simply the maximum absolute column sum of the matrix.
<math display="block"> \|A\|_\infty = \max_{1 \leq i \leq m} \sum _{j=1}^n \left| a_{ij} \right|\ ,</math>
which is simply the maximum absolute row sum of the matrix.

For example, for
<math display="block">A = \begin{bmatrix} -3 & 5 & 7 \\ ~~2 & 6 & 4 \\ ~~0 & 2 & 8 \\ \end{bmatrix}\ ,</math>
we have that
<math display="block">\|A\|_1 = \max\bigl\{\ |{-3}|+2+0\ ,~ 5+6+2\ ,~ 7+4+8\ \bigr\} = \max\bigl\{\ 5\ ,~ 13\ ,~ 19\ \bigr\} = 19\ ,</math>
<math display="block">\|A\|_\infty = \max\bigl\{\ |{-3}|+5+7\ ,~ 2+6+4\ ,~ 0+2+8\ \bigr\} = \max\bigl\{\ 15\ ,~ 12\ ,~ 10\ \bigr\} = 15 ~.</math>

==== Spectral norm (''p'' = 2) ====
{{anchor|Spectral norm}}
When <math>p = 2</math> (the [[Euclidean norm]] or <math>\ell_2</math>-norm for vectors), the induced matrix norm is the ''spectral norm''. The two values do ''not'' coincide in infinite dimensions &mdash; see [[Spectral radius]] for further discussion. The spectral radius should not be confused with the spectral norm. The spectral norm of a matrix <math>A</math> is the largest [[singular value]] of <math>A</math>, i.e., the square root of the largest [[eigenvalue]] of the matrix <math>A^*A,</math> where <math>A^*</math> denotes the [[conjugate transpose]] of <math>A</math>:<ref>Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, §5.2, p.281, Society for Industrial & Applied Mathematics, June 2000.</ref><math display="block"> \|A\|_2 = \sqrt{\lambda_{\max}\left(A^* A\right)} = \sigma_{\max}(A).</math>where <math>\sigma_{\max}(A)</math> represents the largest singular value of matrix <math>A.</math>

There are further properties:
* <math display="inline">\|A \|_2 = \sup\{x^* A y : x \in K^m, y \in K^n \text{ with }\|x\|_2 = \|y\|_2 = 1\}.</math> Proved by the [[Cauchy–Schwarz inequality]].
* <math display="inline"> \| A^* A\|_2 = \| A A^* \|_2 = \|A\|_2^2</math>. Proven by [[singular value decomposition]] (SVD) on <math>A</math>.
* <math display="inline"> \|A\| _2 = \sigma_{\mathrm{max}}(A) \leq \|A\|_{\rm F} = \sqrt{\sum_i \sigma_{i}(A)^2}</math>, where <math>\|A\|_\textrm{F}</math> is the [[#Frobenius norm|Frobenius norm]]. Equality holds if and only if the matrix <math>A</math> is a rank-one matrix or a zero matrix.
* Conversely, <math>\|A\|_\textrm{F} \leq \min(m,n)^{1/2}\|A\|_2</math>.

* <math> \|A\|_2 = \sqrt{\rho(A^{*}A)}\leq\sqrt{\|A^{*}A\|_\infty}\leq\sqrt{\|A\|_1\|A\|_\infty} </math>.

===Matrix norms induced by vector ''α''- and ''β''-norms===
We can generalize the above definition. Suppose we have vector norms <math>\|\cdot\|_{\alpha}</math> and <math>\|\cdot\|_{\beta}</math> for spaces <math>K^n</math> and <math>K^m</math> respectively; the corresponding operator norm is
<math display="block">
\|A\|_{\alpha, \beta} = \sup\{ \|Ax\|_\beta : x \in K^n \text{ such that } \|x\|_\alpha \leq 1 \}
</math>
In particular, the <math>\|A\|_{p}</math> defined previously is the special case of <math>\|A\|_{p, p}</math>.

In the special cases of <math>\alpha = 2</math> and <math>\beta=\infty</math>, the induced matrix norms can be computed by<math display="block"> \|A\|_{2,\infty}= \max_{1\le i\le m}\|A_{i:}\|_2, </math> where <math>A_{i:}</math> is the i-th row of matrix <math> A </math>.

In the special cases of <math>\alpha = 1</math> and <math>\beta=2</math>, the induced matrix norms can be computed by<math display="block"> \|A\|_{1, 2} = \max_{1\le j\le n}\|A_{:j}\|_2, </math> where <math>A_{:j}</math> is the j-th column of matrix <math> A </math>.

Hence, <math> \|A\|_{2,\infty} </math> and <math> \|A\|_{1, 2} </math> are the maximum row and column 2-norm of the matrix, respectively.

===Properties===

Any operator norm is [[#Consistent and compatible norms|consistent]]  with the vector norms that induce it, giving
<math display="block">\|Ax\|_\beta \leq \|A\|_{\alpha,\beta}\|x\|_\alpha.</math>

Suppose <math>\|\cdot\|_{\alpha,\beta}</math>; <math>\|\cdot\|_{\beta,\gamma}</math>; and <math>\|\cdot\|_{\alpha,\gamma}</math> are operator norms induced by the respective pairs of vector norms <math>(\|\cdot\|_\alpha, \|\cdot\|_\beta)</math>; <math>(\|\cdot\|_\beta, \|\cdot\|_{\gamma})</math>; and <math>(\|\cdot\|_\alpha, \|\cdot\|_\gamma)</math>.  Then,
:<math>\|AB\|_{\alpha,\gamma} \leq \|A\|_{\beta, \gamma} \|B\|_{\alpha, \beta} ;</math>
this follows from
<math display="block">\|ABx\|_\gamma \leq \|A\|_{\beta, \gamma} \|Bx\|_\beta \leq \|A\|_{\beta, \gamma} \|B\|_{\alpha, \beta} \|x\|_\alpha </math>
and
<math display="block">\sup_{\|x\|_\alpha = 1} \|ABx \|_\gamma = \|AB\|_{\alpha, \gamma} .</math>

===Square matrices===
Suppose <math>\|\cdot\|_{\alpha, \alpha}</math> is an operator norm on the space of square matrices <math>K^{n \times n}</math>
induced by vector norms <math>\|\cdot\|_{\alpha}</math> and <math>\|\cdot\|_\alpha</math>.
Then, the operator norm is a sub-multiplicative matrix norm: 
<math display="block">\|AB\|_{\alpha, \alpha} \leq \|A\|_{\alpha, \alpha} \|B\|_{\alpha, \alpha}.</math>

Moreover, any such norm satisfies the inequality
{{NumBlk||<math display="block">(\|A^r\|_{\alpha, \alpha})^{1/r} \ge \rho(A) </math>  | {{EquationRef|1}}}}
for all positive integers ''r'', where {{math|''ρ''(''A'')}} is the [[spectral radius]] of {{mvar|A}}. For [[Symmetric matrix|symmetric]] or [[Hermitian matrix|hermitian]] {{mvar|A}}, we have equality in ({{EquationNote|1}}) for the 2-norm, since in this case the 2-norm ''is'' precisely the spectral radius of {{mvar|A}}. For an arbitrary matrix, we may not have equality for any norm; a counterexample would be
<math display="block">A = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix},</math>
which has vanishing spectral radius. In any case, for any matrix norm, we have the [[Spectral radius#Gelfand's formula|spectral radius formula]]:
<math display="block">\lim_{r\to\infty}\|A^r\|^{1/r}=\rho(A). </math>

===Energy norms===

If the vector norms <math>\|\cdot\|_{\alpha}</math> and <math>\|\cdot\|_{\beta}</math> are given in terms of [[Norm_(mathematics)#Energy_norm|energy norms]] based on [[Symmetric_matrix|symmetric]] [[Definite_matrix|positive definite]] matrices <math>P</math> and <math>Q</math> respectively, the resulting operator norm is given as
<math display="block">
\|A\|_{P, Q} = \sup \{ \|Ax\|_Q : \|x\|_P \leq 1 \}.
</math>

Using the symmetric [[Square_root_of_a_matrix|matrix square roots]] of <math>P</math> and <math>Q</math> respectively, the operator norm can be expressed as the spectral norm of a modified matrix:

<math display="block">
\|A\|_{P, Q} = \|Q^{1/2} A P^{-1/2}\|_{2}.
</math>