Operator norm

Template:Short description In mathematics, the operator norm measures the "size" of certain linear operators by assigning each a real number called its Template:Em. Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm <math>\|T\|</math> of a linear map <math>T : X \to Y</math> is the maximum factor by which it "lengthens" vectors.

Introduction and definition

Given two normed vector spaces <math>V</math> and <math>W</math> (over the same base field, either the real numbers <math>\R</math> or the complex numbers <math>\Complex</math>), a linear map <math>A : V \to W</math> is continuous if and only if there exists a real number <math>c</math> such that<ref>Template:Citation</ref> <math display="block">\|Av\| \leq c \|v\| \quad \text{ for all } v\in V.</math>

The norm on the left is the one in <math>W</math> and the norm on the right is the one in <math>V</math>. Intuitively, the continuous operator <math>A</math> never increases the length of any vector by more than a factor of <math>c.</math> Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of <math>A,</math> one can take the infimum of the numbers <math>c</math> such that the above inequality holds for all <math>v \in V.</math> This number represents the maximum scalar factor by which <math>A</math> "lengthens" vectors. In other words, the "size" of <math>A</math> is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of <math>A</math> as <math display="block">\|A\|_\text{op} = \inf\{ c \geq 0 : \|Av\| \leq c \|v\| \text{ for all } v \in V \}.</math>

The infimum is attained as the set of all such <math>c</math> is closed, nonempty, and bounded from below.<ref>See e.g. Lemma 6.2 of Template:Harvtxt.</ref>

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces <math>V</math> and <math>W</math>.

Examples

Every real <math>m</math>-by-<math>n</math> matrix corresponds to a linear map from <math>\R^n</math> to <math>\R^m.</math> Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all <math>m</math>-by-<math>n</math> matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both <math>\R^n</math> and <math>\R^m,</math> then the matrix norm given to a matrix <math>A</math> is the square root of the largest eigenvalue of the matrix <math>A^{*} A</math> (where <math>A^{*}</math> denotes the conjugate transpose of <math>A</math>).<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> This is equivalent to assigning the largest singular value of <math>A.</math>

Passing to a typical infinite-dimensional example, consider the sequence space <math>\ell^2,</math> which is an L^p space, defined by <math display="block">\ell^2 = \left\{ (a_n)_{n \geq 1} : \; a_n \in \Complex, \; \sum_n |a_n|^2 < \infty \right\}.</math>

This can be viewed as an infinite-dimensional analogue of the Euclidean space <math>\Complex^n.</math> Now consider a bounded sequence <math>s_{\bull} = \left(s_n\right)_{n=1}^\infty.</math> The sequence <math>s_{\bull}</math> is an element of the space <math>\ell^\infty,</math> with a norm given by <math display="block">\left\|s_{\bull}\right\|_\infty = \sup _n \left|s_n\right|.</math>

Define an operator <math>T_s</math> by pointwise multiplication: <math display="block">\left(a_n\right)_{n=1}^{\infty} \;\stackrel{T_s}{\mapsto}\;\ \left(s_n \cdot a_n\right)_{n=1}^{\infty}.</math>

The operator <math>T_s</math> is bounded with operator norm <math display="block">\left\|T_s\right\|_\text{op} = \left\|s_{\bull}\right\|_\infty.</math>

This discussion extends directly to the case where <math>\ell^2</math> is replaced by a general <math>L^p</math> space with <math>p > 1</math> and <math>\ell^\infty</math> replaced by <math>L^\infty.</math>

Equivalent definitions

Let <math>A : V \to W</math> be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition <math>V \neq \{0\}</math> then they are all equivalent:

<math>

\begin{alignat}{4} \|A\|_\text{op} &= \inf &&\{ c \geq 0 ~&&:~ \| A v \| \leq c \| v \| ~&&~ \text{ for all } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| \leq 1 ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| < 1 ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| \in \{0,1\} ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| = 1 ~&&~\mbox{ and } ~&&v \in V \} \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \} \\ &= \sup &&\bigg\{ \frac{\| Av \|}{\| v \|} ~&&:~ v \ne 0 ~&&~\mbox{ and } ~&&v \in V \bigg\} \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \}. \\ \end{alignat} </math> If <math>V = \{0\}</math> then the sets in the last two rows will be empty, and consequently their supremums over the set <math>[-\infty, \infty]</math> will equal <math>-\infty</math> instead of the correct value of <math>0.</math> If the supremum is taken over the set <math>[0, \infty]</math> instead, then the supremum of the empty set is <math>0</math> and the formulas hold for any <math>V.</math>

Importantly, a linear operator <math>A : V \to W</math> is not, in general, guaranteed to achieve its norm <math>\|A\|_\text{op} = \sup \{\|A v\| : \|v\| \leq 1, v \in V\}</math> on the closed unit ball <math>\{v \in V : \|v\| \leq 1\},</math> meaning that there might not exist any vector <math>u \in V</math> of norm <math>\|u\| \leq 1</math> such that <math>\|A\|_\text{op} = \|A u\|</math> (if such a vector does exist and if <math>A \neq 0,</math> then <math>u</math> would necessarily have unit norm <math>\|u\| = 1</math>). R.C. James proved James's theorem in 1964, which states that a Banach space <math>V</math> is reflexive if and only if every bounded linear functional <math>f \in V^*</math> achieves its norm on the closed unit ball.Template:Sfn It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If <math>A : V \to W</math> is bounded thenTemplate:Sfn <math display="block">\|A\|_\text{op} = \sup \left\{\left|w^*(A v)\right| : \|v\| \leq 1, \left\|w^*\right\| \leq 1 \text{ where } v \in V, w^* \in W^*\right\}</math> andTemplate:Sfn <math display="block">\|A\|_\text{op} = \left\|{}^tA\right\|_\text{op}</math> where <math>{}^t A : W^* \to V^*</math> is the transpose of <math>A : V \to W,</math> which is the linear operator defined by <math>w^* \,\mapsto\, w^* \circ A.</math>

Properties

The operator norm is indeed a norm on the space of all bounded operators between <math>V</math> and <math>W</math>. This means <math display="block">\|A\|_\text{op} \geq 0 \mbox{ and } \|A\|_\text{op} = 0 \mbox{ if and only if } A = 0,</math> <math display="block">\|aA\|_\text{op} = |a| \|A\|_\text{op} \mbox{ for every scalar } a ,</math> <math display="block">\|A + B\|_\text{op} \leq \|A\|_\text{op} + \|B\|_\text{op}.</math>

The following inequality is an immediate consequence of the definition: <math display="block">\|Av\| \leq \|A\|_\text{op} \|v\| \ \mbox{ for every }\ v \in V.</math>

The operator norm is also compatible with the composition, or multiplication, of operators: if <math>V</math>, <math>W</math> and <math>X</math> are three normed spaces over the same base field, and <math>A : V \to W</math> and <math>B : W \to X</math> are two bounded operators, then it is a sub-multiplicative norm, that is: <math display="block">\|BA\|_\text{op} \leq \|B\|_\text{op} \|A\|_\text{op}.</math>

For bounded operators on <math>V</math>, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

Table of common operator norms

By choosing different norms for the codomain, used in computing <math>\|Av\|</math>, and the domain, used in computing <math>\|v\|</math>, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in <math>N^2</math> operations (for an <math>N \times N</math> matrix), with the exception of the <math>\ell_2 - \ell_2</math> norm (which requires <math>N^3</math> operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

Computability of Operator Norms<ref>section 4.3.1, Joel Tropp's PhD thesis, [1]</ref>

		Co-domain
		<math>\ell_1</math>	<math>\ell_2</math>	<math>\ell_\infty</math>
Domain	<math>\ell_1</math>	Maximum <math>\ell_1</math> norm of a column	Maximum <math>\ell_2</math> norm of a column	Maximum <math>\ell_{\infty}</math> norm of a column
	<math>\ell_2</math>	NP-hard	Maximum singular value	Maximum <math>\ell_2</math> norm of a row
	<math>\ell_\infty</math>	NP-hard	NP-hard	Maximum <math>\ell_1</math> norm of a row

The norm of the adjoint or transpose can be computed as follows. We have that for any <math>p, q,</math> then <math>\|A\|_{p\rightarrow q} = \|A^*\|_{q'\rightarrow p'}</math> where <math>p', q'</math> are Hölder conjugate to <math>p, q,</math> that is, <math>1/p + 1/p' = 1</math> and <math>1/q + 1/q' = 1.</math>

Operators on a Hilbert space

Suppose <math>H</math> is a real or complex Hilbert space. If <math>A : H \to H</math> is a bounded linear operator, then we have <math display="block">\|A\|_\text{op} = \left\|A^*\right\|_\text{op}</math> and <math display="block">\left\|A^* A\right\|_\text{op} = \|A\|_\text{op}^2,</math> where <math>A^{*}</math> denotes the adjoint operator of <math>A</math> (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix <math>A</math>).

In general, the spectral radius of <math>A</math> is bounded above by the operator norm of <math>A</math>: <math display="block">\rho(A) \leq \|A\|_\text{op}.</math>

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator <math>A</math> has spectrum <math>\{0\}.</math> So <math>\rho(A) = 0</math> while <math>\|A\|_\text{op} > 0.</math>

However, when a matrix <math>N</math> is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that <math display="block">\rho(N) = \|N\|_\text{op}.</math>

This formula can sometimes be used to compute the operator norm of a given bounded operator <math>A</math>: define the Hermitian operator <math>B = A^{*} A,</math> determine its spectral radius, and take the square root to obtain the operator norm of <math>A.</math>

The space of bounded operators on <math>H,</math> with the topology induced by operator norm, is not separable. For example, consider the Lp space <math>L^2[0, 1],</math> which is a Hilbert space. For <math>0 < t \leq 1,</math> let <math>\Omega_t</math> be the characteristic function of <math>[0, t],</math> and <math>P_t</math> be the multiplication operator given by <math>\Omega_t,</math> that is, <math display="block">P_t (f) = f \cdot \Omega_t.</math>

Then each <math>P_t</math> is a bounded operator with operator norm 1 and <math display="block">\left\|P_t - P_s\right\|_\text{op} = 1 \quad \mbox{ for all } \quad t \neq s.</math>

But <math>\{P_t : 0 < t \leq 1\}</math> is an uncountable set. This implies the space of bounded operators on <math>L^2([0, 1])</math> is not separable, in operator norm. One can compare this with the fact that the sequence space <math>\ell^{\infty}</math> is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

See also

• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link

References

• Template:Citation.
• Template:Citation
• Template:Cite book
• Template:Rudin Walter Functional Analysis