Editing Orthonormal basis

{{Short description|Specific linear basis (mathematics)}}

In [[mathematics]], particularly [[linear algebra]], an '''orthonormal basis''' for an [[inner product space]] <math>V</math> with finite [[Dimension (linear algebra)|dimension]] is a [[Basis (linear algebra)|basis]] for <math>V</math> whose vectors are [[orthonormal]], that is, they are all [[unit vector]]s and [[Orthogonality_(mathematics)|orthogonal]] to each other.<ref>{{cite book|last=Lay|first=David C.|title=Linear Algebra and Its Applications|url=https://archive.org/details/studyguidetoline0000layd|url-access=registration|publisher=[[Addison–Wesley]]|year=2006|edition = 3rd|isbn=0-321-28713-4}}</ref><ref>{{cite book|last=Strang|first=Gilbert|author-link=Gilbert Strang|title=Linear Algebra and Its Applications|publisher=[[Brooks Cole]]|year=2006|edition = 4th|isbn=0-03-010567-6}}</ref><ref>{{cite book|last = Axler|first = Sheldon|title = Linear Algebra Done Right|publisher = [[Springer Science+Business Media|Springer]]|year = 2002|edition = 2nd|isbn = 0-387-98258-2}}</ref> For example, the [[standard basis]] for a [[Euclidean space]] <math>\R^n</math> is an orthonormal basis, where the relevant inner product is the [[dot product]] of vectors. The [[Image (mathematics)|image]] of the standard basis under a [[Rotation (mathematics)|rotation]] or [[Reflection (mathematics)|reflection]] (or any [[orthogonal transformation]]) is also orthonormal, and every orthonormal basis for <math>\R^n</math> arises in this fashion.
An orthonormal basis can be derived from an [[orthogonal basis]] via [[Normalize (linear algebra)|normalization]].
{{anchor|Frame}}The choice of an [[origin (mathematics)|origin]] and an orthonormal basis forms a [[coordinate frame]] known as an '''''orthonormal frame'''''.

For a general inner product space <math>V,</math> an orthonormal basis can be used to define normalized [[orthogonal coordinates]] on <math>V.</math> Under these coordinates, the inner product becomes a dot product of vectors. Thus the presence of an orthonormal basis reduces the study of a [[Dimension (vector space)|finite-dimensional]] inner product space to the study of <math>\R^n</math> under the dot product. Every finite-dimensional inner product space has an orthonormal basis, which may be obtained from an arbitrary basis using the [[Gram–Schmidt process]].

In [[functional analysis]], the concept of an orthonormal basis can be generalized to arbitrary (infinite-dimensional) [[inner product space]]s.<ref>{{cite book|last=Rudin|first=Walter|author-link=Walter Rudin|title=Real & Complex Analysis|publisher=[[McGraw-Hill]]|year=1987|isbn=0-07-054234-1}}</ref> Given a pre-Hilbert space <math>H,</math> an ''orthonormal basis'' for <math>H</math> is an orthonormal set of vectors with the property that every vector in <math>H</math> can be written as an [[infinite linear combination]] of the vectors in the basis. In this case, the orthonormal basis is sometimes called a '''Hilbert basis''' for <math>H.</math> Note that an orthonormal basis in this sense is not generally a [[Hamel basis]], since infinite linear combinations are required.{{sfn|Roman|2008|p=218|loc=ch. 9}} Specifically, the [[linear span]] of the basis must be [[Dense set|dense]] in <math>H,</math> although not necessarily the entire space.

If we go on to [[Hilbert space]]s, a non-orthonormal set of vectors having the same linear span as an orthonormal basis may not be a basis at all. For instance, any [[square-integrable function]] on the interval <math>[-1,1]</math> can be expressed ([[almost everywhere]]) as an infinite sum of [[Legendre polynomials]] (an orthonormal basis), but not necessarily as an infinite sum of the [[monomial]]s <math>x^n.</math>

A different generalisation is to pseudo-inner product spaces, finite-dimensional vector spaces <math>M</math> equipped with a non-degenerate [[symmetric bilinear form]] known as the [[metric tensor]]. In such a basis, the metric takes the form <math>\text{diag}(+1,\cdots,+1,-1,\cdots,-1)</math> with <math>p</math> positive ones and <math>q</math> negative ones.

==Examples==

* For <math>\mathbb{R}^3</math>, the set of vectors <math>\left\{ \mathbf{e_1} =
\begin{pmatrix} 1 & 0 & 0 \end{pmatrix} \ ,

\ \mathbf{e_2} =
\begin{pmatrix} 0 & 1 & 0 \end{pmatrix} \ ,

\ \mathbf{e_3} =
\begin{pmatrix} 0 & 0 & 1 \end{pmatrix}

\right\},</math> is called the '''standard basis''' and forms an orthonormal basis of <math>\mathbb{R}^3</math> with respect to the standard dot product. Note that both the standard basis and standard dot product rely on viewing <math>\mathbb{R}^3</math> as the [[Cartesian product]] <math>\mathbb{R}\times\mathbb{R}\times\mathbb{R}</math>
*:'''Proof:''' A straightforward computation shows that the inner products of these vectors equals zero, <math>\left\langle \mathbf{e_1}, \mathbf{e_2} \right\rangle = \left\langle \mathbf{e_1}, \mathbf{e_3} \right\rangle = \left\langle \mathbf{e_2}, \mathbf{e_3} \right\rangle = 0</math> and that each of their magnitudes equals one, <math>\left\|\mathbf{e_1}\right\| = \left\|\mathbf{e_2}\right\| = \left\|\mathbf{e_3}\right\| = 1.</math> This means that <math>\left\{\mathbf{e_1}, \mathbf{e_2}, \mathbf{e_3}\right\}</math> is an orthonormal set. All vectors <math>(\mathbf{x}, \mathbf{y}, \mathbf{z}) \in \R^3</math> can be expressed as a sum of the basis vectors scaled <math display="block"> (\mathbf{x},\mathbf{y},\mathbf{z}) = \mathbf{x e_1} + \mathbf{y e_2} + \mathbf{z e_3},</math> so <math>\left\{\mathbf{e_1}, \mathbf{e_2}, \mathbf{e_3}\right\}</math> spans <math>\R^3</math> and hence must be a basis. It may also be shown that the standard basis rotated about an axis through the origin or reflected in a plane through the origin also forms an orthonormal basis of <math>\R^3</math>.
* For <math>\mathbb{R}^n</math>, the standard basis and inner product are similarly defined. Any other orthonormal basis is related to the standard basis by an [[orthogonal transformation]] in the group O(n).
* For [[pseudo-Euclidean space]] <math>\mathbb{R}^{p,q},</math>, an orthogonal basis <math>\{e_\mu\}</math> with metric <math>\eta</math> instead satisfies <math>\eta(e_\mu,e_\nu) = 0</math> if <math>\mu\neq \nu</math>, <math>\eta(e_\mu,e_\mu) = +1</math> if <math>1\leq\mu\leq p</math>, and <math>\eta(e_\mu,e_\mu) =-1</math> if <math>p+1\leq\mu\leq p+q</math>. Any two orthonormal bases are related by a pseudo-orthogonal transformation. In the case <math>(p,q) = (1,3)</math>, these are Lorentz transformations.
* The set <math>\left\{f_n : n \in \Z\right\}</math> with <math>f_n(x) = \exp(2 \pi inx),</math> where <math>\exp</math> denotes the [[exponential function]], forms an orthonormal basis of the space of functions with finite Lebesgue integrals, <math>L^2([0,1]),</math> with respect to the [[2-norm]]. This is fundamental to the study of [[Fourier series]].
* The set <math>\left\{e_b : b \in B\right\}</math> with <math>e_b(c) = 1</math> if <math>b = c</math> and <math>e_b(c) = 0</math> otherwise forms an orthonormal basis of <math>\ell^2(B).</math>
* [[Eigenfunction|Eigenfunctions]] of a [[Sturm–Liouville eigenproblem]].
* The [[column vectors]] of an [[orthogonal matrix]] form an orthonormal set.

==Basic formula==

If <math>B</math> is an orthogonal basis of <math>H,</math> then every element <math>x \in H</math> may be written as
<math display=block>x = \sum_{b\in B} \frac{\langle x,b\rangle}{\lVert b\rVert^2} b.</math>

When <math>B</math> is orthonormal, this simplifies to
<math display=block>x = \sum_{b\in B}\langle x,b\rangle b</math>
and the square of the [[Norm (mathematics)|norm]] of <math>x</math> can be given by
<math display=block>\|x\|^2 = \sum_{b\in B}|\langle x,b\rangle |^2.</math>

Even if <math>B</math> is [[Uncountable set|uncountable]], only countably many terms in this sum will be non-zero, and the expression is therefore well-defined. This sum is also called the ''[[Generalized Fourier series|Fourier expansion]]'' of <math>x,</math> and the formula is usually known as [[Parseval's identity]].

If <math>B</math>  is an orthonormal basis of <math>H,</math> then <math>H</math> is ''isomorphic'' to <math>\ell^2(B)</math> in the following sense: there exists a [[bijective]] [[linear operator|linear]] map <math>\Phi : H \to \ell^2(B)</math> such that
<math display="block">\langle\Phi(x),\Phi(y)\rangle=\langle x,y\rangle \ \ \forall \ x, y \in H.</math>

==Orthonormal system==

A set <math>S</math> of mutually orthonormal vectors in a Hilbert space <math>H</math> is called an orthonormal system. An orthonormal basis is an orthonormal system with the additional property that the linear span of <math>S</math> is dense in <math>H</math>.{{sfn | Steinwart | Christmann | 2008 | p=503}} Alternatively, the set <math>S</math> can be regarded as either ''complete'' or ''incomplete'' with respect to <math>H</math>. That is, we can take the smallest closed linear subspace <math>V \subseteq H</math> containing <math>S.</math> Then <math>S</math> will be an orthonormal basis of <math>V;</math> which may of course be smaller than <math>H</math> itself, being an ''incomplete'' orthonormal set, or be <math>H,</math> when it is a ''complete'' orthonormal set.

==Existence==

Using [[Zorn's lemma]] and the [[Gram–Schmidt process]] (or more simply well-ordering and transfinite recursion), one can show that ''every'' Hilbert space admits an orthonormal basis;<ref> [https://books.google.com/books?id=-m3jBwAAQBAJ Linear Functional Analysis] Authors: Rynne, Bryan, Youngson, M.A. page 79</ref> furthermore, any two orthonormal bases of the same space have the same [[Cardinal number|cardinality]] (this can be proven in a manner akin to that of the proof of the usual [[dimension theorem for vector spaces]], with separate cases depending on whether  the larger basis candidate is countable or not). A Hilbert space is [[Separable metric space|separable]] if and only if it admits a [[countable]] orthonormal basis. (One can prove this last statement without using the [[axiom of choice]]. However, one would have to use the [[axiom of countable choice]].)

==Choice of basis as a choice of isomorphism==

For concreteness we discuss orthonormal bases for a real, <math>n</math>-dimensional vector space <math>V</math> with a positive definite symmetric bilinear form <math>\phi=\langle\cdot,\cdot\rangle</math>. 

One way to view an orthonormal basis with respect to <math>\phi</math> is as a set of vectors <math>\mathcal{B} = \{e_i\}</math>, which allow us to write <math>v = v^ie_i \ \ \forall \ v \in V</math> , and <math>v^i\in \mathbb{R}</math> or <math>(v^i) \in \mathbb{R}^n</math>. With respect to this basis, the components of <math>\phi</math> are particularly simple: <math>\phi(e_i,e_j) = \delta_{ij}</math> (where <math>\delta_{ij}</math> is the [[Kronecker delta]]).

We can now view the basis as a map <math>\psi_\mathcal{B}:V\rightarrow \mathbb{R}^n</math> which is an isomorphism of inner product spaces: to make this more explicit we can write
:<math>\psi_\mathcal{B}:(V,\phi)\rightarrow (\mathbb{R}^n,\delta_{ij}).</math>
Explicitly we can write <math>(\psi_\mathcal{B}(v))^i = e^i(v) = \phi(e_i,v)</math> where <math>e^i</math> is the dual basis element to <math>e_i</math>.

The inverse is a component map
:<math>C_\mathcal{B}:\mathbb{R}^n\rightarrow V, (v^i)\mapsto \sum_{i=1}^n v^ie_i.</math>
These definitions make it manifest that there is a bijection
:<math>\{\text{Space of orthogonal bases } \mathcal{B}\}\leftrightarrow \{\text{Space of isomorphisms }V\leftrightarrow \mathbb{R}^n\}.</math>

The space of isomorphisms admits actions of orthogonal groups at either the <math>V</math> side or the <math>\mathbb{R}^n</math> side. For concreteness we fix the isomorphisms to point in the direction <math>\mathbb{R}^n\rightarrow V</math>, and consider the space of such maps, <math>\text{Iso}(\mathbb{R}^n\rightarrow V)</math>. 

This space admits a left action by the group of isometries of <math>V</math>, that is, <math>R\in \text{GL}(V)</math> such that <math>\phi(\cdot,\cdot) = \phi(R\cdot,R\cdot)</math>, with the action given by composition: <math>R*C=R\circ C.</math>

This space also admits a right action by the group of isometries of <math>\mathbb{R}^n</math>, that is, <math>R_{ij} \in \text{O}(n)\subset \text{Mat}_{n\times n}(\mathbb{R})</math>, with the action again given by composition: <math>C*R_{ij} = C\circ R_{ij}</math>.

==As a principal homogeneous space==
{{Main|Stiefel manifold}}

The set of orthonormal bases for <math>\mathbb{R}^n</math> with the standard inner product is a [[principal homogeneous space]] or G-torsor for the [[orthogonal group]] <math>G = \text{O}(n),</math> and is called the [[Stiefel manifold]] <math>V_n(\R^n)</math> of orthonormal [[k-frame|<math>n</math>-frames]].<ref>{{Cite web|title=CU Faculty|url=https://engfac.cooper.edu/fred|access-date=2021-04-15|website=engfac.cooper.edu}}</ref>

In other words, the space of orthonormal bases is like the orthogonal group, but without a choice of base point: given the space of orthonormal bases, there is no natural choice of orthonormal basis, but once one is given one, there is a one-to-one correspondence between bases and the orthogonal group.
Concretely, a linear map is determined by where it sends a given basis: just as an invertible map can take any basis to any other basis, an orthogonal map can take any ''orthogonal'' basis to any other ''orthogonal'' basis.

The other Stiefel manifolds <math>V_k(\R^n)</math> for <math>k < n</math> of ''incomplete'' orthonormal bases (orthonormal <math>k</math>-frames) are still homogeneous spaces for the orthogonal group, but not ''principal'' homogeneous spaces: any <math>k</math>-frame can be taken to any other <math>k</math>-frame by an orthogonal map, but this map is not uniquely determined.

* The set of orthonormal bases for <math>\mathbb{R}^{p,q}</math> is a G-torsor for <math>G = \text{O}(p,q)</math>.
* The set of orthonormal bases for <math>\mathbb{C}^n</math> is a G-torsor for <math>G = \text{U}(n)</math>.
* The set of orthonormal bases for <math>\mathbb{C}^{p,q}</math> is a G-torsor for <math>G = \text{U}(p,q)</math>.
* The set of right-handed orthonormal bases for <math>\mathbb{R}^n</math> is a G-torsor for <math>G = \text{SO}(n)</math>

==See also==

* {{annotated link|Orthogonal basis}}
* {{annotated link|Basis (linear algebra)}}
* {{annotated link|Affine space#Affine coordinates|Orthonormal frame}}
* {{annotated link|Schauder basis}}
* {{annotated link|Total set}}

==Notes==
{{reflist}}

==References==
*{{cite book
 | last       = Roman
 | first      = Stephen
 | title      = Advanced Linear Algebra
 | edition    = Third
 | series     =[[Graduate Texts in Mathematics]] 
 | publisher  = Springer
 | date       = 2008
 | pages      = 
 | isbn       = 978-0-387-72828-5
 |author-link =Steven Roman
}} (page 218, ch.9)
* {{Rudin Walter Functional Analysis|edition=2}} <!-- {{sfn|Rudin|1991|p=}} -->
* {{cite book | last=Steinwart | first=Ingo | last2=Christmann | first2=Andreas | title=Support vector machines | publisher=Springer | publication-place=New York | date=2008 | isbn=978-0-387-77241-7 | doi=10.1007/978-0-387-77242-4}}

==External links==

* This [https://math.stackexchange.com/q/1805184 Stack Exchange Post] discusses why the set of Dirac Delta functions is not a basis of L<sup>2</sup>([0,1]).

{{linear algebra}}
{{Hilbert space}}
{{Functional analysis}}

{{DEFAULTSORT:Orthonormal Basis}}
[[Category:Fourier analysis]]
[[Category:Functional analysis]]
[[Category:Linear algebra]]