Gaussian integral

Template:Use American English Template:Short description {{#invoke:Hatnote|hatnote}}

A graph of the function <math>f(x) = e^{-x^2}</math> and the area between it and the <math>x</math>-axis, (i.e. the entire real line) which is equal to <math>\sqrt{\pi}</math>.

The Gaussian integral, also known as the Euler–Poisson integral, is the integral of the Gaussian function <math>f(x) = e^{-x^2}</math> over the entire real line. Named after the German mathematician Carl Friedrich Gauss, the integral is <math display="block">\int_{-\infty}^\infty e^{-x^2}\,dx = \sqrt{\pi}.</math>

Abraham de Moivre originally discovered this type of integral in 1733, while Gauss published the precise integral in 1809,<ref name="The Evolution of the Normal Distribution">{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> attributing its discovery to Laplace. The integral has a wide range of applications. For example, with a slight change of variables it is used to compute the normalizing constant of the normal distribution. The same integral with finite limits is closely related to both the error function and the cumulative distribution function of the normal distribution. In physics this type of integral appears frequently, for example, in quantum mechanics, to find the probability density of the ground state of the harmonic oscillator. This integral is also used in the path integral formulation, to find the propagator of the harmonic oscillator, and in statistical mechanics, to find its partition function.

Although no elementary function exists for the error function, as can be proven by the Risch algorithm,<ref>Template:Cite journal</ref> the Gaussian integral can be solved analytically through the methods of multivariable calculus. That is, there is no elementary indefinite integral for <math display="block">\int e^{-x^2}\,dx,</math> but the definite integral <math display="block">\int_{-\infty}^\infty e^{-x^2}\,dx</math> can be evaluated. The definite integral of an arbitrary Gaussian function is <math display="block">\int_{-\infty}^{\infty} e^{-a(x+b)^2}\,dx= \sqrt{\frac{\pi}{a}}.</math>

ComputationEdit

By polar coordinatesEdit

A standard way to compute the Gaussian integral, the idea of which goes back to Poisson,<ref name="york.ac.uk">{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> is to make use of the property that:

<math display="block">\left(\int_{-\infty}^{\infty} e^{-x^2}\,dx\right)^2 = \int_{-\infty}^{\infty} e^{-x^2}\,dx \int_{-\infty}^{\infty} e^{-y^2}\,dy = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} e^{-\left(x^2+y^2\right)}\, dx\,dy. </math>

Consider the function <math>e^{-\left(x^2 + y^2\right)} = e^{-r^{2}}</math>on the plane <math>\mathbb{R}^2</math>, and compute its integral two ways:

on the one hand, by double integration in the Cartesian coordinate system, its integral is a square: <math display="block">\left(\int e^{-x^2}\,dx\right)^2;</math>
on the other hand, by shell integration (a case of double integration in polar coordinates), its integral is computed to be <math>\pi</math>

Comparing these two computations yields the integral, though one should take care about the improper integrals involved.

<math display="block">\begin{align}

  \iint_{\R^2} e^{-\left(x^2 + y^2\right)}dx\,dy
  &= \int_0^{2\pi} \int_0^{\infty} e^{-r^2}r\,dr\,d\theta\\[6pt]
  &= 2\pi \int_0^\infty re^{-r^2}\,dr\\[6pt]
  &= 2\pi \int_{-\infty}^0 \tfrac{1}{2} e^s\,ds && s = -r^2\\[6pt]
  &= \pi \int_{-\infty}^0 e^s\,ds \\[6pt]
  &= \pi \, \left[ e^s\right]_{-\infty}^{0} \\[6pt]
  &= \pi \,\left(e^0 - e^{-\infty}\right) \\[6pt]
  &= \pi \,\left(1 - 0\right) \\[6pt]
  &=\pi,
\end{align}</math>

where the factor of Template:Mvar is the Jacobian determinant which appears because of the transform to polar coordinates (Template:Math is the standard measure on the plane, expressed in polar coordinates Wikibooks:Calculus/Polar Integration#Generalization), and the substitution involves taking Template:Math, so Template:Math.

Combining these yields <math display="block">\left ( \int_{-\infty}^\infty e^{-x^2}\,dx \right )^2=\pi,</math> so <math display="block">\int_{-\infty}^\infty e^{-x^2} \, dx = \sqrt{\pi}.</math>

Complete proofEdit

To justify the improper double integrals and equating the two expressions, we begin with an approximating function: <math display="block">I(a) = \int_{-a}^a e^{-x^2}dx.</math>

If the integral <math display="block">\int_{-\infty}^\infty e^{-x^2} \, dx</math> were absolutely convergent we would have that its Cauchy principal value, that is, the limit <math display="block">\lim_{a\to\infty} I(a) </math> would coincide with <math display="block">\int_{-\infty}^\infty e^{-x^2}\,dx.</math> To see that this is the case, consider that

<math display="block">\int_{-\infty}^\infty \left|e^{-x^2}\right| dx < \int_{-\infty}^{-1} -x e^{-x^2}\, dx + \int_{-1}^1 e^{-x^2}\, dx+ \int_{1}^{\infty} x e^{-x^2}\, dx < \infty .</math>

So we can compute <math display="block">\int_{-\infty}^\infty e^{-x^2} \, dx</math> by just taking the limit <math display="block">\lim_{a\to\infty} I(a).</math>

Taking the square of <math>I(a)</math> yields

<math display="block">\begin{align} I(a)^2 & = \left ( \int_{-a}^a e^{-x^2}\, dx \right ) \left ( \int_{-a}^a e^{-y^2}\, dy \right ) \\[6pt] & = \int_{-a}^a \left ( \int_{-a}^a e^{-y^2}\, dy \right )\,e^{-x^2}\, dx \\[6pt] & = \int_{-a}^a \int_{-a}^a e^{-\left(x^2+y^2\right)}\,dy\,dx. \end{align}</math>

Using Fubini's theorem, the above double integral can be seen as an area integral <math display="block">\iint_{[-a, a] \times [-a, a]} e^{-\left(x^2+y^2\right)}\,d(x,y),</math> taken over a square with vertices Template:Math on the xy-plane.

Since the exponential function is greater than 0 for all real numbers, it then follows that the integral taken over the square's incircle must be less than <math>I(a)^2</math>, and similarly the integral taken over the square's circumcircle must be greater than <math>I(a)^2</math>. The integrals over the two disks can easily be computed by switching from Cartesian coordinates to polar coordinates:

<math display="block">\begin{align} x &= r \cos \theta, & y &= r \sin\theta \end{align}</math> <math display="block"> \mathbf J(r, \theta) = \begin{bmatrix}

 \dfrac{\partial x}{\partial r} & \dfrac{\partial x}{\partial\theta}\\[1em]
 \dfrac{\partial y}{\partial r} & \dfrac{\partial y}{\partial\theta} \end{bmatrix}

= \begin{bmatrix}

 \cos\theta &           -  r\sin \theta \\
 \sin\theta & \hphantom{-} r\cos \theta

\end{bmatrix} </math> <math display="block">d(x,y) = \left|J(r, \theta)\right| d(r,\theta) = r\, d(r,\theta).</math> <math display="block">\int_0^{2\pi} \int_0^a re^{-r^2} \, dr \, d\theta < I^2(a) < \int_0^{2\pi} \int_0^{a\sqrt{2}} re^{-r^2} \, dr\, d\theta.</math>

(See to polar coordinates from Cartesian coordinates for help with polar transformation.)

Integrating, <math display="block">\pi \left(1-e^{-a^2}\right) < I^2(a) < \pi \left(1 - e^{-2a^2}\right). </math>

By the squeeze theorem, this gives the Gaussian integral <math display="block">\int_{-\infty}^\infty e^{-x^2}\, dx = \sqrt{\pi}.</math>

By Cartesian coordinatesEdit

A different technique, which goes back to Laplace (1812),<ref name="york.ac.uk" /> is the following. Let <math display="block">\begin{align} y & = xs \\ dy & = x\,ds. \end{align}</math>

Since the limits on Template:Mvar as Template:Math depend on the sign of Template:Mvar, it simplifies the calculation to use the fact that Template:Math is an even function, and, therefore, the integral over all real numbers is just twice the integral from zero to infinity. That is,

<math display="block">\int_{-\infty}^{\infty} e^{-x^2} \, dx = 2\int_{0}^{\infty} e^{-x^2}\,dx.</math>

Thus, over the range of integration, Template:Math, and the variables Template:Mvar and Template:Mvar have the same limits. This yields: <math display="block">\begin{align} I^2 &= 4 \int_0^\infty \int_0^\infty e^{-\left(x^2 + y^2\right)} dy\,dx \\[6pt] &= 4 \int_0^\infty \left( \int_0^\infty e^{-\left(x^2 + y^2\right)} \, dy \right) \, dx \\[6pt] &= 4 \int_0^\infty \left( \int_0^\infty e^{-x^2\left(1+s^2\right)} x\,ds \right) \, dx \\[6pt] \end{align}</math> Then, using Fubini's theorem to switch the order of integration: <math display="block">\begin{align} I^2 &= 4 \int_0^\infty \left( \int_0^\infty e^{-x^2\left(1 + s^2\right)} x \, dx \right) \, ds \\[6pt] &= 4 \int_0^\infty \left[ \frac{e^{-x^2\left(1+s^2\right)} }{-2 \left(1+s^2\right)} \right]_{x=0}^{x=\infty} \, ds \\[6pt] &= 4 \left (\frac{1}{2} \int_0^\infty \frac{ds}{1+s^2} \right) \\[6pt] &= 2 \arctan(s)\Big |_0^\infty \\[6pt] &= \pi. \end{align}</math>

Therefore, <math>I = \sqrt{\pi}</math>, as expected.

By Laplace's methodEdit

In Laplace approximation, we deal only with up to second-order terms in Taylor expansion, so we consider <math>e^{-x^2}\approx 1-x^2 \approx (1+x^2)^{-1}</math>.

In fact, since <math>(1+t)e^{-t} \leq 1</math> for all <math>t</math>, we have the exact bounds:<math display="block">1-x^2 \leq e^{-x^2} \leq (1+x^2)^{-1}</math>Then we can do the bound at Laplace approximation limit:<math display="block">\int_{[-1, 1]}(1-x^2)^n dx \leq \int_{[-1, 1]}e^{-nx^2} dx \leq \int_{[-1, 1]}(1+x^2)^{-n} dx</math>

That is, <math display="block">2\sqrt n\int_{[0, 1]}(1-x^2)^n dx \leq \int_{[-\sqrt n, \sqrt n]}e^{-x^2} dx \leq 2\sqrt n\int_{[0, 1]}(1+x^2)^{-n} dx</math>

By trigonometric substitution, we exactly compute those two bounds: <math>2\sqrt n(2n)!!/(2n+1)!!</math> and <math>2\sqrt n (\pi/2)(2n-3)!!/(2n-2)!!</math>

By taking the square root of the Wallis formula, <math display="block">\frac \pi 2 = \prod_{n=1} \frac{(2n)^2}{(2n-1)(2n+1)}</math>we have <math>\sqrt \pi = 2 \lim_{n\to \infty} \sqrt{n} \frac{(2n)!!}{(2n+1)!!}</math>, the desired lower bound limit. Similarly we can get the desired upper bound limit. Conversely, if we first compute the integral with one of the other methods above, we would obtain a proof of the Wallis formula.

Relation to the gamma functionEdit

The integrand is an even function,

<math display="block">\int_{-\infty}^{\infty} e^{-x^2} dx = 2 \int_0^\infty e^{-x^2} dx</math>

Thus, after the change of variable <math display="inline">x = \sqrt{t}</math>, this turns into the Euler integral

<math display="block">2 \int_0^\infty e^{-x^2} dx = 2\int_0^\infty \frac{1}{2}\ e^{-t} \ t^{-\frac{1}{2}} dt = \Gamma{\left(\frac{1}{2}\right)} = \sqrt{\pi}</math>

where <math display="inline"> \Gamma(z) = \int_{0}^{\infty} t^{z-1} e^{-t} dt </math> is the gamma function. This shows why the factorial of a half-integer is a rational multiple of <math display="inline">\sqrt \pi</math>. More generally, <math display="block">\int_0^\infty x^n e^{-ax^b} dx = \frac{\Gamma{\left((n+1)/b\right)}}{b a^{(n+1)/b}}, </math> which can be obtained by substituting <math>t=a x^b</math> in the integrand of the gamma function to get <math display="inline"> \Gamma(z) = a^z b \int_0^{\infty} x^{bz-1} e^{-a x^b} dx </math>.

GeneralizationsEdit

The integral of a Gaussian functionEdit

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} The integral of an arbitrary Gaussian function is <math display="block">\int_{-\infty}^{\infty} e^{-a(x+b)^2}\,dx= \sqrt{\frac{\pi}{a}}.</math>

An alternative form is <math display="block">\int_{-\infty}^{\infty}e^{- (a x^2 + b x + c)}\,dx=\sqrt{\frac{\pi}{a}}\,e^{\frac{b^2}{4a}-c}.</math>

This form is useful for calculating expectations of some continuous probability distributions related to the normal distribution, such as the log-normal distribution, for example.

Complex formEdit

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} <math display="block">\int_{-\infty}^{\infty} e^{\frac 12 it^2} dt = e^{i\pi/4} \sqrt{2\pi}</math>and more generally,<math display="block">\int_{\mathbb{R}^N} e^{\frac{1}{2} i \mathbf{x}^T A \mathbf{x}}dx = \det(A)^{-\frac{1}{2}} {\left(e^{i\pi/4} \sqrt{2\pi}\right)}^N</math>for any positive-definite symmetric matrix <math>A</math>.

n-dimensional and functional generalizationEdit

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} Suppose A is a symmetric positive-definite (hence invertible) Template:Math precision matrix, which is the matrix inverse of the covariance matrix. Then,

<math display="block">\begin{align} \int_{\mathbb{R}^n} \exp{\left(-\frac 1 2 \mathbf{x}^\mathsf{T} A \mathbf{x} \right)} \, d^n \mathbf{x} &= \int_{\mathbb{R}^n} \exp{\left(-\frac 1 2 \sum\limits_{i,j=1}^{n} A_{ij} x_i x_j \right)} \, d^n \mathbf{x} \\[1ex] &= \sqrt{\frac{{\left(2\pi\right)}^n}{\det A}} = \sqrt{\frac{1}{\det \left(A / 2\pi\right)}} \\[1ex] &= \sqrt{\det \left(2 \pi A^{-1}\right)} \end{align}</math>By completing the square, this generalizes to<math display="block">\int_{\mathbb{R}^n} \exp{\left(-\tfrac 1 2 \mathbf{x}^\mathsf{T} A \mathbf{x} + \mathbf{b}^\mathsf{T} \mathbf{x} + c\right)} \, d^n \mathbf{x} = \sqrt{\det \left(2 \pi A^{-1}\right)} \exp\left(\tfrac{1}{2} \mathbf{b}^\mathsf{T} A^{-1} \mathbf{b} + c\right)</math>

This fact is applied in the study of the multivariate normal distribution.

Also, <math display="block">\int x_{k_1}\cdots x_{k_{2N}} \, \exp{\left( -\frac{1}{2} \sum\limits_{i,j=1}^{n}A_{ij} x_i x_j \right)} \, d^nx =\sqrt{\frac{(2\pi)^n}{\det A}} \, \frac{1}{2^N N!} \, \sum_{\sigma \in S_{2N}}(A^{-1})_{k_{\sigma(1)}k_{\sigma(2)}} \cdots (A^{-1})_{k_{\sigma(2N-1)}k_{\sigma(2N)}}</math> where σ is a permutation of Template:Math and the extra factor on the right-hand side is the sum over all combinatorial pairings of Template:Math of N copies of A⁻¹.

Alternatively,<ref name="Central identity explanation">{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>

<math display="block">\int f(\mathbf x) \exp{\left( - \frac 1 2 \sum_{i,j=1}^n A_{ij} x_i x_j \right)} d^n\mathbf{x} = \sqrt{\frac{{\left(2\pi\right)}^n}{\det A}} \, \left. \exp\left(\frac{1}{2} \sum_{i,j=1}^{n}\left(A^{-1}\right)_{ij}{\partial \over \partial x_i}{\partial \over \partial x_j}\right) f(\mathbf{x})\right|_{\mathbf{x}=0}</math>

for some analytic function f, provided it satisfies some appropriate bounds on its growth and some other technical criteria. (It works for some functions and fails for others. Polynomials are fine.) The exponential over a differential operator is understood as a power series.

While functional integrals have no rigorous definition (or even a nonrigorous computational one in most cases), we can define a Gaussian functional integral in analogy to the finite-dimensional case. Template:Citation needed There is still the problem, though, that <math>(2\pi)^\infty</math> is infinite and also, the functional determinant would also be infinite in general. This can be taken care of if we only consider ratios:

<math display="block">\begin{align} & \frac{\displaystyle\int f(x_1)\cdots f(x_{2N}) \exp\left[{-\iint \frac{1}{2}A(x_{2N+1},x_{2N+2}) f(x_{2N+1}) f(x_{2N+2}) \, d^dx_{2N+1} \, d^dx_{2N+2}}\right] \mathcal{D}f}{\displaystyle\int \exp\left[{-\iint \frac{1}{2} A(x_{2N+1}, x_{2N+2}) f(x_{2N+1}) f(x_{2N+2}) \, d^dx_{2N+1} \, d^dx_{2N+2}}\right] \mathcal{D}f} \\[6pt] = {} & \frac{1}{2^N N!}\sum_{\sigma \in S_{2N}}A^{-1}(x_{\sigma(1)},x_{\sigma(2)})\cdots A^{-1}(x_{\sigma(2N-1)},x_{\sigma(2N)}). \end{align}</math>

In the DeWitt notation, the equation looks identical to the finite-dimensional case.

n-dimensional with linear termEdit

If A is again a symmetric positive-definite matrix, then (assuming all are column vectors) <math display="block">\begin{align} \int \exp\left(-\frac{1}{2}\sum_{i,j=1}^n A_{ij} x_i x_j+\sum_{i=1}^n b_i x_i\right) d^n \mathbf{x} &= \int \exp\left(-\tfrac{1}{2} \mathbf{x}^\mathsf{T} A \mathbf{x} + \mathbf{b}^\mathsf{T} \mathbf{x}\right) d^n \mathbf{x} \\ &= \sqrt{ \frac{(2\pi)^n}{\det A} } \exp\left(\tfrac{1}{2} \mathbf{b}^\mathsf{T} A^{-1} \mathbf{b}\right). \end{align}</math>

Integrals of similar formEdit

<math display="block">\int_0^\infty x^{2n} e^{-{x^2}/{a^2}}\,dx = \sqrt{\pi}\frac{a^{2n+1} (2n-1)!!}{2^{n+1}}</math> <math display="block">\int_0^\infty x^{2n+1} e^{-{x^2}/{a^2}} \, dx = \frac{n!}{2} a^{2n+2}</math> <math display="block">\int_0^\infty x^{2n}e^{-bx^2}\,dx = \frac{(2n-1)!!}{b^n 2^{n+1}} \sqrt{\frac{\pi}{b}}</math> <math display="block">\int_0^\infty x^{2n+1}e^{-bx^2}\,dx = \frac{n!}{2b^{n+1}}</math> <math display="block">\int_0^\infty x^{n}e^{-bx^2}\,dx = \frac{\Gamma(\frac{n+1}{2})}{2b^{\frac{n+1}{2}}}</math> where <math>n</math> is a positive integer

An easy way to derive these is by differentiating under the integral sign.

<math display="block">\begin{align} \int_{-\infty}^\infty x^{2n} e^{-\alpha x^2}\,dx &= \left(-1\right)^n\int_{-\infty}^\infty \frac{\partial^n}{\partial \alpha^n} e^{-\alpha x^2}\,dx \\[1ex] &= \left(-1\right)^n\frac{\partial^n}{\partial \alpha^n} \int_{-\infty}^\infty e^{-\alpha x^2}\,dx\\[1ex] &= \sqrt{\pi} \left(-1\right)^n\frac{\partial^n}{\partial \alpha^n}\alpha^{-\frac{1}{2}} \\[1ex] &= \sqrt{\frac{\pi}{\alpha}}\frac{(2n-1)!!}{\left(2\alpha\right)^n} \end{align}</math>

One could also integrate by parts and find a recurrence relation to solve this.

Higher-order polynomialsEdit

Applying a linear change of basis shows that the integral of the exponential of a homogeneous polynomial in n variables may depend only on SL(n)-invariants of the polynomial. One such invariant is the discriminant, zeros of which mark the singularities of the integral. However, the integral may also depend on other invariants.<ref name="morozov2009">Template:Cite journal</ref>

Exponentials of other even polynomials can numerically be solved using series. These may be interpreted as formal calculations when there is no convergence. For example, the solution to the integral of the exponential of a quartic polynomial isTemplate:Citation needed

<math display="block">\int_{-\infty}^{\infty} e^{a x^4+b x^3+c x^2+d x+f}\,dx = \frac{1}{2} e^f \sum_{\begin{smallmatrix}n,m,p=0 \\ n+p=0 \bmod 2\end{smallmatrix}}^{\infty} \frac{b^n}{n!} \frac{c^m}{m!} \frac{d^p}{p!} \frac{\Gamma{\left (\frac{3n+2m+p+1}{4} \right)}}{{\left(-a\right)}^{\frac{3n+2m+p+1}4}}.</math>

The Template:Math mod 2 requirement is because the integral from −∞ to 0 contributes a factor of Template:Math to each term, while the integral from 0 to +∞ contributes a factor of 1/2 to each term. These integrals turn up in subjects such as quantum field theory.