Editing Functional derivative (section)

==Determining functional derivatives==
A formula to determine functional derivatives for a common class of functionals can be written as the integral of a function and its derivatives. This is a generalization of the [[Euler–Lagrange equation]]: indeed, the functional derivative was introduced in [[physics]] within the derivation of the [[Joseph-Louis Lagrange|Lagrange]] equation of the second kind from the [[principle of least action]] in [[Lagrangian mechanics]] (18th century). The first three examples below are taken from [[density functional theory]] (20th century), the fourth from [[statistical mechanics]] (19th century).

===Formula===
Given a functional
<math display="block">F[\rho] = \int f( \boldsymbol{r}, \rho(\boldsymbol{r}), \nabla\rho(\boldsymbol{r}) )\, d\boldsymbol{r},</math>
and a function <math>\phi(\boldsymbol{r})</math> that vanishes on the boundary of the region of integration, from a previous section [[#Definition|Definition]],
<math display="block">\begin{align}
\int \frac{\delta F}{\delta\rho(\boldsymbol{r})} \, \phi(\boldsymbol{r}) \, d\boldsymbol{r}
& = \left [ \frac{d}{d\varepsilon} \int f( \boldsymbol{r}, \rho + \varepsilon \phi, \nabla\rho+\varepsilon\nabla\phi )\, d\boldsymbol{r} \right ]_{\varepsilon=0} \\
& = \int \left( \frac{\partial f}{\partial\rho} \, \phi + \frac{\partial f}{\partial\nabla\rho} \cdot \nabla\phi \right) d\boldsymbol{r} \\
& = \int \left[ \frac{\partial f}{\partial\rho} \, \phi + \nabla \cdot \left( \frac{\partial f}{\partial\nabla\rho} \, \phi \right) - \left( \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi \right] d\boldsymbol{r} \\
& = \int \left[ \frac{\partial f}{\partial\rho} \, \phi - \left( \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi \right] d\boldsymbol{r} \\
& = \int \left( \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial\nabla\rho} \right) \phi(\boldsymbol{r}) \ d\boldsymbol{r} \, .
\end{align}</math>

The second line is obtained using the [[total derivative]], where {{math|''∂f'' /''∂∇ρ''}} is a [[Matrix calculus#Scalar-by-vector|derivative of a scalar with respect to a vector]].<ref group="Note">For a three-dimensional Cartesian coordinate system,
<math display="block">\frac{\partial f}{\partial\nabla\rho} = \frac{\partial f}{\partial\rho_x} \mathbf{\hat{i}} + \frac{\partial f}{\partial\rho_y} \mathbf{\hat{j}} + \frac{\partial f}{\partial\rho_z} \mathbf{\hat{k}}\, ,</math>
where <math>\rho_x = \frac{\partial \rho}{\partial x}\, , \ \rho_y = \frac{\partial \rho}{\partial y}\, , \ \rho_z = \frac{\partial \rho}{\partial z}</math> and <math>\mathbf{\hat{i}}</math>, <math>\mathbf{\hat{j}}</math>, <math>\mathbf{\hat{k}}</math> are unit vectors along the x, y, z axes.</ref>

The third line was obtained by use of a [[Divergence#Properties|product rule for divergence]]. The fourth line was obtained using the [[divergence theorem]] and the condition that <math>\phi=0</math> on the boundary of the region of integration. Since <math>\phi</math> is also an arbitrary function, applying the [[fundamental lemma of calculus of variations]] to the last line, the functional derivative is
<math display="block">\frac{\delta F}{\delta\rho(\boldsymbol{r})} = \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial\nabla\rho} </math>

where {{math|1=''ρ'' = ''ρ''('''''r''''')}} and {{math|1=''f'' = ''f'' ('''''r''''', ''ρ'', &nabla;''ρ'')}}. This formula is for the case of the functional form given by {{math|''F''[''ρ'']}} at the beginning of this section. For other functional forms, the definition of the functional derivative can be used as the starting point for its determination. (See the example [[#Coulomb potential energy functional|Coulomb potential energy functional]].)

The above equation for the functional derivative can be generalized to the case that includes higher dimensions and higher order derivatives. The functional would be,
<math display="block">F[\rho(\boldsymbol{r})] = \int f( \boldsymbol{r}, \rho(\boldsymbol{r}), \nabla\rho(\boldsymbol{r}), \nabla^{(2)}\rho(\boldsymbol{r}), \dots, \nabla^{(N)}\rho(\boldsymbol{r}))\, d\boldsymbol{r},</math>

where the vector {{math|'''''r''''' &isin; '''R'''<sup>''n''</sup>}}, and {{math|&nabla;<sup>(''i'')</sup>}} is a tensor whose {{math|''n<sup>i</sup>''}} components are partial derivative operators of order {{math|''i''}},
<math display="block"> \left [ \nabla^{(i)} \right ]_{\alpha_1 \alpha_2 \cdots \alpha_i} = \frac {\partial^{\, i}} {\partial r_{\alpha_1} \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } \qquad \qquad \text{where} \quad \alpha_1, \alpha_2, \dots, \alpha_i = 1, 2, \dots , n \ . </math><ref group="Note">For example, for the case of three dimensions ({{math|1=''n'' = 3}}) and second order derivatives ({{math|1=''i'' = 2}}), the tensor {{math|&nabla;<sup>(2)</sup>}} has components,
<math display="block"> \left [ \nabla^{(2)} \right ]_{\alpha \beta} = \frac {\partial^{\,2}} {\partial r_{\alpha} \, \partial r_{\beta}} </math>where <math>\alpha</math> and <math>\beta</math> can be <math>1,2,3</math>.</ref>

An analogous application of the definition of the functional derivative yields
<math display="block">\begin{align}
\frac{\delta F[\rho]}{\delta \rho} &{} = \frac{\partial f}{\partial\rho} - \nabla \cdot \frac{\partial f}{\partial(\nabla\rho)} + \nabla^{(2)} \cdot \frac{\partial f}{\partial\left(\nabla^{(2)}\rho\right)} + \dots + (-1)^N \nabla^{(N)} \cdot \frac{\partial f}{\partial\left(\nabla^{(N)}\rho\right)} \\
&{} = \frac{\partial f}{\partial\rho} + \sum_{i=1}^N (-1)^{i}\nabla^{(i)} \cdot \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} \ .
\end{align}</math>

In the last two equations, the {{math|''n<sup>i</sup>''}} components of the tensor <math> \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} </math> are partial derivatives of {{math|''f''}} with respect to partial derivatives of ''ρ'',
<math display="block"> \left [ \frac {\partial f} {\partial \left (\nabla^{(i)}\rho \right ) } \right ]_{\alpha_1 \alpha_2 \cdots \alpha_i} = \frac {\partial f} {\partial \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} }  </math>
where <math>  \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} \equiv \frac {\partial^{\,i}\rho} {\partial r_{\alpha_1} \, \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } </math>, and the tensor scalar product is,
<math display="block"> \nabla^{(i)} \cdot \frac{\partial f}{\partial\left(\nabla^{(i)}\rho\right)} = \sum_{\alpha_1, \alpha_2, \cdots, \alpha_i = 1}^n \ \frac {\partial^{\, i} } {\partial r_{\alpha_1} \, \partial r_{\alpha_2} \cdots \partial r_{\alpha_i} } \ \frac {\partial f} {\partial \rho_{\alpha_1 \alpha_2 \cdots \alpha_i} } \ . </math> <ref group="Note">For example, for the case {{math|1=''n'' = 3}} and {{math|1=''i'' = 2}}, the tensor scalar product is,
<math display="block"> \nabla^{(2)} \cdot \frac{\partial f}{\partial\left(\nabla^{(2)}\rho\right)} = \sum_{\alpha, \beta = 1}^3 \ \frac {\partial^{\, 2} } {\partial r_{\alpha} \, \partial r_{\beta} } \, \frac {\partial f} {\partial \rho_{\alpha \beta} } , </math>where <math>\rho_{\alpha \beta} \equiv \frac {\partial^{\, 2}\rho} {\partial r_{\alpha} \, \partial r_{\beta} }</math>.</ref>

===Examples===

====Thomas–Fermi kinetic energy functional====
The [[Thomas–Fermi model]] of 1927 used a kinetic energy functional for a noninteracting uniform [[free electron model|electron gas]] in a first attempt of [[density-functional theory]] of electronic structure:
<math display="block">T_\mathrm{TF}[\rho] = C_\mathrm{F} \int \rho^{5/3}(\mathbf{r}) \, d\mathbf{r} \, .</math>
Since the integrand of {{math|''T''<sub>TF</sub>[''ρ'']}} does not involve derivatives of {{math|''ρ''('''''r''''')}}, the functional derivative of {{math|''T''<sub>TF</sub>[''ρ'']}} is,<ref name=ParrYangP247A.6>{{harvp|Parr|Yang|1989|loc=p. 247, Eq. A.6}}.</ref>
<math display="block">\frac{\delta T_{\mathrm{TF}}}{\delta \rho (\boldsymbol{r}) }
= C_\mathrm{F} \frac{\partial \rho^{5/3}(\mathbf{r})}{\partial \rho(\mathbf{r})}
= \frac{5}{3} C_\mathrm{F} \rho^{2/3}(\mathbf{r}) \, .</math>

====Coulomb potential energy functional====
The '''electron-nucleus''' potential energy is
<math display="block">V[\rho] = \int \frac{\rho(\boldsymbol{r})}{|\boldsymbol{r}|} \ d\boldsymbol{r}.</math>

Applying the definition of functional derivative,
<math display="block">\begin{align}
\int \frac{\delta V}{\delta \rho(\boldsymbol{r})} \ \phi(\boldsymbol{r}) \ d\boldsymbol{r}
& {} = \left [ \frac{d}{d\varepsilon} \int \frac{\rho(\boldsymbol{r}) + \varepsilon \phi(\boldsymbol{r})}{|\boldsymbol{r}|} \ d\boldsymbol{r} \right ]_{\varepsilon=0} \\[1ex]
& {} = \int \frac {\phi(\boldsymbol{r})} {|\boldsymbol{r}|} \ d\boldsymbol{r} \, .
\end{align}</math>
So,
<math display="block"> \frac{\delta V}{\delta \rho(\boldsymbol{r})} = \frac{1}{|\boldsymbol{r}|} \ . </math>

The functional derivative of the classical part of the '''electron-electron interaction''' (often called Hartree energy) is 
<math display="block">J[\rho] = \frac{1}{2}\iint \frac{\rho(\mathbf{r}) \rho(\mathbf{r}')}{| \mathbf{r}-\mathbf{r}' |}\, d\mathbf{r} d\mathbf{r}' \, .</math>
From the [[#Functional derivative|definition of the functional derivative]],
<math display="block">\begin{align}
\int \frac{\delta J}{\delta\rho(\boldsymbol{r})} \phi(\boldsymbol{r})d\boldsymbol{r}
& {} = \left [ \frac {d \ }{d\varepsilon} \, J[\rho + \varepsilon\phi] \right ]_{\varepsilon = 0} \\
& {} = \left [ \frac {d \ }{d\varepsilon} \, \left ( \frac{1}{2}\iint \frac {[\rho(\boldsymbol{r}) + \varepsilon \phi(\boldsymbol{r})] \, [\rho(\boldsymbol{r}') + \varepsilon \phi(\boldsymbol{r}')] }{| \boldsymbol{r}-\boldsymbol{r}' |}\, d\boldsymbol{r} d\boldsymbol{r}' \right ) \right ]_{\varepsilon = 0} \\
& {} = \frac{1}{2}\iint \frac {\rho(\boldsymbol{r}') \phi(\boldsymbol{r}) }{| \boldsymbol{r}-\boldsymbol{r}' |}\, d\boldsymbol{r} d\boldsymbol{r}' + \frac{1}{2}\iint \frac {\rho(\boldsymbol{r}) \phi(\boldsymbol{r}') }{| \boldsymbol{r}-\boldsymbol{r}' |}\, d\boldsymbol{r} d\boldsymbol{r}' \\
\end{align}</math>
The first and second terms on the right hand side of the last equation are equal, since {{math|'''''r'''''}} and {{math|'''''r&prime;'''''}} in the second term can be interchanged without changing the value of the integral. Therefore,
<math display="block"> \int \frac{\delta J}{\delta\rho(\boldsymbol{r})} \phi(\boldsymbol{r})d\boldsymbol{r} = \int \left ( \int \frac {\rho(\boldsymbol{r}') }{| \boldsymbol{r}-\boldsymbol{r}' |} d\boldsymbol{r}' \right ) \phi(\boldsymbol{r}) d\boldsymbol{r} </math>
and the functional derivative of the electron-electron Coulomb potential energy functional {{math|''J''}}[''ρ''] is,<ref name=ParrYangP248A.11>{{harvp|Parr|Yang|1989|loc=p. 248, Eq. A.11}}.</ref>
<math display="block"> \frac{\delta J}{\delta\rho(\boldsymbol{r})} = \int \frac {\rho(\boldsymbol{r}') }{| \boldsymbol{r}-\boldsymbol{r}' |} d\boldsymbol{r}' \, . </math>

The second functional derivative is
<math display="block">\frac{\delta^2 J[\rho]}{\delta \rho(\mathbf{r}')\delta\rho(\mathbf{r})} = \frac{\partial}{\partial \rho(\mathbf{r}')} \left ( \frac{\rho(\mathbf{r}')}{| \mathbf{r}-\mathbf{r}' |} \right ) = \frac{1}{| \mathbf{r}-\mathbf{r}' |}.</math>

====von Weizsäcker kinetic energy functional====
In 1935 [[Carl Friedrich von Weizsacker|von Weizsäcker]] proposed to add a gradient correction to the Thomas-Fermi kinetic energy functional to make it better suit a molecular electron cloud:
<math display="block">T_\mathrm{W}[\rho] = \frac{1}{8} \int \frac{\nabla\rho(\mathbf{r}) \cdot \nabla\rho(\mathbf{r})}{ \rho(\mathbf{r}) } d\mathbf{r} = \int t_\mathrm{W}(\mathbf{r}) \ d\mathbf{r} \, ,</math>
where
<math display="block"> t_\mathrm{W} \equiv \frac{1}{8} \frac{\nabla\rho \cdot \nabla\rho}{ \rho } \qquad \text{and} \ \ \rho = \rho(\boldsymbol{r}) \ . </math>
Using a previously derived [[#Formula|formula]] for the functional derivative,
<math display="block">\begin{align}
\frac{\delta T_\mathrm{W}}{\delta \rho}
& = \frac{\partial t_\mathrm{W}}{\partial \rho} - \nabla\cdot\frac{\partial t_\mathrm{W}}{\partial \nabla \rho} \\
& = -\frac{1}{8}\frac{\nabla\rho \cdot \nabla\rho}{\rho^2} - \left ( \frac {1}{4} \frac {\nabla^2\rho} {\rho} - \frac {1}{4} \frac {\nabla\rho \cdot \nabla\rho} {\rho^2} \right ) \qquad \text{where} \ \ \nabla^2 = \nabla \cdot \nabla \ ,
\end{align}</math>
and the result is,<ref name=ParrYangP247A.9>{{harvp|Parr|Yang|1989|loc= p. 247, Eq. A.9}}.</ref>
<math display="block"> \frac{\delta T_\mathrm{W}}{\delta \rho} = \ \ \, \frac{1}{8}\frac{\nabla\rho \cdot \nabla\rho}{\rho^2} - \frac{1}{4}\frac{\nabla^2\rho}{\rho} \ . </math>

====Entropy====
The [[information entropy|entropy]] of a discrete [[random variable]] is a functional of the [[probability mass function]].

<math display="block">H[p(x)] = -\sum_x p(x) \log p(x)</math>
Thus,
<math display="block">\begin{align}
\sum_x \frac{\delta H}{\delta p(x)} \, \phi(x)
& {} = \left[ \frac{d}{d\varepsilon} H[p(x) + \varepsilon\phi(x)] \right]_{\varepsilon=0}\\
& {} = \left [- \, \frac{d}{d\varepsilon} \sum_x \, [p(x) + \varepsilon\phi(x)] \ \log [p(x) + \varepsilon\phi(x)] \right]_{\varepsilon=0} \\
& {} = -\sum_x \, [1+\log p(x)] \ \phi(x) \, .
\end{align}</math>
Thus,
<math display="block">\frac{\delta H}{\delta p(x)} = -1-\log p(x).</math>

==== Exponential ====

Let
<math display="block"> F[\varphi(x)]= e^{\int \varphi(x) g(x)dx}.</math>

Using the delta function as a test function,
<math display="block">\begin{align}
\frac{\delta F[\varphi(x)]}{\delta \varphi(y)}
& {} = \lim_{\varepsilon\to 0}\frac{F[\varphi(x)+\varepsilon\delta(x-y)]-F[\varphi(x)]}{\varepsilon}\\
& {} = \lim_{\varepsilon\to 0}\frac{e^{\int (\varphi(x)+\varepsilon\delta(x-y)) g(x)dx}-e^{\int \varphi(x) g(x)dx}}{\varepsilon}\\
& {} = e^{\int \varphi(x) g(x)dx}\lim_{\varepsilon\to 0}\frac{e^{\varepsilon \int \delta(x-y) g(x)dx}-1}{\varepsilon}\\
& {} = e^{\int \varphi(x) g(x)dx}\lim_{\varepsilon\to 0}\frac{e^{\varepsilon g(y)}-1}{\varepsilon}\\
& {} = e^{\int \varphi(x) g(x)dx}g(y).
\end{align}</math>

Thus,
<math display="block"> \frac{\delta F[\varphi(x)]}{\delta \varphi(y)} = g(y) F[\varphi(x)]. </math>

This is particularly useful in calculating the [[Correlation function (quantum field theory)|correlation functions]] from the [[Partition function (quantum field theory)|partition function]] in [[quantum field theory]].

====Functional derivative of a function====
A function can be written in the form of an integral like a functional. For example,
<math display="block">\rho(\boldsymbol{r}) = F[\rho] = \int \rho(\boldsymbol{r}') \delta(\boldsymbol{r}-\boldsymbol{r}')\, d\boldsymbol{r}'.</math>
Since the integrand does not depend on derivatives of ''ρ'', the functional derivative of ''ρ''{{math|('''''r''''')}} is,
<math display="block">\frac {\delta \rho(\boldsymbol{r})} {\delta\rho(\boldsymbol{r}')} \equiv \frac {\delta F} {\delta\rho(\boldsymbol{r}')}
= \frac{\partial \ \ }{\partial \rho(\boldsymbol{r}')} \, [\rho(\boldsymbol{r}') \delta(\boldsymbol{r}-\boldsymbol{r}')]
= \delta(\boldsymbol{r}-\boldsymbol{r}').</math>

==== Functional derivative of iterated function====
The functional derivative of the iterated function <math>f(f(x))</math> is given by:
<math display="block">\frac{\delta f(f(x))}{\delta f(y) } = f'(f(x))\delta(x-y) + \delta(f(x)-y)</math>
and
<math display="block">\frac{\delta f(f(f(x)))}{\delta f(y) } = f'(f(f(x))(f'(f(x))\delta(x-y) + \delta(f(x)-y)) + \delta(f(f(x))-y)</math>

In general:
<math display="block">\frac{\delta f^N(x)}{\delta f(y)} = f'( f^{N-1}(x) ) \frac{ \delta f^{N-1}(x)}{\delta f(y)} + \delta( f^{N-1}(x) - y ) </math>

Putting in {{math|1=''N'' = 0}} gives:
<math display="block"> \frac{\delta f^{-1}(x)}{\delta f(y) } = - \frac{ \delta(f^{-1}(x)-y ) }{ f'(f^{-1}(x)) }</math>