Editing Likelihood function (section)

===Exponential families===
{{further|Exponential family}}
The log-likelihood is also particularly useful for [[exponential families]] of distributions, which include many of the common [[parametric model|parametric probability distributions]]. The probability distribution function (and thus likelihood function) for exponential families contain products of factors involving [[exponentiation]]. The logarithm of such a function is a sum of products, again easier to differentiate than the original function.

An exponential family is one whose probability density function is of the form (for some functions, writing <math display="inline">\langle -, - \rangle</math> for the [[inner product]]):

<math display="block"> p(x \mid \boldsymbol \theta) = h(x) \exp\Big(\langle \boldsymbol\eta({\boldsymbol \theta}), \mathbf{T}(x)\rangle -A({\boldsymbol \theta}) \Big).</math>

Each of these terms has an interpretation,{{efn|See {{slink|Exponential family|Interpretation}}}} but simply switching from probability to likelihood and taking logarithms yields the sum:

<math display="block"> \ell(\boldsymbol \theta \mid x) = \langle \boldsymbol\eta({\boldsymbol \theta}), \mathbf{T}(x)\rangle - A({\boldsymbol \theta}) + \log h(x).</math>

The <math display="inline">\boldsymbol \eta(\boldsymbol \theta)</math> and <math display="inline">h(x)</math> each correspond to a [[change of coordinates]], so in these coordinates, the log-likelihood of an exponential family is given by the simple formula:

<math display="block"> \ell(\boldsymbol \eta \mid x) = \langle \boldsymbol\eta, \mathbf{T}(x)\rangle - A({\boldsymbol \eta}).</math>

In words, the log-likelihood of an exponential family is inner product of the natural parameter {{tmath|\boldsymbol\eta}} and the [[sufficient statistic]] {{tmath|\mathbf{T}(x)}}, minus the normalization factor ([[log-partition function]]) {{tmath|A({\boldsymbol \eta})}}. Thus for example the maximum likelihood estimate can be computed by taking derivatives of the sufficient statistic {{math|''T''}} and the log-partition function {{math|''A''}}.

====Example: the gamma distribution====
The [[gamma distribution]] is an exponential family with two parameters, <math display="inline">\alpha</math> and <math display="inline">\beta</math>. The likelihood function is

<math display="block">\mathcal{L} (\alpha, \beta \mid x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}.</math>

Finding the maximum likelihood estimate of <math display="inline">\beta</math> for a single observed value <math display="inline">x</math> looks rather daunting. Its logarithm is much simpler to work with:

<math display="block">\log \mathcal{L}(\alpha,\beta \mid x) = \alpha \log \beta - \log \Gamma(\alpha) + (\alpha-1) \log x  - \beta x. \, </math>

To maximize the log-likelihood, we first take the [[partial derivative]] with respect to <math display="inline">\beta</math>:

<math display="block">\frac{\partial \log \mathcal{L}(\alpha,\beta \mid x)}{\partial \beta} = \frac{\alpha}{\beta} - x.</math>

If there are a number of independent observations <math display="inline">x_1, \ldots, x_n</math>, then the joint log-likelihood will be the sum of individual log-likelihoods, and the derivative of this sum will be a sum of derivatives of each individual log-likelihood:

<math display="block">
\begin{align}
& \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_1, \ldots, x_n)}{\partial \beta} \\
&= \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_1)}{\partial \beta} + \cdots + \frac{\partial \log \mathcal{L}(\alpha,\beta \mid x_n)}{\partial \beta} \\
&= \frac{n \alpha} \beta - \sum_{i=1}^n x_i.
\end{align}
</math>

To complete the maximization procedure for the joint log-likelihood, the equation is set to zero and solved for <math display="inline">\beta</math>:

<math display="block">\widehat\beta = \frac{\alpha}{\bar{x}}.</math>

Here <math display="inline">\widehat\beta</math> denotes the maximum-likelihood estimate, and <math display="inline">\textstyle \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i</math> is the [[sample mean]] of the observations.