Editing Estimation theory (section)

====Maximum likelihood====
{{main|Maximum likelihood}}

Continuing the example using the [[maximum likelihood]] estimator, the [[probability density function]] (pdf) of the noise for one sample <math>w[n]</math> is
<math display="block">p(w[n]) = \frac{1}{\sigma \sqrt{2 \pi}} \exp\left(- \frac{1}{2 \sigma^2} w[n]^2 \right)</math>
and the probability of <math>x[n]</math> becomes (<math>x[n]</math> can be thought of a <math>\mathcal{N}(A, \sigma^2)</math>)
<math display="block">p(x[n]; A) = \frac{1}{\sigma \sqrt{2 \pi}} \exp\left(- \frac{1}{2 \sigma^2} (x[n] - A)^2 \right)</math>
By [[statistical independence|independence]], the probability of <math>\mathbf{x}</math> becomes
<math display="block">
p(\mathbf{x}; A)
=
\prod_{n=0}^{N-1} p(x[n]; A)
=
\frac{1}{\left(\sigma \sqrt{2\pi}\right)^N}
\exp\left(- \frac{1}{2 \sigma^2} \sum_{n=0}^{N-1}(x[n] - A)^2 \right)
</math>
Taking the [[natural logarithm]] of the pdf
<math display="block">
\ln p(\mathbf{x}; A)
=
-N \ln \left(\sigma \sqrt{2\pi}\right)
- \frac{1}{2 \sigma^2} \sum_{n=0}^{N-1}(x[n] - A)^2
</math>
and the maximum likelihood estimator is
<math display="block">\hat{A} = \arg \max \ln p(\mathbf{x}; A)</math>

Taking the first [[derivative]] of the log-likelihood function
<math display="block">
\frac{\partial}{\partial A} \ln p(\mathbf{x}; A)
=
\frac{1}{\sigma^2} \left[ \sum_{n=0}^{N-1}(x[n] - A) \right]
=
\frac{1}{\sigma^2} \left[ \sum_{n=0}^{N-1}x[n] - N A \right]
</math>
and setting it to zero
<math display="block">
0
=
\frac{1}{\sigma^2} \left[ \sum_{n=0}^{N-1}x[n] - N A \right]
=
\sum_{n=0}^{N-1}x[n] - N A
</math>

This results in the maximum likelihood estimator
<math display="block">\hat{A} = \frac{1}{N} \sum_{n=0}^{N-1}x[n]</math>
which is simply the sample mean.
From this example, it was found that the sample mean is the maximum likelihood estimator for <math>N</math> samples of a fixed, unknown parameter corrupted by AWGN.