Editing Estimation theory (section)

== Basics ==
For a given model, several statistical "ingredients" are needed so the estimator can be implemented. The first is a [[statistical sample]] – a set of data points taken from a [[random vector]] (RV) of size ''N''. Put into a [[vector (geometric)|vector]],
<math display="block">\mathbf{x} = \begin{bmatrix} x[0] \\ x[1] \\ \vdots \\ x[N-1] \end{bmatrix}.</math>
Secondly, there are ''M'' parameters
<math display="block">\boldsymbol{\theta} = \begin{bmatrix} \theta_1 \\ \theta_2 \\ \vdots \\ \theta_M \end{bmatrix},</math>
whose values are to be estimated. Third, the continuous [[probability density function]] (pdf) or its discrete counterpart, the [[probability mass function]] (pmf), of the underlying distribution that generated the data must be stated conditional on the values of the parameters:
<math display="block">p(\mathbf{x} | \boldsymbol{\theta}).\,</math>
It is also possible for the parameters themselves to have a probability distribution (e.g., [[Bayesian statistics]]). It is then necessary to define the [[Bayesian probability]]
<math display="block">\pi( \boldsymbol{\theta}).\,</math>
After the model is formed, the goal is to estimate the parameters, with the estimates commonly denoted <math>\hat{\boldsymbol{\theta}}</math>, where the "hat" indicates the estimate.

One common estimator is the [[minimum mean squared error]] (MMSE) estimator, which utilizes the error between the estimated parameters and the actual value of the parameters
<math display="block">\mathbf{e} = \hat{\boldsymbol{\theta}} - \boldsymbol{\theta}</math>
as the basis for optimality.  This error term is then squared and the [[expected value]] of this squared value is minimized for the MMSE estimator.