Editing Mean squared error (section)

====Proof of variance and bias relationship====

<math>\begin{align}
\operatorname{MSE}(\hat{\theta})
&= \operatorname{E}_\theta \left [(\hat{\theta}-\theta)^2 \right ] \\
&= \operatorname{E}_\theta\left[\left(\hat{\theta}-\operatorname{E}_\theta [\hat\theta]+\operatorname{E}_\theta[\hat\theta]-\theta\right)^2\right]\\ 
&= \operatorname{E}_\theta\left[\left(\hat{\theta}-\operatorname{E}_\theta[\hat\theta]\right)^2 +2\left (\hat{\theta}-\operatorname{E}_\theta[\hat\theta] \right ) \left (\operatorname{E}_\theta[\hat\theta]-\theta \right )+\left( \operatorname{E}_\theta[\hat\theta]-\theta \right)^2\right] \\ 
&= \operatorname{E}_\theta\left[\left(\hat{\theta}-\operatorname{E}_\theta[\hat\theta]\right)^2\right]+\operatorname{E}_\theta\left[2 \left (\hat{\theta}-\operatorname{E}_\theta[\hat\theta] \right ) \left (\operatorname{E}_\theta[\hat\theta]-\theta \right ) \right] + \operatorname{E}_\theta\left [ \left(\operatorname{E}_\theta[\hat\theta]-\theta\right)^2 \right] \\
&= \operatorname{E}_\theta\left[\left(\hat{\theta}-\operatorname{E}_\theta[\hat\theta]\right)^2\right]+ 2 \left(\operatorname{E}_\theta[\hat\theta]-\theta\right) \operatorname{E}_\theta\left[\hat{\theta}-\operatorname{E}_\theta[\hat\theta] \right] +  \left(\operatorname{E}_\theta[\hat\theta]-\theta\right)^2 && \operatorname{E}_\theta[\hat\theta]-\theta = \text{constant} \\
&= \operatorname{E}_\theta\left[\left(\hat{\theta}-\operatorname{E}_\theta[\hat\theta]\right)^2\right]+ 2 \left(\operatorname{E}_\theta [\hat\theta]-\theta\right) \left ( \operatorname{E}_\theta[\hat{\theta}]-\operatorname{E}_\theta[\hat\theta] \right )+  \left(\operatorname{E}_\theta[\hat\theta]-\theta\right)^2 && \operatorname{E}_\theta[\hat\theta] = \text{constant} \\
&= \operatorname{E}_\theta\left[\left(\hat\theta-\operatorname{E}_\theta[\hat\theta]\right)^2\right]+\left(\operatorname{E}_\theta [\hat\theta]-\theta\right)^2\\ 
&= \operatorname{Var}_\theta(\hat\theta)+ \operatorname{Bias}_\theta(\hat\theta,\theta)^2
\end{align}</math>

An even shorter proof can be achieved using the well-known formula that for a random variable <math display="inline">X</math>, <math display="inline">\mathbb{E}(X^2) = \operatorname{Var}(X) + (\mathbb{E}(X))^2</math>. By substituting <math display="inline">X</math> with, <math display="inline">\hat\theta-\theta</math>, we have
:<math display="block">\begin{aligned}
\operatorname{MSE}(\hat{\theta}) &= \mathbb{E}[(\hat\theta-\theta)^2] \\
&= \operatorname{Var}(\hat{\theta} - \theta) + (\mathbb{E}[\hat\theta - \theta])^2 \\
&= \operatorname{Var}(\hat\theta) + \operatorname{Bias}^2(\hat\theta,\theta)
\end{aligned}</math>
But in real modeling case, MSE could be described as the addition of model variance, model bias, and irreducible uncertainty (see [[Bias–variance tradeoff]]). According to the relationship, the MSE of the estimators could be simply used for the [[Efficiency (statistics)|efficiency]] comparison, which includes the information of estimator variance and bias. This is called MSE criterion.