Editing Consistent estimator (section)

== Bias versus consistency ==

=== Unbiased but not consistent ===
An estimator can be [[biased estimator|unbiased]] but not consistent. For example, for an [[iid]] sample {''x''{{su|b=1}},..., ''x{{su|b=n}}''} one can use ''T{{su|b=n}}''(''X'') = ''x''{{su|b=n}} as the estimator of the mean E[''X'']. Note that here the sampling distribution of ''T{{su|b=n}}'' is the same as the underlying distribution (for any ''n,'' as it ignores all points but the last). So E[''T{{su|b=n}}''(''X'')] = E[''X''] for any ''n,'' hence it is unbiased, but it does not converge to any value.

However, if a sequence of estimators is unbiased ''and'' converges to a value, then it is consistent, as it must converge to the correct value.

=== Biased but consistent ===
Alternatively, an estimator can be biased but consistent. For example, if the mean is estimated by <math>{1 \over n} \sum x_i + {1 \over n}</math> it is biased, but as <math>n \rightarrow \infty</math>, it approaches the correct value, and so it is consistent.

Important examples include the [[sample variance]] and [[sample standard deviation]]. Without [[Bessel's correction]] (that is, when using the sample size <math>n</math> instead of the [[Degrees of freedom (statistics)|degrees of freedom]] <math>n-1</math>), these are both negatively biased but consistent estimators. With the correction, the corrected sample variance is unbiased, while the corrected sample standard deviation is still biased, but less so, and both are still consistent: the correction factor converges to 1 as sample size grows.

Here is another example. Let <math>T_n</math> be a sequence of estimators for <math>\theta</math>.

:<math>\Pr(T_n) = \begin{cases}
  1 - 1/n, & \mbox{if }\, T_n = \theta \\
  1/n,  & \mbox{if }\, T_n = n\delta + \theta
\end{cases}</math>

We can see that <math>T_n \xrightarrow{p} \theta</math>, <math>\operatorname{E}[T_n] = \theta + \delta </math>, and the bias does not converge to zero.