Editing Estimator (section)

===Bias===
The [[bias of an estimator|bias]] of <math>\widehat{\theta}</math> is defined as <math>B(\widehat{\theta}) = \operatorname{E}(\widehat{\theta}) - \theta</math>. It is the distance between the average of the collection of estimates, and the single parameter being estimated. The bias of <math>\widehat{\theta}</math> is a function of the true value of <math>\theta</math> so saying that the bias of <math>\widehat{\theta}</math> is <math>b</math> means that for every <math>\theta</math> the bias of <math>\widehat{\theta}</math> is <math>b</math>.

There are two kinds of estimators: biased estimators and unbiased estimators. Whether an estimator is biased or not can be identified by the relationship between <math>\operatorname{E}(\widehat{\theta}) - \theta</math> and 0:
* If <math>\operatorname{E}(\widehat{\theta}) - \theta\neq0</math>, <math>\widehat{\theta}</math> is biased.
* If <math>\operatorname{E}(\widehat{\theta}) - \theta=0</math>, <math>\widehat{\theta}</math> is unbiased.

The bias is also the expected value of the error, since <math> \operatorname{E}(\widehat{\theta}) - \theta = \operatorname{E}(\widehat{\theta} - \theta ) </math>. If the parameter is the bull's eye of a target and the arrows are estimates, then a relatively high absolute value for the bias means the average position of the arrows is off-target, and a relatively low absolute bias means the average position of the arrows is on target. They may be dispersed, or may be clustered. The relationship between bias and variance is analogous to the relationship between [[accuracy and precision]].

The estimator <math>\widehat{\theta}</math> is an [[estimator bias|unbiased estimator]] of <math>\theta</math> [[if and only if]] <math>B(\widehat{\theta}) = 0</math>. Bias is a property of the estimator, not of the estimate. Often, people refer to a "biased estimate" or an "unbiased estimate", but they really are talking about an "estimate from a biased estimator", or an "estimate from an unbiased estimator". Also, people often confuse the "error" of a single estimate with the "bias" of an estimator. That the error for one estimate is large, does not mean the estimator is biased. In fact, even if all estimates have astronomical absolute values for their errors, if the expected value of the error is zero, the estimator is unbiased. Also, an estimator's being biased does not preclude the error of an estimate from being zero in a particular instance. The ideal situation is to have an unbiased estimator with low variance, and also try to limit the number of samples where the error is extreme (that is, to have few [[Outlier|outliers]]). Yet unbiasedness is not essential. Often, if just a little bias is permitted, then an estimator can be found with lower mean squared error and/or fewer outlier sample estimates.

An alternative to the version of "unbiased" above, is "median-unbiased", where the [[median]] of the distribution of estimates agrees with the true value; thus, in the long run half the estimates will be too low and half too high. While this applies immediately only to scalar-valued estimators, it can be extended to any measure of [[central tendency]] of a distribution: see [[Bias of an estimator#Median-unbiased estimators, and bias with respect to other loss functions|median-unbiased estimators]].

In a practical problem, <math>\widehat{\theta}</math> can always have functional relationship with <math>\theta</math>. For example, if a genetic theory states there is a type of leaf (starchy green) that occurs with probability <math>p_1=1/4\cdot(\theta + 2)</math>, with <math>0<\theta<1</math>.
Then, for <math>n</math> leaves, the random variable <math>N_1</math>, or the number of starchy green leaves, can be modeled with a <math>Bin(n,p_1)</math> distribution. The number can be used to express the following estimator for <math>\theta</math>: <math>\widehat{\theta}=4/n\cdot N_1-2</math>. One can show that <math>\widehat{\theta}</math> is an unbiased estimator for <math>\theta</math>:
<math>E[\widehat{\theta}]=E[4/n\cdot N_1-2]</math>
<math>=4/n\cdot E[N_1]-2</math>
<math>=4/n\cdot np_1-2</math>
<math>=4\cdot p_1-2</math>
<math>=4\cdot1/4\cdot(\theta+2)-2</math>
<math>=\theta+2-2</math>
<math>=\theta</math>.