What Is an Unbiased Estimator?

An estimator $\hat{\theta}$ of an unknown quantity $\theta$ is unbiased if its expectation is equal to $\theta$:

$$\mathbb{E}(\hat{\theta})=\theta.$$

This is generally a good thing, since it says that the estimator is “correct on average”.

Roughly, one way to think about this is to imagine repeatedly collecting new samples in the same way and calculating a new estimate each time. The individual estimates would vary from sample to sample – but if we repeated the process many times, their average would likely be close to the true value $\theta$.

Example: The Sample Mean

For example, consider a random sample

$$Y_1,Y_2,\dots,Y_n,$$

where each $Y_i$ has the same mean

$$\mu=\mathbb{E}(Y_i).$$

The sample mean is

$$\bar Y=\frac{Y_1+Y_2+\dots+Y_n}{n}.$$

Taking its expected value gives

$$\mathbb{E}(\bar Y)=\frac{\mathbb{E}(Y_1)+\mathbb{E}(Y_2)+\dots+\mathbb{E}(Y_n)}{n}=\frac{\mu+\mu+\dots+\mu}{n}=\mu.$$

So the sample mean $\bar Y$ is an unbiased estimator of the population mean $\mu$.

This does not mean that any particular estimate $\bar y$ is sure to equal $\mu$ exactly. This would be too much to hope for. Rather, $\bar Y$ is “correct on average” – in the precise sense that its expected value is $\mu$.

Bias

We can also define the bias of an estimator as the difference between its expected value and its target:

$$\operatorname{Bias}_{\theta}(\hat{\theta})=\mathbb{E}(\hat{\theta})-\theta.$$

An estimator is therefore unbiased when its bias is $0$.

Notice that bias always depends on the target $\theta$. The same estimator can be unbiased for one quantity and biased for another.

Background

• Expected Values

• Estimators as Random Variables