Recap of Key Concepts

The distinction between an estimand, an estimate, and an estimator is fundamental to statistics, so we give a short recap here.

Estimand

An estimand is the fixed quantity we want to estimate.

For example, if $Y$ is a population random variable, possible estimands include its mean

$$\mu=\mathbb{E}(Y)$$

or its variance

$$\sigma^2=\operatorname{Var}(Y).$$

Usually these quantities are unknown. In particular, for real-world random variables, we generally do not know the full pmf or pdf needed to calculate them directly using probability theory.

Instead, we try to estimate them from data.

Estimate

An estimate is a numerical value calculated from a particular observed sample

$$y_1,y_2,\dots,y_n.$$

We can think of it as our informed numerical guess of the estimand.

For instance, the observed sample mean is

$$\bar y=\frac{y_1+y_2+\dots+y_n}{n},$$

while the observed sample variance is

$$s_n^2=\frac{(y_1-\bar y)^2+(y_2-\bar y)^2+\dots+(y_n-\bar y)^2}{n-1}.$$

Once the values $y_1,y_2,\dots,y_n$ have been observed, both of these can be evaluated as actual numbers.

If we collected a different sample, we would usually obtain different estimates.

Estimator

An estimator is the corresponding quantity considered before collecting the data, when our observations are still random variables.

It is therefore a function of the random sample

$$Y_1,Y_2,\dots,Y_n.$$

For instance, the sample mean estimator is

$$\bar Y=\frac{Y_1+Y_2+\dots+Y_n}{n},$$

and the sample variance estimator is

$$S_n^2=\frac{(Y_1-\bar Y)^2+(Y_2-\bar Y)^2+\dots+(Y_n-\bar Y)^2}{n-1}.$$

Both $\bar Y$ and $S_n^2$ are random variables in their own right. We can therefore study their probability distributions, expected values, and variances.

After we collect our data, the estimator takes a particular realised value: this is precisely an estimate.