What Is the Method of Moments?

The method of moments is a systematic procedure for constructing estimators of unknown population quantities. It works by first writing the quantity of interest in terms of the moments of our population random variable, and then replacing these moments with their standard estimators.

Recall that if $Y$ is a random variable, its moments are the expected values of its powers:

$$\mathbb{E}(Y),\ \mathbb{E}(Y^2),\ \mathbb{E}(Y^3),\dots$$

Let’s write the $r^{th}$ moment $\mathbb{E}(Y^r)$ as

$$M_r=\mathbb{E}(Y^r).$$

For instance, the third moment is

$$M_3=\mathbb{E}(Y^3).$$

Moments are useful because many other quantities describing a distribution can be written in terms of them. As an example, the variance of a random variable may be written:

$$\operatorname{Var}(Y) = \mathbb{E} \left( (Y-\mathbb{E}(Y))^2 \right) = \mathbb{E} ( Y^2 ) - \left( \mathbb{E} ( Y) \right)^2 = M_2 - M_1^2. $$

Estimating Population Moments

Suppose we have some data:

$$y_1,y_2,\dots,y_n.$$

Since a moment $M_r$ is the expected value of $Y^r$, it is natural to estimate it using the sample average of $y^r$:

$$m_r=\frac{y_1^r + y_2^r + \dots + y_n^r}{n}.$$

Now, before collecting our data, consider the random sample

$$Y_1,Y_2,\dots,Y_n.$$

The same procedure gives us the following formula:

$$\widehat M_r=\frac{1}{n}\sum_{i=1}^nY_i^r.$$

Here the hat over $\widehat M_r$ indicates that this is an estimator of the unknown population moment $M_r$.

So, for example, if we want to estimate the third population moment

$$M_3=\mathbb{E}(Y^3),$$

we use the standard estimator

$$\widehat M_3=\frac{1}{n}\sum_{i=1}^nY_i^3,$$

From Moments to Other Quantities

We now have a standard way to estimate population moments. Next we combine this with the important observation that many other quantities of interest can themselves be written in terms of moments, as we saw with the variance above.

This gives us the method of moments:

1. Write the quantity of interest in terms of population moments such as $M_1,M_2,\dots$.

2. Replace these population moments with their standard estimators $\widehat M_1,\widehat M_2,\dots$.

The resulting expression is a method of moments estimator of the quantity we are interested in.

Example: Estimating Skewness

As a more interesting example, consider the skewness of a random variable:

$$\gamma_1(Y)=\mathbb{E}\left(\left(\frac{Y-\mu}{\sigma}\right)^3\right).$$

This measures the asymmetry, or “skew”, of a distribution.

We can write skewness entirely in terms of population moments:

$$\gamma_1(Y)=\frac{M_3-3M_1M_2+2M_1^3}{(M_2-M_1^2)^{3/2}}.$$

The method of moments now tells us what to do: replace each unknown population moment $M_r$ with its estimator $\widehat M_r$.

This gives the method of moments estimator of the population skewness:

$$\widehat\gamma_1=\frac{\widehat M_3-3\widehat M_1\widehat M_2+2\widehat M_1^3}{(\widehat M_2-\widehat M_1^2)^{3/2}}.$$

Again, the hat on $\widehat\gamma_1$ indicates that this is an estimator of the unknown population quantity $\gamma_1(Y)$.

Using

$$\widehat M_r=\frac{1}{n}\sum_{i=1}^nY_i^r,$$

we could write the numerator explicitly as

$$\left(\frac{1}{n}\sum_{i=1}^nY_i^3\right)-3\left(\frac{1}{n}\sum_{i=1}^nY_i\right)\left(\frac{1}{n}\sum_{i=1}^nY_i^2\right)+2\left(\frac{1}{n}\sum_{i=1}^nY_i\right)^3,$$

and the denominator as

$$\left(\left(\frac{1}{n}\sum_{i=1}^nY_i^2\right)-\left(\frac{1}{n}\sum_{i=1}^nY_i\right)^2\right)^{3/2}.$$

The formula looks rather formidable when written out in full, but the idea behind it is quite straightforward: write the quantity in terms of population moments, then replace those moments with their sample counterparts.

Background

• Moments Definition