What Is the Law of Iterated Expectation?

The law of iterated expectation says that we can find the expected value of a random variable $Y$ by first finding its conditional expectation given another random variable $X$, and then taking an expectation again:

$$\mathbb{E}(Y)=\mathbb{E}(\mathbb{E}(Y\mid X)).$$

It is also known as the tower property.

The inner expectation,

$$\mathbb{E}(Y\mid X),$$

is itself a random variable: it represents our prediction for $Y$ based on the information contained in $X$, but while $X$ is still random. In fact, it is a function of $X$.

The outer expectation then averages this “random predictor” $\mathbb{E}(Y\mid X)$ over the possible values of $X$.

So the law says that the expectation of this conditional “random predictor” is the same as the overall expected value of $Y$.

Let’s use our balls-and-coins example to see this in action.

Example: Balls and Coin Flips

Suppose we have three balls labelled $1$, $2$ and $3$ in a bag, and a fair coin.

We pick one ball at random and let $X$ be the number written on it.

We then flip the coin $X$ times, and let $Y$ be the total number of heads obtained.

Since each fair coin toss contributes $0.5$ heads on average, we have:

$$\mathbb{E}(Y\mid X=1)=0.5,$$

$$\mathbb{E}(Y\mid X=2)=1,$$

and

$$\mathbb{E}(Y\mid X=3)=1.5.$$

We can summarise these three cases as

$$\mathbb{E}(Y\mid X=x)=0.5x,\qquad x=1,2,3.$$

Once $x$ is fixed, this is just a number.

Before we know which ball will be drawn, however, $X$ is still random. Replacing the fixed value $x$ with the random variable $X$ gives

$$\mathbb{E}(Y\mid X)=0.5X.$$

This is our random predictor for $Y$.

Taking an Expectation Again

The law of iterated expectation tells us to take the expected value of this random predictor:

$$\mathbb{E}(Y)=\mathbb{E}(\mathbb{E}(Y\mid X)).$$

In our example,

$$\mathbb{E}(Y)=\mathbb{E}(0.5X).$$

Since the three balls are equally likely,

$$\mathbb{E}(X)=1\times\frac{1}{3}+2\times\frac{1}{3}+3\times\frac{1}{3}=2.$$

Therefore,

$$\mathbb{E}(Y)=\mathbb{E}(0.5X)=0.5\mathbb{E}(X)=0.5\times2=1.$$

So the expected number of heads is $1$.

This is the law of iterated expectations in action: taking the expectation of $\mathbb{E}(Y\mid X)$ gives us $\mathbb{E}(Y)$.

Background:

• Expected Values

• Conditional Expectations