How Do We Apply the Law of Iterated Expectation?

The law of iterated expectation is often useful when we want to find the expected value of a random variable $Y$, and $Y$ depends on another random variable $X$.

The problem often becomes much easier if we first treat $X$ as taking some fixed value $x$.

We can apply this idea systematically in three steps.

Step 1: Fix $X=x$

Treat $X$ as taking some fixed numerical value $x$, and find

$$\mathbb{E}(Y\mid X=x).$$

This will usually be a function of $x$; say

$$\mathbb{E}(Y\mid X=x)=g(x).$$

Step 2: Replace $x$ with $X$

Replace the fixed value $x$ with its random counterpart $X$:

$$\mathbb{E}(Y\mid X)=g(X).$$

This is now a random variable: a function of $X$.

Step 3: Take an Expectation Again

Finally, use the law of iterated expectation:

$$\mathbb{E}(Y)=\mathbb{E}(\mathbb{E}(Y\mid X))=\mathbb{E}(g(X)).$$

In short: condition first, then take an expectation again.

Example: A Normal Distribution with a Random Mean

Suppose we first draw a number $X$ uniformly from the interval $[10,20]$:

$$X\sim\mathcal{U}[10,20].$$

Then, given $X=x$, we draw $Y$ from a normal distribution with mean $x$ and variance $1$:

$$Y\big|_{X=x}\sim\mathcal{N}(x,1).$$

The unconditional distribution of $Y$ is quite complicated: it is like a normal distribution, but with a “random parameter” (later we will call distributions like this compound distributions).

Fortunately, with our method, finding its expectation is much easier.

Step 1: Fix $X=x$

Given $X=x$,

$$Y\big|_{X=x}\sim\mathcal{N}(x,1).$$

The mean of this normal distribution is $x$, so

$$\mathbb{E}(Y\mid X=x)=x.$$

Step 2: Replace $x$ with $X$

Replacing the fixed value $x$ with its random counterpart $X$ gives

$$\mathbb{E}(Y\mid X)=X.$$

Step 3: Take an Expectation Again

Using the law of iterated expectation,

$$\mathbb{E}(Y)=\mathbb{E}(\mathbb{E}(Y\mid X))=\mathbb{E}(X).$$

For a uniform random variable on $[10,20]$,

$$\mathbb{E}(X)=\frac{10+20}{2}=15.$$

Therefore,

$$\mathbb{E}(Y)=15.$$

We have found the expected value of $Y$ without first having to work out its unconditional distribution.

Background:

• Expected Values

• Uniform Distribution

• Normal Distribution

• Conditional Expectations

• The Law of Iterated Expectation