What Is a Conditional Expectation?

A conditional expectation is what the expected value of one random variable becomes after we have learned some information about another.

Suppose $X$ and $Y$ are related random variables. Before observing either of them, we may have a prediction for $Y$ – its expected value. Once we learn the value of $X$, this new information may change what we expect $Y$ to be.

We write

$$\mathbb{E}(Y\mid X=x)$$

for the expected value of $Y$ given that $X=x$.

The vertical line $\mid$ is read as “given”.

For example,

$$\mathbb{E}(Y\mid X=1)=5$$

means that the expected value of $Y$, given that $X=1$, is $5$.

A Simple Example

Suppose we roll a fair six-sided die, and let $Y$ be the outcome.

Before rolling the die, imagine that we visit a witch. The witch has magical powers, and can see the future! However, she can only tell us whether the outcome will be low or high, not the exact number.

Let $X$ represent the witch’s prediction:

• $X=0$ if the outcome will be low, meaning $Y\in{1,2,3}$;

• $X=1$ if the outcome will be high, meaning $Y\in{4,5,6}$.

If the witch tells us that $X=0$, then we know that $Y$ must be $1$, $2$ or $3$.

These three values are equally likely, so we may take an average:

$$\mathbb{E}(Y\mid X=0)=\frac{1+2+3}{3}=2.$$

If instead we learn that $X=1$, the possible values of $Y$ are $4$, $5$ and $6$. In this case,

$$\mathbb{E}(Y\mid X=1)=\frac{4+5+6}{3}=5.$$

Learning the value of $X$ has changed our prediction for $Y$.

Conditional Expectation Given $X=x$

We can summarise the two cases above with one formula.

For a possible value $x$ of $X$,

$$\mathbb{E}(Y\mid X=x)=2+3x,$$

where $x=0$ or $x=1$.

If $x=0$, this gives

$$\mathbb{E}(Y\mid X=0)=2,$$

while if $x=1$, it gives

$$\mathbb{E}(Y\mid X=1)=5.$$

Note that once we fix a particular value $x$, the quantity $\mathbb{E}(Y\mid X=x)$ is a number – either $2$ or $5$, depending on $x$.

Conditional Expectation Given $X$

Now imagine the moment before we visit the witch.

We know what will happen once she gives us her prediction:

• if she says the outcome will be low, our prediction for $Y$ will be $2$;

• if she says the outcome will be high, our prediction for $Y$ will be $5$.

But we do not yet know what the witch will say!

So, before learning the value of her prediction $X$, the prediction we will make for $Y$ once we know $X$ is itself uncertain. It depends on which value the random variable $X$ will take.

From above, we know that this prediction can be written as $2+3X$.

We may call the expression $2+3X$ a random predictor. It has its own special notation:

$$\mathbb{E}(Y\mid X)=2+3X.$$

In particular, $\mathbb{E}(Y\mid X)$ is not yet a single number. It is itself a random variable, because $X$ is still random.

More generally, if

$$\mathbb{E}(Y\mid X=x)=g(x),$$

for the possible values of $x$, then

$$\mathbb{E}(Y\mid X)=g(X).$$

We can think of this as “bringing $x$ back to life”: replacing the fixed value $x$ with the random variable $X$. The result is generally a random variable rather than a number.

This is a tricky idea, but will be very important later!

A Second Example

Now consider a different experiment.

Suppose we have three balls labelled $1$, $2$ and $3$ in a bag, and a fair coin.

We pick a ball from the bag at random, look at the number, and flip the coin as many times as the number we drew.

We use this experiment to define two new random variables, $U$ and $V$.

Firstly, let $U$ be the number written on the ball we pick. We then flip the fair coin $U$ times.

Next, let $V$ be the number of heads obtained from these $U$ coin flips.

If $U=1$, we flip the coin once. The expected number of heads is

$$\mathbb{E}(V\mid U=1)=0.5.$$

If $U=2$, we flip the coin twice, giving

$$\mathbb{E}(V\mid U=2)=1.$$

If $U=3$, we flip the coin three times, giving

$$\mathbb{E}(V\mid U=3)=1.5.$$

These three results can be summarised as

$$\mathbb{E}(V\mid U=u)=0.5u,$$

for $u=1,2,3$.

Once a particular value $u$ is known, this conditional expectation is just a number: either $0.5$, $1$, or $1.5$, depending on $u$.

Before the ball has been drawn, $U$ is still random. As above, we write

$$\mathbb{E}(V\mid U)=0.5U.$$

This is itself a random variable: a function of the random variable $U$.

Why Conditional Expectations Matter

Conditional expectation lets us formalise the idea of using information about one random variable to update our prediction of another.

Before observing $X$, we do not yet know the value of $\mathbb{E}(Y\mid X)$, because it depends on the random value that $X$ will take.

This idea appears throughout probability and statistics whenever we want to describe what we expect to happen after learning some additional information.

It is also important in econometrics, where conditional expectations have a central role in the most important theorem of linear regression.

Background:

• Expected Values

• Joint PMFs