How Do We Find Conditional Expectations?

Sometimes we can find a conditional expectation intuitively, just by looking at the situation.

In other cases, we need to calculate it explicitly.

The basic method for finding the conditional expectation of a random variable is essentially the same as for an ordinary expected value:

1. Find the possible values the random variable can take, given the information we are conditioning on.

2. Multiply each possible value by its conditional probability.

3. Add the resulting terms.

For a discrete random variable $Y$,

$$\mathbb{E}(Y\mid X=x)=\sum_y y \ \mathbb{P}(Y=y\mid X=x),$$

provided $\mathbb{P}(X=x)>0$.

Let’s work through a simple example from the previous page, to see how the explicit route works in practice.

Example: Balls and Coin Flips

Suppose we have three balls labelled $1$, $2$ and $3$ in a bag, and a fair coin.

We pick one ball at random and let $X$ be the number written on it.

We then flip the coin $X$ times, and let $Y$ be the total number of heads obtained. The coin tosses are independent of the ball draw.

Suppose we want to calculate

$$\mathbb{E}(Y\mid X=2).$$

Knowing that $X=2$ tells us that the coin was flipped exactly twice.

Step 1: Find the Possible Values

Given that $X=2$, there are four possible coin-toss outcomes:

$$HH,\ HT,\ TH,\ TT.$$

One gives no heads, two give one head, and one gives two heads.

The possible values of $Y$ are therefore

$$0,1,2.$$

Step 2: Multiply by the Conditional Probabilities

We now need the probabilities of these possible values of $Y$ given that $X=2$.

The four outcomes $HH,\ HT,\ TH,\ TT$ are equally likely, so we can already see that

$$\mathbb{P}(Y=0\mid X=2)=\frac{1}{4},$$

$$\mathbb{P}(Y=1\mid X=2)=\frac{2}{4},$$

and

$$\mathbb{P}(Y=2\mid X=2)=\frac{1}{4}.$$

For the arithmetic, it is often convenient to keep the probabilities over the same denominator rather than simplifying them immediately.

We can also find these probabilities explicitly using the conditional probability formula.

For example,

$$\mathbb{P}(Y=0\mid X=2)=\frac{\mathbb{P}(Y=0\cap X=2)}{\mathbb{P}(X=2)}.$$

Since the three balls are equally likely,

$$\mathbb{P}(X=2)=\frac{1}{3}.$$

For both $X=2$ and $Y=0$ to occur, we must first choose the ball labelled $2$, and then obtain two tails.

The probability of choosing the ball labelled $2$ is $\frac{1}{3}$, while the probability of two tails is $\frac{1}{4}$. Since the coin tosses are independent of the ball draw,

$$\mathbb{P}(Y=0\cap X=2)=\frac{1}{3}\times\frac{1}{4}=\frac{1}{12}.$$

Hence,

$$\mathbb{P}(Y=0\mid X=2)=\frac{1/12}{1/3}=\frac{1}{4}.$$

In this example, it was easier to see the conditional probabilities directly from the four possible coin-toss outcomes. This is often the case in probability: if you understand the story of what is going on, the calculations become easier! In more complicated problems, the conditional probability formula can be useful to fall back on.

We now multiply each possible value of $Y$ by its conditional probability:

$$0\times\frac{1}{4},\qquad 1\times\frac{2}{4},\qquad 2\times\frac{1}{4}.$$

Step 3: Add the Terms

Finally, we add the weighted values:

$$\mathbb{E}(Y\mid X=2)=0\times\frac{1}{4}+1\times\frac{2}{4}+2\times\frac{1}{4}.$$

So,

$$\mathbb{E}(Y\mid X=2)=1.$$

Given that we drew the ball labelled $2$, the expected number of heads is $1$.

The General Method

For a discrete conditional expectation:

1. Find the possible values the random variable can take, given the information we are conditioning on.

2. Multiply each value by its conditional probability.

3. Add the resulting terms.

In symbols,

$$\mathbb{E}(Y\mid X=x)=\sum_y y \ \mathbb{P}(Y=y\mid X=x).$$

In summary: a conditional expectation is calculated in much the same way as an ordinary expected value, but using the probabilities that apply after we have conditioned on the new information.

Background:

• Conditional Probability

• Expected Values

• Calculating Expected Values

• Joint PMFs

• Conditional Expectations