The law of total variance decomposes the variance of a random variable $Y$ into two parts:

$$\operatorname{Var}(Y)=\mathbb{E}(\operatorname{Var}(Y\mid X))+\operatorname{Var}(\mathbb{E}(Y\mid X))$$

Intuitively,

total variance = average within-group variance + between-group variance.

The first term measures how much $Y$ varies within groups defined by $X$, on average. The second measures how much the conditional mean of $Y$ varies between those groups.

To fully understand the formula, we first need to master the idea of conditional variance.

What Is Conditional Variance?

The conditional variance of $Y$ given $X=x$ measures how much $Y$ varies once we know that $X=x$.

Recall that

$$\operatorname{Var}(Y)=\mathbb{E}(Y^2)-(\mathbb{E}(Y))^2.$$

Conditional variance uses the same formula, but with conditional expectations:

$$\operatorname{Var}(Y\mid X=x)=\mathbb{E}(Y^2\mid X=x)-(\mathbb{E}(Y\mid X=x))^2.$$

So $\operatorname{Var}(Y\mid X=x)$ is a number once $x$ is fixed.

For example, suppose $Y$ is a fair die roll. Let

• $X=0$ if the roll is low, so $Y\in{1,2,3}$;

• $X=1$ if the roll is high, so $Y\in{4,5,6}$.

Given $X=0$,

$$\mathbb{E}(Y\mid X=0)=1\times\frac{1}{3}+2\times\frac{1}{3}+3\times\frac{1}{3}=2.$$

Also,

$$\mathbb{E}(Y^2\mid X=0)=1^2\times\frac{1}{3}+2^2\times\frac{1}{3}+3^2\times\frac{1}{3}=\frac{14}{3}.$$

Hence,

$$\operatorname{Var}(Y\mid X=0)=\frac{14}{3}-2^2=\frac{2}{3}.$$

The same variance applies when $X=1$, since the values $4,5,6$ have exactly the same spread as $1,2,3$.

Conditional variances are very important in econometrics, particularly the conditional variance of an error term given another variable:

$$\operatorname{Var}(U\mid X=x).$$

Conditional Variance as a Random Variable

In general, $\operatorname{Var}(Y\mid X=x)$ may depend on the value of $x$.

For example, we might have

$$\operatorname{Var}(Y\mid X=x)=2+x^2.$$

Once $x$ is fixed, this is a number.

Replacing the fixed value $x$ with the random variable $X$ gives

$$\operatorname{Var}(Y\mid X)=2+X^2.$$

This is itself a random variable: its value depends on the random value of $X$.

We can therefore take its expectation:

$$\mathbb{E}(\operatorname{Var}(Y\mid X)).$$

This measures the average conditional variance of $Y$.

Conditional Expectation as a Random Variable

Recall that the conditional expectation $\mathbb{E}(Y\mid X)$ is also a random variable.

If, for a fixed value $x$,

$$\mathbb{E}(Y\mid X=x)=g(x),$$

then replacing $x$ with its random counterpart $X$ gives

$$\mathbb{E}(Y\mid X)=g(X).$$

Since $X$ is random, $\mathbb{E}(Y\mid X)$ is also random: its value changes according to the value taken by $X$.

Provided its variance exists, we can therefore calculate

$$\operatorname{Var}(\mathbb{E}(Y\mid X)).$$

This measures how much the conditional mean of $Y$ varies as $X$ varies.

What Is the Law of Total Variance?

The law of total variance says

$$\operatorname{Var}(Y)=\mathbb{E}(\operatorname{Var}(Y\mid X))+\operatorname{Var}(\mathbb{E}(Y\mid X))$$

provided the relevant second moments exist.

The two terms have different meanings. These are easiest to explain if $X$ represents membership in a certain group (including low vs. high rolls of a die).

In such cases, the first term,

$$\mathbb{E}(\operatorname{Var}(Y\mid X)),$$

is the average amount of variation in $Y$ within the groups defined by $X$.

The second term,

$$\operatorname{Var}(\mathbb{E}(Y\mid X)),$$

measures how much the conditional mean of $Y$ varies as $X$ changes, i.e. between the groups defined by $X$.

So the total variance formula separates the variation in $Y$ into:

• variation within groups;

• variation between the conditional means of those groups.

Example: A Fair Die

In the fair-die example above,

$$\operatorname{Var}(Y\mid X)=\frac{2}{3}.$$

Since this is constant,

$$\mathbb{E}(\operatorname{Var}(Y\mid X))=\frac{2}{3}.$$

We also have

$$\mathbb{E}(Y\mid X)=2+3X.$$

Since $X$ takes the values $0$ and $1$ with equal probability,

$$\operatorname{Var}(X)=\frac{1}{4}.$$

Therefore,

$$\operatorname{Var}(\mathbb{E}(Y\mid X))=\operatorname{Var}(2+3X)=9\operatorname{Var}(X)=\frac{9}{4}.$$

Applying the law of total variance,

$$\operatorname{Var}(Y)=\frac{2}{3}+\frac{9}{4}=\frac{35}{12}.$$

This is exactly the variance of a fair six-sided die.

The overall variation therefore consists of two parts: the variation among outcomes within the low or high group, and the variation between the mean of the low group and the mean of the high group.

Within-Group and Between-Group Variation

The same interpretation is useful in real-world settings.

Suppose $Y$ represents an employee’s wages and $X$ represents the industry they work in.

Then

$$\mathbb{E}(\operatorname{Var}(Y\mid X))$$

measures the average variation in wages within industries, while

$$\operatorname{Var}(\mathbb{E}(Y\mid X))$$

measures the variation in average wages between industries.

The law of total variance tells us that both sources contribute to the overall variation in wages:

$$\text{total variance}=\text{average within-group variance}+\text{between-group variance}$$

Background:

• Variance and Standard Deviation

• Conditional Expectations