What is the Variance of a Random Variable?

The variance of a random variable measures how spread out its values tend to be around its expected value. A small variance means they tend to stay fairly close to the mean; a large variance means they are more widely spread.

The variance gives us information that the expected value alone cannot provide. For example, suppose $X$ is equally likely to be 4 or 6, while $Y$ is equally likely to be 0 or 10. Both have the same expected value:

$$\mathbb{E}(X)=\mathbb{E}(Y)=5.$$

But $Y$ is clearly much more spread out around 5 than $X$. We would therefore like a way to measure this difference.

Write the mean of $X$ as

$$\mu=\mathbb{E}(X).$$

A natural first thought is to look at the difference between $X$ and its own mean, $X-\mu$. However,

$$\mathbb{E}(X-\mu)=\mathbb{E}(X)-\mu=\mu-\mu=0.$$

The positive and negative differences from the mean cancel out. So this does not tell us anything about the spread.

To avoid this problem, we square the differences. This gives the definition of the variance:

$$\operatorname{Var}(X)=\mathbb{E}\left((X-\mu)^2\right).$$

So variance is the expected squared distance from the mean. The further the values of $X$ tend to lie from $\mu$, the larger the variance will be.

For our simple examples above,

$$\operatorname{Var}(X)=1$$

whereas

$$\operatorname{Var}(Y)=25.$$

This captures the fact that $Y$ is much more spread out, even though $X$ and $Y$ have exactly the same expected value.

There is an equivalent formula which is often easier to calculate with:

$$\operatorname{Var}(X)=\mathbb{E}(X^2)-\left(\mathbb{E}(X)\right)^2.$$

For a discrete random variable,

$$\mathbb{E}(X^2)=\sum_x x^2 \ \mathbb{P}(X=x),$$

where the sum is taken over all the possible values of $X$. This is calculated in much the same way as an ordinary expected value, except that we square each possible value before weighting it by its probability.

Variance also behaves in an important way when we add or multiply by constants. Adding a constant shifts every possible value by the same amount, but does not change the spread:

$$\operatorname{Var}(X+b)=\operatorname{Var}(X).$$

Multiplying by a constant $a$ changes the variance by a factor of $a^2$:

$$\operatorname{Var}(aX)=a^2\operatorname{Var}(X).$$

Combining these gives

$$\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X).$$

Standard Deviation

Because variance is based on squared differences, it is measured in squared units. If $X$ is measured in kilograms, for example, its variance is measured in kilogams squared.

The standard deviation returns us to the original units by taking the square root of the variance:

$$\operatorname{SD}(X)=\sqrt{\operatorname{Var}(X)}.$$

Like the variance, a larger standard deviation means that the random variable tends to be more spread out around its mean.

For continuous random variables, the ideas of variance and standard deviation are exactly the same, although we calculate the required expected values a little differently.

Background:

• Random Variables

• Expected Values

Extensions

• Properties of Variance and Covariance