When Are Two Random Variables Independent?

Two random variables are independent when learning the value of one gives us no information about the probabilities for the other.

Intuitively, if $X$ and $Y$ do not affect each other, they will be independent. For example, suppose $X$ is the number shown when we roll a fair die, while $Y$ records the result of an unrelated fair coin flip. If we discover that $Y=1$, meaning that the coin landed heads, this tells us nothing about which number appeared on the die. Each value of $X$ is still equally likely.

For discrete random variables, independence can be expressed using their pmfs. If $X$ and $Y$ are independent, then their joint pmf factorises into the product of their individual pmfs:

$$\mathbb{P}(X=x,Y=y)=\mathbb{P}(X=x)\mathbb{P}(Y=y)$$

for all values $x$ and $y$.

In other words, for two independent random variables, the joint distribution is obtained by multiplying their individual probabilities.

Continuous Random Variables

For continuous random variables, individual point probabilities are zero, so we instead express independence using their pdfs.

If $X$ and $Y$ have a joint pdf, independence means that the joint density factorises into the two marginal densities:

$$f_{X,Y}(x,y)=f_X(x)f_Y(y).$$

This is the continuous analogue of multiplying the individual probabilities in the discrete case.

A Definition That Covers Both Cases

We can also describe independence using cdfs, which gives us a definition that works for both discrete and continuous random variables.

Two random variables $X$ and $Y$ are independent if

$$\mathbb{P}(X\leq x,Y\leq y)=\mathbb{P}(X\leq x)\mathbb{P}(Y\leq y)$$

for all $x$ and $y$.

Equivalently,

$$F_{X,Y}(x,y)=F_X(x)F_Y(y).$$

Again, the idea is that knowing how one variable behaves gives us no extra information about how the other behaves.

More Than Two Random Variables

The same idea extends to any number of random variables. We say that $X_1,\dots,X_n$ are independent if their joint cdf factorises:

$$\mathbb{P}(X_1\leq x_1,\dots,X_n\leq x_n)=\mathbb{P}(X_1\leq x_1)\dots\mathbb{P}(X_n\leq x_n)$$

for all $x_1,\dots,x_n$.

This is stronger than simply requiring every pair of random variables to be independent. It is possible for every pair to be independent without all the random variables being jointly independent.

Background:

• Random Variables

• Joint PMFs

• Joint PDFs and Double integrals

• Cumulative Distribution Functions