How Does the Sum of Two Random Variables Behave?

Suppose we have two independent random variables $X$ and $Y$. We form the new random variable $X+Y$, and would like to know its probability distribution.

We can find the answer using a convolution!

We begin with the discrete case, where the idea is easiest to see.

The Discrete Case

Suppose $X$ and $Y$ are independent random variables which take non-negative integer values.

For $X$, we have the sequence of probabilities

$$p_0=\mathbb{P}(X=0), \quad p_1=\mathbb{P}(X=1), \quad p_2=\mathbb{P}(X=2), \dots$$

For $Y$, we have

$$q_0=\mathbb{P}(Y=0), \quad q_1=\mathbb{P}(Y=1), \quad q_2=\mathbb{P}(Y=2), \dots$$

More generally,

$$p_k=\mathbb{P}(X=k)$$

and

$$q_k=\mathbb{P}(Y=k).$$

Now suppose we want to find

$$\mathbb{P}(X+Y=3).$$

There are four possible ways for this to happen:

• $X=0$ and $Y=3$
• $X=1$ and $Y=2$
• $X=2$ and $Y=1$
• $X=3$ and $Y=0$

Since $X$ and $Y$ are independent, we multiply the probabilities within each case. We then add the four possibilities together:

$$\mathbb{P}(X+Y=3)=p_0q_3+p_1q_2+p_2q_1+p_3q_0.$$

More generally, to get $X+Y=n$, the possible combinations are

$$X=0,\ Y=n$$

$$X=1,\ Y=n-1$$

$$X=2,\ Y=n-2$$

$$\dots$$

$$X=n,\ Y=0.$$

So,

$$\mathbb{P}(X+Y=n)=p_0q_n+p_1q_{n-1}+p_2q_{n-2}+\dots+p_nq_0.$$

We can write the same thing more compactly as

$$\mathbb{P}(X+Y=n)=\sum_{k=0}^n p_kq_{n-k}.$$

The probabilities for $X+Y$ form a new sequence. This sequence is called the convolution of the original probability sequences for $X$ and $Y$.

The key idea is simple: we consider every combination of values that adds up to the required total.

The Continuous Case

Now suppose instead that $X$ and $Y$ are independent continuous random variables, with pdfs $f_X$ and $f_Y$.

We again want to find the distribution of their sum $X+Y$.

Suppose

$$X+Y=z.$$

For example, some possible combinations are

$$X=0,\ Y=z$$

$$X=1,\ Y=z-1$$

$$X=2,\ Y=z-2$$

and so on.

More generally, if $X=t$, then

$$Y=z-t.$$

Since $X$ and $Y$ are independent, their joint pdf is the product of their individual pdfs. For a particular value of $t$, this gives

$$f_X(t)f_Y(z-t).$$

In the discrete case, we added over all possible values. In the continuous case, we instead integrate over all possible values of $t$.

This gives the pdf of the sum:

$$f_{X+Y}(z)=\int_{-\infty}^{\infty} f_X(t)f_Y(z-t) \ dt.$$

This new function is called the convolution of $f_X$ and $f_Y$.

It is often written

$$f_{X+Y}=f_X*f_Y.$$

So the discrete and continuous cases follow the same basic pattern:

• in the discrete case, we sum over all combinations which give the required total;
• in the continuous case, we integrate over all combinations which give the required total.

When the Random Variables Are Non-Negative

Suppose $X$ and $Y$ are still independent and continuous, but can only take non-negative values.

For $z>0$, we have

$$X=t$$

and

$$Y=z-t.$$

For both of these values to be non-negative, $t$ must lie between $0$ and $z$.

So in this case,

$$f_{X+Y}(z)=\int_0^z f_X(t)f_Y(z-t) \ dt.$$

This is the same convolution formula, but the limits now only cover values of $t$ which are actually possible.

Background:

• Probability Basics

• Independence