The Experiment

We now use a single example to compare our three notions of convergence, and see how they can give different answers about whether a sequence of random variables converges.

Suppose we flip a fair coin three times, so that our sample space is:

$$\Omega=\{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT\}.$$

Our target random variable $X$ records the number of heads on the second flip.

Now define a sequence of random variables:

• $X_1$ records the total number of heads in all three flips.

• $X_2$ records the number of heads in the final two flips.

• For every $n\geq3$, $X_n$ records the number of heads on the third flip.

For example, if

$$\omega=HTH,$$

then

$$X_1(\omega)=2,\qquad X_2(\omega)=1,\qquad X_3(\omega)=X_4(\omega)=\dots=1.$$

So the realised sequence is

$$2,1,1,1,\dots$$

which converges to $1$.

However, our target $X$ looks at the second flip. So, since $\omega=HTH$, the second flip is tails, and we have:

$$X(\omega)=0.$$

Thus, for this particular outcome,

$$\lim_{n\to\infty}X_n(\omega)=1\neq0=X(\omega).$$

That is, for $\omega=HTH$, our sequence of numbers converges — but not to the target!

We now ask whether the sequence of random variables $X_n$ converges to our target $X$ according to each of our three notions of convergence.

Does $X_n$ Converge to $X$ Almost Surely?

For every $n\geq3$, $X_n$ records whether the third flip is heads, while $X$ records whether the second flip is heads.

Therefore, the condition

$$\lim_{n\to\infty}X_n(\omega)=X(\omega)$$

will hold precisely when the second and third flips are the same.

This happens for the outcomes

$$\{HHH,THH,HTT,TTT\}.$$

Hence

$$\mathbb{P}(\{\omega\in\Omega:\lim_{n\to\infty}X_n(\omega)=X(\omega)\})=\frac{4}{8}=\frac{1}{2}.$$

Almost sure convergence would require this probability to equal $1$.

Therefore, $X_n$ does not converge almost surely to $X$.

Does $X_n$ Converge to $X$ in Probability?

Take

$$\varepsilon=0.1.$$

Since both $X_n$ and $X$ are either $0$ or $1$, the condition

$$|X_n-X|>0.1$$

holds exactly when they are different: $X_n\neq X$.

For every $n\geq3$, this happens precisely when the second and third flips differ, which has probability

$$\mathbb{P}(|X_n-X|>0.1)=\frac{4}{8}=\frac{1}{2}.$$

So

$$\lim_{n\to\infty}\mathbb{P}(|X_n-X|>0.1)=\frac{1}{2}.$$

Convergence in probability would require this limit to be $0$. Therefore, $X_n$ does not converge in probability to $X$.

Does $X_n$ Converge to $X$ in Distribution?

Here we get a different answer.

For every $n\geq3$, $X_n$ records whether the third flip is heads. Since the coin is fair,

$$X_n\sim\operatorname{Bernoulli}\left(\frac{1}{2}\right).$$

Our target $X$ records whether the second flip is heads, so likewise

$$X\sim\operatorname{Bernoulli}\left(\frac{1}{2}\right).$$

Thus $X_n$ and $X$ have exactly the same distribution for every $n\geq3$.

Therefore,

$$X_n\overset{d}{\longrightarrow}X.$$

What Does This Show?

In this example, $X_n$ converges in distribution to $X$, but not in probability and not almost surely.

So, we have a concrete example showing that the implications in

$$\text{Almost Sure Convergence}\Rightarrow\text{Convergence in Probability}\Rightarrow\text{Convergence in Distribution}$$

cannot, in general, be reversed.

Background:

• Random Variables (Semi-Formal)