How Do We Think About Random Variables Mathematically?

In formal terms, a random variable is a kind of function.

It takes as its input a member of a sample space: the set of all possible outcomes. The sample space is denoted by the Greek letter $\Omega$.

For instance, suppose we plan to flip a coin three times. Then

$$\Omega=\{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT\}.$$

Here, $HHH$ means getting three heads in a row, and so on.

A random variable assigns a numerical value to each outcome $\omega\in\Omega$.

For instance, suppose we define the random variable $X$ to be the number of heads obtained. Then

$$X:\Omega\to\mathbb{R},$$

and, for example,

$$X(HTT)=1.$$

So a random variable takes an outcome of the experiment and turns it into a number.

What Does $\mathbb{P}(X=1)$ Mean?

Probabilities are assigned to subsets of $\Omega$ – though perhaps not to all of them. Subsets of $\Omega$ to which probabilities are assigned are called “events”.

For this experiment, though, let us assign probabilities to all subsets $A\subseteq\Omega$.

Since the coin is fair, all $8$ possible outcomes of the experiment are equally likely, and these probabilities should depend only on how many of these outcomes are in $A$. For example,

$$\mathbb{P}(\{HTT,THT,TTH\})=\frac{3}{8}.$$

Now suppose we look to find

$$\mathbb{P}(X=1).$$

What this really means is the probability of the set of outcomes $\omega$ for which $X(\omega)=1$.

In this example,

$$\{\omega\in\Omega:X(\omega)=1\}=\{HTT,THT,TTH\}.$$

Therefore,

$$\mathbb{P}(X=1)\stackrel{\text{def}}{=}\mathbb{P}(\{\omega\in\Omega:X(\omega)=1\})=\mathbb{P}(\{HTT,THT,TTH\})=\frac{3}{8}.$$

So probability statements involving a random variable are really probability statements about corresponding sets of outcomes in the original sample space.

Towards a Fully Rigorous Treatment: Measure Theory

For a richer sample space $\Omega$, such as $\mathbb{R}$, often we cannot assign probabilities to every subset while retaining all the properties of probability we need.

We therefore need to ensure that ordinary probability statements of the kind we want to make about $X$ correspond to subsets of $\Omega$ to which probabilities have actually been assigned.

It turns out that, for a real-valued random variable, it is enough to require that, for each real number $a$, the set of outcomes $\omega$ such that

$$X(\omega)\leq a$$

is assigned a probability.

This ensures that the CDF

$$F_X(a)=\mathbb{P}(X\leq a)$$

is well-defined for every real number $a$.

A function satisfying the appropriate condition is called measurable.

The condition is necessary because the CDF determines the distribution of $X$, which we need to be well-defined. Understanding why this condition is also sufficient for an adequate definition leads us into measure theory.

Background:

• Introduction to Random Variables

• Cumulative Distribution Functions