What is a Joint Probability Mass Function?

A joint probability mass function, or joint pmf, describes how two discrete random variables behave together. Given possible values $x$ and $y$, it tells us the probability that $X=x$ and $Y=y$ at the same time:

$$p_{X,Y}(x,y)=\mathbb{P}(X=x,Y=y).$$

For example, suppose we roll a fair die and let $X$ be the number rolled, while $Y$ records whether the number is even, taking the value 1 if it is even and 0 otherwise.

Then

$$\mathbb{P}(X=4,Y=1)=\frac{1}{6},$$

because rolling a 4 automatically gives us $Y=1$. On the other hand,

$$\mathbb{P}(X=4,Y=0)=0,$$

since these two values cannot occur together.

More generally, a joint pmf assigns a probability to every possible pair of values. These probabilities are often organised in a table, with values of one random variable along the rows and values of the other along the columns.

As with an ordinary pmf, all the probabilities must be non-negative, and the probabilities over all possible pairs must add up to 1.

Marginal Distributions

Once we know the joint pmf, we can recover the ordinary distribution of either random variable on its own.

For a particular value $x$, to find the probability that $X=x$, we add up all the joint probabilities involving that value of $X$:

$$\mathbb{P}(X=x)=\sum_y \mathbb{P}(X=x,Y=y).$$

Similarly,

$$\mathbb{P}(Y=y)=\sum_x \mathbb{P}(X=x,Y=y).$$

These are called the marginal distributions of $X$ and $Y$. In a joint probability table, we obtain them simply by adding across the appropriate rows or columns.

Expected Values with Two Random Variables

A joint pmf also lets us find expected values involving both random variables.

For example,

$$\mathbb{E}(XY)=\sum_x \sum_y xy \ \mathbb{P}(X=x,Y=y).$$

Here we simply follow the usual definition of an expected value: we take each possible pair $x$ and $y$, calculate the product $xy$, weight it by the probability of that pair occurring, and then add everything together.

More generally, for a function $g(X,Y)$,

$$\mathbb{E}(g(X,Y))=\sum_x \sum_y g(x,y) \ \mathbb{P}(X=x,Y=y).$$

So, this is the same weighted-average idea we used for a single discrete random variable. The only difference is that we are now summing over pairs of possible values rather than individual values.

Background:

• Random Variables

• Discrete Random Variables

• Probability Mass Functions