MLE With More Than One Observation

So far, our maximum likelihood example involved a single random variable.

However, in statistics, it is more common to start from a random sample:

$$X_1,X_2,\dots,X_n,$$

and then form our estimates from the corresponding observed data:

$$x_1,x_2,\dots,x_n.$$

For discrete data, if the observations are independent, the probability of obtaining all of them together is the product of their individual probabilities.

This leads us to extend our work so far by defining the joint likelihood:

$$L(\theta\mid\mathbf{x})=\mathbb{P}(X_1=x_1)\times\mathbb{P}(X_2=x_2)\times\dots\times\mathbb{P}(X_n=x_n),$$

where $\mathbf{x}=(x_1,x_2,\dots,x_n)$ represents the whole dataset.

We can then maximise this likelihood with respect to the unknown parameter $\theta$, just as before.

Example: Counting Ducklings

Suppose that on four different days we sit beside a river for an hour and count the number of ducklings that pass.

We model these counts as a random sample from a Poisson distribution:

$$X_i\sim\operatorname{Po}(\lambda).$$

Suppose we observe

$$x_1=3,\qquad x_2=4,\qquad x_3=3,\qquad x_4=6.$$

For a Poisson random variable,

$$\mathbb{P}(X_i=x_i)=\frac{e^{-\lambda}\lambda^{x_i}}{x_i!}.$$

Using independence, the likelihood of our complete dataset is therefore

$$L(\lambda\mid\mathbf{x})=\frac{e^{-\lambda}\lambda^3}{3!}\frac{e^{-\lambda}\lambda^4}{4!}\frac{e^{-\lambda}\lambda^3}{3!}\frac{e^{-\lambda}\lambda^6}{6!}.$$

Collecting the terms gives

$$L(\lambda\mid\mathbf{x})=\frac{e^{-4\lambda}\lambda^{16}}{622080}.$$

Finding the Maximum Likelihood Estimate

As before, taking logs makes the calculation easier:

$$l(\lambda\mid\mathbf{x})=-4\lambda+16\log(\lambda)-\log(622080).$$

Differentiating with respect to $\lambda$ and setting the result equal to $0$ gives

$$-4+\frac{16}{\lambda}=0.$$

Hence,

$$\hat\lambda=4.$$

This makes intuitive sense: over the four hours we saw a total of $16$ ducklings, giving an average rate of

$$\frac{16}{4}=4$$

ducklings per hour.

More generally, it is easy to show that carrying out the same calculation for a Poisson random sample gives

$$\hat\lambda=\frac{x_1+x_2+\dots+x_n}{n}=\bar x.$$

So in this case, the maximum likelihood estimate of the Poisson rate is simply the sample mean.