Gauss-Markov For Simple Linear Regression

Having developed ordinary least squares (OLS) in detail, we now ask a crucial question: under what circumstances should we expect it to work well?

The answer depends on the assumptions we make about our simple linear regression model and the process generating our data. We introduce these assumptions carefully, including random sampling, variation in $X$, exogeneity, and homoskedasticity. The most important concern the error term, $U$, and its relationship with $X$.

Before getting going, we draw on probability theory to look more closely at what it means to condition on $X$. This means treating the values of $X$ as already fixed, while studying the remaining randomness in our OLS estimators.

We then explain the assumptions one by one, making clear the precise role of each. Using them together, we establish the Gauss-Markov Theorem in two parts.

In Part I, we show that the OLS estimators are unbiased. We then derive their conditional variances and, in Part II, obtain the famous BLUE result: amongst linear unbiased estimators, OLS has the smallest variance. In a sense, OLS is the best game in town!

The aim throughout is not simply to memorise a list of assumptions, but to understand what each one actually does – and, through examples and visualisation, what can go wrong when it fails.