10.1 Consequences for least squares
Let \(Y = X\beta + U\) with \(E(U \mid X) = 0\) and \(\Var (U \mid X) = \Sigma = \operatorname {diag}(\sigma _1^2, \dots , \sigma _n^2)\). This is the case \(\sigma ^2\Omega = \Sigma \) of Section 9.4, so by Theorem 9.4.1 least squares is unbiased, and \begin {equation} \label {eq:sandwich} \Var (b) = (X'X)^{-1}X'\Sigma X(X'X)^{-1} = (X'X)^{-1}\Bigl (\sum _{i=1}^n\sigma _i^2x_ix_i'\Bigr )(X'X)^{-1}, \end {equation} where \(x_i'\) is the \(i\)th row of \(X\). This is called a sandwich: the “meat” \(X'\Sigma X\) between two slices of “bread” \((X'X)^{-1}\). The conventional formula \(s^2(X'X)^{-1}\) is wrong, and it is worth seeing in which direction.
Theorem 10.1.1. In the two-variable model with deviations \(x_i\), \[ \Var (b) = \frac {\sum x_i^2\sigma _i^2}{\bigl (\sum x_i^2\bigr )^2}, \qquad \text {while}\qquad E\!\left (\frac {s^2}{\sum x_i^2}\right ) = \frac {\sum (1 - p_{ii})\sigma _i^2}{(n - 2)\sum x_i^2} , \] where \(p_{ii}\) are the leverages of Problem 5.7.6. Ignoring the leverage correction, the conventional estimate is too small when the larger variances \(\sigma _i^2\) occur at the observations with larger \(x_i^2\), and too large in the opposite case.
Proof. The first formula is \(\Var (\sum w_iu_i) = \sum w_i^2\sigma _i^2\) with \(w_i = x_i/\sum x_j^2\). For the second, \(E(e'e) = E(U'MU) = \tr (M\Sigma ) = \sum _i (1 - p_{ii})\sigma _i^2\) by Theorem 5.2.1(iii), since \(\Sigma \) is diagonal. Replacing \((1 - p_{ii})\) by its average \((n - 2)/n\), the conventional estimate has expectation about \(\bar \sigma ^2/\sum x_i^2\), where \(\bar \sigma ^2\) is the average of the \(\sigma _i^2\). The ratio of the true variance to this is \[ \frac {\sum x_i^2\sigma _i^2}{\bar \sigma ^2\sum x_i^2} = 1 + \frac {\sum x_i^2(\sigma _i^2 - \bar \sigma ^2)}{\bar \sigma ^2\sum x_i^2}, \] which exceeds one exactly when the \(\sigma _i^2\) are above average where \(x_i^2\) is large, that is when \(\sigma _i^2\) and \(x_i^2\) are positively associated. □
The common case in economics is the first: variance grows with the size of the regressor. The conventional standard errors are then too small, \(t\) ratios too large, and confidence intervals too narrow. Inference is misleading even though the estimates themselves are unbiased. A second consequence is inefficiency: by Aitken’s theorem, generalised least squares has a smaller variance.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.