10.2 Heteroscedasticity-robust standard errors
The sandwich (10.1) needs the \(n\) unknown \(\sigma _i^2\), which cannot be estimated individually. White’s insight (1980), anticipated by Eicker and Huber, was that it is not necessary to estimate them: only the \(k \times k\) matrix \(\frac 1n\sum \sigma _i^2x_ix_i'\) is needed, and that average can be estimated consistently by replacing each \(\sigma _i^2\) with the squared residual.
Theorem 10.2.1 (White). Under (A1), independent observations, \(\frac 1nX'X \to Q\) positive definite, and suitable moment conditions, \[ \frac 1n\sum _{i=1}^ne_i^2x_ix_i' - \frac 1n\sum _{i=1}^n\sigma _i^2x_ix_i' \xrightarrow {p} 0 , \] so that \[ \widehat \Var _{\text {HC}}(b) = (X'X)^{-1}\Bigl (\sum _{i=1}^ne_i^2x_ix_i'\Bigr )(X'X)^{-1} \] is a consistent estimator of the variance of \(b\), in the sense that \(n\widehat \Var _{\text {HC}}(b) - n\Var (b) \xrightarrow {p} 0\), whatever the form of the heteroscedasticity.
Sketch of proof. Write \(e_i = u_i - x_i'(b - \beta )\), so that \[ e_i^2 = u_i^2 - 2u_ix_i'(b - \beta ) + \bigl (x_i'(b - \beta )\bigr )^2 . \] First, \(\frac 1n\sum (u_i^2 - \sigma _i^2)x_ix_i' \xrightarrow {p} 0\) by a law of large numbers for independent, non-identically distributed terms with mean zero, since \(E(u_i^2) = \sigma _i^2\). Second, the remaining terms are of the form \(\frac 1n\sum u_ix_i'(b - \beta )\,x_ix_i'\) and \(\frac 1n\sum \bigl (x_i'(b - \beta )\bigr )^2x_ix_i'\). Each is a product of \(b - \beta \xrightarrow {p} 0\) with an average that is bounded in probability under the moment conditions, so each converges to zero. Multiplying by \(\bigl (\frac 1nX'X\bigr )^{-1}\) on both sides and using the continuous mapping theorem gives the result. □
The individual \(e_i^2\) are terrible estimates of the individual \(\sigma _i^2\), being single observations, but their weighted average is a good estimate of the weighted average of the \(\sigma _i^2\). The same idea underlies Theorem 4.4.1(iii) for the two-variable model. In finite samples the squared residuals are too small on average, since \(E(e_i^2) = (1 - p_{ii})\sigma _i^2\) under homoscedasticity, and several corrections are in use:
| HC0 | \(e_i^2\) (White’s original) |
| HC1 | \(e_i^2\,n/(n - k)\) (the default in Stata) |
| HC2 | \(e_i^2/(1 - p_{ii})\) |
| HC3 | \(e_i^2/(1 - p_{ii})^2\) (recommended for small samples) |
HC3 inflates the residuals at high-leverage points most, because those points pull the fitted line towards themselves and so have the most deflated residuals.
Robust standard errors are used in robust \(t\) tests and confidence intervals, and robust Wald tests replace \(s^2(X'X)^{-1}\) by \(\widehat \Var _{\text {HC}}\) in the quadratic form of Theorem 7.3.1. These tests are justified asymptotically, so the reference distributions are \(N(0, 1)\) and \(\chi ^2_q\). Many applied economists now report robust standard errors routinely in cross-section work, on the grounds that they are valid whether or not the disturbances are heteroscedastic, while the conventional ones are valid only if they are not.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.