8.1 Omitted and irrelevant variables
Partition the regressors as \(X = [X_1\ \ X_2]\) with \(k_1\) and \(k_2\) columns, and suppose the true model is \[ Y = X_1\beta _1 + X_2\beta _2 + U , \] satisfying (A1)–(A4). Let \(P_{12} = (X_1'X_1)^{-1}X_1'X_2\), the \(k_1 \times k_2\) matrix of coefficients from regressing each column of \(X_2\) on \(X_1\).
Theorem 8.1.1 (Omitted variables). Let \(\tilde b_1 = (X_1'X_1)^{-1}X_1'Y\) be the estimator from the short regression of \(Y\) on \(X_1\) alone, and let \(b_1\) be the corresponding part of the long regression on \([X_1\ \ X_2]\). Then:
- (i)
- \(\tilde b_1 = b_1 + P_{12}b_2\) exactly, in every sample;
- (ii)
- \(E(\tilde b_1) = \beta _1 + P_{12}\beta _2\), so \(\tilde b_1\) is biased unless \(P_{12}\beta _2 = 0\);
- (iii)
- \(\Var (\tilde b_1) = \sigma ^2(X_1'X_1)^{-1}\), and \(\Var (b_1) - \Var (\tilde b_1)\) is psd;
- (iv)
- the short regression’s \(s^2\) has \(E(\tilde s^2) = \sigma ^2 + \beta _2'X_2'M_1X_2\beta _2/(n - k_1) \geq \sigma ^2\).
Proof. (i) Substitute the long-regression decomposition \(Y = X_1b_1 + X_2b_2 + e\), with \(X_1'e = 0\), into \(\tilde b_1\): \(\tilde b_1 = b_1 + (X_1'X_1)^{-1}X_1'X_2b_2 + (X_1'X_1)^{-1}X_1'e = b_1 + P_{12}b_2\).
(ii) Take expectations, using \(E(b) = \beta \) for the long regression, which is correctly specified.
(iii) \(\tilde b_1 = \beta _1 + P_{12}\beta _2 + (X_1'X_1)^{-1}X_1'U\), with variance \(\sigma ^2(X_1'X_1)^{-1}\). In the long regression, by the Frisch–Waugh–Lovell theorem applied with the roles of \(X_1\) and \(X_2\) exchanged, \(\Var (b_1) = \sigma ^2(X_1'M_2X_1)^{-1}\). Since \(X_1'X_1 - X_1'M_2X_1 = X_1'(I - M_2)X_1\) is psd, \((X_1'M_2X_1)^{-1} - (X_1'X_1)^{-1}\) is psd (inversion reverses the order of pd matrices).
(iv) The short residuals are \(\tilde e = M_1Y = M_1X_2\beta _2 + M_1U\). By Theorem 5.2.1(iii), \(E(\tilde e'\tilde e) = \sigma ^2\tr M_1 + \beta _2'X_2'M_1X_2\beta _2 = \sigma ^2(n - k_1) + \beta _2'X_2'M_1X_2\beta _2\). □
Part (i) is the omitted variable formula. With one included and one omitted regressor (each with an intercept) it says \[ \tilde b_1 = b_1 + \delta \,b_2 , \] where \(\delta \) is the slope from regressing the omitted variable on the included one. The bias \(\delta \beta _2\) has the sign of the product of the omitted variable’s effect and its correlation with the included variable:
| \(\Corr (X_1, X_2) > 0\) | \(\Corr (X_1, X_2) < 0\) | |
| \(\beta _2 > 0\) | upward bias | downward bias |
| \(\beta _2 < 0\) | downward bias | upward bias |
In the wage equation of Example 1.1.5, ability raises wages (\(\beta _2 > 0\)) and is positively correlated with schooling, so omitting it biases the return to schooling upwards.
Parts (iii) and (iv) describe a trade-off. The short regression is biased, but its estimates have smaller variance, while its estimate of \(\sigma ^2\) is too large, so its reported standard errors are not even right for the smaller true variance. Omitting a relevant variable is almost never a good bargain for inference, because the conventional standard errors misrepresent the uncertainty, whatever the mean squared error comparison.
Theorem 8.1.2 (Irrelevant variables). If \(\beta _2 = 0\), the long regression’s \(b_1\) is still unbiased for \(\beta _1\), and its \(s^2\) is unbiased for \(\sigma ^2\), but \(\Var (b_1) - \Var (\tilde b_1)\) is psd: the estimates are less precise than those from the correctly specified short regression.
Proof. The long regression is correctly specified (with \(\beta _2 = 0\)), so Theorem 7.1.1 and Theorem 5.5.1 apply. The variance comparison is Theorem 8.1.1(iii), which did not depend on \(\beta _2\). □
Including an irrelevant variable costs precision, by the variance inflation factor of Problem 7.8.6: nothing if the irrelevant variable is uncorrelated with the others, a great deal if it is highly correlated with them. Omitting a relevant one costs unbiasedness. The asymmetry is why applied economists usually err towards including a variable when theory suggests it.
Example 8.1.3. In the savings data, regress sr on pop75 alone (short), on pop15 and pop75 (long), and pop15 on pop75 (auxiliary). The slopes are \[ \tilde b_{75} = 1.0987, \qquad b_{75} = -1.9341,\ b_{15} = -0.4708, \qquad \delta = -6.4412 . \] The formula gives \(b_{75} + \delta b_{15} = -1.9341 + (-6.4412)(-0.4708) = 1.0987\), exactly the short coefficient. The positive short-regression coefficient on the elderly share is entirely omitted-variable bias. A country with more elderly people has many fewer children (\(\delta = -6.44\)), and children reduce saving (\(b_{15} < 0\)), so the bias \(\delta b_{15} = +3.03\) is large and positive. It more than reverses the true negative partial effect.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.