13.3 Two-stage least squares

With more instruments than regressors, \(Z'X\) is not square, and there are many ways to combine the instruments. The best, under homoscedasticity, is to use the fitted values from the regression of \(X\) on \(Z\).

Theorem 13.3.1. Let \(P_Z = Z(Z'Z)^{-1}Z'\) and \(\hat X = P_ZX\). The two-stage least squares (2SLS) estimator is \[ b_{\text {2SLS}} = (\hat X'X)^{-1}\hat X'Y = (X'P_ZX)^{-1}X'P_ZY . \]

(i): It is the least squares coefficient in the regression of \(Y\) on \(\hat X\) (the “second stage”), and it equals \(b_{\text {IV}}\) when \(\ell = k\).
(ii): Under the conditions of Theorem 13.2.2, \(\sqrt n(b_{\text {2SLS}} - \beta ) \xrightarrow {d} N\bigl (0,\ \sigma ^2(Q_{XZ}Q_{ZZ}^{-1}Q_{ZX})^{-1}\bigr )\), estimated by \(s^2(\hat X'\hat X)^{-1}\) with \(s^2\) computed from \(Y - Xb_{\text {2SLS}}\).
(iii): Among all IV estimators that use \(k\) linear combinations \(ZA\) of the instruments, 2SLS has the smallest asymptotic variance.

Proof. (i) \(P_Z\) is symmetric and idempotent, so \(\hat X'\hat X = X'P_ZX = \hat X'X\), and the least squares coefficient of \(Y\) on \(\hat X\) is \((\hat X'\hat X)^{-1}\hat X'Y = b_{\text {2SLS}}\). If \(\ell = k\), \(Z'X\) is square and invertible, and \begin {align*} \bigl (X'Z(Z'Z)^{-1}Z'X\bigr )^{-1}X'Z(Z'Z)^{-1}Z'Y &= (Z'X)^{-1}(Z'Z)(X'Z)^{-1}X'Z(Z'Z)^{-1}Z'Y\\ &= (Z'X)^{-1}Z'Y . \end {align*}

(ii) \(b_{\text {2SLS}}\) is the IV estimator of Theorem 13.2.2 with the \(k\) instruments \(\hat X\). Writing \(\hat X = Z\hat \Pi \) with \(\hat \Pi = (Z'Z)^{-1}Z'X \xrightarrow {p} Q_{ZZ}^{-1}Q_{ZX}\), its asymptotic variance is \(\sigma ^2(\Pi 'Q_{ZX})^{-1}\Pi 'Q_{ZZ}\Pi (Q_{XZ}\Pi )^{-1}\) with \(\Pi = Q_{ZZ}^{-1}Q_{ZX}\), and substituting gives \(\sigma ^2(Q_{XZ}Q_{ZZ}^{-1}Q_{ZX})^{-1}\).

(iii) With instruments \(W = ZA\), the IV estimator has asymptotic variance \(\sigma ^2\) times the probability limit of \(n(W'X)^{-1}W'W(X'W)^{-1} = n(X'P_WX)^{-1}\), where \(P_W\) is the projection on the columns of \(W\). Since the columns of \(W\) lie in the column space of \(Z\), \(P_Z - P_W\) is the projection on the part of that space orthogonal to \(W\), and is positive semi-definite. Hence \(X'P_ZX - X'P_WX\) is psd, and so \((X'P_WX)^{-1} - (X'P_ZX)^{-1}\) is psd, by the rule that \(A - B\) psd implies \(B^{-1} - A^{-1}\) psd for positive definite \(A\) and \(B\). □

The name describes the computation: regress each endogenous regressor on all the instruments, then regress \(Y\) on the fitted values. But running the second stage by hand gives the wrong standard errors. Its residuals are \(Y - \hat Xb\), not \(Y - Xb\), and they estimate the wrong variance; Software 13.5.1 shows the error. The first stage also has diagnostic value: its \(F\) statistic for the excluded instruments measures how strongly they move the endogenous regressor.

Weak instruments

The asymptotic theory can be a poor guide in finite samples when the instruments are only weakly correlated with the regressor. 2SLS is then biased towards least squares; with \(\ell \) excluded instruments and a first-stage \(F\) statistic \(F\), the bias of 2SLS relative to that of least squares is roughly \(1/F\). The standard errors are too small, and \(t\) tests over-reject. In the exactly identified case the IV estimator has no finite mean at all, and its distribution has very fat tails. Staiger and Stock (1997) proposed the rule of thumb that the first-stage \(F\) should exceed \(10\); Bound, Jaeger and Baker (1995) showed that a famous study using quarter of birth as an instrument for schooling gave estimates close to least squares even with random instruments. Software 13.5.2 illustrates both effects. A weak instrument is worse than useless if it is also slightly endogenous: the inconsistency of IV is \(\plim \frac 1n z'u/\plim \frac 1n z'x\), and a small denominator magnifies a small numerator.

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.