6.4 Partitioned regression: the Frisch–Waugh–Lovell theorem
Partition the regressors into two groups, \(X = [X_1\ \ X_2]\), with \(k_1\) and \(k_2\) columns, and correspondingly \(b = (b_1', b_2')'\). Let \(M_1 = I - X_1(X_1'X_1)^{-1}X_1'\) be the residual maker for \(X_1\) alone.
Theorem 6.4.1 (Frisch–Waugh–Lovell). The subvector \(b_2\) of the least squares estimates in the regression of \(Y\) on \([X_1\ \ X_2]\) is \[ b_2 = (X_2'M_1X_2)^{-1}X_2'M_1Y . \] Equivalently, \(b_2\) is obtained by regressing the residuals \(M_1Y\) (of \(Y\) on \(X_1\)) on the residuals \(M_1X_2\) (of each column of \(X_2\) on \(X_1\)). The residuals of that regression are identical to the residuals \(e\) of the full regression.
Proof. The full residual vector satisfies \(Y = X_1b_1 + X_2b_2 + e\) with \(X_1'e = 0\) and \(X_2'e = 0\). Premultiply by \(M_1\). Since \(M_1X_1 = 0\), and \(M_1e = e - X_1(X_1'X_1)^{-1}X_1'e = e\), \[ M_1Y = M_1X_2b_2 + e . \] Premultiply by \(X_2'M_1 = (M_1X_2)'\) and use \(X_2'M_1e = X_2'e = 0\): \(X_2'M_1Y = X_2'M_1X_2b_2\). The matrix \(X_2'M_1X_2\) is non-singular because \(X\) has full rank, which gives the formula. Finally \(e = M_1Y - M_1X_2b_2\) is exactly the residual vector of the regression of \(M_1Y\) on \(M_1X_2\), since \((M_1X_2)'e = 0\) shows that \(b_2\) satisfies that regression’s normal equations. □
The theorem is the precise meaning of “holding the other variables constant”. A multiple regression coefficient measures the relation between \(Y\) and \(X_2\) after the linear influence of \(X_1\) has been removed from both. Nothing is literally held fixed; the effect of \(X_1\) is partialled out. Two special cases are used constantly.
- (i)
- (Deviation form.) Take \(X_1 = \iota \), so \(M_1 = M_0\) and \(M_0X_2\), \(M_0Y\) are deviations from means. The slope coefficients are obtained by regressing the deviations of \(Y\) on the deviations of the regressors, \(b_2 = (X_2'M_0X_2)^{-1}X_2'M_0Y\), and the intercept is \(b_1 = \bar Y - b_2\bar X_2 - \dots - b_k\bar X_k\). With two regressors in deviation form, \[ b_2 = \frac {\sum x_2y\sum x_3^2 - \sum x_3y\sum x_2x_3}{\sum x_2^2\sum x_3^2 - \bigl (\sum x_2x_3\bigr )^2}, \qquad b_3 = \frac {\sum x_3y\sum x_2^2 - \sum x_2y\sum x_2x_3}{\sum x_2^2\sum x_3^2 - \bigl (\sum x_2x_3\bigr )^2} . \]
- (ii)
- (Detrending.) Including a time trend in a regression gives the same slope coefficients as first removing a linear trend from every variable and then regressing the detrended series on one another.
Proof. (i) is the theorem with \(X_1 = \iota \); the explicit formulas are the inverse of the \(2 \times 2\) matrix of deviation sums of squares and cross-products, as in Example 6.2.3. (ii) is the theorem with \(X_1 = [\iota \ \ t]\). □
Part (ii) was Frisch and Waugh’s original result in 1933, and it answered a debate of the time about whether economic series should be detrended before a regression is run. The answer is that it makes no difference to the slopes, provided the trend is in the regression.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.