6.5 Partial correlation
Definition 6.5.1. The partial correlation of \(Y\) and \(X_2\) given \(X_3\), written \(r_{12\cdot 3}\) with \(Y\) labelled \(1\), is the correlation between the residuals of \(Y\) on \(X_3\) and the residuals of \(X_2\) on \(X_3\) (both regressions with an intercept). More generally, \(r_{12\cdot 34\ldots k}\) partials out \(X_3, \dots , X_k\).
Theorem 6.5.2. In terms of the simple correlations, \[ r_{12\cdot 3} = \frac {r_{12} - r_{13}r_{23}}{\sqrt {1 - r_{13}^2}\sqrt {1 - r_{23}^2}} , \] and the coefficient of determination of the regression of \(Y\) on \(X_2\) and \(X_3\) satisfies \[ 1 - R^2_{1\cdot 23} = (1 - r_{12}^2)(1 - r_{13\cdot 2}^2), \] that is, \[ R^2_{1\cdot 23} = r_{12}^2 + r_{13\cdot 2}^2(1 - r_{12}^2) . \]
Proof. Work in deviations and write \(S_{jk} = \sum x_jx_k\), with \(x_1 = y\). The residual of \(y\) on \(x_3\) is \(y - (S_{13}/S_{33})x_3\), and that of \(x_2\) on \(x_3\) is \(x_2 - (S_{23}/S_{33})x_3\). Their cross-product and sums of squares are \[ S_{12} - \frac {S_{13}S_{23}}{S_{33}}, \qquad S_{11} - \frac {S_{13}^2}{S_{33}} = S_{11}(1 - r_{13}^2), \qquad S_{22} - \frac {S_{23}^2}{S_{33}} = S_{22}(1 - r_{23}^2) . \] Dividing the cross-product by the square roots of the sums of squares, and \(S_{12}\), \(S_{13}S_{23}/S_{33}\) by \(\sqrt {S_{11}S_{22}}\), gives the first formula.
For the second, by Theorem 6.4.1 the residuals of the full regression equal those of regressing \(e_{1\cdot 2}\), the residual of \(y\) on \(x_2\), on \(e_{3\cdot 2}\), the residual of \(x_3\) on \(x_2\). A simple regression of \(e_{1\cdot 2}\) on \(e_{3\cdot 2}\) leaves the fraction \(1 - r^2\) of the sum of squares of \(e_{1\cdot 2}\) unexplained, where \(r\) is the correlation of the two residual series, which is \(r_{13\cdot 2}\) by definition (Theorem 2.2.3). And the sum of squares of \(e_{1\cdot 2}\) is \(S_{11}(1 - r_{12}^2)\). So \(\text {RSS} = S_{11}(1 - r_{12}^2)(1 - r_{13\cdot 2}^2)\); divide by \(\text {TSS} = S_{11}\). □
The second identity says that the variation left unexplained by \(X_2\) alone, the fraction \(1 - r_{12}^2\), is reduced by the fraction \(r_{13\cdot 2}^2\) when \(X_3\) is added. The partial correlation is the correlation of \(Y\) with what is new in \(X_3\).
Example 6.5.3. In Example 6.2.3, \(r_{12} = 25/\sqrt {640} = 0.9882\), \(r_{13} = 22/\sqrt {640} = 0.8696\) and \(r_{23} = 8/10 = 0.8\). Then \[ r_{13\cdot 2} = \frac {0.8696 - 0.9882(0.8)}{\sqrt {1 - 0.9766}\sqrt {1 - 0.64}} = \frac {0.0790}{0.0919} = 0.861 , \] and \(R^2_{1\cdot 23} = 0.9766 + 0.861^2(0.0234) = 0.9939\). Directly, \(\text {ESS} = b_2\sum x_2y + b_3\sum x_3y = 2.0556(25) + 0.5556(22) = 63.611\) and \(\text {TSS} = 64\), so \(R^2 = 0.9939\), agreeing.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.