5.3 DFFITS and DFBETAS
Cook’s distance summarises the effect of a deletion on all of \(\hat {\boldsymbol \beta }\) at once. Two further measures look at particular effects: on the fitted value at the deleted point, and on each coefficient separately. Both are scaled by the deleted estimate \(S^2_{(i)}\), for the reason given in the discussion of R-student: a bad point should not be allowed to inflate the yardstick used to judge it.
Definition 5.3.1. \[ \mathrm {DFFITS}_i = \frac {\hat y_i - \hat y_{(i)}}{\sqrt {S^2_{(i)}\,h_{ii}}} , \qquad \mathrm {DFBETAS}_{j,i} = \frac {\hat \beta _j - \hat \beta _{j(i)}}{\sqrt {S^2_{(i)}\,C_{jj}}} , \] where \(\hat y_{(i)} = \mathbf {x}_i^\top \hat {\boldsymbol \beta }_{(i)}\) and \(\hat \beta _{j(i)}\) is entry \(j\) of \(\hat {\boldsymbol \beta }_{(i)}\).
\(\mathrm {DFFITS}_i\) is the change in the fitted value at \(\mathbf {x}_i\), in units of (an estimate of) the standard deviation of \(\hat y_i\), which is \(\sigma \sqrt {h_{ii}}\). Similarly \(\mathrm {DFBETAS}_{j,i}\) is the change in \(\hat \beta _j\), in units of its standard error. Both can be computed from the full fit.
Theorem 5.3.2. Suppose \(h_{ii} < 1\), and let \(\mathbf {R} = \mathbf {C}\mathbf {X}^\top \), a \(p \times n\) matrix with entries \(\rho _{ji}\). Then
- (i)
- \(\mathrm {DFFITS}_i = t_i\sqrt {\dfrac {h_{ii}}{1 - h_{ii}}}\);
- (ii)
- \(\mathrm {DFBETAS}_{j,i} = \dfrac {\rho _{ji}}{\sqrt {C_{jj}}}\cdot \dfrac {t_i}{\sqrt {1 - h_{ii}}}\), and \(\sum _{i=1}^n \rho _{ji}^2 = C_{jj}\).
Proof. (i) By Theorem 4.2.3(iii), \(\hat y_i - \hat y_{(i)} = (y_i - \hat y_{(i)}) - (y_i - \hat y_i)\) \(= e_i/(1 - h_{ii}) - e_i = h_{ii}e_i/(1 - h_{ii})\). So \begin {align*} \mathrm {DFFITS}_i &= \frac {h_{ii}e_i}{(1 - h_{ii})\sqrt {S^2_{(i)}h_{ii}}} = \frac {e_i}{\sqrt {S^2_{(i)}(1 - h_{ii})}}\cdot \sqrt {\frac {h_{ii}}{1 - h_{ii}}} \\ &= t_i\sqrt {\frac {h_{ii}}{1 - h_{ii}}} . \end {align*}
(ii) Column \(i\) of \(\mathbf {R}\) is \(\mathbf {C}\mathbf {x}_i\), so by Theorem 4.2.3(ii), \(\hat \beta _j - \hat \beta _{j(i)} = \rho _{ji}e_i/(1 - h_{ii})\). Hence \[ \mathrm {DFBETAS}_{j,i} = \frac {\rho _{ji}}{\sqrt {C_{jj}}}\cdot \frac {e_i}{(1 - h_{ii})\sqrt {S^2_{(i)}}} = \frac {\rho _{ji}}{\sqrt {C_{jj}}}\cdot \frac {t_i}{\sqrt {1 - h_{ii}}} . \] Finally \(\mathbf {R}\mathbf {R}^\top = \mathbf {C}\mathbf {X}^\top \mathbf {X}\mathbf {C} = \mathbf {C}\), whose \((j,j)\) entry is \(\sum _i \rho _{ji}^2 = C_{jj}\). □
DFFITS has the same structure as Cook’s distance, a residual times a leverage factor; indeed \(\mathrm {DFFITS}_i^2 = p\,D_i\,MS_{\mathrm {Res}}/S^2_{(i)}\) (Problem 5.7.3). The difference is that it uses R-student, which is not bounded, so DFFITS reacts more strongly to a gross outlier. In DFBETAS, the factor \(\rho _{ji}/\sqrt {C_{jj}}\) is the share of observation \(i\) in the estimate \(\hat \beta _j = \sum _i \rho _{ji}y_i\): by part (ii) the squares of these shares add up to \(1\) over \(i\).
The usual cut-offs, which allow for the fact that every measure shrinks as \(n\) grows, are \[ |\mathrm {DFFITS}_i| > 2\sqrt {p/n} , \qquad |\mathrm {DFBETAS}_{j,i}| > 2/\sqrt {n} . \]
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.