5.4 The effect on precision: COVRATIO
The measures so far look at the estimates. An observation can also matter through the precision of the estimates: a sound point at high leverage reduces the variances of the coefficients (panel 2 of Figure 10), while a wild point inflates \(MS_{\mathrm {Res}}\) and so the standard errors. A single number for the precision of \(\hat {\boldsymbol \beta }\) is the determinant of its estimated covariance matrix, \(\det (MS_{\mathrm {Res}}\mathbf {C})\), the generalised variance. It is proportional to the squared volume of the confidence ellipsoid of Lemma 5.2.1.
Definition 5.4.1. \[ \mathrm {COVRATIO}_i = \frac {\det \bigl (S^2_{(i)}(\mathbf {X}_{(i)}^\top \mathbf {X}_{(i)})^{-1}\bigr )}{\det \bigl (MS_{\mathrm {Res}}(\mathbf {X}^\top \mathbf {X})^{-1}\bigr )} . \]
\(\mathrm {COVRATIO}_i > 1\) means the estimates are less precise without observation \(i\): the point improves precision. \(\mathrm {COVRATIO}_i < 1\) means they are more precise without it: the point degrades precision. The computation needs one fact about determinants.
Lemma 5.4.2 (Matrix determinant lemma). If \(\mathbf {A}\) is an invertible \(p \times p\) matrix and \(\mathbf {u}\) a \(p\)-vector, then \(\det (\mathbf {A} - \mathbf {u}\mathbf {u}^\top ) = \det (\mathbf {A})\,(1 - \mathbf {u}^\top \mathbf {A}^{-1}\mathbf {u})\).
Proof. Factorise the \((p + 1) \times (p + 1)\) matrix \(\mathbf {M} = \begin {bmatrix} \mathbf {A} & \mathbf {u} \\ \mathbf {u}^\top & 1 \end {bmatrix}\) in two ways, by eliminating below the top-left block and above the bottom-right entry: \begin {align*} \mathbf {M} &= \begin {bmatrix} \mathbf {I} & \mathbf {0} \\ \mathbf {u}^\top \mathbf {A}^{-1} & 1 \end {bmatrix} \begin {bmatrix} \mathbf {A} & \mathbf {u} \\ \mathbf {0}^\top & 1 - \mathbf {u}^\top \mathbf {A}^{-1}\mathbf {u} \end {bmatrix} , \\ \mathbf {M} &= \begin {bmatrix} \mathbf {I} & \mathbf {u} \\ \mathbf {0}^\top & 1 \end {bmatrix} \begin {bmatrix} \mathbf {A} - \mathbf {u}\mathbf {u}^\top & \mathbf {0} \\ \mathbf {u}^\top & 1 \end {bmatrix} . \end {align*}
Each factor is block triangular, and the determinant of a block-triangular matrix is the product of the determinants of its diagonal blocks. The first line gives \(\det \mathbf {M} = \det (\mathbf {A})(1 - \mathbf {u}^\top \mathbf {A}^{-1}\mathbf {u})\) and the second \(\det \mathbf {M} = \det (\mathbf {A} - \mathbf {u}\mathbf {u}^\top )\). □
Theorem 5.4.3. Suppose \(h_{ii} < 1\). Then \[ \mathrm {COVRATIO}_i = \left (\frac {S^2_{(i)}}{MS_{\mathrm {Res}}}\right )^p\frac {1}{1 - h_{ii}} = \left (\frac {n - p - r_i^2}{n - p - 1}\right )^p\frac {1}{1 - h_{ii}} . \]
Proof. For a \(p \times p\) matrix, \(\det (c\mathbf {B}) = c^p\det \mathbf {B}\) and \(\det (\mathbf {B}^{-1}) = 1/\det \mathbf {B}\). By Lemma 5.4.2 with \(\mathbf {A} = \mathbf {X}^\top \mathbf {X}\) and \(\mathbf {u} = \mathbf {x}_i\), \(\det (\mathbf {X}_{(i)}^\top \mathbf {X}_{(i)}) = \det (\mathbf {X}^\top \mathbf {X} - \mathbf {x}_i\mathbf {x}_i^\top ) = \det (\mathbf {X}^\top \mathbf {X})(1 - h_{ii})\). So \[ \mathrm {COVRATIO}_i = \frac {(S^2_{(i)})^p\big /\bigl [\det (\mathbf {X}^\top \mathbf {X})(1 - h_{ii})\bigr ]}{MS_{\mathrm {Res}}^p\big /\det (\mathbf {X}^\top \mathbf {X})} , \] which is the first form. The second follows from \((n - p - 1)S^2_{(i)}/MS_{\mathrm {Res}} = n - p - r_i^2\), shown in the proof of Theorem 4.2.5(iii). □
The two factors pull in opposite directions. High leverage makes \(1/(1 - h_{ii})\) large, pushing COVRATIO above \(1\): the point extends the design and sharpens the estimates. A large residual makes \(S^2_{(i)}/MS_{\mathrm {Res}}\) small, pushing COVRATIO below \(1\). The usual rule looks closer at observation \(i\) if \[ \mathrm {COVRATIO}_i > 1 + \frac {3p}{n} \quad \text {or}\quad \mathrm {COVRATIO}_i < 1 - \frac {3p}{n} , \] the lower limit being of use only when \(n > 3p\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.