4.4 Outliers
An outlier is an observation that is unlike the rest: in a regression, one whose response is far from where the model fitted to the other observations would put it. It shows up as a large scaled residual, conventionally three or four standard deviations from zero.
Outliers have causes, and the cause decides what is done. A bad value, a reading recorded wrongly, a faulty instrument, a unit that does not belong to the population, should be corrected if possible, and otherwise removed. But an outlier may be a perfectly genuine observation, telling us that the model is wrong in some region, or that the errors have heavier tails than the normal. Such points should never be deleted merely to improve the fit; they are often the most informative points in the data. The safe course is to fit the model with and without the point, report both, and look for an explanation.
A test for an outlier
Theorem 4.2.5(ii) turns R-student into a test. If observation \(i\) was singled out in advance, compare \(|t_i|\) with \(t_{\alpha /2, n - p - 1}\). Usually it was not: we look at all \(n\) residuals and test the largest. The chance that at least one of \(n\) statistics exceeds the critical value is then far above \(\alpha \). The Bonferroni correction allows for this. Testing each at level \(\alpha /n\), the probability of any false alarm is at most \(n \times \alpha /n = \alpha \) (the probability of a union of events is at most the sum of their probabilities). So \[ \text {declare observation } i \text { an outlier if } |t_i| > t_{\alpha /(2n),\, n - p - 1} . \] The test is conservative, but critical values of this size are what a single extreme residual among \(n\) should be measured against.
Example 4.4.1. In the chemical-yield data of Example 2.2.7, suppose the yield at \(40^\circ \)C had been recorded as \(80\) instead of \(70\):
| Temperature \(x\) | 10 | 20 | 30 | 40 | 50 | 60 |
| Yield \(y\) | 52 | 58 | 63 | 80 | 74 | 81 |
Fit the line and look for an outlier.
Solution. As before, \(\bar x = 35\) and \(S_{xx} = 1750\). Now \(\sum y_i = 408\), \(\bar y = 68\), and \begin {align*} S_{xy} = \sum (x_i - 35)y_i &= -1300 - 870 - 315 + 400 + 1110 + 2025 \\ &= 1050 , \end {align*}
so \(\hat \beta _1 = 1050/1750 = 0.6\), \(\hat \beta _0 = 68 - 0.6 \times 35 = 47\) and \(\hat y = 47 + 0.6x\). By Example 4.1.2, \(h_{ii} = 1/6 + (x_i - 35)^2/1750\).
| \(x_i\) | 10 | 20 | 30 | 40 | 50 | 60 |
| \(\hat y_i\) | 53 | 59 | 65 | 71 | 77 | 83 |
| \(e_i\) | \(-1\) | \(-1\) | \(-2\) | \(9\) | \(-3\) | \(-2\) |
| \(h_{ii}\) | 0.524 | 0.295 | 0.181 | 0.181 | 0.295 | 0.524 |
\(SS_{\mathrm {Res}} = 1 + 1 + 4 + 81 + 9 + 4 = 100\) and \(MS_{\mathrm {Res}} = 100/4 = 25\). For the fourth point, \begin {align*} r_4 &= \frac {9}{\sqrt {25 \times 0.819}} = \frac {9}{4.525} = 1.989, \\ S^2_{(4)} &= \frac {100 - 81/0.819}{3} = \frac {100 - 98.895}{3} = 0.368, \\ t_4 &= \frac {9}{\sqrt {0.368 \times 0.819}} = \frac {9}{0.549} = 16.39 . \end {align*}
The other \(|t_i|\) are all below \(0.7\).
The studentized residual \(r_4 = 1.99\) looks entirely unremarkable. But here \(n - p = 4\), so no \(r_i\) can exceed \(2\) in size: \(r_4\) is as large as a studentized residual can be. R-student shows what is happening. With the fourth point removed, the other five lie very close to a line (\(S^2_{(4)} = 0.37\), against \(MS_{\mathrm {Res}} = 25\) with it), and the fourth point is \(16\) standard errors from that line. The Bonferroni critical value at \(\alpha = 0.05\) is \(t_{0.05/12,\,3} = 6.23\), and \(16.39\) is far beyond it: observation 4 is an outlier. Checking the laboratory record would reveal the transcription error.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.