3.12 Confidence and prediction intervals

Theorem 3.12.1. Under A1–A4, \(100(1 - \alpha )\%\) intervals are:

(i): for a coefficient \(\beta _j\): \(\hat \beta _j \pm t_{\alpha /2, n-p}\sqrt {\hat \sigma ^2 C_{jj}}\);
(ii): for the mean response at \(\mathbf {x}_0 = (1, x_{01}, \dots , x_{0k})^\top \): \[ \hat y_0 \pm t_{\alpha /2, n-p}\sqrt {\hat \sigma ^2\, \mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0}, \qquad \hat y_0 = \mathbf {x}_0^\top \hat {\boldsymbol \beta } ; \]
(iii): for a new observation at \(\mathbf {x}_0\): \[ \hat y_0 \pm t_{\alpha /2, n-p}\sqrt {\hat \sigma ^2 \left (1 + \mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0\right )} . \]

Proof. (i) follows from Theorem 3.10.1 exactly as in simple regression.

(ii) \(\hat y_0 = \mathbf {x}_0^\top \hat {\boldsymbol \beta }\) is normal with mean \(\mathbf {x}_0^\top \boldsymbol \beta = E(y \mid \mathbf {x}_0)\) and, by Lemma 3.1.3, variance \(\mathbf {x}_0^\top (\sigma ^2\mathbf {C})\mathbf {x}_0\). It is a function of \(\hat {\boldsymbol \beta }\), so is independent of \(SS_{\mathrm {Res}}\), and the \(t\) argument applies.

(iii) A new observation \(y_0 = \mathbf {x}_0^\top \boldsymbol \beta + \varepsilon _0\) is independent of the sample, so \(y_0 - \hat y_0\) has mean \(0\) and variance \(\sigma ^2 + \sigma ^2\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0\), and the \(t\) argument applies again. □

With \(k = 1\), \(\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0 = \frac 1n + \frac {(x_0 - \bar x)^2}{S_{xx}}\), recovering Theorems 2.11.3 and 2.11.4. Note that \(\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0\) is the leverage that the point \(\mathbf {x}_0\) would have if it were added to the data.

Example 3.12.2. For the crop data, find 95% confidence intervals for \(\beta _1\) and \(\beta _2\). Then, for a plot with \(300\) kg of fertiliser per hectare and \(300\) mm of rain, find a 95% confidence interval for the mean yield and a 95% prediction interval for the yield of a single plot.

Solution. With \(t_{0.025,3} = 3.182\): \begin {align*} \beta _1&\colon \ 2 \pm 3.182 \times 0.3412 = 2 \pm 1.086, \quad \text {that is, } (0.914,\ 3.086); \\ \beta _2&\colon \ 1 \pm 3.182 \times 0.5819 = 1 \pm 1.852, \quad \text {that is, } (-0.852,\ 2.852). \end {align*}

The interval for \(\beta _2\) contains \(0\), matching the \(t\)-test.

For the new plot, \(\mathbf {x}_0 = (1, 3, 3)^\top \) and \(\hat y_0 = 3 + 2(3) + 3 = 12\). Next, \[ \mathbf {C}\mathbf {x}_0 = \frac {1}{378}\begin {bmatrix} 510 - 72 - 450 \\ -24 + 99 - 90 \\ -150 - 90 + 288 \end {bmatrix} = \frac {1}{378}\begin {bmatrix} -12 \\ -15 \\ 48 \end {bmatrix}, \] so \[ \mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0 = \frac {1}{378}\bigl (-12 + 3(-15) + 3(48)\bigr ) = \frac {87}{378} = 0.2302 . \]

Mean yield: \(12 \pm 3.182\sqrt {1.3333 \times 0.2302} = 12 \pm 3.182 \times 0.5540 = 12 \pm 1.763\), that is, \((10.24,\ 13.76)\) tonnes per hectare.
A single plot: \(12 \pm 3.182\sqrt {1.3333 \times 1.2302} = 12 \pm 3.182 \times 1.2807 = 12 \pm 4.076\), that is, \((7.92,\ 16.08)\) tonnes per hectare.

Remark (Hidden extrapolation). With several regressors it is easy to extrapolate without noticing. A point can lie inside the range of each regressor separately and still be far from the data jointly: here, a plot with \(x_1 = 6\) and \(x_2 = 1\) is within both ranges, but no plot combined so much fertiliser with so little rain, and indeed \(\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0 = 846/378 = 2.24\) there. A useful check is to compare \(\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0\) with the largest leverage in the data, \(h_{\max }\); if it is larger, the prediction is an extrapolation. For \(\mathbf {x}_0 = (1, 3, 3)^\top \), \(0.2302 < h_{\max } = 0.778\), so the intervals above are safe.

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.