Solution. (a) \(\sum x_{i1} = \sum x_{i2} = \sum x_{i1}x_{i2} = 0\) and \(\sum x_{i1}^2 = \sum x_{i2}^2 = 4\), so \[ \mathbf {X}^\top \mathbf {X} = \begin {bmatrix} 6 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 4 \end {bmatrix}, \qquad \mathbf {X}^\top \mathbf {y} = \begin {bmatrix} 174 \\ 22 \\ 14 \end {bmatrix} . \] Here \(\sum x_{i1}y_i = -20 + 30 - 26 + 38 = 22\) and \(\sum x_{i2}y_i = -20 - 30 + 26 + 38 = 14\). A diagonal matrix is inverted by inverting its diagonal, so \[ \hat {\boldsymbol \beta } = \begin {bmatrix} 174/6 \\ 22/4 \\ 14/4 \end {bmatrix} = \begin {bmatrix} 29 \\ 5.5 \\ 3.5 \end {bmatrix}, \qquad \hat y = 29 + 5.5x_1 + 3.5x_2 . \] (b) The fitted values are \(20, 31, 27, 38, 29, 29\), and the residuals \(0, -1, -1, 0, 0, 2\). \(SS_{\mathrm {Res}} = 0 + 1 + 1 + 0 + 0 + 4 = 6\) (check: \(\mathbf {y}^\top \mathbf {y} - \hat {\boldsymbol \beta }^\top \mathbf {X}^\top \mathbf {y} = 5222 - 5216 = 6\)), so \(\hat \sigma ^2 = 6/3 = 2\). Then \(\operatorname {se}(\hat \beta _0) = \sqrt {2/6} = 0.577\) and \(\operatorname {se}(\hat \beta _1) = \operatorname {se}(\hat \beta _2) = \sqrt {2/4} = 0.707\).

(c) \(\operatorname {Var}(\hat {\boldsymbol \beta }) = \sigma ^2\operatorname {diag}(1/6, 1/4, 1/4)\) has zeros off the diagonal. The cause is that the columns of \(\mathbf {X}\) are orthogonal: each pair has zero inner product. This is the main reason designed experiments use such patterns of \(x\) values: each effect is estimated separately from the others, and the calculation is trivial.

Solution. \(\mathbf {e} = (\mathbf {I} - \mathbf {H})\mathbf {y} = (\mathbf {I} - \mathbf {H})(\mathbf {X}\boldsymbol \beta + \boldsymbol \varepsilon ) = (\mathbf {I} - \mathbf {H})\boldsymbol \varepsilon \), since \((\mathbf {I} - \mathbf {H})\mathbf {X} = \mathbf {0}\). By Lemma 3.1.3, \(E(\mathbf {e}) = (\mathbf {I} - \mathbf {H})\mathbf {0} = \mathbf {0}\) and \[ \operatorname {Var}(\mathbf {e}) = (\mathbf {I} - \mathbf {H})\sigma ^2\mathbf {I}(\mathbf {I} - \mathbf {H})^\top = \sigma ^2(\mathbf {I} - \mathbf {H})^2 = \sigma ^2(\mathbf {I} - \mathbf {H}) . \] So \(\operatorname {Var}(e_i) = \sigma ^2(1 - h_{ii})\). For plot 6, \(h_{66} = 7/9\), so \(\operatorname {Var}(e_6) = \frac 29\sigma ^2\). Plot 6 has high leverage: the fitted plane is pulled close to it, so its residual is small whatever its error was. Residuals do not all have the same variance, even though the errors do. This is why, in the section on model adequacy, residuals are scaled before they are compared.

Solution. (a) The first entry of \((\mathbf {X}^\top \mathbf {X})^{-1}\mathbf {X}^\top \mathbf {y}\) is \[ \hat \beta _0 = \frac {\sum x_i^2 \sum y_i - \sum x_i \sum x_i y_i}{nS_{xx}} . \] Put \(\sum x_i^2 = S_{xx} + n\bar x^2\) and \(\sum x_i y_i = S_{xy} + n\bar x\bar y\), with \(\sum x_i = n\bar x\) and \(\sum y_i = n\bar y\). The numerator becomes \[ (S_{xx} + n\bar x^2)n\bar y - n\bar x(S_{xy} + n\bar x\bar y) = nS_{xx}\bar y - n\bar x S_{xy}, \] and dividing by \(nS_{xx}\) gives \(\bar y - \bar x S_{xy}/S_{xx} = \bar y - \hat \beta _1\bar x\).

(b) The \((1,1)\) entry is \(\sigma ^2\sum x_i^2/(nS_{xx}) = \sigma ^2(S_{xx} + n\bar x^2)/(nS_{xx}) = \sigma ^2\bigl (\frac 1n + \frac {\bar x^2}{S_{xx}}\bigr )\); the \((2,2)\) entry is \(\sigma ^2 n/(nS_{xx}) = \sigma ^2/S_{xx}\); and the off-diagonal entry is \(-\sigma ^2\sum x_i/(nS_{xx}) = -\sigma ^2 n\bar x/(nS_{xx}) = -\bar x\sigma ^2/S_{xx}\).

Solution. The new hat matrix is \[ \mathbf {XD}\bigl ((\mathbf {XD})^\top \mathbf {XD}\bigr )^{-1}(\mathbf {XD})^\top = \mathbf {XD}\,\mathbf {D}^{-1}(\mathbf {X}^\top \mathbf {X})^{-1}(\mathbf {D}^\top )^{-1}\mathbf {D}^\top \mathbf {X}^\top = \mathbf {H}, \] using \((\mathbf {ABC})^{-1} = \mathbf {C}^{-1}\mathbf {B}^{-1}\mathbf {A}^{-1}\). So \(\hat {\mathbf {y}} = \mathbf {Hy}\) and \(\mathbf {e} = (\mathbf {I} - \mathbf {H})\mathbf {y}\) are unchanged. The new coefficients are \[ \bigl ((\mathbf {XD})^\top \mathbf {XD}\bigr )^{-1}(\mathbf {XD})^\top \mathbf {y} = \mathbf {D}^{-1}(\mathbf {X}^\top \mathbf {X})^{-1}\mathbf {X}^\top \mathbf {y} = \mathbf {D}^{-1}\hat {\boldsymbol \beta } . \] Measuring fertiliser in kg instead of 100 kg multiplies its column by \(100\) and divides its coefficient by \(100\); the fit itself is untouched.

Solution. \(\mathbf {y}^\top \bar {\mathbf {J}}\mathbf {y} = \frac 1n(\mathbf {1}^\top \mathbf {y})^2 = \frac 1n\left (\sum y_i\right )^2 = n\bar y^2\), so \(\mathbf {y}^\top (\mathbf {I} - \bar {\mathbf {J}})\mathbf {y} = \sum y_i^2 - n\bar y^2 = \sum (y_i - \bar y)^2\) by Lemma 2.2.3. Then, writing \(\mathbf {I} - \bar {\mathbf {J}} = (\mathbf {H} - \bar {\mathbf {J}}) + (\mathbf {I} - \mathbf {H})\), \[ SS_T = \mathbf {y}^\top (\mathbf {H} - \bar {\mathbf {J}})\mathbf {y} + \mathbf {y}^\top (\mathbf {I} - \mathbf {H})\mathbf {y} = SS_R + SS_{\mathrm {Res}} , \] using the forms of \(SS_R\) and \(SS_{\mathrm {Res}}\) found in the proofs of Theorems 3.8.1 and 3.6.1.

Solution. From Problem 3.13.1: \(\hat y = 29 + 5.5x_1 + 3.5x_2\), \(SS_{\mathrm {Res}} = 6\), \(\hat \sigma ^2 = 2\) with \(3\) degrees of freedom, \(\mathbf {C} = \operatorname {diag}(1/6, 1/4, 1/4)\), \(\mathbf {y}^\top \mathbf {y} = 5222\) and \(\hat {\boldsymbol \beta }^\top \mathbf {X}^\top \mathbf {y} = 5216\).

(a) \((\sum y_i)^2/n = 174^2/6 = 5046\), so \(SS_T = 5222 - 5046 = 176\) and \(SS_R = 5216 - 5046 = 170\).

\(42.5 > F_{0.05,2,3} = 9.55\): reject \(H_0\colon \beta _1 = \beta _2 = 0\).

(b) \(t_1 = 5.5/0.7071 = 7.78\) and \(t_2 = 3.5/0.7071 = 4.95\). Both exceed \(t_{0.025,3} = 3.182\), so both regressors are significant.

(c) \(R^2 = 170/176 = 0.966\), and \(R^2_{\mathrm {adj}} = 1 - \dfrac {2}{176/5} = 1 - 0.0568 = 0.943\).

(d) For the model with \(x_1\) alone, \(\mathbf {X}_1\) has orthogonal columns too, and \(\hat \beta _1 = 22/4 = 5.5\) as before, so \(SS_R(\beta _1) = \hat \beta _1 \sum x_{i1}y_i = 5.5 \times 22 = 121\). Then \(SS_R(\beta _2 \mid \beta _1) = 170 - 121 = 49\). In the same way \(SS_R(\beta _2) = 3.5 \times 14 = 49\), so \(SS_R(\beta _1 \mid \beta _2) = 170 - 49 = 121\). Each regressor contributes the same amount whatever else is in the model, because with orthogonal columns the estimate of one coefficient does not change when another regressor is added or removed. In the crop data, fertiliser and rainfall are correlated, and they share credit for the yield.

(e) \(\mathbf {x}_0 = (1, 0.5, 0.5)^\top \), \(\hat y_0 = 29 + 2.75 + 1.75 = 33.5\), and \(\mathbf {x}_0^\top \mathbf {C}\mathbf {x}_0 = \frac 16 + \frac {0.25}{4} + \frac {0.25}{4} = 0.2917\).

• Mean: \(33.5 \pm 3.182\sqrt {2 \times 0.2917} = 33.5 \pm 2.431\), that is, \((31.07,\ 35.93)\).
• New observation: \(33.5 \pm 3.182\sqrt {2 \times 1.2917} = 33.5 \pm 5.115\), that is, \((28.38,\ 38.62)\).

Solution. \(H_0\colon \beta _1 = 1.5\) against \(H_1\colon \beta _1 > 1.5\). \(t_0 = (2 - 1.5)/0.3412 = 1.466\). The test is one-sided with \(3\) degrees of freedom, so the critical value is \(t_{0.05,3} = 2.353\). As \(1.466 < 2.353\), do not reject \(H_0\): six plots are not enough to establish the claim, although the estimate \(\hat \beta _1 = 2\) is in its favour.

Solution. (a) \(R^2_{\mathrm {adj}} = 1 - \dfrac {SS_{\mathrm {Res}}/(n-p)}{SS_T/(n-1)} = 1 - \dfrac {n-1}{n-p}\cdot \dfrac {SS_{\mathrm {Res}}}{SS_T}\) and \(SS_{\mathrm {Res}}/SS_T = 1 - R^2\). Since \(\frac {n-1}{n-p} \ge 1\), we subtract at least \(1 - R^2\) from \(1\), so \(R^2_{\mathrm {adj}} \le R^2\). With \(n = 6\), \(p = 3\) and \(R^2 = 0.1\), \(R^2_{\mathrm {adj}} = 1 - \frac 53 \times 0.9 = -0.5\): a weak model with many parameters scores below zero.

(b) With \(x_1\) alone (Example 3.11.3), \(SS_{\mathrm {Res}} = 7.9375\) on \(4\) degrees of freedom, so \(R^2 = 85.5625/93.5 = 0.915\) and \(R^2_{\mathrm {adj}} = 1 - \frac 54(0.0849) = 0.894\). With both, \(R^2 = 0.957\) and \(R^2_{\mathrm {adj}} = 0.929\). Both measures prefer the larger model here, even though the \(t\)-test did not find \(x_2\) significant.

In fact \(R^2_{\mathrm {adj}}\) rises exactly when the partial \(F\) statistic for the added term exceeds \(1\). Adding one term to a model with \(p\) parameters takes \(SS_{\mathrm {Res}}\) from \(S_0\) to \(S_1\), and \(R^2_{\mathrm {adj}}\) rises exactly when the residual mean square falls: \begin {align*} \frac {S_1}{n-p-1} < \frac {S_0}{n-p} &\iff (n-p)S_1 < (n-p-1)S_0 \\ &\iff S_1 < (n-p-1)(S_0 - S_1) \\ &\iff F_0 = \frac {S_0 - S_1}{S_1/(n-p-1)} > 1 . \end {align*}

Here \(F_0 = 2.95 > 1\). That is a much weaker requirement than significance, which needed \(F_0 > 10.13\).