9.5 Practice problems
Problem 9.5.1. In Example 9.2.3, suppose the indicator is defined the other way round, \(w = 1\) for variety \(A\) and \(0\) for variety \(B\). Write down the fitted parallel-lines model without refitting, and show in general that the fitted values do not depend on which level is the reference.
Show solution
Solution. \(w = 1 - z\), so \(9.1 + 2.1x + 3.0z = 9.1 + 2.1x + 3.0(1 - w) = 12.1 + 2.1x - 3.0w\). In general, the columns \(\mathbf {1}, \mathbf {x}, \mathbf {z}\) and \(\mathbf {1}, \mathbf {x}, \mathbf {1} - \mathbf {z}\) span the same space, and the least-squares fitted values are the projection of \(\mathbf {y}\) onto that space, whatever basis describes it. Only the meaning of the coefficients changes: the intercept becomes the mean for the new reference level, and the indicator coefficient changes sign.
Problem 9.5.2. For a factor with \(a\) levels, show that the model \(y = \gamma _1z_1 + \cdots + \gamma _az_a + \varepsilon \), with an indicator for every level and no intercept, has full column rank, and that \(\hat \gamma _i = \bar y_i\). How is it related to the model with an intercept and \(a - 1\) indicators?
Show solution
Solution. The columns of the \(a\) indicators are non-zero and have no \(1\) in a common row, so they are orthogonal and hence linearly independent. \(\mathbf {X}^\top \mathbf {X} = \operatorname {diag}(n_1, \dots , n_a)\) and \(\mathbf {X}^\top \mathbf {y}\) has entries \(\sum _j y_{ij} = n_i\bar y_i\), so \(\hat \gamma _i = \bar y_i\). It spans the same column space as the model with an intercept and \(a - 1\) indicators (Theorem 9.3.1), so it gives the same fit, reparametrised: \(\gamma _A = \beta _0\), and \(\gamma _i = \beta _0 + \beta _i\) for the other levels. This cell-means form avoids the dummy-variable trap by dropping the intercept instead of an indicator. Note that its \(R^2\), computed by software as for a model without an intercept, is not comparable with the usual \(R^2\).
Problem 9.5.3. A model for weekly earnings \(y\) (kwacha) of traders, with \(x\) years of experience and \(z = 1\) for traders in a city market and \(0\) for a rural market, was fitted as \[ \hat y = 420 + 35x + 150z + 20xz . \]
- (a)
- Write down the fitted lines for the two markets.
- (b)
- What is the estimated difference between city and rural earnings for a trader with \(5\) years’ experience? With none?
- (c)
- Why would it be misleading to report “city traders earn \(150\) kwacha more”?
Show solution
Solution. (a) Rural (\(z = 0\)): \(\hat y = 420 + 35x\). City (\(z = 1\)): \(\hat y = 570 + 55x\).
(b) The difference is \(150 + 20x\): \(250\) kwacha at \(x = 5\), and \(150\) at \(x = 0\).
(c) With an interaction, \(\beta _2 = 150\) is the difference only at \(x = 0\), for traders with no experience. The gap grows by \(20\) kwacha with each year of experience, so there is no single difference to report; it must be given as a function of \(x\), or at stated values of \(x\) that are typical of the data.
Problem 9.5.4. The output \(y\) of a process was measured at eight settings, and the engineers believe the response steepens beyond \(x = 4\):
| \(x\) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| \(y\) | 3 | 4 | 6 | 6 | 9 | 11 | 15 | 17 |
Fit \(y = \beta _0 + \beta _1x + \beta _2(x - 4)_+ + \varepsilon \) and test whether the slope changes. The straight line \(y = \beta _0 + \beta _1x + \varepsilon \) has \(SS_{\mathrm {Res}} = 8.821\).
Show solution
Solution. The values of \((x - 4)_+\) are \(0, 0, 0, 0, 1, 2, 3, 4\). Then \[ \mathbf {X}^\top \mathbf {X} = \begin {bmatrix} 8 & 36 & 10 \\ 36 & 204 & 70 \\ 10 & 70 & 30 \end {bmatrix} , \qquad \mathbf {X}^\top \mathbf {y} = \begin {bmatrix} 71 \\ 405 \\ 144 \end {bmatrix} , \] with \(\det (\mathbf {X}^\top \mathbf {X}) = 880\). By cofactors, as in Example 3.3.3, \[ (\mathbf {X}^\top \mathbf {X})^{-1} = \frac {1}{880}\begin {bmatrix} 1220 & -380 & 480 \\ -380 & 140 & -200 \\ 480 & -200 & 336 \end {bmatrix} , \] and \[ \hat {\boldsymbol \beta } = \frac {1}{880}\begin {bmatrix} 1220(71) - 380(405) + 480(144) \\ -380(71) + 140(405) - 200(144) \\ 480(71) - 200(405) + 336(144) \end {bmatrix} = \frac {1}{880}\begin {bmatrix} 1840 \\ 920 \\ 1464 \end {bmatrix} = \begin {bmatrix} 2.091 \\ 1.045 \\ 1.664 \end {bmatrix} . \] So the slope is \(1.045\) up to \(x = 4\) and \(1.045 + 1.664 = 2.709\) beyond it. The residual sum of squares is \(1.573\) on \(5\) degrees of freedom, and \[ F = \frac {8.821 - 1.573}{1.573/5} = \frac {7.248}{0.3145} = 23.0 > F_{0.05,1,5} = 6.61 . \] The slope does change at \(x = 4\): the response rises nearly three times as fast beyond it.
Problem 9.5.5. A model is to compare three schools, allowing for each learner’s previous score \(x\), with possibly different slopes in each school. How many parameters does it have, and what are the regressors? Which test examines whether the slopes are equal, and on how many degrees of freedom?
Show solution
Solution. With indicators \(z_1\), \(z_2\) for two of the schools, the regressors are \(x\), \(z_1\), \(z_2\), \(xz_1\) and \(xz_2\), and with the intercept there are \(6\) parameters: an intercept and slope for each school. Equal slopes is \(H_0\colon \beta _{xz_1} = \beta _{xz_2} = 0\), a partial \(F\) comparing this model with the parallel-lines model in \(x\), \(z_1\), \(z_2\), on \(2\) and \(n - 6\) degrees of freedom.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.