6.5 Practice problems
- (a)
- Use the delta method to show that if \(y\) is Poisson with mean \(\mu \), then \(\sqrt {y}\) has variance approximately \(\frac 14\), whatever \(\mu \).
- (b)
- Find a transformation that stabilises the variance when \(\sigma ^2(\mu ) = \mu + \mu ^2\) (counts with extra variation).
Show solution
Solution. (a) \(g(y) = \sqrt y\) has \(g'(\mu ) = 1/(2\sqrt \mu )\), so by Lemma 6.1.1 \(\operatorname {Var}(\sqrt y) \approx \mu /(4\mu ) = \frac 14\).
(b) We need \(g'(\mu ) = 1/\sqrt {\mu + \mu ^2}\). Substituting \(\mu = u^2\), \(d\mu = 2u\,du\), \[ g(\mu ) = \int \frac {d\mu }{\sqrt {\mu (1 + \mu )}} = \int \frac {2u\,du}{u\sqrt {1 + u^2}} = 2\operatorname {arsinh}u = 2\operatorname {arsinh}\sqrt \mu . \] So \(g(y) = \operatorname {arsinh}\sqrt {y} = \ln \bigl (\sqrt y + \sqrt {1 + y}\bigr )\). For small \(\mu \) it behaves like \(\sqrt y\), and for large \(\mu \) like \(\frac 12\ln y + \ln 2\): between the square root and the log, as the variance function is between Poisson and constant-percentage.
- (a)
- Show that \((y^\lambda - 1)/\lambda \to \ln y\) as \(\lambda \to 0\), so that the Box–Cox transformation is continuous in \(\lambda \).
- (b)
- Show that \(SS_{\mathrm {Res}}(1)\) equals the residual sum of squares of the untransformed fit, and that for any \(\lambda \neq 0\), \(SS_{\mathrm {Res}}(\lambda )\) is \(1/(\lambda \dot y^{\lambda - 1})^2\) times the residual sum of squares from regressing \(y^\lambda \) itself.
Show solution
Solution. (a) Write \(y^\lambda = e^{\lambda \ln y}\). The limit is \(0/0\), and by l’Hôpital’s rule (differentiating in \(\lambda \)) it equals \(\lim _{\lambda \to 0}\ln y\,e^{\lambda \ln y} = \ln y\). Since also \(\dot y^{\lambda - 1} \to \dot y^{-1}\), \(y^{(\lambda )} \to \ln y/\dot y^{-1} = \dot y\ln y = y^{(0)}\).
(b) \(y^{(1)} = y - 1\). Subtracting a constant from every response changes only the intercept, so the residuals, and \(SS_{\mathrm {Res}}\), are unchanged. In general \(y^{(\lambda )} = a y^\lambda + b\) with \(a = 1/(\lambda \dot y^{\lambda - 1})\) and \(b = -a\). Adding \(b\) leaves the residuals alone, and multiplying the response by \(a\) multiplies \(\hat {\boldsymbol \beta }\), the fitted values and the residuals by \(a\) (the least-squares estimator is linear in \(\mathbf {y}\)), so \(SS_{\mathrm {Res}}\) is multiplied by \(a^2\).
Problem 6.5.3. The time \(y\) (minutes) to dry a coating of thickness \(x\) (mm) was measured:
| \(x\) | 1 | 2 | 4 | 8 | 16 |
| \(y\) | 3 | 4 | 6 | 9 | 12 |
Fit the power model \(y = \beta _0 x^{\beta _1}\varepsilon \) and interpret \(\beta _1\).
Show solution
Solution. Let \(u = \ln x\) and \(v = \ln y\):
| \(u = \ln x\) | 0 | 0.6931 | 1.3863 | 2.0794 | 2.7726 |
| \(v = \ln y\) | 1.0986 | 1.3863 | 1.7918 | 2.1972 | 2.4849 |
The \(u_i\) are \(0\), \(1\), \(2\), \(3\), \(4\) times \(\ln 2 = 0.6931\), so \(\bar u = 2\ln 2 = 1.3863\) and \(S_{uu} = 10(\ln 2)^2 = 4.8045\). Then \(\bar v = 8.9588/5 = 1.79176\) and \begin {align*} S_{uv} &= \ln 2\,\bigl [-2(1.0986) - 1(1.3863) + 0 + 1(2.1972) + 2(2.4849)\bigr ] \\ &= 0.6931 \times 3.5835 = 2.4839 , \end {align*}
so \(\hat \beta _1 = 2.4839/4.8045 = 0.517\), and \(\widehat {\ln \beta _0} = 1.79176 - 0.517 \times 1.3863 = 1.0751\), giving \(\hat \beta _0 = e^{1.0751} = 2.93\): \[ \hat y = 2.93\,x^{0.517} . \] \(\beta _1\) is an elasticity: a \(1\%\) increase in thickness goes with an increase of about \(\beta _1\% = 0.52\%\) in drying time; doubling the thickness multiplies the drying time by \(2^{0.517} = 1.43\). The estimate is close to \(\frac 12\), which would mean drying time proportional to the square root of thickness, a hypothesis that could be tested with the \(t\) statistic for \(\beta _1 = 0.5\).
Problem 6.5.4. Suppose \(y_i = \beta _0 + \beta _1 x_i + \varepsilon _i\) with \(x_i > 0\) and \(\operatorname {Var}(\varepsilon _i) = \sigma ^2x_i^2\).
- (a)
- Show that weighted least squares with \(w_i = 1/x_i^2\) is the same as ordinary least squares for the model \(y_i/x_i = \beta _1 + \beta _0(1/x_i) + \delta _i\), in which the roles of intercept and slope are exchanged, and that the \(\delta _i\) have constant variance.
- (b)
- Fit both ways to the data \(x = 1, 2, 3, 4\), \(y = 2, 4, 6, 11\), and compare with ordinary least squares on the original data.
Show solution
Solution. (a) Dividing the model by \(x_i\), which is multiplying by \(\sqrt {w_i} = 1/x_i\), gives \(y_i/x_i = \beta _0/x_i + \beta _1 + \varepsilon _i/x_i\), and \(\operatorname {Var}(\varepsilon _i/x_i) = \sigma ^2\). This is the transformed model in the proof of Theorem 6.4.1: its regressor columns are \(1/x_i\) (from the intercept) and \(x_i/x_i = 1\) (from the slope). So the old slope \(\beta _1\) is the new intercept, and the old intercept \(\beta _0\) the new slope.
(b) Let \(u = 1/x\) and \(v = y/x\): \(u = 1, \frac 12, \frac 13, \frac 14\) and \(v = 2, 2, 2, 2.75\). Then \(\bar u = 25/48 = 0.52083\), \(\bar v = 2.1875\), \begin {align*} S_{uu} &= \textstyle \sum u_i^2 - 4\bar u^2 = 1.42361 - 1.08507 = 0.33854, \\ S_{uv} &= \textstyle \sum (u_i - \bar u)v_i \\ &= 0.95833 - 0.04167 - 0.37500 - 0.74479 = -0.20313 , \end {align*}
so the new slope is \(-0.20313/0.33854 = -0.600\) and the new intercept \(2.1875 + 0.600 \times 0.52083 = 2.500\). Hence \(\hat \beta _0 = -0.6\), \(\hat \beta _1 = 2.5\): \[ \hat y_W = -0.6 + 2.5x . \] Corollary 6.4.2 gives the same: with \(w = 1, \frac 14, \frac 19, \frac 1{16}\) the weighted means are \(\bar x_w = 2.0833/1.4236 = 1.4634\) and \(\bar y_w = 4.3542/1.4236 = 3.0585\), and the slope is \(2.5\). Ordinary least squares gives \(\bar x = 2.5\), \(\bar y = 5.75\), \(S_{xx} = 5\) and \(S_{xy} = 14.5\), so \(\hat y = -1.5 + 2.9x\). The last point, \((4, 11)\), has the largest variance under the model; ordinary least squares lets it raise the slope, while the weighted fit discounts it.
- (a)
- Show that the weighted residuals \(e_i = y_i - \hat y_i\) of a weighted fit with an intercept satisfy \(\sum w_i e_i = 0\) and \(\sum w_i x_{ij}e_i = 0\), but in general \(\sum e_i \neq 0\).
- (b)
- Check this on Example 6.4.4.
Show solution
Solution. (a) The normal equations of Theorem 6.4.1 are \(\mathbf {X}^\top \mathbf {W}(\mathbf {y} - \mathbf {X}\hat {\boldsymbol \beta }_W) = \mathbf {0}\), whose rows are \(\sum w_i x_{ij}e_i = 0\), the first (\(x_{i0} = 1\)) being \(\sum w_i e_i = 0\). These are weighted sums; the unweighted sum \(\sum e_i\) is not constrained.
(b) The residuals are \(-2\), \(1\), \(0\), \(-1\), \(2\) with weights \(1\), \(4\), \(6\), \(4\), \(1\): \(\sum w_ie_i = -2 + 4 + 0 - 4 + 2 = 0\) and \(\sum w_ix_ie_i = -2 + 8 + 0 - 16 + 10 = 0\). Here \(\sum e_i = 0\) too, by the symmetry of the design, but with the data of Problem 6.5.4 the residuals from \(-0.6 + 2.5x\) are \(0.1\), \(-0.4\), \(-0.9\), \(1.6\), which add to \(0.4\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.