Solution. This is Theorem 9.2.1. (i) \(f(u) = (2\pi \sigma ^2)^{-n/2}\exp (-u'u/2\sigma ^2)\). (ii) The Jacobian of \(u = y - X\beta \) is \(1\), so \(f(y \mid X) = (2\pi \sigma ^2)^{-n/2}\exp \{-(y - X\beta )'(y - X\beta )/2\sigma ^2\}\). (iii) \(\ell = -\frac n2\log 2\pi - \frac n2\log \sigma ^2 - \frac {1}{2\sigma ^2}(y - X\beta )'(y - X\beta )\). (iv) \(\hat \beta = (X'X)^{-1}X'y\) and \(\hat \sigma ^2 = e'e/n\). (v) \(I(\theta ) = \begin {pmatrix} X'X/\sigma ^2 & 0 \\ 0 & n/(2\sigma ^4) \end {pmatrix}\), from the second derivatives \(-X'X/\sigma ^2\), \(-X'u/\sigma ^4\) (expectation zero) and \(n/(2\sigma ^4) - u'u/\sigma ^6\) (expectation \(-n/(2\sigma ^4)\)).

Solution. (i) \(\Cov (PU) = P\Cov (U)P' = \sigma ^2P\Omega P'\). From \(P'P = \Omega ^{-1}\), \(\Omega = P^{-1}(P')^{-1}\), so \(P\Omega P' = PP^{-1}(P')^{-1}P' = I\).

(ii) The transformed model satisfies the assumptions of the previous problem, so its MLE is least squares on the transformed data: \(\hat \beta = (X_*'X_*)^{-1}X_*'Y_* = (X'P'PX)^{-1}X'P'PY = (X'\Omega ^{-1}X)^{-1}X'\Omega ^{-1}Y\), the GLS estimator.

Solution. (a) \(\Omega = \operatorname {diag}(X_i)\), so \(b_{\text {GLS}} = (X'\Omega ^{-1}X)^{-1}X'\Omega ^{-1}Y = \bigl (\sum X_i^2/X_i\bigr )^{-1}\sum X_iY_i/X_i = \sum Y_i/\sum X_i = \bar Y/\bar X\), with variance \(\sigma ^2(X'\Omega ^{-1}X)^{-1} = \sigma ^2/\sum X_i\).

(b) By Theorem 9.4.1, \(\Var (b) = \sigma ^2\sum X_i^2X_i/(\sum X_i^2)^2 = \sigma ^2\sum X_i^3/(\sum X_i^2)^2\). GLS is at least as precise if \((\sum X_i^2)^2 \leq \sum X_i\sum X_i^3\), which is the Cauchy–Schwarz inequality applied to the vectors \((X_i^{1/2})\) and \((X_i^{3/2})\). Equality holds only if all \(X_i\) are equal. The GLS estimator is the familiar ratio estimator of survey sampling, which is optimal when the variance is proportional to the size of the unit.

Solution. \(F = (18/3)/(120/55) = 6/2.182 = 2.75\), against \(F_{0.05,3,55} = 2.77\): not rejected, just. With \(x = 18/120 = 0.15\), \(W = 60(0.15) = 9.00\), \(\text {LR} = 60\log 1.15 = 8.39\) and \(\text {LM} = 60(0.15/1.15) = 7.83\). The \(5\%\) point of \(\chi ^2_3\) is \(7.81\), so all three asymptotic tests reject, LM only barely. The example shows the ordering \(W > \text {LR} > \text {LM}\) and also why the exact \(F\) test is preferred when it is available: the asymptotic tests ignore the degrees-of-freedom correction and reject a little too readily in small samples.

Solution. \(\ell (\lambda ) = \sum y_i\log \lambda - n\lambda - \sum \log y_i!\). The score \(\sum y_i/\lambda - n = 0\) gives \(\hat \lambda = \bar y\). The second derivative is \(-\sum y_i/\lambda ^2\), with expectation \(-n/\lambda \), so \(I(\lambda ) = n/\lambda \) and the bound is \(\lambda /n\). \(\hat \lambda = \bar y\) is unbiased with variance \(\Var (y)/n = \lambda /n\), so it attains the bound exactly, in every sample size. Such cases, where an unbiased estimator attains the bound in finite samples, occur precisely when the score is linear in the estimator, here \(s = (n/\lambda )(\bar y - \lambda )\).

Solution. \(e'e/\sigma ^2 \sim \chi ^2_{n-k}\) with mean \(n - k\) and variance \(2(n - k)\). So \(E(\hat \sigma ^2) = \frac {n-k}{n}\sigma ^2\), a bias of \(-k\sigma ^2/n\), and \(\Var (\hat \sigma ^2) = 2(n - k)\sigma ^4/n^2\). The mean squared error is the variance plus the squared bias, \(\sigma ^4\bigl (2(n - k) + k^2\bigr )/n^2\). For \(s^2\), unbiased, it is \(2\sigma ^4/(n - k)\). With \(n = 50\) and \(k = 5\): \((90 + 25)/2500 = 0.046\) against \(2/45 = 0.044\). So \(s^2\) is slightly better here; but with \(k = 1\) the MLE wins (\(0.0396\) against \(0.0408\)). Neither dominates, and the estimator \(e'e/(n - k + 2)\) has the smallest mean squared error of all estimators of the form \(e'e/c\).