Solution. (a) \((\sigma ^2I)^{-1} = \sigma ^{-2}I\), so \(X'(\sigma ^2I)^{-1}X = \sum _{i=1}^k(X_i/\sigma )^2\), a sum of squares of \(k\) independent \(N(0, 1)\) variables: \(\chi ^2_k\).

(b) \(\chi ^2_k\). Let \(Z = \Sigma ^{-1/2}X\), where \(\Sigma ^{-1/2} = Q\Lambda ^{-1/2}Q'\) from the spectral decomposition \(\Sigma = Q\Lambda Q'\). Then \(Z \sim N(0, \Sigma ^{-1/2}\Sigma \Sigma ^{-1/2}) = N(0, I_k)\) and \(X'\Sigma ^{-1}X = Z'Z \sim \chi ^2_k\) (Theorem 5.4.2).

Solution. See Theorem 5.4.3: diagonalise \(A = Q\Lambda Q'\) with \(\Lambda \) having \(r\) ones, and \(X'AX = \sum _{i\leq r}Y_i^2\) with \(Y = Q'X \sim N(0, I)\). For the example, \(A^2 = A\) and \(\tr A = 1\), so \(r = 1\). Directly, \(X'AX = \frac 12(X_1 + X_2)^2 = \bigl ((X_1 + X_2)/\sqrt 2\bigr )^2\), and \((X_1 + X_2)/\sqrt 2 \sim N(0, 1)\), so \(X'AX \sim \chi ^2_1\). The eigenvector for eigenvalue \(1\) is \((1, 1)'/\sqrt 2\), which is exactly the combination that appears.

Solution. By Theorem 5.2.1(iii), \(E(X'BX) = \tr (B\Sigma ) + \mu 'B\mu \). Here \(B\Sigma = \begin {pmatrix} 2 & 1 \\ 2 & 6 \end {pmatrix}\) with trace \(8\), and \(\mu 'B\mu = 1 + 2(4) = 9\), so \(E(X'BX) = 17\). Directly, \(E(X_1^2) = \Var (X_1) + \mu _1^2 = 2 + 1 = 3\) and \(E(X_2^2) = 3 + 4 = 7\), so \(E(X_1^2 + 2X_2^2) = 3 + 14 = 17\). Note that the covariance \(\Sigma _{12} = 1\) does not enter, because \(B\) is diagonal.

Solution. (a) \(|\Omega | = 0.75\), so \(\Omega ^{-1} = \frac {1}{0.75}\begin {pmatrix} 1 & -0.5 \\ -0.5 & 1 \end {pmatrix} = \begin {pmatrix} \frac 43 & -\frac 23 \\ -\frac 23 & \frac 43 \end {pmatrix}\). Write \(P = \begin {pmatrix} p & q \\ 0 & r \end {pmatrix}\); then \(P'P = \begin {pmatrix} p^2 & pq \\ pq & q^2 + r^2 \end {pmatrix}\). Matching, \(p = 2/\sqrt 3 = 1.1547\), \(q = -\frac 23/p = -1/\sqrt 3 = -0.5774\) and \(r^2 = \frac 43 - \frac 13 = 1\), so \(r = 1\). (This is the Cholesky factor that R’s chol(solve(Omega)) returns.)

(b) Multiplying out gives \(P\Omega P' = I\), as Corollary 5.1.4(iii) guarantees for any non-singular \(P\) with \(P'P = \Omega ^{-1}\). If \(\Var (U) = \sigma ^2\Omega \), then \(\Var (PU) = \sigma ^2P\Omega P' = \sigma ^2I\): the transformed disturbances are uncorrelated with constant variance. Premultiplying the whole regression by \(P\) therefore produces a model that satisfies the classical assumptions. That is generalised least squares, developed with maximum likelihood.

Solution. (a) This is Example 5.4.5.

(b) With \(\mu = 0\), \(X/\sigma \sim N(0, I)\). \(I - M_0 = \frac 1n\iota \iota '\) has rank \(1\), \(M_0\) has rank \(n - 1\), they sum to \(I\) and the ranks sum to \(n\). By Cochran’s theorem \(X'(I - M_0)X/\sigma ^2 = n\bar X^2/\sigma ^2 \sim \chi ^2_1\) and \(X'M_0X/\sigma ^2 \sim \chi ^2_{n-1}\), independently. Their ratio, scaled by the degrees of freedom, \(n\bar X^2/S^2\), is \(F_{1,n-1}\): the square of the \(t\) statistic for \(\mu = 0\).

Solution. (a) \(p_{ii} = \sum _jp_{ij}^2\) because \(P = P^2 = PP'\), so \(p_{ii} = p_{ii}^2 + \sum _{j\neq i}p_{ij}^2 \geq p_{ii}^2\), giving \(0 \leq p_{ii} \leq 1\). The sum is \(\tr P = k\).

(b) \(P\) is unchanged if \(X\) is replaced by \(XT\) for any non-singular \(T\), since \(XT(T'X'XT)^{-1}T'X' = P\). Use the columns \(\iota \) and \(x = X - \bar X\iota \), which are orthogonal. Then \(X'X = \operatorname {diag}(n, \sum x_j^2)\) and \(P = \frac 1n\iota \iota ' + xx'/\sum x_j^2\), whose \(i\)th diagonal element is the formula.

(c) The average is \(k/n\). \(p_{ii}\) is the leverage of observation \(i\): by (b), it is large when \(X_i\) is far from \(\bar X\), and an observation with large leverage pulls the fitted line towards itself. Compare the prediction variance of Theorem 2.5.1, which at \(X_0 = X_i\) is \(\sigma ^2p_{ii}\).