Proof. (i) \(E(a'Y) = a'X\beta \). This equals \(c'\beta \) for every \(\beta \) if and only if \(a'X = c'\), that is \(X'a = c\).

(ii) \(\Var (a'Y) = a'\Var (Y)a = \sigma ^2a'a\).

(iii) Form the Lagrangian \(\mathcal L = a'a - 2\lambda '(X'a - c)\), with a \(k\)-vector of multipliers \(\lambda \) (the factor \(2\) is for convenience). By Lemma 5.1.8, \(\partial \mathcal L/\partial a = 2a - 2X\lambda = 0\), so \(a = X\lambda \). The constraint gives \(X'X\lambda = c\), so \(\lambda = (X'X)^{-1}c\) and \(a^* = X(X'X)^{-1}c\). It is a minimum, because for any feasible \(a\), writing \(a = a^* + d\) with \(X'd = 0\), \[ a'a = a^{*\prime }a^* + 2d'X(X'X)^{-1}c + d'd = a^{*\prime }a^* + d'd \geq a^{*\prime }a^* . \] Finally \(a^{*\prime }Y = c'(X'X)^{-1}X'Y = c'b\).

For the matrix statement, a linear unbiased \(\tilde \beta = AY\) has \(AX = I\). For any \(c\), \(c'\tilde \beta = (A'c)'Y\) is linear and unbiased for \(c'\beta \), so by (ii) and (iii) \(c'\Var (\tilde \beta )c = \Var (c'\tilde \beta ) \geq \Var (c'b) = c'\Var (b)c\). That is the definition of \(\Var (\tilde \beta ) - \Var (b)\) being psd. □