9.2 Least Squares Estimation

Given the outcomes \(Y\) and the values of the predictor valuables with full column rank, we determine the least squares estimate \(\widehat {B}_{(i)}\) modal \(i\) exclusively from the \(\underline {Y}_{(i)}\) on the \(i^{\text {th}}\) responses. Conforming to the single.

\begin {equation} \tag {4} \widehat {B}_{(i)} = \big (Z'Z\big )^{-1}Z'Y_{(i)} \end {equation}

\[Y_{(i)} = Z\underline {B}_{(i)} + \underline {\varepsilon }_{(i)}\]

\[\min f\big (B_i\big ) = \big (\underline {Y}_{(i)}-ZB_{(i)}\big )'\big (\underline {Y}_{(i)}-ZB_{(i)}\big )\]

collecting these univariate various least square estimates we obtain \begin {align*} \widehat {B} & = \begin {bmatrix} \widehat {\underline {B}}_{(1)} & \vdots & \widehat {\underline {B}}_{(2)} & \vdots & \cdots & \vdots & \widehat {\underline {B}}_{(m)}\\ \end {bmatrix}\\ & = \big (Z'Z\big )^{-1}Z'\begin {bmatrix} \underline {Y}_{(1)} & \vdots & \underline {Y}_{(2)} & \vdots & \cdots & \vdots & \underline {Y}_{(m)}\\ \end {bmatrix} \end {align*}

\begin {equation} \tag {5} \text {or}\quad \underbrace {\widehat {B}}_{(r+1)\times m} = \big (Z'Z\big )^{-1}Z'Y_{n\times m} \end {equation}

For any choice of parameters \( B = \begin {bmatrix} b_{(1)} & \vdots & b_{(2)} & \vdots & \cdots& \vdots & b_{(m)}\\ \end {bmatrix} \)

The matrix of errors is \(Y-ZB.\)
The error sum of squares and cross product is \[\big (Y-ZB\big )'\big (Y-ZB\big )= \begin {bmatrix} \big (Y_{(1)}-Zb_{(1)}\big )'\big (Y_{(1)}-Zb_{(1)}\big ) & \cdots& \big (Y_{(1)}-Zb_{(1)}\big )'\big (Y_{(m)}-Zb_{(m)}\big )\\ \vdots & & \vdots \\ \big (Y_{(m)}-Zb_{(m)}\big )'\big (Y_{(1)}-Zb_{(1)}\big ) & \cdots& \big (Y_{(m)}-Zb_{(n)}\big )'\big (Y_{(m)}-Zb_{(n)}\big )\\ \end {bmatrix} \]

The selection \(b_{(i)}=\widehat {B}_{(i)}\) minimises the \(i^{\text {th}}\) diagonal sum of squares \[\big (Y_{(i)} - Zb_{(i)}\big )'\big (Y_{(i)}-Zb_{(i)}\big )\] Consequently, \(tr\big [\big (Y-ZB\big )'\big (Y-ZB\big )\big ]\) is minimised by the choice \(B=\widehat {B}\). Also the generalised variance \(\big |\big (Y-ZB\big )'\big (Y-ZB\big )|\) is minimised by the least squares estimate \(\widehat {B}\).

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.