Proof. Setting the partial derivatives to zero gives the normal equations \begin {align*} \frac {\partial S}{\partial a} &= -2\sum (Y_i - a - bX_i) = 0 ,\\ \frac {\partial S}{\partial b} &= -2\sum X_i(Y_i - a - bX_i) = 0 , \end {align*}

that is, \begin {align*} \sum Y_i &= na + b\sum X_i ,\\ \sum X_iY_i &= a\sum X_i + b\sum X_i^2 . \end {align*}

The first gives \(a = \bar Y - b\bar X\). Substituting into the second and using \(\sum X_iY_i - n\bar X\bar Y = \sum x_iy_i\) and \(\sum X_i^2 - n\bar X^2 = \sum x_i^2\) gives \(b\sum x_i^2 = \sum x_iy_i\). Under (A4) this has the unique solution (2.1).

The point is a minimum because \(S\) is a convex quadratic: its Hessian is \[ 2\begin {pmatrix} n & \sum X_i \\ \sum X_i & \sum X_i^2 \end {pmatrix}, \] with positive diagonal and determinant \(4\bigl (n\sum X_i^2 - (\sum X_i)^2\bigr ) = 4n\sum x_i^2 > 0\). □

Proof. (i) and the first part of (ii) are the two normal equations. Then \(\sum x_ie_i = \sum X_ie_i - \bar X\sum e_i = 0\). (iii) \(\sum \hat Y_ie_i = a\sum e_i + b\sum X_ie_i = 0\). (iv) is \(a = \bar Y - b\bar X\). □

Proof. Square \(y_i = bx_i + e_i\) and sum: \(\sum y_i^2 = b^2\sum x_i^2 + 2b\sum x_ie_i + \sum e_i^2\). The middle term is zero by Theorem 2.2.2(ii). Finally \(b^2\sum x_i^2 = b\sum x_iy_i\) by (2.1). □

Proof. \(R^2 = \dfrac {b\sum x_iy_i}{\sum y_i^2} = \dfrac {(\sum x_iy_i)^2}{\sum x_i^2\sum y_i^2} = r^2\). □