2.3 Properties of the fitted line
The normal equations force the residuals to satisfy some exact identities. These are useful for checking hand calculations, and they are used repeatedly in the proofs that follow.
Theorem 2.3.1. For the least-squares fit of a model with an intercept:
- (i)
- \(\sum e_i = 0\): the residuals sum to zero;
- (ii)
- \(\sum y_i = \sum \hat y_i\);
- (iii)
- \(\sum x_i e_i = 0\);
- (iv)
- \(\sum \hat y_i e_i = 0\);
- (v)
- the fitted line passes through the point \((\bar x, \bar y)\).
Proof. The normal equations of Theorem 2.2.4 are \(\sum (y_i - \hat \beta _0 - \hat \beta _1 x_i) = 0\) and \(\sum x_i (y_i - \hat \beta _0 - \hat \beta _1 x_i) = 0\), and the bracket in each is exactly \(e_i\).
(i) and (iii) are therefore the two normal equations themselves.
(ii) By (i), \(0 = \sum (y_i - \hat y_i) = \sum y_i - \sum \hat y_i\).
(iv) \(\sum \hat y_i e_i = \sum (\hat \beta _0 + \hat \beta _1 x_i) e_i = \hat \beta _0 \sum e_i + \hat \beta _1 \sum x_i e_i = 0\) by (i) and (iii).
(v) At \(x = \bar x\) the line gives \(\hat \beta _0 + \hat \beta _1 \bar x = (\bar y - \hat \beta _1 \bar x) + \hat \beta _1 \bar x = \bar y\). □
Example 2.3.2. Check (i) and (iii) for the residuals of Example 2.2.6.
Solution. \(\sum e_i = 0.4 - 0.3 + 0 - 0.7 + 0.6 = 0\), and \begin {align*} \sum x_i e_i &= 1(0.4) + 2(-0.3) + 3(0) + 4(-0.7) + 5(0.6) \\ &= 0.4 - 0.6 + 0 - 2.8 + 3.0 = 0 . \end {align*}
Both hold. Had either failed, there would be an arithmetic error in \(\hat \beta _0\) or \(\hat \beta _1\).
Note. (i) and (ii) depend on the model having an intercept: they come from the normal equation for \(\beta _0\). For a line forced through the origin they fail in general (Problem 2.13.3).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.