Proof. (i) \(\sum (x_i - \bar x) = \sum x_i - n\bar x = n\bar x - n\bar x = 0\).

(ii) Expanding the square, \begin {align*} S_{xx} &= \sum \left (x_i^2 - 2\bar x x_i + \bar x^2\right ) = \sum x_i^2 - 2\bar x \sum x_i + n\bar x^2 \\ &= \sum x_i^2 - 2n\bar x^2 + n\bar x^2 = \sum x_i^2 - n\bar x^2 , \end {align*}

and \(n\bar x^2 = n\left (\sum x_i / n\right )^2 = \left (\sum x_i\right )^2/n\).

(iii) \(S_{xy} = \sum (x_i - \bar x) y_i - \bar y \sum (x_i - \bar x) = \sum (x_i - \bar x) y_i\) by (i). Then \(\sum (x_i - \bar x) y_i = \sum x_i y_i - \bar x \sum y_i = \sum x_i y_i - n\bar x \bar y\). □

Proof. At a minimum of \(S\) both partial derivatives vanish: \begin {align*} \frac {\partial S}{\partial \beta _0} &= -2 \sum \left ( y_i - \beta _0 - \beta _1 x_i \right ) = 0, \\ \frac {\partial S}{\partial \beta _1} &= -2 \sum x_i \left ( y_i - \beta _0 - \beta _1 x_i \right ) = 0 . \end {align*}

Dividing by \(-2\) and rearranging gives the normal equations \begin {align*} n\hat \beta _0 + \hat \beta _1 \sum x_i &= \sum y_i, \\ \hat \beta _0 \sum x_i + \hat \beta _1 \sum x_i^2 &= \sum x_i y_i . \end {align*}

Dividing the first normal equation by \(n\) gives \(\hat \beta _0 + \hat \beta _1 \bar x = \bar y\), that is, \(\hat \beta _0 = \bar y - \hat \beta _1 \bar x\). Substituting this into the second, \[ (\bar y - \hat \beta _1 \bar x)\, n\bar x + \hat \beta _1 \sum x_i^2 = \sum x_i y_i , \] and collecting the terms in \(\hat \beta _1\), \[ \hat \beta _1 \left ( \sum x_i^2 - n\bar x^2 \right ) = \sum x_i y_i - n\bar x\bar y , \] which by Lemma 2.2.3 reads \(\hat \beta _1 S_{xx} = S_{xy}\). Since the \(x_i\) are not all equal, \(S_{xx} > 0\) and we may divide.

It remains to check that this stationary point is a minimum. The second derivatives are \[ \frac {\partial ^2 S}{\partial \beta _0^2} = 2n, \qquad \frac {\partial ^2 S}{\partial \beta _1^2} = 2\sum x_i^2, \qquad \frac {\partial ^2 S}{\partial \beta _0 \, \partial \beta _1} = 2\sum x_i , \] constants, so the Hessian matrix is the same at every point. Its leading entry \(2n\) is positive and its determinant is \[ 4n\sum x_i^2 - 4\left (\sum x_i\right )^2 = 4n\left ( \sum x_i^2 - n\bar x^2 \right ) = 4nS_{xx} > 0 . \] The Hessian is therefore positive definite everywhere, \(S\) is a convex function, and the stationary point is its global minimum. □

Solution. Set out the working in a table, one row per observation, and add up the columns:

So \(n = 5\), \(\bar x = 15/5 = 3\) and \(\bar y = 10/5 = 2\). By Lemma 2.2.3, \begin {align*} S_{xx} &= 55 - \frac {15^2}{5} = 55 - 45 = 10, \\ S_{xy} &= 37 - \frac {15 \times 10}{5} = 37 - 30 = 7 . \end {align*}

By Theorem 2.2.4, \[ \hat \beta _1 = \frac {S_{xy}}{S_{xx}} = \frac {7}{10} = 0.7, \qquad \hat \beta _0 = \bar y - \hat \beta _1 \bar x = 2 - 0.7 \times 3 = -0.1 , \] and the fitted line is \[ \hat y = -0.1 + 0.7x . \] Interpretation. The slope says that each extra dollar of advertising goes with an increase of \(0.7\) units in mean sales. The intercept would be the mean sales with no advertising at all, but \(x = 0\) lies outside the data, which run from \(1\) to \(5\), and negative sales are impossible. The intercept here is simply where the line has to cross the axis to fit the observed range; it should not be read as a prediction. Using a fitted line outside the range of the data is called extrapolation, and it is unreliable.

The fitted values and residuals are

They are shown as the vertical segments in Figure 3.

Solution.

Here \(n = 6\), \(\bar x = 210/6 = 35\) and \(\bar y = 398/6 = 66.3333\). Then \begin {align*} S_{xx} &= 9100 - \frac {210^2}{6} = 9100 - 7350 = 1750, \\ S_{xy} &= 14930 - \frac {210 \times 398}{6} = 14930 - 13930 = 1000, \end {align*}

so \[ \hat \beta _1 = \frac {1000}{1750} = 0.5714, \qquad \hat \beta _0 = 66.3333 - 0.5714 \times 35 = 46.3333 , \] and the fitted line is \(\hat y = 46.3333 + 0.5714x\). Each extra degree raises the mean yield by about \(0.57\) percentage points.

At \(x = 45\), inside the range of the data, \(\hat y = 46.3333 + 0.5714 \times 45 = 72.05\%\).