4.1 Introduction

Simple linear regression model \[Y_i=\beta _0+\beta _1X_i+e_i,\quad i=1,2,\ldots ,n\quad e_i\thicksim N(0,\sigma ^2)\] \[L\big (Y,B\big )=\big (Y-XB\big )^t\big (Y-XB\big )\]

The normal equations \(X^tX\widehat {B}=X^tY\)

If \(X\) is full column rank

\begin {align*} \widehat {B} & = \big (X^tX\big )^{-1}X^tY\\ X^tX & = \begin {pmatrix} n & \sum X_i\\ \sum X_i & \sum X_i^2\\ \end {pmatrix}\\ \big (X^tX\big )^{-1} & =\frac {1}{n\sum x^2_i-\big (\sum X_i\big )^2} \begin {pmatrix} \sum X_i^2 & -\sum X_i\\ -\sum X_i & n\\ \end {pmatrix}\\ X^tY & = \begin {pmatrix} \sum Y_i\\ \sum X_iY_i\\ \end {pmatrix}\\ \widehat {B} & = \frac {1}{n\sum X_i^2-\big (\sum X_i\big )^2} \begin {pmatrix} \sum X_i^2& -\sum X_i\\ -\sum X_i & n\\ \end {pmatrix} \begin {pmatrix} \sum Y_i\\ \sum X_iY_i\\ \end {pmatrix}\\ & = \frac {1}{n\sum X_i^2-\big (\sum X_i\big )^2} \begin {pmatrix} \sum X_i^2\sum Y_i-\sum X_i\sum X_iY_i\\ n\sum X_iY_i -\big (\sum X_i\big )\big (\sum Y_i\big )\\ \end {pmatrix}\\ \therefore \quad \widehat {\beta }_0 & = \frac {\sum X^2_i\sum Y_i-\sum X_i\sum X_iY_i}{n\sum X_i^2-\big (\sum X_i\big )^2}\\ \widehat {\beta }_1 & = \frac {n\sum X_iY_i-\big (\sum X_i\big )\big (\sum Y_i\big )}{n\sum X_i^2-\big (\sum X_i\big )^2} \end {align*}

\(Y_i=\beta _0+\beta _1X_i\) is a regression line which is a straight line.

\(Y_i=\beta _0+\beta _1X_i+e_i\) is a regression fitted line which is a straight line.

\(\widehat {Y}=\widehat {\beta }_0+\widehat {\beta }_1X\).

\(E\big (Y/X\big )=\beta _0+\beta _1X \)

\(\beta _0=E\big (Y/X=0\big )\)

\(\beta _0=\) rate of change in \(Y\) per unit (increase) change in \(X\).

\(\frac {\partial }{\partial X}E\big (Y/X\big )=\beta _1\)

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.