3 Multiple linear regression
Most responses depend on more than one variable. Crop yield depends on fertiliser and on rainfall; the time to make a delivery depends on the number of items and on the distance. A multiple linear regression model with \(k\) regressors is \begin {equation} \label {eq:mlr} y = \beta _0 + \beta _1 x_1 + \beta _2 x_2 + \cdots + \beta _k x_k + \varepsilon . \end {equation} It has \(p = k + 1\) parameters \(\beta _0, \beta _1, \dots , \beta _k\), called the regression coefficients.
Remark. \(\beta _j\) is the change in the mean response \(E(y)\) per unit increase in \(x_j\) when all the other regressors are held fixed. For this reason the \(\beta _j\) are called partial regression coefficients. With one regressor, \(E(y)\) is a line; with two it is a plane, as in Figure 5; with more it is a hyperplane, which cannot be drawn but is handled in exactly the same way.
As in the last section, “linear” refers to the parameters. A model such as \(y = \beta _0 + \beta _1 x + \beta _2 x^2 + \varepsilon \) is a multiple linear regression model with \(x_1 = x\) and \(x_2 = x^2\), and everything in this section applies to it.
Multiple regression is carried out with matrices. The formulas of simple linear regression were already long with one regressor; written out term by term for \(k\) regressors they would be unmanageable. In matrix form, one formula covers every \(k\), and simple linear regression becomes the special case \(k = 1\).
3.2 The model in matrix form
3.3 Least-squares estimation
3.4 Fitted values, residuals and the hat matrix
3.5 Properties of the least-squares estimator
3.6 Estimation of \(\sigma ^2\) in multiple regression
3.7 Distribution theory under normality
3.8 Test for significance of regression
3.9 \(R^2\) and adjusted \(R^2\)
3.10 Tests on individual coefficients
3.11 The extra-sum-of-squares principle
3.12 Confidence and prediction intervals
3.13 Practice problems
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.