4.3 General Multiple Line Regression

Introduction

Multiple linear regression analysis is one of the most widely used of all statistical tools. The problem is estimating or predicting the values of a dependent variable \(Y\) on the basis of a set of measurements taken on several independent variables \(X_1,X_2,\ldots ,X_{k-1}\). For example we wish to estimate the speed of wind \((Y)\) as a function of height above ground \((X_1)\), temperature \((X_2)\) and pressure \((X_3)\). The predication equation is obtained using least squares estimator on data collected at various heights temperatures and pressures to evaluate the necessary coefficients in the assumed model. \[E\big (Y/X_1,X_2,X_3\big )=\beta _0+\beta _1X_1+\beta _2X_2+\beta _3X_3\] \[Y=XB+\varepsilon ,\quad \varepsilon \thicksim N\big (0,\sigma ^2 I_n\big )\] \[Y_i=\beta _0+\beta _1X_{i1}+\beta _2X_{i2}+\beta _3X_{i3}+e_i\,,\quad e_i\thicksim ^{iid}N(0,\sigma ^2)\]

Multiple linear regression models are often used as approximating function, relationship between \(Y\) and \(X_1,X_2,\ldots ,X_{k-1}\) is known, but over certain ranges of the regression variables \(\big (X_1,X_2,\ldots ,X_{k-1}\big )\) the linear regression model is an adequate approximation.
Models that are complex is structure then \[Y_i=\beta _0+\beta _1X_{i1}+\beta _2X_{i2}+\cdots + \beta _{k-1}X{ik-1}+e_{i1}\] may still be analysed by linear regression techniques.

For example, a cubic polynomial model \[ Y=\beta _0+\beta _1X+\beta _2X^2+\beta _3X^3+e\] Let \(X_1=X,\quad X_2=X\) and \(X_3=X^3\) then we have \[Y=\beta _0+\beta _1X_1+\beta _2X_2+\beta _3X_3+e\] we will generally write the multiple linear regression model \[Y_i=\beta _0+X_{i1}\beta _1+\cdots + X_{ik-1}\beta _{k-1}+e_i\quad \text {as}\quad Y_i=\sum ^{k-1}_{j=0}X_{ij}\beta _j+e_j,\quad X_{i0}=1,2,3\cdots ,n\] \[E\big (Y/X\big )=\sum ^{k-1}_{j=0}X_{ij}\beta _j\] \[Y=XB+\varepsilon \,,\quad B=\big (\beta _0,\beta _1,\ldots ,\beta _{k-1}\big )^t\] \[X= \begin {pmatrix} 1_n & X_1 & X_2 & \cdots & X_{k-1}\\ \end {pmatrix} \,,\quad X_j= \begin {pmatrix} X_{1j} & X_{2j} & X_{nj}\\ \end {pmatrix}^t,\quad j=1,2,\ldots ,k-1 \]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.