8.1 The correlation form of the model

Multicollinearity is a property of the regressors alone, and it is best studied on a common scale. Centre and scale each regressor to unit length: \[ w_{ij} = \frac {x_{ij} - \bar x_j}{\sqrt {S_{jj}}} , \qquad S_{jj} = \sum _{i=1}^n (x_{ij} - \bar x_j)^2 , \] and likewise \(y_i^* = (y_i - \bar y)/\sqrt {SS_T}\). Let \(\mathbf {W}\) be the \(n \times k\) matrix of the \(w_{ij}\). Each column of \(\mathbf {W}\) has mean \(0\) and length \(1\), and the inner product of columns \(j\) and \(l\) is \[ \sum _i w_{ij}w_{il} = \frac {\sum _i (x_{ij} - \bar x_j)(x_{il} - \bar x_l)}{\sqrt {S_{jj}S_{ll}}} = r_{jl} , \] the correlation between \(x_j\) and \(x_l\). So \(\mathbf {W}^\top \mathbf {W} = \mathbf {R}\), the correlation matrix of the regressors, and similarly \(\mathbf {W}^\top \mathbf {y}^*\) is the vector \(\mathbf {r}_y\) of correlations between each regressor and \(y\). As in the proof of Theorem 5.1.1, centring separates the intercept from the other coefficients, and the least-squares estimates of the coefficients of the \(w_j\) in a regression of \(y^*\) are \[ \hat {\mathbf {b}} = \mathbf {R}^{-1}\mathbf {r}_y , \qquad \hat \beta _j = \hat b_j\sqrt {SS_T/S_{jj}} . \] The \(\hat b_j\) are the standardized regression coefficients. They are free of units, and \(\operatorname {Var}(\hat {\mathbf {b}}) = \sigma _*^2\mathbf {R}^{-1}\), where \(\sigma _*^2 = \sigma ^2/SS_T\) is the error variance on the scale of \(y^*\).

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.