3.9 \(R^2\) and adjusted \(R^2\)
The coefficient of determination is defined as before, \(R^2 = SS_R/SS_T = 1 - SS_{\mathrm {Res}}/SS_T\). For the crop data, \(R^2 = 89.5/93.5 = 0.957\).
Proof. \(SS_T\) does not depend on the model. The larger model can reproduce any fit of the smaller one by setting the new coefficient to zero, so its minimum of \(S(\boldsymbol \beta )\), which is its \(SS_{\mathrm {Res}}\), is no greater. So \(1 - SS_{\mathrm {Res}}/SS_T\) cannot fall. □
So \(R^2\) always favours the bigger model, even when the added regressor is pure noise. To compare models with different numbers of parameters, \(R^2\) is adjusted by replacing each sum of squares with its mean square: \[ R^2_{\mathrm {adj}} = 1 - \frac {SS_{\mathrm {Res}}/(n - p)}{SS_T/(n - 1)} . \] An extra regressor lowers \(SS_{\mathrm {Res}}\) but also lowers \(n - p\), so \(R^2_{\mathrm {adj}}\) rises only if the regressor earns its degree of freedom. For the crop data, \[ R^2_{\mathrm {adj}} = 1 - \frac {4/3}{93.5/5} = 1 - \frac {1.3333}{18.7} = 0.929 . \]
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.