2.12 Correlation
In regression, \(x\) is fixed and \(y\) is random. When both are random, measured together on each unit, their linear association is measured by the correlation coefficient.
Definition 2.12.1. The sample correlation coefficient is \[ r = \frac {S_{xy}}{\sqrt {S_{xx} S_{yy}}} . \]
By the Cauchy–Schwarz inequality (Problem 2.13.4), \(-1 \le r \le 1\). It estimates the population correlation \(\rho \).
- (i)
- \(R^2 = r^2 = \dfrac {S_{xy}^2}{S_{xx} S_{yy}}\);
- (ii)
- \(\hat \beta _1 = r \sqrt {S_{yy}/S_{xx}}\), so \(\hat \beta _1\) and \(r\) have the same sign;
- (iii)
- the \(t\) statistic for \(H_0\colon \beta _1 = 0\) can be written \(t_0 = \dfrac {r\sqrt {n-2}}{\sqrt {1 - r^2}}\).
Proof. (i) \(R^2 = SS_R/SS_T = \hat \beta _1 S_{xy}/S_{yy} = S_{xy}^2/(S_{xx} S_{yy}) = r^2\).
(ii) \(r\sqrt {S_{yy}/S_{xx}} = \dfrac {S_{xy}}{\sqrt {S_{xx}S_{yy}}} \cdot \dfrac {\sqrt {S_{yy}}}{\sqrt {S_{xx}}} = \dfrac {S_{xy}}{S_{xx}} = \hat \beta _1\).
(iii) By (i), \(SS_{\mathrm {Res}} = SS_T(1 - R^2) = S_{yy}(1 - r^2)\), so \(\hat \sigma ^2 = S_{yy}(1 - r^2)/(n-2)\). Using (ii), \[ t_0 = \frac {\hat \beta _1}{\sqrt {\hat \sigma ^2/S_{xx}}} = \frac {r\sqrt {S_{yy}/S_{xx}}\, \sqrt {S_{xx}}\, \sqrt {n-2}}{\sqrt {S_{yy}(1 - r^2)}} = \frac {r\sqrt {n-2}}{\sqrt {1 - r^2}} . \qedhere \] □
When \(x\) and \(y\) are jointly normal, (iii) is the test of \(H_0\colon \rho = 0\): under \(H_0\) it has the \(t_{n-2}\) distribution. So testing for zero correlation and testing for zero slope are the same test.
Example 2.12.3. Find \(r\) for the advertising data and test \(H_0\colon \rho = 0\) at the 5% level.
Solution. \(r = 7/\sqrt {10 \times 6} = 7/\sqrt {60} = 0.9037\), and \(r^2 = 0.8167 = R^2\). Then \[ t_0 = \frac {0.9037\sqrt {3}}{\sqrt {1 - 0.8167}} = \frac {1.5652}{0.4282} = 3.656 , \] the same value as in Example 2.8.2. As \(3.656 > t_{0.025,3} = 3.182\), reject \(H_0\): the correlation is significant.
Remark. Correlation measures linear association only. Points lying exactly on the parabola \(y = x^2\) for \(x = -2, -1, 0, 1, 2\) have \(S_{xy} = \sum x_i^3 - n\bar x \bar y = 0\), so \(r = 0\), though \(y\) is completely determined by \(x\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.