2.2 The two-sample \(t\) test
Write \(y_{ij}\) for the \(j\)th observation on treatment \(i\), \(i = 1, 2\), \(j = 1, \dots , n_i\). The model is \[ y_{ij} = \mu _i + \varepsilon _{ij} , \qquad \varepsilon _{ij} \sim N(0, \sigma ^2) \text { independent} , \] so the two treatments have possibly different means but a common variance. The randomisation of Section 1 is what entitles us to treat the errors as independent. We wish to test \[ H_0 : \mu _1 = \mu _2 \qquad \text {against}\qquad H_1 : \mu _1 \ne \mu _2 . \] Both sample variances estimate \(\sigma ^2\), and they are combined into the pooled estimate \[ S_p^2 = \frac {(n_1 - 1)S_1^2 + (n_2 - 1)S_2^2}{n_1 + n_2 - 2} . \]
Theorem 2.2.1. Under the model, \[ T = \frac {\bar Y_1 - \bar Y_2 - (\mu _1 - \mu _2)}{S_p\sqrt {\frac {1}{n_1} + \frac {1}{n_2}}} \sim t_{n_1+n_2-2} . \]
Proof. By Theorem 2.1.2 applied to each sample, \(\bar Y_1\), \(\bar Y_2\), \(S_1^2\) and \(S_2^2\) are independent, and \begin {gather*} Z = \frac {\bar Y_1 - \bar Y_2 - (\mu _1 - \mu _2)}{\sigma \sqrt {1/n_1 + 1/n_2}} \sim N(0, 1) , \\ V = \frac {(n_1 - 1)S_1^2 + (n_2 - 1)S_2^2}{\sigma ^2} \sim \chi ^2_{n_1+n_2-2} , \end {gather*} the second because a sum of independent \(\chi ^2_{n_1-1}\) and \(\chi ^2_{n_2-1}\) variables is \(\chi ^2_{n_1+n_2-2}\). Since \(Z\) and \(V\) are independent and \(V/(n_1 + n_2 - 2) = S_p^2/\sigma ^2\), the ratio \(Z/\sqrt {V/(n_1 + n_2 - 2)} = T\) has the \(t_{n_1+n_2-2}\) distribution. The unknown \(\sigma \) cancels, which is the point of the construction. □
Under \(H_0\) the statistic \(t_0 = (\bar y_1 - \bar y_2)/\bigl (s_p\sqrt {1/n_1 + 1/n_2}\bigr )\) is an observation from \(t_{n_1+n_2-2}\). We reject \(H_0\) at level \(\alpha \) if \(|t_0| > t_{\alpha /2,\,n_1+n_2-2}\), the upper \(\alpha /2\) point of the distribution. Inverting the test gives the \(100(1 - \alpha )\%\) confidence interval \[ \bar y_1 - \bar y_2 \pm t_{\alpha /2,\,n_1+n_2-2}\,s_p\sqrt {\frac {1}{n_1} + \frac {1}{n_2}} . \]
Example 2.2.2. For the catalyst data of Example 2.0.1, \(n_1 = n_2 = 8\) and \[ s_p^2 = \frac {7(5.6741) + 7(2.7286)}{14} = 4.2013 , \qquad s_p = 2.0497 . \] The standard error of the difference is \(s_p\sqrt {1/8 + 1/8} = 2.0497(0.5) = 1.0249\), and \[ t_0 = \frac {92.2625 - 90.0500}{1.0249} = \frac {2.2125}{1.0249} = 2.159 . \] With \(t_{0.025,14} = 2.145\), we have \(|t_0| > 2.145\) and reject \(H_0\) at the \(5\%\) level; the two-sided \(p\)-value is \(0.049\). The \(95\%\) confidence interval for \(\mu _1 - \mu _2\) is \[ 2.2125 \pm 2.145(1.0249) , \qquad \text {that is,}\qquad (0.014,\ 4.411) . \] The evidence is just significant, and the interval, which nearly reaches \(0\), says the same in more useful form: catalyst \(A\) is better by somewhere between almost nothing and \(4.4\) percentage points.
Remark. The \(t\) test can be checked against the randomisation argument of Section 1, which assumes no normal model. Under \(H_0\) the treatments do not matter, the sixteen yields are fixed, and the only randomness is which eight were labelled \(A\). Of the \(\binom {16}{8} = 12870\) equally likely labellings, \(668\) give a difference of means at least as large in absolute value as the observed \(2.2125\), so the exact randomisation \(p\)-value is \(668/12870 = 0.052\). The agreement with \(0.049\) is typical: Fisher showed that the \(t\) test is a good approximation to the randomisation test, and this is the justification for using it even when normality is only roughly true.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.