6.2 Two Independent Samples
Now let \(\underline {X}_{11},\dots ,\underline {X}_{1n_1}\) be a random sample from \(N_p(\underline {\mu }_1,\Sigma )\) and \(\underline {X}_{21},\dots ,\underline {X}_{2n_2}\) an independent random sample from \(N_p(\underline {\mu }_2,\Sigma )\) — with the same covariance matrix.
Definition 6.3 (Pooled covariance matrix). \[S_{\text {pooled}} = \frac {(n_1-1)S_1 + (n_2-1)S_2}{n_1+n_2-2}.\]
Note 6.4. This is the matrix referred to later as \(\left (n_1+n_2-2\right )S\) in the MANOVA of Section 8. Its justification is the additivity of the Wishart, Theorem 5.17(ii): \((n_1-1)S_1\) and \((n_2-1)S_2\) are independent \(W_p(n_1-1,\Sigma )\) and \(W_p(n_2-1,\Sigma )\) variates, so their sum is \(W_p(n_1+n_2-2,\Sigma )\), and dividing by its degrees of freedom gives an unbiased estimator of the common \(\Sigma \) using both samples at once.
Theorem 6.5 (Two-sample \(T^{2}\)). Under \(H_0:\underline {\mu }_1=\underline {\mu }_2\) and equal covariance matrices, \[T^{2} = \left (\overline {\underline {X}}_1-\overline {\underline {X}}_2\right )' \left [\left (\tfrac {1}{n_1}+\tfrac {1}{n_2}\right )S_{\text {pooled}}\right ]^{-1} \left (\overline {\underline {X}}_1-\overline {\underline {X}}_2\right ) \ \sim \ \frac {\left (n_1+n_2-2\right )p}{n_1+n_2-p-1}\,F_{p,\,n_1+n_2-p-1}.\]
Note 6.6. Set \(p=1\) and the statistic becomes the square of the ordinary two-sample \(t\), with \(\left (\tfrac {1}{n_1}+\tfrac {1}{n_2}\right )s^{2}_{\text {pooled}}\) in the denominator. Every multivariate statistic in this course collapses to its univariate ancestor in this way, and checking that it does is the surest way to remember the degrees of freedom.
Note 6.7 (The assumption of equal covariance). The pooling is legitimate only if \(\Sigma _1=\Sigma _2\). When the covariance matrices differ and the sample sizes are unequal, the test does not hold its nominal level — and the direction of the error depends on which sample is the larger, so it cannot even be relied upon to be conservative. This is the multivariate Behrens–Fisher problem, and it has no exact solution. Approximate procedures replace the pooled matrix by \(\tfrac {1}{n_1}S_1+\tfrac {1}{n_2}S_2\) and adjust the degrees of freedom.
Equality can be examined by Box’s \(M\) test, though that test is itself sensitive to non-normality, so a significant result may reflect the shape of the data rather than a genuine difference in covariance. When \(n_1\) and \(n_2\) are close to equal the pooled procedure is fairly robust to moderate inequality, which is one practical argument for balanced designs.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.