3.4 Two-Sample \(U\)-Statistics
The theory developed so far concerns a single sample. Most of the rank procedures in practical use compare two samples, and they are covered by an almost identical construction. The extension is worth carrying out in full, because it delivers the Mann–Whitney statistic — and with it the Wilcoxon rank-sum test — as a special case, together with its exact mean and variance.
Definition 3.4.1 (Two-sample estimable parameter). Let \(X_1,\dots ,X_m\) be \(iid\) with distribution \(F\) and \(Y_1,\dots ,Y_n\) be \(iid\) with distribution \(G\), the two samples independent. A parameter \(\gamma = \gamma (F,G)\) is estimable of degree \((r,s)\) if \(r\) and \(s\) are the smallest sample sizes for which there is a function \(h(x_1,\dots ,x_r;\,y_1,\dots ,y_s)\) with \[E\left [h(X_1,\dots ,X_r;\,Y_1,\dots ,Y_s)\right ] = \gamma \qquad \text {for all } F,G\in \mathscr {Z}.\]
Definition 3.4.2 (Two-sample \(U\)-statistic). For a kernel \(h\) symmetric within each of its two arguments, the corresponding \(U\)-statistic is \[U = \frac {1}{\binom {m}{r}\binom {n}{s}} \sum _{\alpha }\sum _{\beta } h\left (X_{\alpha _1},\dots ,X_{\alpha _r};\ Y_{\beta _1},\dots ,Y_{\beta _s}\right ),\] the sums running over all \(\binom {m}{r}\) subsets \(\alpha \) of \(\{1,\dots ,m\}\) and all \(\binom {n}{s}\) subsets \(\beta \) of \(\{1,\dots ,n\}\).
As in the one-sample case \(U\) is unbiased for \(\gamma \) by construction, since it is an average of terms each with expectation \(\gamma \).
Example 3.4.3 (The Mann–Whitney statistic). Take \(\gamma = P(X<Y)\), and the kernel of degree \((1,1)\) \[h(x;y) = \mathbb {I}\{x<y\},\] which has \(E\left [h(X;Y)\right ] = P(X<Y) = \gamma \) directly. The associated \(U\)-statistic is \[U = \frac {1}{mn}\sum ^{m}_{i=1}\sum ^{n}_{j=1}\mathbb {I}\left \{X_i<Y_j\right \} = \frac {U_{XY}}{mn},\] where \(U_{XY}\) counts the pairs in which the \(X\) observation is the smaller. This count is the Mann–Whitney statistic, and \(U\) estimates \(P(X<Y)\) without bias for every pair of continuous distributions.
Note 3.4.4. Example 3.4.3 identifies what the Mann–Whitney and Wilcoxon rank-sum tests actually test. The null hypothesis is \(\gamma = P(X<Y) = \tfrac 12\), and the parameter estimated is a probability of precedence, not a difference of means or of medians. Under the additional assumption that \(G\) is \(F\) shifted by a constant — the location-shift model — the two statements coincide, and only then. A significant Mann–Whitney test on samples differing in shape says that one variable tends to exceed the other, which is a weaker and different claim than a difference in centre.
To obtain the variance we need the same conditional-variance components as in the one-sample theory, now indexed by how many observations are shared in each sample.
Definition 3.4.5. For \(0\leq c\leq r\) and \(0\leq d\leq s\), write \[\zeta _{c,d} = \operatorname {cov}\left ( h(X_1,\dots ,X_r;\,Y_1,\dots ,Y_s),\ h(X_1',\dots ,X_r';\,Y_1',\dots ,Y_s')\right ),\] where the two sets of \(X\) arguments have exactly \(c\) observations in common and the two sets of \(Y\) arguments exactly \(d\).
Theorem 3.4.6 (Variance of a two-sample \(U\)-statistic). \[\operatorname {var}(U) = \frac {1}{\binom {m}{r}\binom {n}{s}} \sum ^{r}_{c=0}\sum ^{s}_{d=0} \binom {r}{c}\binom {m-r}{r-c}\binom {s}{d}\binom {n-s}{s-d}\ \zeta _{c,d} .\]
The argument is the one used for Theorem in the one-sample case: expand the double sum, classify each pair of terms by the number of shared observations in each sample, and count how many pairs fall into each class. Since \(\zeta _{0,0}=0\), the leading behaviour for large \(m,n\) is governed by \(\zeta _{1,0}\) and \(\zeta _{0,1}\), giving \[\operatorname {var}(U)\ \approx \ \frac {r^{2}}{m}\,\zeta _{1,0} + \frac {s^{2}}{n}\,\zeta _{0,1} ,\] which is the two-sample analogue of the projection result and yields asymptotic normality by the same route.
Example 3.4.7 (Mean and variance of Mann–Whitney under \(H_0\)). For the kernel of Example 3.4.3, \(r=s=1\) and \(\gamma = P(X<Y)\). Under \(H_0: F=G\) continuous, symmetry gives \(\gamma = \tfrac 12\), hence \[E\left (U_{XY}\right ) = mn\gamma = \frac {mn}{2}.\] For the variance, note \(h\) is an indicator so \(\zeta _{0,0}=0\) and \[\zeta _{1,0} = \operatorname {var}\left (P(X<Y\mid X)\right ) = \operatorname {var}\left (1-F(X)\right ) = \tfrac {1}{12},\] because \(F(X)\) is uniform on \((0,1)\) by Theorem 2.2.1. By symmetry \(\zeta _{0,1}=\tfrac {1}{12}\) also. Substituting into Theorem 3.4.6 and multiplying by \((mn)^2\) to return from \(U\) to the count \(U_{XY}\) gives \[\operatorname {var}\left (U_{XY}\right ) = \frac {mn(m+n+1)}{12},\] the familiar expression, here derived rather than asserted.
Remark 3.4.8. The appearance of \(\tfrac {1}{12}\) — the variance of a uniform variable on \((0,1)\) — is not a coincidence and is worth noticing. It enters through Theorem 2.2.1, and it is the same \(\tfrac {1}{12}\) that will reappear in the efficacy of the Wilcoxon test in Chapter 4. Distribution-freeness and the efficiency of rank procedures trace back to the same fact about the probability integral transform.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.