4.1 Kruskal-Wallis Test
The parametric one-way analysis of variance fits the linear model \[Y_{ij} = \mu + \alpha _i + e_{ij},\] with \(\mu \) the overall mean, \(\alpha _i\) the effect of group \(i\), and \(e_{ij}\) normal errors. The Kruskal–Wallis test is its distribution-free counterpart: it replaces the observations by their ranks and asks the same question without assuming normality.
Definition 4.1. Let there be \(k\) independent samples of sizes \(n_1,\ldots ,n_k\), with \(N = \sum n_i\). Rank all \(N\) observations together, averaging ties, and let \(R_i\) be the sum of the ranks in group \(i\). The Kruskal–Wallis statistic is \[H = \dfrac {12}{N(N+1)}\sum ^{k}_{i=1}\dfrac {R_i^{2}}{n_i} \;-\; 3(N+1).\] It tests \[H_0 : \text {all } k \text { samples come from the same distribution.}\]
Theorem 4.2. Under \(H_0\), and provided the group sizes are not too small, \(H\) is distributed approximately as \(\chi ^{2}\) on \(k-1\) degrees of freedom. \(H_0\) is rejected at level \(\alpha \) when \[H \geq \chi ^{2}_{\alpha ,\,k-1}.\]
Note 4.3. Ties. With tied observations the ranks are averaged and \(H\) is divided by \[C = 1 - \dfrac {\sum _{p}\left (t_p^{3} - t_p\right )}{N^{3} - N},\] the sum running over the groups of tied values and \(t_p\) being the size of the \(p\)-th group; the corrected statistic is \(H^{*} = H/C\). Note the denominator: \(N^{3} - N = N(N^{2}-1)\), not \(N(N+1)\). Since \(C \leq 1\), the correction can only increase \(H\), which is the right direction — ties reduce the spread of the rank sums and the statistic must be inflated to compensate.
Multiple comparisons. When \(H_0\) is rejected, any two groups \(i\) and \(j\) may be compared by \[Z_{ij} = \dfrac {\left |\overline {R}_i - \overline {R}_j\right |} {\sqrt {\Bigg [\dfrac {N(N+1)}{12}\Bigg ]\Bigg (\dfrac {1}{n_i}+\dfrac {1}{n_j}\Bigg )}},\] declaring them different when \(Z_{ij}\) exceeds \(Z^{*} = Z_{\alpha /(k(k-1))}\). The quantity \(\alpha \) is then the experimentwise error rate — the probability of at least one false declaration among the \(\frac {k(k-1)}{2}\) pairwise comparisons.
Example 4.4. Tensile strengths of parachutes from four suppliers, five each, ranked together over \(N = 20\) observations.
| Supplier 1 | Supplier 2 | Supplier 3 | Supplier 4
| ||||
| Amount | Rank | Amount | Rank | Amount | Rank | Amount | Rank |
| 18.5 | 4 | 26.3 | 20 | 20.6 | 8 | 25.4 | 19 |
| 24.0 | 13.5 | 25.3 | 18 | 25.2 | 17 | 19.9 | 5.5 |
| 17.2 | 1 | 24.0 | 13.5 | 20.8 | 9 | 22.6 | 11 |
| 19.9 | 5.5 | 21.2 | 10 | 24.7 | 16 | 17.5 | 2 |
| 18.0 | 3 | 24.5 | 15 | 22.9 | 12 | 20.4 | 7 |
Two values are tied at \(19.9\), sharing ranks \(5\) and \(6\) as \(5.5\), and two at \(24.0\), sharing ranks \(13\) and \(14\) as \(13.5\).
Rank sums. \[R_1 = 27,\qquad R_2 = 76.5,\qquad R_3 = 62,\qquad R_4 = 44.5,\] and the check holds: \(27 + 76.5 + 62 + 44.5 = 210 = \dfrac {20(21)}{2}\).
The statistic. With every \(n_i = 5\), \[H = \dfrac {12}{20(21)}\cdot \dfrac {27^{2} + 76.5^{2} + 62^{2} + 44.5^{2}}{5} \;-\; 3(21) = 70.8886 - 63 = 7.8886 .\]
Decision. With \(k - 1 = 3\) degrees of freedom, \(\chi ^{2}_{0.05,\,3} = 7.8147\). Since \(7.8886 > 7.8147\) we reject \(H_0\): the suppliers do not all deliver the same distribution of tensile strength.
Note 4.5. The decision is marginal. The statistic exceeds the critical value by \(0.074\). A single observation reordered could reverse it, and the honest report is the \(P\)-value, \(P(\chi ^{2}_{3} \geq 7.8886) = 0.0484\), rather than a bare “significant at \(5\%\)”.
Example 4.6 (The tie correction applied). There are two tied groups, each of size \(2\), so \[\sum _{p}\left (t_p^{3}-t_p\right ) = \left (2^{3}-2\right ) + \left (2^{3}-2\right ) = 6 + 6 = 12 ,\] and with \(N = 20\), \[C = 1 - \dfrac {12}{20^{3}-20} = 1 - \dfrac {12}{7980} = 0.998496 .\] Hence \[H^{*} = \dfrac {H}{C} = \dfrac {7.8886}{0.998496} = 7.9005 ,\] still above \(7.8147\), so the conclusion stands.
Note 4.7. Three things to get right in that correction. The denominator is \(N^{3}-N = 20 \times 399 = 7980\), not \(N(N+1) = 420\) — an error here is a factor of nearly twenty. Every tied group counts, so two groups of size two give \(12\), not \(6\). And with only four tied observations out of twenty the correction is tiny: it moves \(H\) from \(7.8886\) to \(7.9005\), a change of about one part in seven hundred. Where ties are heavy it will matter much more, which is why it should be computed rather than waved away.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.