4.5 Practice Problems

Problem 4.1. Describe how the problem of tied observations is resolved when calculating the Kruskal–Wallis statistic \(H\).

Show solution

Solution. \(H\) is computed from the ranks of the pooled sample, and ties leave the ranking undefined until a convention is imposed.

Step one: mid-ranks. Observations that are equal are given the average of the ranks they would jointly have occupied. If the fourth, fifth and sixth smallest values in the pooled sample are all equal, each receives \(\frac {4+5+6}{3}=5\). The rank sums \(R_j\) and the statistic \[H=\dfrac {12}{N(N+1)}\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j}-3(N+1)\] are then formed from these mid-ranks in the usual way. Note that the total of all ranks is still \(N(N+1)/2\), so the arithmetic check is unaffected by ties.

Step two: correct the statistic. Averaging ranks reduces the variance of the rank sums, which makes \(H\) systematically too small. The corrected statistic divides by a factor computed from the tie pattern: \[H^{*}=\dfrac {H}{1-\dfrac {\sum _{p}\left (t_p^{3}-t_p\right )}{N^{3}-N}},\] the sum running over every group of tied values in the pooled sample and \(t_p\) being the number of observations in the \(p^{th}\) group. Untied observations form groups of size \(1\) and contribute nothing, since \(1^{3}-1=0\).

\(H^{*}\) is then referred to \(\chi ^{2}_{k-1}\) as before.

Direction and size of the correction. The divisor is at most \(1\), so \(H^{*}\geq H\): the correction always inflates the statistic and makes rejection easier. Omitting it is therefore conservative — the test loses power but does not overstate significance — which is why the omission so often goes unnoticed. The effect is negligible unless ties are numerous, but with heavily tied data, such as ordinal scores on a five-point scale, it can be substantial.

Problem 4.2. The Kruskal–Wallis statistic may be written \[F=A\sum ^{k}_{j=1}\dfrac {1}{n_j}\left [R_j-\dfrac {n_j(N+1)}{2}\right ]^{2},\] where \(R_j\) is the rank sum of the \(j^{th}\) sample, \(n_j\) its size, \(k\) the number of samples, \(N=n_1+n_2+\dots +n_k\), and \(A=\dfrac {12}{N(N+1)}\). Show that \[F=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j}-3(N+1).\]

Show solution

Solution. Two facts about the pooled ranking are needed. Every rank from \(1\) to \(N\) is used exactly once, so \[\sum ^{k}_{j=1}R_j=\sum ^{N}_{i=1}i=\dfrac {N(N+1)}{2}, \hspace {1cm}\text {and}\hspace {1cm}\sum ^{k}_{j=1}n_j=N .\]

Expand the square: \begin {align*} F &=A\sum ^{k}_{j=1}\dfrac {1}{n_j} \left [R_j^{2}-2R_j\cdot \dfrac {n_j(N+1)}{2}+\dfrac {n_j^{2}(N+1)^{2}}{4}\right ]\\[4pt] &=A\sum ^{k}_{j=1}\left [\dfrac {R_j^{2}}{n_j}-R_j(N+1) +\dfrac {n_j(N+1)^{2}}{4}\right ]\\[4pt] &=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j} -A(N+1)\sum ^{k}_{j=1}R_j +\dfrac {A(N+1)^{2}}{4}\sum ^{k}_{j=1}n_j . \end {align*}

Substituting the two sums, \begin {align*} F &=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j} -A(N+1)\cdot \dfrac {N(N+1)}{2} +\dfrac {A(N+1)^{2}}{4}\cdot N\\[4pt] &=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j} -\dfrac {AN(N+1)^{2}}{2}+\dfrac {AN(N+1)^{2}}{4}\\[4pt] &=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j}-\dfrac {AN(N+1)^{2}}{4} . \end {align*}

Finally, with \(A=\dfrac {12}{N(N+1)}\), \[\dfrac {AN(N+1)^{2}}{4} =\dfrac {12}{N(N+1)}\cdot \dfrac {N(N+1)^{2}}{4} =\dfrac {12(N+1)}{4}=3(N+1),\] so \[F=A\sum ^{k}_{j=1}\dfrac {R_j^{2}}{n_j}-3(N+1). \qquad \blacksquare \]

Note 4.14. The two forms say different things about the same quantity. The first shows what the statistic measures: \(n_j(N+1)/2\) is the rank sum expected for sample \(j\) under \(H_0\), so \(F\) is a weighted sum of squared departures from expectation — visibly a \(\chi ^{2}\)-like quantity, which is why \(\chi ^{2}_{k-1}\) is the reference distribution. The second is what you actually compute, needing only the \(R_j\) and \(n_j\) with no expected values worked out separately. Derivations of this kind are worth doing once: the memorable form and the computable form are rarely the same expression.

Problem 4.3. Describe how multiple comparisons may be made in Friedman’s test when the null hypothesis has been rejected.

Four treatments are compared in \(n=10\) blocks, giving rank sums \[R_{\cdot 1}=34,\hspace {0.5cm}R_{\cdot 2}=18,\hspace {0.5cm} R_{\cdot 3}=32,\hspace {0.5cm}R_{\cdot 4}=16 .\] Confirm that \(H_0\) is rejected at \(\alpha =0.05\), then determine which treatments differ.

Show solution

Solution. The procedure. A significant Friedman test says only that the \(k\) treatments are not all alike. To find which pairs differ, the Nemenyi test — the rank analogue of Tukey’s — compares every pair of mean ranks against a single critical difference \[CD=q_{\alpha }\sqrt {\dfrac {k(k+1)}{6n}}, \hspace {0.8cm}q_{\alpha }=\dfrac {q_{\infty ,k,\alpha }}{\sqrt {2}},\] declaring treatments \(i\) and \(j\) different when \(\left |\overline {R}_{\cdot i}-\overline {R}_{\cdot j}\right |\geq CD\), where \(\overline {R}_{\cdot j}=R_{\cdot j}/n\). Using one threshold for all \(\binom {k}{2}\) pairs is what keeps the family-wise error rate at \(\alpha \); testing each pair at \(\alpha \) separately would not.

If instead every treatment is compared only with a control, there are \(k-1\) comparisons rather than \(\binom {k}{2}\), and a Bonferroni adjustment — each comparison at \(\alpha /(k-1)\) — is available and more powerful than Nemenyi, which pays for comparisons that are not being made.

The omnibus test. First check the rank sums: \(34+18+32+16=100=\dfrac {nk(k+1)}{2}=\dfrac {10\times 4\times 5}{2}\). Then \begin {align*} F_R &=\dfrac {12}{nk(k+1)}\sum ^{k}_{j=1}R_{\cdot j}^{2}-3n(k+1)\\ &=\dfrac {12}{10\times 4\times 5}\left (34^{2}+18^{2}+32^{2}+16^{2}\right ) -3\times 10\times 5\\ &=0.06\left (1156+324+1024+256\right )-150\\ &=0.06(2760)-150=165.6-150=15.6 . \end {align*}

With \(k-1=3\) degrees of freedom, \(\chi ^{2}_{(0.05,3)}=7.815\), and \(15.6>7.815\), so \(H_0\) is rejected.

The comparisons. The mean ranks are \(3.4\), \(1.8\), \(3.2\), \(1.6\). With \(q_{0.05}=3.633/\sqrt {2}=2.569\), \[CD=2.569\sqrt {\dfrac {4\times 5}{6\times 10}}=2.569\times 0.577=1.483 .\]

pair 1–2 1–3 1–4 2–3 2–4 3–4
\(\left |\overline {R}_{\cdot i}-\overline {R}_{\cdot j}\right |\) 1.6 0.2 1.8 1.4 0.2 1.6
\(\geq CD=1.483\)? yes no yes no no yes

Treatments \(1\) and \(3\) form one group and treatments \(2\) and \(4\) another: each of \(1\) and \(3\) differs from \(4\), and \(1\) differs from \(2\), while the members of each pair cannot be separated. Note that \(2\) against \(3\), at \(1.4\), falls just short of the threshold even though \(2\) against \(1\) at \(1.6\) clears it — with overlapping groups like this the honest report is the set of differences that reached \(CD\), not a clean ranking of all four.

Problem 4.4. The following data are on three treatments:

\(T_1\) 1 1 2 8
\(T_2\) 0 0 7 9
\(T_3\) 0 0 1 8

(a).
Use the Kruskal–Wallis test to determine whether the treatments are the same. Take \(\alpha = 0.05\) and critical value \(4.51\).
(b).
Explain why this test is preferable here to the analysis of a completely randomised design.

Show solution

Solution. (a). Rank all \(N = 12\) observations together, averaging tied ranks. The four \(0\)s share ranks \(1\) to \(4\), so each gets \(2.5\). The three \(1\)s share ranks \(5\) to \(7\), so each gets \(6\). The \(2\) gets rank \(8\) and the \(7\) gets \(9\). The two \(8\)s share ranks \(10\) and \(11\), so each gets \(10.5\), and the \(9\) gets \(12\). The rank sums are \begin {align*} R_1 &= 6+6+8+10.5 = 30.5,\\ R_2 &= 2.5+2.5+9+12 = 26,\\ R_3 &= 2.5+2.5+6+10.5 = 21.5, \end {align*}

with check \(R_1+R_2+R_3 = 78 = \frac {12(13)}{2}\). Then \[H = \frac {12}{12(13)}\cdot \frac {30.5^{2}+26^{2}+21.5^{2}}{4}-3(13) = \frac {2189.5}{52}-39 = 0.779.\] There are tie groups of sizes \(4\), \(3\) and \(2\), so \[C = 1-\frac {(4^{3}-4)+(3^{3}-3)+(2^{3}-2)}{12^{3}-12} = 1-\frac {90}{1716} = 0.948, \qquad H^{*} = \frac {0.779}{0.948} = 0.822.\] Since \(0.822 < 4.51\) (and far below \(\chi ^{2}_{0.05,2} = 5.991\)), \(H_0\) is not rejected: there is no evidence that the treatments differ. A glance at the data agrees. Each treatment has two low values and two high ones, and the rank sums are close to their common expectation \(\frac {n_i(N+1)}{2} = 26\).

(b). The analysis of variance for a completely randomised design assumes normal errors with equal variances in the groups. These data cannot support that. The values are clumped at \(0\)–\(2\) and \(7\)–\(9\) with nothing between, there are many ties, and four observations per group are too few to check anything. The Kruskal–Wallis test needs only independent samples from continuous distributions of the same shape. It is also unaffected by outliers, since an extreme value only gets the extreme rank.

Problem 4.5. A fast-food chain wants to evaluate the service at three of its restaurants. Six investigators with varied experience are hired as raters. Each rater visits the three restaurants in a random order and rates each on a scale of \(1\) to \(10\):

Rater A B C
1 7 6 8
2 7 7 8
3 7 9 6
4 8 6 9
5 8 6 9
6 7 8 9

(a).
Using the Friedman test, can it be concluded at the \(5\%\) level (critical value \(5.991\)) that the median ratings of the three restaurants differ?
(b).
Explain why the randomised block design analysis is not appropriate here.

Show solution

Solution. (a). Each rater is a block, so rank within each row. Rater 2 gives A and B the same score, so they share ranks \(1\) and \(2\) as \(1.5\) each:

Rater A B C
1 2 1 3
2 1.5 1.5 3
3 2 3 1
4 2 1 3
5 2 1 3
6 1 2 3
\(R_{\cdot j}\) 10.5 9.5 16

Check: \(10.5+9.5+16 = 36 = \frac {nk(k+1)}{2} = \frac {6(3)(4)}{2}\). With \(n = 6\) and \(k = 3\), \[F_R = \frac {12}{6(3)(4)}\left (10.5^{2}+9.5^{2}+16^{2}\right )-3(6)(4) = \frac {456.5}{6}-72 = 4.083.\] One tie group of size 2 gives the correction \(1-\frac {2^{3}-2}{6(3^{3}-3)} = 1-\frac {6}{144} = 0.958\), so \(F_R^{*} = 4.261\). Since \(4.261 < 5.991\), \(H_0\) is not rejected: at the \(5\%\) level the raters do not show a difference between the restaurants. Restaurant C is ranked top by five of the six raters, which looks persuasive, but with only six blocks it is not enough. The \(p\)-value is about \(0.12\).

(b). The randomised block analysis is a two-way analysis of variance, and it treats the ratings as measurements on an interval scale with normal errors of constant variance. A rating from 1 to 10 is ordinal: an 8 is better than a 7, but the gap from 7 to 8 need not equal the gap from 8 to 9, and raters with “varied experience” use the scale differently. Ranking within each rater removes exactly that difference. Six raters are, besides, far too few to check normality.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.