4.5 Practice problems

Problem 4.1. [Sample Questions] Three systems were compared, each on six occasions:

System A 55 60 63 56 59 55
System B 54 57 53 64 49 62
System C 66 52 61 60 57 58

(a).
Obtain the analysis of variance table, showing all calculations.
(b).
Are the three systems equally effective? Use \(\alpha =0.05\).
(c).
Is it necessary to carry out pairwise tests? Give your reasons.

Show solution

Solution. (a). The totals are \(T_A=348\), \(T_B=339\), \(T_C=354\), so the grand total is \(T=1041\) with \(n=18\) observations in \(k=3\) groups of \(n_j=6\).

Start with the correction factor, which appears in every sum of squares: \[CF=\frac {T^2}{n}=\frac {1041^2}{18}=\frac {1083681}{18}=60204.5.\] With \(\sum \sum x_{ij}^2=60545\), \[SS_T=\sum \sum x_{ij}^2-CF=60545-60204.5=340.5,\] \[SS_B=\sum _{j}\frac {T_j^2}{n_j}-CF=\frac {348^2+339^2+354^2}{6}-60204.5=60223.5-60204.5=19.0,\] \[SS_W=SS_T-SS_B=340.5-19.0=321.5.\] Note \(SS_B\) uses each group’s own total \(T_j\), not the grand total.

Source \(SS\) df \(MS\) \(F\)
Between systems \(19.0\) \(k-1=2\) \(9.50\) \(0.44\)
Within systems \(321.5\) \(n-k=15\) \(21.43\)
Total \(340.5\) \(n-1=17\)

The degrees of freedom add up, \(2+15=17\), which is worth checking every time. Note that \(MS_W\) divides \(SS_W\) by \(n-k=15\), matching its own row.

(b). Testing \(H_0:\mu _A=\mu _B=\mu _C\) against the alternative that at least one differs: \[F=\frac {MS_B}{MS_W}=\frac {9.50}{21.43}=0.44.\] The critical value is \(F_{2,15,\,0.05}=3.68\), and \(0.44\) falls far short of it; the \(P\)-value is \(0.65\). We fail to reject \(H_0\). There is no evidence that the three systems differ in effectiveness.

Look at why. The group means are \(58.0\), \(56.5\) and \(59.0\) – a spread of only \(2.5\) – while individual readings within a single system run from \(49\) to \(66\). The variation between systems is far smaller than the variation within them, and \(F\) is precisely the ratio of those two. An \(F\) below 1 says the group means are closer together than one would expect from chance alone.

(c). No. Pairwise tests are what one does after a significant \(F\), to find which groups differ. Here \(F\) is not significant, so there is nothing to locate: we have no evidence that any of the systems differ from any other.

Running pairwise tests anyway would be a mistake, and not a harmless one. Three comparisons each at \(5\%\) give a much higher chance than \(5\%\) of at least one false positive, which is the whole reason for testing all groups together with a single \(F\) first.

Problem 4.2. [Assignment] A stimulus-response experiment with three treatments was laid out in a randomised block design using four subjects. The response is reaction time in seconds; the treatment applied is shown in brackets.

Subject 1
Subject 2
Subject 3
Subject 4
1.7 (1) 2.1 (3) 0.1 (1) 2.3 (2)
2.3 (3) 1.5 (1) 2.3 (2) 0.6 (1)
3.4 (2) 2.6 (2) 0.8 (3) 1.6 (3)

Do the data present sufficient evidence of a difference in mean response between the stimuli?

Show solution

Solution. The subjects are the blocks and the stimuli the treatments. As always in a randomised block design, the treatments appear in a different order in each block, so totals must be gathered by label and not by position in the column.

Subject 1 Subject 2 Subject 3 Subject 4 Treatment total
Treatment 1 1.7 1.5 0.1 0.6 3.9
Treatment 2 3.4 2.6 2.3 2.3 10.6
Treatment 3 2.3 2.1 0.8 1.6 6.8
Block total 7.4 6.2 3.2 4.5 21.3
Table 35: Reaction times rearranged by treatment. Both sets of totals are needed.

With \(N=12\) and \(T=21.3\), the correction factor is \[CF=\frac {T^2}{N}=\frac {453.69}{12}=37.8075.\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=47.31-37.8075=9.5025\\ SS_{\text {Blocks}} &=\frac {7.4^2+6.2^2+3.2^2+4.5^2}{3}-CF=41.23-37.8075=3.4225\\ SS_{\text {Treat}} &=\frac {3.9^2+10.6^2+6.8^2}{4}-CF=43.4525-37.8075=5.6450\\ SS_{\text {Error}} &=9.5025-3.4225-5.6450=0.4350 \end {align*}

Each total is divided by the number of observations that went into it: block totals by the \(3\) treatments, treatment totals by the \(4\) blocks.

Source df \(SS\) \(MS\) \(F\)
Blocks (subjects) 3 3.4225 1.1408 15.74
Treatments (stimuli) 2 5.6450 2.8225 38.93
Error 6 0.4350 0.0725
Total 11 9.5025
Table 36: ANOVA table for the randomised block design.

The degrees of freedom add, \(3+2+6=11\), and so do the sums of squares.

At \(\alpha =0.05\), \(F_{2,6}=5.14\) for treatments and \(F_{3,6}=4.76\) for blocks. \[38.93>5.14\quad \text {and}\quad 15.74>4.76,\] so both are significant. There is very strong evidence of a difference in mean reaction time between the stimuli (\(P<0.001\)), and the subjects differ from one another too.

That the blocks are significant is worth noticing: blocking earned its keep here. Subject 3 is much faster than Subject 1 across every stimulus, and had that variation been left in the error term the treatment effect would have been far harder to see. Compare the randomised block example earlier in this chapter, where the blocks showed no difference and the blocking bought nothing.

Problem 4.3. [Assignment] Four groups of students were taught by different techniques and tested afterwards. Drop-outs left the groups unequal in size. Do the data indicate a difference in mean achievement between the four techniques?

Group 1 Group 2 Group 3 Group 4
65 75 59 94
87 69 78 89
73 83 67 80
79 81 62 88
81 72 82
69 79 73
90

Show solution

Solution. The groups have \(n_i=6, 7, 6, 4\), so \(N=23\). Unequal group sizes change nothing in principle — each treatment total is simply divided by its own \(n_i\).

Group 1 Group 2 Group 3 Group 4 Overall
\(n_i\) 6 7 6 4 23
\(T_i\) 454 549 421 351 1775
Mean 75.67 78.43 70.17 87.75 77.17
Table 37: Group totals and means for the four teaching techniques.

\[CF=\frac {T^2}{N}=\frac {1775^2}{23}=136\,983.7\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=138\,899-136\,983.7=1915.30\\ SS_{\text {Between}} &=\sum _i\frac {T_i^2}{n_i}-CF =\frac {454^2}{6}+\frac {549^2}{7}+\frac {421^2}{6}+\frac {351^2}{4}-CF=766.67\\ SS_{\text {Within}} &=1915.30-766.67=1148.63 \end {align*}

Source df \(SS\) \(MS\) \(F\)
Between techniques \(k-1=3\) 766.67 255.56 4.23
Within (error) \(N-k=19\) 1148.63 60.45
Total 22 1915.30
Table 38: One-way ANOVA for the teaching techniques.

At \(\alpha =0.05\), \(F_{3,19}=3.13\). Since \(4.23>3.13\) we reject \(H_0\): there is evidence of a difference in mean achievement between the four techniques (\(P\approx 0.019\)).

The \(F\) test says only that the four means are not all equal. Reading the group means, Group 4 at \(87.75\) and Group 3 at \(70.17\) are the extremes and are the likely source, but establishing which pairs differ requires a follow-up procedure such as the least significant difference — not a series of unadjusted \(t\) tests.

Problem 4.4. [Assignment] Four assemblers each assembled four electronic devices, in the Latin square below. The response is assembly time in minutes and the device is shown in brackets. Two sources of unwanted variation are controlled: differences between people, and fatigue over the assembly sequence.

Position
Assembler 1
Assembler 2
Assembler 3
Assembler 4
1 44 (3) 41 (1) 30 (2) 40 (4)
2 41 (2) 42 (3) 49 (4) 49 (1)
3 59 (1) 41 (4) 59 (3) 34 (2)
4 58 (4) 37 (2) 53 (1) 59 (3)

Do the data indicate a difference in mean assembly time between the four devices?

Show solution

Solution. First check the design really is a Latin square: each device must appear exactly once in every row and every column. Reading across and down, each of \(1,2,3,4\) does. That property is what allows three effects — position, assembler and device — to be separated from only \(16\) observations.

Gathering the three sets of totals, with \(N=16\) and \(T=736\):

1 2 3 4
Row (position) totals 155 181 193 207
Column (assembler) totals 202 161 191 182
Treatment (device) totals 202 142 204 188
Table 39: The three sets of totals a Latin square requires.

\[CF=\frac {736^2}{16}=33\,856\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=35\,186-33\,856=1330.0\\ SS_{\text {Rows}} &=\frac {155^2+181^2+193^2+207^2}{4}-CF=365.0\\ SS_{\text {Cols}} &=\frac {202^2+161^2+191^2+182^2}{4}-CF=226.5\\ SS_{\text {Treat}} &=\frac {202^2+142^2+204^2+188^2}{4}-CF=626.0\\ SS_{\text {Error}} &=1330.0-365.0-226.5-626.0=112.5 \end {align*}

The error degrees of freedom are \((p-1)(p-2)=3\times 2=6\).

Source df \(SS\) \(MS\) \(F\)
Rows (position) 3 365.0 121.67 6.49
Columns (assemblers) 3 226.5 75.50 4.03
Treatments (devices) 3 626.0 208.67 11.13
Error 6 112.5 18.75
Total 15 1330.0
Table 40: ANOVA table for the Latin square. Degrees of freedom: \(3+3+3+6=15\).

At \(\alpha =0.05\), \(F_{3,6}=4.76\) throughout.

Devices. \(11.13>4.76\): reject. There is strong evidence of a difference in mean assembly time between the four devices. Device 2 is quickest, with a total of \(142\) against \(204\) for device 3.
Position. \(6.49>4.76\): reject. Times lengthen down the sequence, totals running \(155, 181, 193, 207\) — which is the fatigue effect the design was built to control.
Assemblers. \(4.03<4.76\): fail to reject. No significant difference between the four people.

The design justified itself. Fatigue is real and significant; had it been ignored, its \(365\) would have gone into the error term, raising \(MS_{\text {error}}\) from \(18.75\) to \(53.1\) and cutting the device \(F\) from \(11.13\) to \(3.93\), against a critical \(F_{3,9}=3.86\) — barely significant instead of decisively so. Removing a known nuisance source is what makes the effect of interest visible.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.