4.5 Practice problems
Problem 4.1. [Sample Questions] Three systems were compared, each on six occasions:
| System A | 55 | 60 | 63 | 56 | 59 | 55 |
| System B | 54 | 57 | 53 | 64 | 49 | 62 |
| System C | 66 | 52 | 61 | 60 | 57 | 58 |
- (a).
- Obtain the analysis of variance table, showing all calculations.
- (b).
- Are the three systems equally effective? Use \(\alpha =0.05\).
- (c).
- Is it necessary to carry out pairwise tests? Give your reasons.
Show solution
Solution. (a). The totals are \(T_A=348\), \(T_B=339\), \(T_C=354\), so the grand total is \(T=1041\) with \(n=18\) observations in \(k=3\) groups of \(n_j=6\).
Start with the correction factor, which appears in every sum of squares: \[CF=\frac {T^2}{n}=\frac {1041^2}{18}=\frac {1083681}{18}=60204.5.\] With \(\sum \sum x_{ij}^2=60545\), \[SS_T=\sum \sum x_{ij}^2-CF=60545-60204.5=340.5,\] \[SS_B=\sum _{j}\frac {T_j^2}{n_j}-CF=\frac {348^2+339^2+354^2}{6}-60204.5=60223.5-60204.5=19.0,\] \[SS_W=SS_T-SS_B=340.5-19.0=321.5.\] Note \(SS_B\) uses each group’s own total \(T_j\), not the grand total.
| Source | \(SS\) | df | \(MS\) | \(F\) |
| Between systems | \(19.0\) | \(k-1=2\) | \(9.50\) | \(0.44\) |
| Within systems | \(321.5\) | \(n-k=15\) | \(21.43\) | |
| Total | \(340.5\) | \(n-1=17\) |
The degrees of freedom add up, \(2+15=17\), which is worth checking every time. Note that \(MS_W\) divides \(SS_W\) by \(n-k=15\), matching its own row.
(b). Testing \(H_0:\mu _A=\mu _B=\mu _C\) against the alternative that at least one differs: \[F=\frac {MS_B}{MS_W}=\frac {9.50}{21.43}=0.44.\] The critical value is \(F_{2,15,\,0.05}=3.68\), and \(0.44\) falls far short of it; the \(P\)-value is \(0.65\). We fail to reject \(H_0\). There is no evidence that the three systems differ in effectiveness.
Look at why. The group means are \(58.0\), \(56.5\) and \(59.0\) – a spread of only \(2.5\) – while individual readings within a single system run from \(49\) to \(66\). The variation between systems is far smaller than the variation within them, and \(F\) is precisely the ratio of those two. An \(F\) below 1 says the group means are closer together than one would expect from chance alone.
(c). No. Pairwise tests are what one does after a significant \(F\), to find which groups differ. Here \(F\) is not significant, so there is nothing to locate: we have no evidence that any of the systems differ from any other.
Running pairwise tests anyway would be a mistake, and not a harmless one. Three comparisons each at \(5\%\) give a much higher chance than \(5\%\) of at least one false positive, which is the whole reason for testing all groups together with a single \(F\) first.
Problem 4.2. [Assignment] A stimulus-response experiment with three treatments was laid out in a randomised block design using four subjects. The response is reaction time in seconds; the treatment applied is shown in brackets.
| Subject 1 | Subject 2 | Subject 3 | Subject 4
| |||||
| 1.7 | (1) | 2.1 | (3) | 0.1 | (1) | 2.3 | (2) | |
| 2.3 | (3) | 1.5 | (1) | 2.3 | (2) | 0.6 | (1) | |
| 3.4 | (2) | 2.6 | (2) | 0.8 | (3) | 1.6 | (3) | |
Do the data present sufficient evidence of a difference in mean response between the stimuli?
Show solution
Solution. The subjects are the blocks and the stimuli the treatments. As always in a randomised block design, the treatments appear in a different order in each block, so totals must be gathered by label and not by position in the column.
| Subject 1 | Subject 2 | Subject 3 | Subject 4 | Treatment total | |
| Treatment 1 | 1.7 | 1.5 | 0.1 | 0.6 | 3.9 |
| Treatment 2 | 3.4 | 2.6 | 2.3 | 2.3 | 10.6 |
| Treatment 3 | 2.3 | 2.1 | 0.8 | 1.6 | 6.8 |
| Block total | 7.4 | 6.2 | 3.2 | 4.5 | 21.3 |
With \(N=12\) and \(T=21.3\), the correction factor is \[CF=\frac {T^2}{N}=\frac {453.69}{12}=37.8075.\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=47.31-37.8075=9.5025\\ SS_{\text {Blocks}} &=\frac {7.4^2+6.2^2+3.2^2+4.5^2}{3}-CF=41.23-37.8075=3.4225\\ SS_{\text {Treat}} &=\frac {3.9^2+10.6^2+6.8^2}{4}-CF=43.4525-37.8075=5.6450\\ SS_{\text {Error}} &=9.5025-3.4225-5.6450=0.4350 \end {align*}
Each total is divided by the number of observations that went into it: block totals by the \(3\) treatments, treatment totals by the \(4\) blocks.
| Source | df | \(SS\) | \(MS\) | \(F\) |
| Blocks (subjects) | 3 | 3.4225 | 1.1408 | 15.74 |
| Treatments (stimuli) | 2 | 5.6450 | 2.8225 | 38.93 |
| Error | 6 | 0.4350 | 0.0725 | |
| Total | 11 | 9.5025 | ||
The degrees of freedom add, \(3+2+6=11\), and so do the sums of squares.
At \(\alpha =0.05\), \(F_{2,6}=5.14\) for treatments and \(F_{3,6}=4.76\) for blocks. \[38.93>5.14\quad \text {and}\quad 15.74>4.76,\] so both are significant. There is very strong evidence of a difference in mean reaction time between the stimuli (\(P<0.001\)), and the subjects differ from one another too.
That the blocks are significant is worth noticing: blocking earned its keep here. Subject 3 is much faster than Subject 1 across every stimulus, and had that variation been left in the error term the treatment effect would have been far harder to see. Compare the randomised block example earlier in this chapter, where the blocks showed no difference and the blocking bought nothing.
Problem 4.3. [Assignment] Four groups of students were taught by different techniques and tested afterwards. Drop-outs left the groups unequal in size. Do the data indicate a difference in mean achievement between the four techniques?
| Group 1 | Group 2 | Group 3 | Group 4 |
| 65 | 75 | 59 | 94 |
| 87 | 69 | 78 | 89 |
| 73 | 83 | 67 | 80 |
| 79 | 81 | 62 | 88 |
| 81 | 72 | 82 | |
| 69 | 79 | 73 | |
| 90 | |||
Show solution
Solution. The groups have \(n_i=6, 7, 6, 4\), so \(N=23\). Unequal group sizes change nothing in principle — each treatment total is simply divided by its own \(n_i\).
| Group 1 | Group 2 | Group 3 | Group 4 | Overall | |
| \(n_i\) | 6 | 7 | 6 | 4 | 23 |
| \(T_i\) | 454 | 549 | 421 | 351 | 1775 |
| Mean | 75.67 | 78.43 | 70.17 | 87.75 | 77.17 |
\[CF=\frac {T^2}{N}=\frac {1775^2}{23}=136\,983.7\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=138\,899-136\,983.7=1915.30\\ SS_{\text {Between}} &=\sum _i\frac {T_i^2}{n_i}-CF =\frac {454^2}{6}+\frac {549^2}{7}+\frac {421^2}{6}+\frac {351^2}{4}-CF=766.67\\ SS_{\text {Within}} &=1915.30-766.67=1148.63 \end {align*}
| Source | df | \(SS\) | \(MS\) | \(F\) |
| Between techniques | \(k-1=3\) | 766.67 | 255.56 | 4.23 |
| Within (error) | \(N-k=19\) | 1148.63 | 60.45 | |
| Total | 22 | 1915.30 | ||
At \(\alpha =0.05\), \(F_{3,19}=3.13\). Since \(4.23>3.13\) we reject \(H_0\): there is evidence of a difference in mean achievement between the four techniques (\(P\approx 0.019\)).
The \(F\) test says only that the four means are not all equal. Reading the group means, Group 4 at \(87.75\) and Group 3 at \(70.17\) are the extremes and are the likely source, but establishing which pairs differ requires a follow-up procedure such as the least significant difference — not a series of unadjusted \(t\) tests.
Problem 4.4. [Assignment] Four assemblers each assembled four electronic devices, in the Latin square below. The response is assembly time in minutes and the device is shown in brackets. Two sources of unwanted variation are controlled: differences between people, and fatigue over the assembly sequence.
| Position | Assembler 1 | Assembler 2 | Assembler 3 | Assembler 4
| ||||
| 1 | 44 | (3) | 41 | (1) | 30 | (2) | 40 | (4) |
| 2 | 41 | (2) | 42 | (3) | 49 | (4) | 49 | (1) |
| 3 | 59 | (1) | 41 | (4) | 59 | (3) | 34 | (2) |
| 4 | 58 | (4) | 37 | (2) | 53 | (1) | 59 | (3) |
Do the data indicate a difference in mean assembly time between the four devices?
Show solution
Solution. First check the design really is a Latin square: each device must appear exactly once in every row and every column. Reading across and down, each of \(1,2,3,4\) does. That property is what allows three effects — position, assembler and device — to be separated from only \(16\) observations.
Gathering the three sets of totals, with \(N=16\) and \(T=736\):
| 1 | 2 | 3 | 4 | |
| Row (position) totals | 155 | 181 | 193 | 207 |
| Column (assembler) totals | 202 | 161 | 191 | 182 |
| Treatment (device) totals | 202 | 142 | 204 | 188 |
\[CF=\frac {736^2}{16}=33\,856\] \begin {align*} SS_{\text {Total}} &=\sum x^2-CF=35\,186-33\,856=1330.0\\ SS_{\text {Rows}} &=\frac {155^2+181^2+193^2+207^2}{4}-CF=365.0\\ SS_{\text {Cols}} &=\frac {202^2+161^2+191^2+182^2}{4}-CF=226.5\\ SS_{\text {Treat}} &=\frac {202^2+142^2+204^2+188^2}{4}-CF=626.0\\ SS_{\text {Error}} &=1330.0-365.0-226.5-626.0=112.5 \end {align*}
The error degrees of freedom are \((p-1)(p-2)=3\times 2=6\).
| Source | df | \(SS\) | \(MS\) | \(F\) |
| Rows (position) | 3 | 365.0 | 121.67 | 6.49 |
| Columns (assemblers) | 3 | 226.5 | 75.50 | 4.03 |
| Treatments (devices) | 3 | 626.0 | 208.67 | 11.13 |
| Error | 6 | 112.5 | 18.75 | |
| Total | 15 | 1330.0 | ||
At \(\alpha =0.05\), \(F_{3,6}=4.76\) throughout.
- Devices. \(11.13>4.76\): reject. There is strong evidence of a difference in mean assembly time between the four devices. Device 2 is quickest, with a total of \(142\) against \(204\) for device 3.
- Position. \(6.49>4.76\): reject. Times lengthen down the sequence, totals running \(155, 181, 193, 207\) — which is the fatigue effect the design was built to control.
- Assemblers. \(4.03<4.76\): fail to reject. No significant difference between the four people.
The design justified itself. Fatigue is real and significant; had it been ignored, its \(365\) would have gone into the error term, raising \(MS_{\text {error}}\) from \(18.75\) to \(53.1\) and cutting the device \(F\) from \(11.13\) to \(3.93\), against a critical \(F_{3,9}=3.86\) — barely significant instead of decisively so. Removing a known nuisance source is what makes the effect of interest visible.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.