11.4 The two-factor mixed model
Now let \(A\) be fixed and \(B\) random. The main effects \(\tau _i\) are constants with \(\sum _i\tau _i = 0\); the \(\beta _j\) are \(N(0, \sigma ^2_\beta )\). The interaction of a fixed level with a random level is random, and there are two conventions for it.
Definition 11.4.1. The restricted mixed model is \begin {gather*} y_{ijk} = \mu + \tau _i + \beta _j + (\tau \beta )_{ij} + \varepsilon _{ijk} , \\ i = 1, \dots , a , \quad j = 1, \dots , b , \quad k = 1, \dots , n , \end {gather*} where
- (i)
- \(\mu \) and the \(\tau _i\) are constants with \(\sum _i\tau _i = 0\);
- (ii)
- the \(\beta _j\) are independent \(N(0, \sigma ^2_\beta )\);
- (iii)
- the interaction effects are normal with mean zero and \(\Var \bigl ((\tau \beta )_{ij}\bigr ) = \frac {a - 1}a\sigma ^2_{\tau \beta }\), they sum to zero over the fixed factor, \(\sum _i(\tau \beta )_{ij} = 0\) for each \(j\), and those in different columns \(j\) are independent;
- (iv)
- the \(\varepsilon _{ijk}\) are independent \(N(0, \sigma ^2)\);
- (v)
- the \(\beta _j\), the interaction effects and the errors are mutually independent.
In the unrestricted mixed model, (iii) is replaced by: the \((\tau \beta )_{ij}\) are independent \(N(0, \sigma ^2_{\tau \beta })\), with no constraint.
A convenient way to build the restricted interaction is to take independent \(w_{ij} \sim N(0, \sigma ^2_{\tau \beta })\) and set \((\tau \beta )_{ij} = w_{ij} - \bar w_{\cdot j}\). This sums to zero over \(i\), has variance \(\frac {a - 1}a\sigma ^2_{\tau \beta }\), and for \(i \ne i'\) has covariance \(\Cov \bigl ((\tau \beta )_{ij}, (\tau \beta )_{i'j}\bigr ) = -\sigma ^2_{\tau \beta }/a\) (Problem 11.6.3). The factor \((a - 1)/a\) is a convention, chosen so that the expected mean squares come out in the same form as in the other models.
Theorem 11.4.2. In the restricted mixed model with \(A\) fixed and \(B\) random, \begin {align*} E(MS_A) &= \sigma ^2 + n\sigma ^2_{\tau \beta } + \frac {bn\sum _i\tau _i^2}{a - 1} , & E(MS_B) &= \sigma ^2 + an\sigma ^2_\beta , \\ E(MS_{AB}) &= \sigma ^2 + n\sigma ^2_{\tau \beta } , & E(MSE) &= \sigma ^2 . \end {align*}
In the unrestricted model the only change is \(E(MS_B) = \sigma ^2 + n\sigma ^2_{\tau \beta } + an\sigma ^2_\beta \).
Proof. Use \((\tau \beta )_{ij} = w_{ij} - \bar w_{\cdot j}\). For \(B\): \(\bar y_{\cdot j\cdot } - \bar y_{\cdot \cdot \cdot } = (\beta _j - \bar \beta ) + (\bar \varepsilon _{\cdot j\cdot } - \bar \varepsilon _{\cdot \cdot \cdot })\), because the interaction averages to zero over \(i\) in every column. So \(E(SS_B) = an\bigl [(b - 1)\sigma ^2_\beta + (b - 1)\sigma ^2/(an)\bigr ]\), with no interaction term.
For \(A\): \(\bar y_{i\cdot \cdot } - \bar y_{\cdot \cdot \cdot } = \tau _i + \overline {(\tau \beta )}_{i\cdot } + (\bar \varepsilon _{i\cdot \cdot } - \bar \varepsilon _{\cdot \cdot \cdot })\), since \(\sum _i\tau _i = 0\) and the interaction has zero grand mean. Here \(\overline {(\tau \beta )}_{i\cdot } = \bar w_{i\cdot } - \bar w_{\cdot \cdot }\), so by the lemma \(E\sum _i\overline {(\tau \beta )}_{i\cdot }^2 = (a - 1)\sigma ^2_{\tau \beta }/b\). The cross terms with the constants \(\tau _i\) have mean zero, and \[ E(SS_A) = bn\Bigl [\sum _i\tau _i^2 + \frac {(a - 1)\sigma ^2_{\tau \beta }}b + \frac {(a - 1)\sigma ^2}{bn}\Bigr ] . \] For \(AB\), the double centring of \((\tau \beta )_{ij}\) equals that of \(w_{ij}\), with expected sum of squares \((a - 1)(b - 1)\sigma ^2_{\tau \beta }\), which gives \(E(MS_{AB})\).
In the unrestricted model the computations for \(A\) and \(AB\) are those of Theorem 11.3.2, giving the same results; but the interaction no longer averages to zero over \(i\), and \(\overline {(\tau \beta )}_{\cdot j}\) contributes \(n\sigma ^2_{\tau \beta }\) to \(E(MS_B)\). □
So in both models the fixed factor is tested against the interaction, \(F_0 = MS_A/MS_{AB}\), and the interaction against error. The random factor is tested against \(MSE\) in the restricted model and against \(MS_{AB}\) in the unrestricted one. The two models are different models of the same data, not different analyses: in the restricted model the column effect \(\beta _j\) is the average response of column \(j\) over the fixed levels, so \(\sigma ^2_\beta \) measures variation of that average; in the unrestricted model the interaction carries part of it. The restricted model is the classical choice; general mixed-model software (REML) fits the unrestricted one. Either is acceptable if it is stated.
The estimates in the restricted model are \begin {gather*} \hat \tau _i = \bar y_{i\cdot \cdot } - \bar y_{\cdot \cdot \cdot } , \qquad \hat \sigma ^2 = MSE , \\ \hat \sigma ^2_{\tau \beta } = \frac {MS_{AB} - MSE}n , \qquad \hat \sigma ^2_\beta = \frac {MS_B - MSE}{an} ; \end {gather*} in the unrestricted model \(\hat \sigma ^2_\beta = (MS_B - MS_{AB})/(an)\). Comparisons among the fixed levels use \(MS_{AB}\) in place of \(MSE\): in both models \[ \Var (\bar y_{i\cdot \cdot } - \bar y_{i'\cdot \cdot }) = \frac {2(\sigma ^2 + n\sigma ^2_{\tau \beta })}{bn} = \frac {2E(MS_{AB})}{bn} , \] so Tukey’s method uses \(q_\alpha (a, (a - 1)(b - 1))\sqrt {MS_{AB}/(bn)}\).
Example 11.4.3. To study the effect of operators and filters in a filtration process, a company measures the percentage of material lost in coffee for each combination of its four operators and three types of filter, with three runs on each. The company has only these four operators, but many types of filter, and the three were selected at random. So operators (\(A\), \(a = 4\)) are fixed and filters (\(B\), \(b = 3\)) random, with \(n = 3\):
| operator | filter \(1\) | filter \(2\) | filter \(3\) | total |
| \(1\) | \(16.2\), \(16.8\), \(17.1\) | \(16.6\), \(16.9\), \(16.8\) | \(16.7\), \(16.9\), \(17.1\) | \(151.1\) |
| \(2\) | \(15.9\), \(15.1\), \(14.5\) | \(16.0\), \(16.3\), \(16.5\) | \(16.5\), \(16.9\), \(16.8\) | \(144.5\) |
| \(3\) | \(15.6\), \(15.9\), \(16.1\) | \(16.1\), \(16.0\), \(17.2\) | \(16.4\), \(17.4\), \(16.9\) | \(147.6\) |
| \(4\) | \(14.9\), \(15.2\), \(14.9\) | \(15.4\), \(14.6\), \(15.9\) | \(16.1\), \(15.4\), \(15.6\) | \(138.0\) |
| total | \(188.2\) | \(194.3\) | \(198.7\) | \(581.2\) |
A model is the restricted mixed model above: \(y_{ijk}\) is the loss on the \(k\)th run by operator \(i\) with filter \(j\), \(\mu \) the overall mean, \(\tau _i\) the fixed effect of operator \(i\), \(\beta _j\) the random effect of filter \(j\), \((\tau \beta )_{ij}\) the random operator–filter interaction and \(\varepsilon _{ijk}\) the run-to-run error, with the assumptions (i)–(v). With \(C = 581.2^2/36 = 9383.1511\), \begin {gather*} SS_A = \frac {151.1^2 + 144.5^2 + 147.6^2 + 138.0^2}9 - C = 10.318 , \\ SS_B = \frac {188.2^2 + 194.3^2 + 198.7^2}{12} - C = 4.634 , \end {gather*} and with \(SS_{AB} = 1.657\), \(SSE = 4.440\) and \(SST = 21.049\), the analysis is
| source | SS | df | MS | \(E(MS)\) | \(F_0\) |
| operators \(A\) | \(10.318\) | \(3\) | \(3.4393\) | \(\sigma ^2 + 3\sigma ^2_{\tau \beta } + 3\sum \tau _i^2\) | \(12.45\) |
| filters \(B\) | \(4.634\) | \(2\) | \(2.3169\) | \(\sigma ^2 + 12\sigma ^2_\beta \) | \(12.52\) |
| \(AB\) | \(1.657\) | \(6\) | \(0.2762\) | \(\sigma ^2 + 3\sigma ^2_{\tau \beta }\) | \(1.49\) |
| error | \(4.440\) | \(24\) | \(0.1850\) | \(\sigma ^2\) | |
| total | \(21.049\) | \(35\) |
(the coefficient of \(\sum \tau _i^2\) is \(bn/(a - 1) = 9/3\)). The three tests are:
- interaction, \(H_0 : \sigma ^2_{\tau \beta } = 0\): \(F_0 = 0.2762/0.1850 = 1.49 < F_{0.05, 6, 24} = 2.51\) (\(P = 0.22\)), not significant;
- operators, \(H_0 : \tau _1 = \dots = \tau _4 = 0\): \(F_0 = 3.4393/0.2762 = 12.45 > F_{0.05, 3, 6} = 4.76\) (\(P = 0.005\)), significant;
- filters, \(H_0 : \sigma ^2_\beta = 0\): \(F_0 = 2.3169/0.1850 = 12.52 > F_{0.05, 2, 24} = 3.40\) (\(P = 0.0002\)), significant.
The variance components are \begin {gather*} \hat \sigma ^2_\beta = \frac {2.3169 - 0.1850}{12} = 0.178 , \qquad \hat \sigma ^2_{\tau \beta } = \frac {0.2762 - 0.1850}3 = 0.030 , \\ \hat \sigma ^2 = 0.185 , \end {gather*} so filter-to-filter variation is about as large as run-to-run variation, and the interaction is small.
The operator means are \(16.79\), \(16.06\), \(16.40\) and \(15.33\). With \(q_{0.05}(4, 6) = 4.90\), Tukey’s critical difference is \(4.90\sqrt {0.2762/9} = 0.86\), so operator \(4\) loses significantly less material than operators \(1\) and \(3\), and no other pair differs.
In the unrestricted model the filter test would be \(F_0 = 2.3169/0.2762 = 8.39\) against \(F_{0.05, 2, 6} = 5.14\) (\(P = 0.018\)), still significant, and \(\hat \sigma ^2_\beta = (2.3169 - 0.2762)/12 = 0.170\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.