3.2 Models for a single-factor experiment
Let \(y_{ij}\) be the \(j\)th observation on treatment \(i\), \(i = 1, \dots , t\), \(j = 1, \dots , n_i\). Two equivalent models are in use.
- (i)
- The cell means model is \[ y_{ij} = \mu _i + \varepsilon _{ij} , \] where \(\mu _i\) is the mean response to treatment \(i\).
- (ii)
- The effects model is \[ y_{ij} = \mu + \tau _i + \varepsilon _{ij} , \] where \(\mu \) is an overall mean and \(\tau _i\) is the effect of treatment \(i\), its deviation from the overall mean.
In both, the errors \(\varepsilon _{ij}\) are independent \(N(0, \sigma ^2)\), and the treatment effects are fixed: the \(t\) treatments are the only ones of interest, and conclusions are about them alone. (Treatments chosen at random from a larger population lead to the random effects model of Section 11.)
The two models describe the same means, \(\mu _i = \mu + \tau _i\), but the effects model has \(t + 1\) parameters for \(t\) means. In matrix form its design matrix has a column of ones followed by the \(t\) treatment indicators, and since the indicators sum to the column of ones, \(X\) has rank \(t\), not \(t + 1\): it is not of full rank. For example, with \(t = 3\) and \(n_i = 2\), \[ \begin {pmatrix} y_{11} \\ y_{12} \\ y_{21} \\ y_{22} \\ y_{31} \\ y_{32} \end {pmatrix} = \begin {pmatrix} 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 1 \\ 1 & 0 & 0 & 1 \end {pmatrix} \begin {pmatrix} \mu \\ \tau _1 \\ \tau _2 \\ \tau _3 \end {pmatrix} + \boldsymbol \varepsilon . \] The normal equations \(X^\top X\hat {\boldsymbol \beta } = X^\top \mathbf Y\) are, writing \(y_{i\cdot } = \sum _jy_{ij}\) and \(y_{\cdot \cdot } = \sum _iy_{i\cdot }\), \begin {align*} N\hat \mu + \sum _{i=1}^tn_i\hat \tau _i &= y_{\cdot \cdot } , \\ n_i\hat \mu + n_i\hat \tau _i &= y_{i\cdot } , \qquad i = 1, \dots , t . \end {align*}
The first equation is the sum of the others, so there are only \(t\) independent equations for \(t + 1\) unknowns and infinitely many solutions. Every solution has \(\hat \mu + \hat \tau _i = \bar y_{i\cdot }\). A unique solution is picked out by imposing one side condition; the usual one is \(\sum n_i\hat \tau _i = 0\), which gives \[ \hat \mu = \bar y_{\cdot \cdot } , \qquad \hat \tau _i = \bar y_{i\cdot } - \bar y_{\cdot \cdot } , \qquad i = 1, \dots , t . \] Statistical software often uses \(\hat \tau _t = 0\) instead, giving \(\hat \mu = \bar y_{t\cdot }\) and \(\hat \tau _i = \bar y_{i\cdot } - \bar y_{t\cdot }\). The two solutions look different, and the next idea explains why it does not matter.
Definition 3.2.2. A linear function \(\psi = \mathbf c^\top \boldsymbol \beta \) of the parameters is estimable if it has a linear unbiased estimator, that is, if there is a vector \(\mathbf a\) such that \(E(\mathbf a^\top \mathbf Y) = \mathbf c^\top \boldsymbol \beta \) for all \(\boldsymbol \beta \).
Since \(E(\mathbf a^\top \mathbf Y) = \mathbf a^\top X\boldsymbol \beta \), the function \(\mathbf c^\top \boldsymbol \beta \) is estimable exactly when \(\mathbf c^\top = \mathbf a^\top X\) for some \(\mathbf a\), that is, when \(\mathbf c^\top \) is a linear combination of the rows of \(X\).
Theorem 3.2.3. If \(\mathbf c^\top \boldsymbol \beta \) is estimable, then \(\mathbf c^\top \hat {\boldsymbol \beta }\) is the same for every solution \(\hat {\boldsymbol \beta }\) of the normal equations. It is unbiased for \(\mathbf c^\top \boldsymbol \beta \), and (Gauss–Markov) it has the smallest variance among all linear unbiased estimators.
Proof. Write \(\mathbf c^\top = \mathbf a^\top X\). Then \(\mathbf c^\top \hat {\boldsymbol \beta } = \mathbf a^\top X\hat {\boldsymbol \beta } = \mathbf a^\top \hat {\mathbf Y}\), and \(\hat {\mathbf Y}\) is the same for every solution by Theorem 3.1.1. Since \(\hat {\mathbf Y} = H\mathbf Y\) and \(HX = X\), \(E(\mathbf a^\top H\mathbf Y) = \mathbf a^\top HX\boldsymbol \beta = \mathbf a^\top X\boldsymbol \beta = \mathbf c^\top \boldsymbol \beta \). For minimum variance, let \(\mathbf b^\top \mathbf Y\) be any linear unbiased estimator, so \(\mathbf b^\top X = \mathbf c^\top \). Then \(\mathbf b^\top \mathbf Y = \mathbf b^\top H\mathbf Y + \mathbf b^\top (I - H)\mathbf Y\), and \(\mathbf b^\top H\mathbf Y = \mathbf c^\top \hat {\boldsymbol \beta }\) because \(\mathbf b^\top H\mathbf Y = \mathbf b^\top X\hat {\boldsymbol \beta }\). The two parts are uncorrelated, since \(\Cov (H\mathbf Y, (I - H)\mathbf Y) = \sigma ^2H(I - H) = 0\). Hence \[ \Var (\mathbf b^\top \mathbf Y) = \Var (\mathbf c^\top \hat {\boldsymbol \beta }) + \sigma ^2\|(I - H)\mathbf b\|^2 \ge \Var (\mathbf c^\top \hat {\boldsymbol \beta }) . \] □
In the effects model each row of \(X\) picks out \(\mu + \tau _i\) for some \(i\), so the estimable functions are exactly the linear combinations \(\sum _ia_i(\mu + \tau _i)\). In particular:
- each treatment mean \(\mu + \tau _i\) is estimable, with estimate \(\bar y_{i\cdot }\);
- a contrast \(\sum _ic_i\tau _i\) with \(\sum _ic_i = 0\) is estimable, since it equals \(\sum _ic_i(\mu + \tau _i)\); differences \(\tau _i - \tau _k\) are the simplest contrasts;
- an individual \(\tau _i\), and \(\mu \) itself, are not estimable. Their values depend on the side condition, which is an arbitrary convention, and no experiment can determine them.
Corollary 3.2.4. For any contrast \(C = \sum _{i=1}^tc_i\tau _i\), the estimate \(\hat C = \sum _{i=1}^tc_i\hat \tau _i\) is the same for the effects model, whatever side condition is used, and for the cell means model, and equals \(\sum _{i=1}^tc_i\bar y_{i\cdot }\).
Proof. Every solution of the normal equations of the effects model has \(\hat \tau _i = \bar y_{i\cdot } - \hat \mu \), so, because \(\sum c_i = 0\), \[ \sum _ic_i\hat \tau _i = \sum _ic_i\bar y_{i\cdot } - \hat \mu \sum _ic_i = \sum _ic_i\bar y_{i\cdot } . \] In the cell means model, which has full rank, \(\hat \mu _i = \bar y_{i\cdot }\), and the same contrast is \(\sum c_i\mu _i = \sum c_i(\mu + \tau _i) = \sum c_i\tau _i\), estimated by \(\sum c_i\bar y_{i\cdot }\). □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.