5.5 A missing observation
If one observation is lost, the design is no longer balanced and the simple formulas no longer apply. A classical remedy, due to Yates, is to estimate the missing value and analyse the completed table.
Proposition 5.5.1. Suppose \(y_{ij}\) is missing, and let \(y_{i\cdot }'\), \(y_{\cdot j}'\) and \(y_{\cdot \cdot }'\) be the treatment, block and grand totals of the observations that are present. The value \(x\) that minimises the error sum of squares of the completed table is \[ \hat x = \frac {t\,y_{i\cdot }' + b\,y_{\cdot j}' - y_{\cdot \cdot }'}{(t - 1)(b - 1)} . \]
Proof. Write the error sum of squares of the completed table as \[ SSE(x) = \sum \sum y^2 - \frac 1b\sum _ky_{k\cdot }^2 - \frac 1t\sum _ly_{\cdot l}^2 + \frac {y_{\cdot \cdot }^2}{tb} , \] in which \(x\) enters \(\sum \sum y^2\) as \(x^2\), the total of treatment \(i\) as \(y_{i\cdot }' + x\), the total of block \(j\) as \(y_{\cdot j}' + x\), and the grand total as \(y_{\cdot \cdot }' + x\). Setting the derivative to zero, \[ 2x - \frac 2b(y_{i\cdot }' + x) - \frac 2t(y_{\cdot j}' + x) + \frac {2}{tb}(y_{\cdot \cdot }' + x) = 0 . \] Multiplying by \(tb/2\) gives \(x(tb - t - b + 1) = t\,y_{i\cdot }' + b\,y_{\cdot j}' - y_{\cdot \cdot }'\), and \(tb - t - b + 1 = (t - 1)(b - 1)\). The second derivative is \(2(t - 1)(b - 1)/(tb) > 0\). □
The completed table is then analysed as usual, with one degree of freedom subtracted from the error. The error sum of squares obtained is exact: it equals the residual sum of squares from fitting the model by least squares to the observations actually present. The treatment sum of squares, however, is biased upwards, by \[ \frac {\bigl (y_{\cdot j}' - (t - 1)\hat x\bigr )^2}{t(t - 1)} , \] which should be subtracted. The exact analysis, which today is just as easy, fits the full and reduced models by least squares and takes the treatment sum of squares as the difference in their residual sums of squares, as in the remark following Theorem 3.3.3.
Example 5.5.2. Suppose the yield of fertiliser \(C\) on field \(3\) (actually \(53\)) had been lost. Then \(y_{C\cdot }' = 191\), \(y_{\cdot 3}' = 142\) and \(y_{\cdot \cdot }' = 858\), and \[ \hat x = \frac {4(191) + 5(142) - 858}{3 \times 4} = \frac {616}{12} = 51.33 . \] From the completed table, \(SSE = 8.033\) on \(11\) degrees of freedom, and the uncorrected treatment sum of squares is \(72.13\). The bias is \(\bigl (142 - 3(51.33)\bigr )^2/12 = 12.00\), so the corrected treatment sum of squares is \(60.13\), which is exactly what the least squares fit to the \(19\) remaining observations gives. Then \(F_0 = (60.13/3)/(8.033/11) = 27.45\) on \(3\) and \(11\) degrees of freedom, and the conclusion is unchanged.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.