1.3 Randomisation as the basis of inference
The claim that randomisation “validates” the analysis can be made precise. Suppose \(N = 2n\) experimental units are available and two treatments \(A\) and \(B\) are to be compared. Assume unit–treatment additivity: unit \(k\) would give response \[ y_k = u_k + \tau _A \quad \text {under } A , \qquad y_k = u_k + \tau _B \quad \text {under } B , \] where \(u_k\) is a fixed but unknown unit value and \(\tau _A\), \(\tau _B\) are the treatment effects. Nothing is assumed about the \(u_k\): they need not be normal, independent or even random. Write \[ \bar u = \frac 1N\sum _{k=1}^Nu_k , \qquad S^2 = \frac {1}{N - 1}\sum _{k=1}^N(u_k - \bar u)^2 . \] In a completely randomised allocation, a set of \(n\) units is chosen at random, every one of the \(\binom Nn\) sets being equally likely, and these receive \(A\); the rest receive \(B\). The only randomness is in the allocation.
Lemma 1.3.1. Let \(\bar u_A\) be the mean of the unit values in a random set of \(n\) of the \(N\) units. Then \[ E(\bar u_A) = \bar u , \qquad \Var (\bar u_A) = \frac {S^2}{n}\Bigl (1 - \frac nN\Bigr ) . \]
Proof. Let \(I_k = 1\) if unit \(k\) is in the set and \(0\) otherwise, so \(\bar u_A = \frac 1n\sum _kI_ku_k\). By symmetry \(P(I_k = 1) = n/N\) for every \(k\), and for \(k \ne l\), \(P(I_k = I_l = 1) = \frac {n(n - 1)}{N(N - 1)}\). Hence \(E(\bar u_A) = \frac 1n\sum _k\frac nNu_k = \bar u\), and \begin {gather*} \Var (I_k) = \frac nN\Bigl (1 - \frac nN\Bigr ) , \\ \Cov (I_k, I_l) = \frac {n(n - 1)}{N(N - 1)} - \frac {n^2}{N^2} = -\frac {n(N - n)}{N^2(N - 1)} . \end {gather*} Since \(\bar u_A - \bar u = \frac 1n\sum _kI_k(u_k - \bar u)\) and \(\sum _k(u_k - \bar u) = 0\), write \(v_k = u_k - \bar u\). Then \begin {align*} n^2\Var (\bar u_A) &= \sum _kv_k^2\Var (I_k) + \sum _{k\ne l}v_kv_l\Cov (I_k, I_l) \\ &= \frac {n(N - n)}{N^2}\Bigl [\sum _kv_k^2 - \frac {1}{N - 1}\sum _{k\ne l}v_kv_l\Bigr ] . \end {align*}
Since \(\sum _{k\ne l}v_kv_l = \bigl (\sum _kv_k\bigr )^2 - \sum _kv_k^2 = -\sum _kv_k^2\), the term in brackets is \(\sum _kv_k^2\bigl (1 + \frac {1}{N - 1}\bigr ) = \frac {N}{N - 1}\sum _kv_k^2 = NS^2\). So \[ \Var (\bar u_A) = \frac {N - n}{nN}S^2 = \frac {S^2}{n}\Bigl (1 - \frac nN\Bigr ) . \] □
Theorem 1.3.2. Under unit–treatment additivity and complete randomisation of \(2n\) units, \(n\) to each treatment, the difference of treatment means satisfies \[ E(\bar y_A - \bar y_B) = \tau _A - \tau _B , \qquad \Var (\bar y_A - \bar y_B) = \frac {2S^2}{n} . \]
Proof. \(\bar y_A - \bar y_B = \tau _A - \tau _B + \bar u_A - \bar u_B\), and since the two groups together contain every unit, \(n\bar u_A + n\bar u_B = N\bar u\), that is \(\bar u_B = 2\bar u - \bar u_A\). Hence \(\bar u_A - \bar u_B = 2(\bar u_A - \bar u)\), which has mean \(0\) and, by Lemma 1.3.1 with \(N = 2n\), variance \(4\cdot \frac {S^2}{n}\cdot \frac 12 = 2S^2/n\). □
Remark. Compare Theorem 1.3.2 with the familiar result for independent observations with variance \(\sigma ^2\), namely \(\Var (\bar Y_A - \bar Y_B) = 2\sigma ^2/n\). The formulas coincide with \(S^2\) in place of \(\sigma ^2\), but the randomisation version assumes nothing about the units: the probability comes from the coin the experimenter tossed, not from a model of nature. This is the precise sense in which randomisation justifies the analysis. Without it, a systematic allocation (say the \(n\) units nearest the door to \(A\)) could make \(\bar u_A - \bar u_B\) anything at all, and no amount of data would reveal the bias.
Example 1.3.3. Eight units have values \(u = (3, 7, 1, 9, 4, 4, 8, 2)\), so \(\bar u = 4.75\) and \(S^2 = 8.5\). There are \(\binom 84 = 70\) ways to choose the four units for \(A\). Taking \(\tau _A = \tau _B\) and listing all \(70\) values of \(\bar u_A - \bar u_B\) gives a distribution with mean exactly \(0\) and variance exactly \(4.25 = 2S^2/n\), as Theorem 1.3.2 says. This randomisation distribution is also the basis of an exact test: an observed difference is significant at level \(\alpha \) if it lies in the extreme \(100\alpha \%\) of the \(70\) values.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.