1.2 One-Way Analysis of Variance
The simplest design: \(N\) experimental units, each receiving one of \(k\) treatments, with nothing else recorded. The aim is a single test of whether the treatment means differ at all.
What follows is an algebraic identity, not a statistical argument. The total sum of squares is split into a between-treatments part and a within-treatments part; the notation is set up first, the identity is derived, and the ratio of the resulting mean squares is the test statistic. Read the derivation as book-keeping about sums of squares — the question of why the ratio has an \(F\) distribution is a separate one, answered in Example 2.5.7.
\[Y_{ij}=\mu _j+\varepsilon _{ij},\quad i=1,2,\ldots ,n_j,\quad j=1,2,\ldots ,k\]
- \(Y_{ij}=\) the observation for the \(i^{\text {th}}\) experiment unit in the \(j^{\text {th}}\) treatment.
- \(K-\) treatments.
- \(n_j=\) experimental units for the \(j^{\text {th}}\) treatments.
- \(\mu _j=E(Y_{ij})=\) the mean for the \(j^{\text {th}}\) treatments.
- \(\varepsilon _{ij}=\) the error for the \(i^{\text {th}}\) experimental unit under the \(j^{\text {th}}\) treatment.
We assume that \(\varepsilon _{ij}\thicksim N(0,\sigma ^2)\) \[Y_{ij}\thicksim N(\mu _j,\sigma ^2)\]
Treatments effects \[Y_{ij}=\mu +\tau _j +\varepsilon _{ij},\quad i=1,2,\ldots ,n_j,\quad j=1,2,\ldots ,k\]
- \(\mu =\) overall mean
- \(\tau _j=\) effect of the \(j^{\text {th}}\) treatment
- \(\tau _j=\mu _j-\mu \)
- Over parametrised model
\[\sum ^k_{j=1}\tau _j=0\]
Notation
\[Y_{ij}=\sum \limits ^{n_j}_{i=1}Y_{ij}\quad \text {and}\quad \overline {Y}._{j}=\frac {Y._{j}}{n_j}=\frac {1}{n_j}\sum \limits ^{n_j}_{i=1}Y_{ij}\] \[Y{..}=\sum ^k_{j=1}\sum ^{n_j}_{i=1}Y_{ij}\quad \text {and}\quad \overline {Y}{..}=\frac {\sum \limits ^k_{j=1}\sum \limits ^{n_j}_{i=1}Y_{ij}}{\sum ^k_{j=1}n_j}\] \[n_1+n_2+\cdots +n_k=\sum ^k_{j=1}n_j=\,\text {total number of observations}\] \begin {align*} Y_{ij}-\overline {Y}.. & = \overline {Y}._j-\overline {Y}..+Y_{ij}-\overline {Y}._j\\ (Y_{ij}-\overline {Y}..)^2 & = (\overline {Y}._j-\overline {Y}..)^2+(Y_{ij}-\overline {Y}._j)^2+2(\overline {Y}._j-\overline {Y}..)(Y_{ij}-\overline {Y}._j)\\ \sum ^k_{j=1}\sum ^{n_j}_{i=1}\big (Y_{ij}-\overline {Y}..\big )^2 & = \sum ^k_{j=1}n_j\big (\overline {Y}._j-\overline {Y}..\big )^2+\sum ^k_{j=1}\sum ^{n_j}_{i=1}\big (Y_{ij}-\overline {Y}._j\big )^2\\ \end {align*}
\begin {align*} SST & = SSTrt + SSE\\ N-1 & = K-1 + N-K\qquad N=\sum ^k_{j=1}n_j\\ \end {align*}
\(H_0:\tau _j=0\qquad j=0, 1,2,\ldots , k\)
\(H_a: \tau _j\neq \tau _{j'}\qquad j\neq j'\)
\[SSE = \sum ^k_{j=1}\Bigg (\sum ^{n_j}_{i=1}\big (Y_{ij}-\overline {Y}._j\big )^2\Bigg )\]
Let \(S^2_j=\frac {\sum \limits ^{n_j}_{i=1}\big (Y_{ij}-\overline {Y}._j\big )^2}{n_j-1}\) we have \(k\) variance estimates. Where \(S^2_j\) is the sample variance for the observations in the \(j^{\text {th}}\) treatment it follows that \(S^2=\frac {\sum \limits ^k_{j=1}(n_j-1)S^2_j}{\sum \limits ^k_{j=1}(n_j-1)}\) is the pooled variance.
\begin {align*} k=2:\quad & \frac {(n_1-1)S^2_1+(n_2-1)S^2_2}{n_1-1+n_2-1}=\frac {(n_1-1)S^2_1+(n_2-1)S^2_2}{n_1+n_2-2}\\ \end {align*}
\(S^2=\frac {\sum \limits ^k_{j=1}(n_j-1)S^2_j}{\sum \limits ^k_{j=1}(n_j-1)}=\frac {SSE}{N-K}\) where \(N=\displaystyle {\sum ^k_{j=1}n_j}\)
\begin {align*} SSTrt & = \sum ^k_{j=1} n_j(\overline {Y}._j-\overline {Y}..)^2\\ \frac {SSTrt}{K-1} & = \sum ^K_{j=1}\frac {n_j\big (\overline {Y}._j-\overline {Y}..\big )^2}{K-1} \end {align*}
under \(H_0:\tau _j = 0,\quad j=1,2,3,\ldots ,k\) then \(\frac {SSTrt}{K-1}\) also estimates \(\sigma ^2\).
\[Y_{ij}\thicksim N(\mu _j,\sigma ^2)\]
\[\overline {Y}._j \thicksim N(\mu ^*,\sigma ^2)\]
\[X_1,X_2,\ldots , X_n\] \[Y_1,Y_2,\ldots , Y_n\]
\(H_0:\sigma _X^2=\sigma ^2_Y\) vs \(H_a:\sigma ^2_X>\sigma ^2_Y\)
\begin {align*} F & = \frac {S^2_X}{S^2_Y}\thicksim f_{n-1,m-1}\\ & = \frac {\frac {(n-1)S_X^2\big /\sigma ^2}{n-1}}{\frac {(m-1)S^2_Y\big /\sigma ^2}{m-1}}\\ \end {align*}
\[E\Bigg (\frac {SSTr(t)}{K-1}\Bigg )=\sigma ^2+\sum ^k_{j=1}\frac {n_j \tau _j^2}{K-1}\]
\[E\Bigg (\frac {SSE}{N-K}\Bigg ) = \sigma ^2\]
The name analysis of variance is derived from partitioning the corrected total sum of squares into its component parts \(SSE\) and \(SSTr(t)\qquad \) \(SST= \displaystyle {\sum ^k_{j=1}\sum ^{n_j}_{i=1}\big (Y_{ij}-\overline {Y}._j\big )^2}\) into \(SSE + SSTr(t)\) and we get two independent estimates of \(\sigma ^2\) then take their ratio divided by their respective degrees of freedom to get.
\begin {align*} F & = \frac {MSTrt}{MSE}\,,\qquad MSTrt = \frac {SSTr(t)}{K-1}\,,\qquad MSE = \frac {SSE}{N-K}\\ & \thicksim f_{K-1,N-K}\\ \end {align*}
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.