2.1 Sampling distributions
The analysis rests on three distributions derived from the normal.
Definition 2.1.1. Let \(Z_1, \dots , Z_k\) be independent \(N(0, 1)\) variables.
- (i)
- \(\chi ^2 = Z_1^2 + \dots + Z_k^2\) has the chi-square distribution with \(k\) degrees of freedom, \(\chi ^2_k\). Its mean is \(k\) and its variance \(2k\).
- (ii)
- If \(Z \sim N(0, 1)\) and \(V \sim \chi ^2_k\) are independent, \(T = Z/\sqrt {V/k}\) has the \(t\) distribution with \(k\) degrees of freedom, \(t_k\).
- (iii)
- If \(V_1 \sim \chi ^2_{k_1}\) and \(V_2 \sim \chi ^2_{k_2}\) are independent, \(F = (V_1/k_1)/(V_2/k_2)\) has the \(F\) distribution with \(k_1\) and \(k_2\) degrees of freedom, \(F_{k_1,k_2}\).
The \(t\) statistics below are of the form in (ii) because of the following theorem, which is used again and again.
Theorem 2.1.2. Let \(Y_1, \dots , Y_n\) be independent \(N(\mu , \sigma ^2)\), with sample mean \(\bar Y\) and sample variance \(S^2 = \sum (Y_i - \bar Y)^2/(n - 1)\). Then
- (i)
- \(\bar Y \sim N(\mu , \sigma ^2/n)\);
- (ii)
- \((n - 1)S^2/\sigma ^2 \sim \chi ^2_{n-1}\);
- (iii)
- \(\bar Y\) and \(S^2\) are independent.
Proof. Let \(Z_i = (Y_i - \mu )/\sigma \), so \(\mathbf Z = (Z_1, \dots , Z_n)^\top \sim N(\mathbf 0, I_n)\). Let \(Q\) be an \(n \times n\) orthogonal matrix whose first row is \((1/\sqrt n, \dots , 1/\sqrt n)\); such a matrix exists, since a unit vector can always be completed to an orthonormal basis. (The Helmert matrix is one explicit choice.) Put \(\mathbf W = Q\mathbf Z\). Then \(\mathbf W \sim N(\mathbf 0, QQ^\top ) = N(\mathbf 0, I_n)\), so \(W_1, \dots , W_n\) are independent \(N(0, 1)\). Now \(W_1 = \sqrt n\,\bar Z\), and because \(Q\) preserves length, \[ \sum _{i=2}^nW_i^2 = \sum _{i=1}^nW_i^2 - W_1^2 = \sum _{i=1}^nZ_i^2 - n\bar Z^2 = \sum _{i=1}^n(Z_i - \bar Z)^2 = \frac {(n - 1)S^2}{\sigma ^2} . \] So \(\bar Y = \mu + \sigma W_1/\sqrt n\) is normal with mean \(\mu \) and variance \(\sigma ^2/n\); \((n - 1)S^2/\sigma ^2\) is a sum of \(n - 1\) squared independent standard normals, hence \(\chi ^2_{n-1}\); and the two depend on disjoint sets of the independent \(W_i\), hence are independent. □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.