1.4 Hypothesis Tests, Recalled
To test a hypothesis about the unknown mean \(\mu \) of a normal distribution we specify a null hypothesis \(H_0 : \mu = \mu _0\), compute the statistic \(t\) from the sample, and compare \(\left |t\right |\) with a tabulated critical value \(t_{\alpha }\).
If \(\left |t\right | \geq t_{\alpha }\) the result is called significant at level \(\alpha \), and we reject \(H_0\). Conventional levels are \(\alpha = 0.05\), \(0.01\) and \(0.001\), described as significant, highly significant and very highly significant.
Note 1.5. What significance does and does not mean. If the result is not significant we fail to reject \(H_0\); we do not accept it, and we certainly do not conclude that it is true. Failing to detect a difference is not the same as demonstrating there is none — a small sample will fail to reject almost any null hypothesis.
Nor does significance establish that \(H_0\) is false. It says the data would be unusual if \(H_0\) held, and at \(\alpha = 0.05\) one test in twenty will say so when \(H_0\) is perfectly true. This is the single most common misreading of a significance test, and Chapter 6 shows what it costs when two hundred tests are run at once.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.