1.5 The Rationale of \(H_0\) and \(\alpha \)
If \(H_0\) is true, values of \(t\) near zero are more likely than large ones of either sign; if \(H_0\) is false, large \(\left |t\right |\) becomes more likely. Choosing \(\alpha = 5\%\) means that over many independent tests performed when \(H_0\) is true, about one in twenty will be declared significant. That is the rate of false positives we agree to tolerate.
Modern software reports the exact probability of a value at least as extreme as the one observed — the \(P\)-value — and \(H_0\) is rejected when \(P \leq \alpha \), which is the same rule as \(\left |t\right | \geq t_\alpha \).
Definition 1.6. A Type I error is rejecting \(H_0\) when it is true; its probability is \(\alpha \). A Type II error is failing to reject \(H_0\) when it is false; its probability is denoted \(\beta \). The power of the test is \(1 - \beta \), the probability of detecting a departure from \(H_0\) when one exists.
Large power is desirable, and it is precisely where non-parametric tests pay for their generality: against normal data the sign test has markedly less power than the \(t\)-test, while against skewed or contaminated data it may have more.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.