2.1 Sign Test
Definition 2.2. Let \(X_1, X_2, \ldots , X_n\) be a random sample from a continuous distribution with median \(M\). The sign test of \[H_0 : M = M_0 \qquad \text {against}\qquad H_1 : M \neq M_0\] discards every observation equal to \(M_0\), replaces each of the remaining \(n'\) observations by the sign of \(X_i - M_0\), and bases the decision on \[N_{+} = \#\{i : X_i > M_0\}, \qquad N_{-} = \#\{i : X_i < M_0\}, \qquad N_{+} + N_{-} = n' .\]
The test uses only the signs of the deviations, never their sizes. That is its strength and its weakness at once: no assumption about the shape of the distribution is needed, but a great deal of information is thrown away, and where the Wilcoxon signed-rank test is applicable it will usually be preferred.
Theorem 2.3. Under \(H_0\) the sign statistic is binomial, \[N_{+} \ \sim \ B\!\left (n', \tfrac 12\right ),\] so that \(E(N_{+}) = \dfrac {n'}{2}\) and \(\operatorname {var}(N_{+}) = \dfrac {n'}{4}\).
Proof. Since the distribution is continuous, \(P(X_i = M_0) = 0\) and each retained observation satisfies \(P(X_i > M_0) = P(X_i < M_0) = \tfrac 12\) under \(H_0\), by the definition of the median. The observations are independent, so \(N_{+}\) counts the successes in \(n'\) independent trials each of probability \(\tfrac 12\). □
Note 2.4. Paired data. For matched pairs \((Y_i, Z_i)\) the test is applied to the differences \(X_i = Y_i - Z_i\) with \(M_0 = 0\), giving a distribution-free alternative to the paired \(t\)-test. For a single sample it replaces the one-sample \(t\)-test, and for ordered categorical data — where a numerical scale is inappropriate but observations can be ranked — it is often the only test available.
Test procedure. Choose whichever of \(N_{+}\), \(N_{-}\) the alternative makes appropriate, and compute the exact binomial tail probability with \(p = \tfrac 12\):
- (i).
- for a one-sided alternative, that tail probability is the \(P\)-value;
- (ii).
- for a two-sided alternative, double it.
Theorem 2.5 (Large-sample approximation). Let \(R = \min \left (N_{+}, N_{-}\right )\). For \(n' \geq 25\) the standardised statistic \[Z = \dfrac {n' - 2R - 1}{\sqrt {n'}}\] is approximately standard normal under \(H_0\), and \(H_0\) is rejected at level \(\alpha \) when \(Z > Z_{\alpha /2}\) for a two-sided test.
Proof. By the previous theorem \(N_{+} \sim B(n',\tfrac 12)\), so the standardised variable is \[\dfrac {N_{+} - n'/2}{\sqrt {n'/4}} = \dfrac {2N_{+} - n'}{\sqrt {n'}} .\] Taking \(R\) to be the smaller count and applying the continuity correction of \(\tfrac 12\) towards the mean replaces \(2R - n'\) by \(2R + 1 - n'\), and changing sign so that the statistic is positive gives \[Z = \dfrac {n' - 2R - 1}{\sqrt {n'}} .\] Equivalently, since \(N_{+} + N_{-} = n'\) gives \(\left |N_{+} - N_{-}\right | = n' - 2R\), \[Z = \dfrac {\left |N_{+}-N_{-}\right | - 1}{\sqrt {n'}} ,\] which is the form quoted in many texts. □
Note 2.6. \(R\) must be the smaller of the two counts. If the larger is used, \(Z\) comes out negative and the test is mis-signed — with \(n' = 30\) and the counts \(20\) and \(10\), the correct value is \(Z = 1.643\) and the wrong choice gives \(-2.008\).
The approximation is good well before \(n' = 25\). At \(n' = 30\) with \(R = 10\) it returns a two-sided \(P\)-value of \(0.1004\) against the exact binomial \(0.0987\); at \(n' = 60\) with \(R = 22\), \(0.0528\) against \(0.0519\). When the decision is close to the boundary, use the exact binomial.
Tied Scores
If a subject scores the same under both conditions the difference is zero and there is no sign to record. The zeros must be dealt with before the test can proceed, and two treatments are in use.
- (i).
- Discard them. Drop the tied subjects and reduce \(n\) accordingly. This is the standard procedure and the one assumed in the tables. Its cost is power: the discarded subjects were observations, and a test on \(n=12\) is weaker than a test on \(n=20\).
- (ii).
- Split them. With an even number of ties, assign half a plus and half a minus; with an odd number greater than one, drop one tied subject at random and split the rest. This keeps more of the sample, at the price of a small arbitrary element.
Discarding is the safer default, and is what should be done unless the ties are numerous. Splitting pulls \(S_+\) towards \(n/2\), which makes rejection harder, so it errs on the conservative side rather than the reckless one.
Note 2.7. Many ties are a warning, not a nuisance to be tidied away. If a quarter of the subjects score identically under both conditions, the measurement is probably too coarse to detect the effect being looked for, and no choice of convention will repair that. Say how many ties there were when reporting the test.
Example 2.8. A random sample of \(10\) students’ weights (kg) from a student population is \[135,\ 119,\ 106,\ 135,\ 180,\ 108,\ 128,\ 160,\ 143,\ 175 .\] Using the sign test, find out whether the population median weight differs from \(140\) kg: \[H_0:\theta =140\hspace {0.6cm}\text {vs}\hspace {0.6cm}H_1:\theta \neq 140 .\]
Solution. Record the sign of \(x_i-140\) for each observation: \[-\hspace {0.3cm}-\hspace {0.3cm}-\hspace {0.3cm}-\hspace {0.3cm}+\hspace {0.3cm}-\hspace {0.3cm}-\hspace {0.3cm}+\hspace {0.3cm}+\hspace {0.3cm}+\] No observation equals \(140\), so there are no ties to discard and \(n=10\), with \[N_+=4,\hspace {1cm}N_+\thicksim B\left (10,\tfrac {1}{2}\right )\ \text { under } H_0 .\]
Since \(N_+=4\) falls below the expected \(5\), the lower tail is the relevant one and the two-sided \(p\)-value is \begin {align*} p &=P\left (N_+\leq 4\right )+P\left (N_+\geq 6\right )\\ &=2\,P\left (N_+\leq 4\right )\hspace {1cm}\text {by symmetry}\\ &=2\sum ^{4}_{i=0}\binom {10}{i}\left (\tfrac {1}{2}\right )^{10}\\ &=2(0.3770)\\ &=0.754 . \end {align*}
Since \(0.754>0.05\) we do not reject \(H_0\). There is no evidence that the median weight differs from \(140\) kg.
For completeness, the critical region: the largest \(c\) with \(2\,P\left (N_+\leq c\right )\leq 0.05\) is \(c=1\), because \(2\,P\left (N_+\leq 1\right )=0.0215\) while \(2\,P\left (N_+\leq 2\right )=0.1094\). So \(H_0\) would be rejected only for \(N_+\leq 1\) or \(N_+\geq 9\), and the observed \(4\) is nowhere near it.
Example 2.9. A psychologist wants to know whether monkeys prefer stimulation of brain area A to stimulation of area B. Fourteen monkeys are taught to press two bars: when a light comes on, presses on bar 1 always stimulate area A and presses on bar 2 always stimulate area B. After the monkeys have learnt the bars they are observed for \(15\) months, and the frequency of presses on each bar is recorded.
| Bar 1 | Bar 2 | Sign of the |
| dev(1-2) | ||
| 20 | 40 | - |
| 18 | 25 | - |
| 24 | 38 | - |
| 14 | 27 | - |
| 5 | 31 | - |
| 26 | 21 | + |
| 15 | 32 | - |
| 29 | 38 | - |
| 15 | 25 | - |
| 9 | 18 | - |
| 25 | 32 | - |
| 31 | 28 | + |
| 35 | 33 | + |
| 12 | 29 | - |
Find out whether there is a difference between the two conditions at the \(5\%\) level of significance.
Solution. Formulation and assumptions. The researcher did not predict a particular direction, only that the two conditions might differ, so the alternative is non-directional: \[H_0:\text {no difference between the two bars},\hspace {0.8cm} H_1:\text {the two bars differ}\hspace {0.3cm}(\text {2-sided}).\] An extreme number of plus signs, whether small or large, supports \(H_1\); a middling number is consistent with \(H_0\).
Procedure
Null distribution. All \(14\) subjects gave a non-zero difference, so \(n=14\). Under \(H_0\) a plus and a minus are equally likely for each subject independently, so the number of plus signs is \[S_+\sim \operatorname {Bin}\left (14,\tfrac {1}{2}\right ).\] Note that \(n=14\) is the sample size and \(S_+\) is the statistic; they are different quantities and must not share a symbol.
Statistic. The table gives \[S_+=3\hspace {0.8cm}\text {and}\hspace {0.8cm}S_-=11,\hspace {0.8cm} S=\min \left (S_+,S_-\right )=3 .\]
Critical region. For a two-sided test at \(\alpha =0.05\) the critical value is the largest \(c\) with \(2\,P\left (S_+\leq c\right )\leq 0.05\). From the binomial table, \[2\,P\left (S_+\leq 2\right )=2(0.00647)=0.0129\leq 0.05,\hspace {0.8cm} 2\,P\left (S_+\leq 3\right )=2(0.02869)=0.0574>0.05,\] so \(c=2\) and the critical region is \(S\leq 2\), that is \(S_+\leq 2\) or \(S_+\geq 12\). The observed \(S=3\) lies outside it.
\(p\)-value. Equivalently, and more informatively, \[p=P\left (S_+\leq 3\right )+P\left (S_+\geq 11\right )=2(0.02869)=0.0574 .\]
Conclusion: \(0.0574>0.05\), so there is not sufficient evidence to reject \(H_0\) at the \(5\%\) level. The monkeys have not been shown to prefer one area to the other.
Note 2.10. The result is a near miss, and it is worth being clear about what that does and does not mean. A \(p\)-value of \(0.0574\) is not evidence that the two conditions agree; it is evidence that fell just short of the conventional threshold with only \(14\) subjects. The same imbalance of \(3\) against \(11\) in a larger study would be decisive. Report the \(p\)-value, not merely the verdict, and let the reader judge.
Note also that \(3\) and \(11\) are the observed statistic and its mirror image, not the critical values. Reading a critical region off the data is circular: every observation lies on the boundary of the region it defines, so the test would always be on the edge of significance.
Example 2.11. Seventeen teachers attend a summer school intended to improve their listening skills, and each is given a pretest and a posttest. Sixteen improved and one did worse. Did attendance improve performance? Take \(\alpha =0.05\).
Solution. Each teacher contributes a plus if the posttest is the higher and a minus otherwise; no teacher scored the same twice, so \(n=17\) with \(S_+=16\) and \(S_-=1\).
The question asks whether performance improved, so the alternative is one-sided: \[H_0:\theta =0\hspace {0.6cm}\text {vs}\hspace {0.6cm}H_a:\theta >0 ,\] \(\theta \) being the median improvement. Under \(H_0\), \(S_-\sim \operatorname {Bin}\left (17,\tfrac {1}{2}\right )\), and evidence of improvement is a small number of minuses: \[p=P\left (S_-\leq 1\right )=\dfrac {\binom {17}{0}+\binom {17}{1}}{2^{17}} =\dfrac {1+17}{131072}=0.000137 .\]
Since \(0.000137<0.05\) we reject \(H_0\) decisively. There is very strong evidence that the summer school improved listening performance.
Note 2.12. Two remarks. First, the significance level is \(\alpha \); \(\theta \) is being used here for the population median and the two must not be confused. Second, the sign test has used only the directions of the seventeen changes and none of their sizes, and it still reaches \(p=0.000137\). When the effect is this one-sided, the crudest test suffices — which is the argument for the sign test as a first look at paired data.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.