2.2 Measuring disease: sensitivity and specificity
A case definition is applied through some procedure (a questionnaire, a clinical examination, a laboratory test) that will classify some people wrongly. Let \(D\) denote true disease status and \(T^+\), \(T^-\) a positive or negative result of the procedure.
Definition 2.2.1. The sensitivity of a classification procedure is \(\text {Se} = P(T^+ \mid D)\), the probability that a true case is classified as a case. Its specificity is \(\text {Sp} = P(T^- \mid \bar D)\), the probability that a non-case is classified as a non-case. The positive predictive value is \(\text {PPV} = P(D \mid T^+)\) and the negative predictive value is \(\text {NPV} = P(\bar D \mid T^-)\).
Sensitivity and specificity describe the procedure and, to a first approximation, do not depend on the population it is applied to. The predictive values describe what a result means for a person, and depend heavily on how common the disease is.
Proposition 2.2.2. If the prevalence of disease is \(p\), then \[ \text {PPV} = \frac {\text {Se}\,p}{\text {Se}\,p + (1 - \text {Sp})(1 - p)}, \qquad \text {NPV} = \frac {\text {Sp}\,(1 - p)}{\text {Sp}\,(1 - p) + (1 - \text {Se})\,p} . \] PPV increases and NPV decreases with \(p\); as \(p \to 0\), \(\text {PPV} \to 0\) whenever \(\text {Sp} < 1\).
Proof. The formulas are Bayes’ theorem. Dividing numerator and denominator of PPV by \(\text {Se}\,p\) gives \(\text {PPV} = \{1 + (1 - \text {Sp})(1 - p)/(\text {Se}\,p)\}^{-1}\), and \((1 - p)/p\) decreases in \(p\), so PPV increases; it tends to \(0\) as \(p \to 0\) because \((1 - p)/p \to \infty \). The argument for NPV is the same with \((1 - p)\) and \(p\) exchanged. □
Example 2.2.3. A test has sensitivity \(0.90\) and specificity \(0.95\). Where the prevalence is \(1\%\), \[ \text {PPV} = \frac {0.90 \times 0.01}{0.90 \times 0.01 + 0.05 \times 0.99} = \frac {0.009}{0.0585} = 0.154 , \] so only about one positive in six is a true case; the false positives from the \(99\%\) who are healthy outnumber the true positives from the \(1\%\) who are ill. Where the prevalence is \(20\%\), as in a clinic of people with symptoms, the same test has \(\text {PPV} = 0.818\). The NPV is \(0.999\) and \(0.974\) in the two settings.
Estimating prevalence from an imperfect test
If a survey classifies \(n\) people with a test of known sensitivity and specificity, the proportion testing positive estimates the apparent prevalence \begin {equation} \label {eq:apparent} p^* = P(T^+) = \text {Se}\,p + (1 - \text {Sp})(1 - p) , \end {equation} not the true prevalence \(p\). Whenever \(\text {Se} + \text {Sp} > 1\) (that is, whenever the test is better than tossing a coin) the relation can be inverted.
Theorem 2.2.4 (Rogan–Gladen estimator). Let \(\hat p^*\) be the proportion testing positive in a random sample of size \(n\), and \(J = \text {Se} + \text {Sp} - 1 > 0\). Then \[ \hat p = \frac {\hat p^* + \text {Sp} - 1}{J} \] is unbiased for \(p\), with variance \(p^*(1 - p^*)/(nJ^2)\).
Proof. From (2.1), \(p^* = (1 - \text {Sp}) + Jp\), so \(p = (p^* + \text {Sp} - 1)/J\). The number testing positive is \(\text {Bin}(n, p^*)\), so \(E(\hat p^*) = p^*\) and \(\Var (\hat p^*) = p^*(1 - p^*)/n\). The estimator is a linear function of \(\hat p^*\) with slope \(1/J\), which gives both results. □
The quantity \(J\) is Youden’s index. The variance is inflated by \(1/J^2\) compared with a perfect test, the price of the misclassification. The estimate can fall outside \([0, 1]\) when \(\hat p^*\) is below \(1 - \text {Sp}\) or above Se; it is then truncated to the nearer end.
Example 2.2.5. In a survey of \(1000\) people, \(130\) test positive on the test of the last example. The apparent prevalence is \(0.130\) and \begin {gather*} \hat p = \frac {0.130 + 0.95 - 1}{0.85} = 0.094, \\ \text {SE}(\hat p) = \frac {\sqrt {0.130 \times 0.870/1000}}{0.85} = 0.0125 , \end {gather*} with approximate \(95\%\) interval \(0.094 \pm 1.96 \times 0.0125 = (0.070, 0.119)\). Taking \(13\%\) at face value would overstate the prevalence by a third.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.