2.6 Problems

Problem 2.6.1. A screening test for a disease has sensitivity \(0.95\) and specificity \(0.98\). Find the positive predictive value when the prevalence is \(0.001\), \(0.01\) and \(0.1\). Explain why mass screening of a general population for a rare disease generates mostly false positives, and what can be done about it.

Show solution

Solution. \(\text {PPV} = 0.95p/\{0.95p + 0.02(1 - p)\}\), which is \(0.045\) at \(p = 0.001\), \(0.324\) at \(p = 0.01\) and \(0.841\) at \(p = 0.1\). At low prevalence the \(2\%\) false positive rate applies to almost the whole population, while the \(95\%\) true positive rate applies to very few, so false positives dominate (Proposition 2.2.2). Remedies are to screen only a higher-prevalence group (by age or risk factors), or to follow a positive result with a second, more specific test; testing in series raises the effective specificity to about \(1 - (1 - \text {Sp}_1)(1 - \text {Sp}_2)\) if the two tests err independently.

Problem 2.6.2. In a serological survey \(84\) of \(600\) people test positive on an assay with sensitivity \(0.92\) and specificity \(0.97\). Estimate the true prevalence with a \(95\%\) confidence interval. What would the estimate be if only \(15\) had tested positive?

Show solution

Solution. \(\hat p^* = 84/600 = 0.140\) and \(J = 0.89\), so \(\hat p = (0.140 - 0.03)/0.89 = 0.124\) and \(\text {SE} = \sqrt {0.14 \times 0.86/600}/0.89 = 0.0159\). The interval is \(0.124 \pm 0.031 = (0.093, 0.155)\). With \(15\) positives, \(\hat p^* = 0.025 < 1 - \text {Sp} = 0.03\), so \(\hat p = (0.025 - 0.03)/0.89 < 0\): fewer people tested positive than the false positive rate alone would produce. The estimate is truncated to \(0\); the data are consistent with no infection at all.

Problem 2.6.3. Show that \(\kappa \) can be low even when the observed agreement is high. Two observers classify \(100\) subjects; they agree that \(90\) are negative, both call \(2\) positive, and they disagree on the remaining \(8\), split equally. Compute \(p_o\) and \(\kappa \), and explain.

Show solution

Solution. Each observer calls \(2 + 4 = 6\) positive, so \(p_e = 0.06^2 + 0.94^2 = 0.8872\) and \(p_o = 0.92\). Then \(\kappa = (0.92 - 0.8872)/(1 - 0.8872) = 0.29\). When one category is rare, two observers agree on most subjects by chance, so \(p_e\) is close to \(1\) and the agreement beyond chance is small: the observers agree on only \(2\) of the \(10\) subjects that either of them called positive. Kappa depends on the prevalence of the trait, and should be reported with the marginal proportions.

Problem 2.6.4. In a cohort study the true risks are \(p_1 = 0.02\) and \(p_0 = 0.01\). Find the observed risk ratio if disease is ascertained (a) with sensitivity \(0.7\) and specificity \(1\), (b) with sensitivity \(1\) and specificity \(0.99\), in both groups. Comment.

Show solution

Solution. (a) \(p_i^* = 0.7p_i\), so \(\RR ^* = 0.014/0.007 = 2\), unbiased (Theorem 2.4.2, \(\text {Sp} = 1\)). (b) \(p_i^* = 0.01 + 0.99p_i\), so \(\RR ^* = (0.01 + 0.0198)/(0.01 + 0.0099) = 0.0298/0.0199 = 1.50\). Losing \(30\%\) of the cases costs only precision; a \(1\%\) false positive rate, applied to a population in which the disease itself has a risk of \(1\)–\(2\%\), adds as many false cases as there are true ones in the unexposed and pulls the ratio a long way towards \(1\). For a rare disease, specificity is what matters.

Problem 2.6.5. Give an example, with numbers, in which differential misclassification of exposure biases the odds ratio away from the null when there is a true association, and one in which it reverses the direction of the association.

Show solution

Solution. Take \(q_1 = 0.4\), \(q_0 = 0.2\) (true \(\OR = 2.67\)) and \(\text {Sp} = 1\) in both groups. If cases report exposure with \(\text {Se} = 1\) and controls with \(\text {Se} = 0.5\), then \(q_1^* = 0.4\), \(q_0^* = 0.1\) and \(\OR ^* = (0.4/0.6)/(0.1/0.9) = 6.0\), biased away from the null. If instead cases under-report with \(\text {Se} = 0.4\) and controls report fully, \(q_1^* = 0.16\), \(q_0^* = 0.2\) and \(\OR ^* = (0.16/0.84)/(0.2/0.8) = 0.76\): a harmful exposure appears protective. Differential error has no predictable direction, which is why it is worse than nondifferential error of the same size.

Problem 2.6.6. In a cohort of \(5000\) people, \(20\%\) are lost to follow-up. Among those followed, the five-year risk is \(0.08\). Using Proposition 2.5.1, find the range of the true five-year risk if the risk among those lost could be anything from \(0\) to \(0.30\). What assumption makes the complete-case risk unbiased?

Show solution

Solution. The true risk is \(0.8 \times 0.08 + 0.2\,\mu _N = 0.064 + 0.2\,\mu _N\), which ranges from \(0.064\) (if no one lost became ill) to \(0.124\) (if \(30\%\) did). The complete-case estimate \(0.08\) is unbiased only if the risk among those lost equals the risk among those followed: loss to follow-up unrelated to outcome. A range like this, computed from extreme but plausible assumptions about the missing, is a simple and honest sensitivity analysis.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.