1.4 Independent Events, Conditional Probability and Bayes’ Theorem

1.4.1 Conditional probability

Often we learn that some event \(B\) has occurred and want to revise the probability of another event \(A\). Knowing that \(B\) occurred restricts the possible outcomes to \(B\), which becomes the new sample space, and only the part of \(A\) inside \(B\) remains possible.

Definition 1.4.1. If \(P(B)>0\), the conditional probability of \(A\) given \(B\) is \[P(A\mid B) = \frac {P(A\cap B)}{P(B)} .\] It is not defined when \(P(B) = 0\).

For fixed \(B\), the function \(A\mapsto P(A\mid B)\) satisfies the three axioms, so all of theorem 1.3.4 applies to it; for example \(P(A'\mid B) = 1 - P(A\mid B)\). Taking \(B = S\) gives \(P(A\mid S) = P(A)\): every probability is conditional on the sample space. Rearranging the definition gives the multiplication rule \begin {equation} \label {eq:multrule} P(A\cap B) = P(B)P(A\mid B) = P(A)P(B\mid A). \end {equation}

Example 1.4.2. Two cards are drawn without replacement from a standard deck of \(52\). Find the probability that both are aces.

Solution. Let \(A_1\) and \(A_2\) be the events that the first and second cards are aces. Then \(P(A_1) = \frac {4}{52}\), and given \(A_1\) the deck has \(51\) cards of which \(3\) are aces, so \(P(A_2\mid A_1) = \frac {3}{51}\). By (1.1), \[P(A_1\cap A_2) = \frac {4}{52}\cdot \frac {3}{51} = \frac {1}{221} = 0.0045 .\]

1.4.2 Independent events

Definition 1.4.3. Events \(A\) and \(B\) are independent if \(P(A\cap B) = P(A)P(B)\).

When \(P(B)>0\) this is the same as \(P(A\mid B) = P(A)\): learning that \(B\) occurred does not change the probability of \(A\). Independence is not the same as mutual exclusiveness. If \(A\) and \(B\) are mutually exclusive and both have positive probability, then \(P(A\cap B) = 0\ne P(A)P(B)\), so they are dependent: learning that \(B\) occurred tells us that \(A\) did not.

Theorem 1.4.4. If \(A\) and \(B\) are independent, then so are \(A\) and \(B'\), \(A'\) and \(B\), and \(A'\) and \(B'\).

Proof. Since \(A\) is the disjoint union of \(A\cap B\) and \(A\cap B'\), \[P(A\cap B') = P(A) - P(A\cap B) = P(A) - P(A)P(B) = P(A)\left [1 - P(B)\right ] = P(A)P(B').\] So \(A\) and \(B'\) are independent. Exchanging the roles of \(A\) and \(B\) shows that \(A'\) and \(B\) are independent. Applying the first result to the independent pair \(A'\), \(B\) shows that \(A'\) and \(B'\) are independent. □

Definition 1.4.5. Events \(A_1, \dots , A_n\) are mutually independent if for every subcollection \(A_{i_1}, \dots , A_{i_k}\) with \(i_1<\dots <i_k\), \[P(A_{i_1}\cap \cdots \cap A_{i_k}) = P(A_{i_1})\cdots P(A_{i_k}).\]

It is not enough that each pair be independent.

Example 1.4.6. Two fair coins are tossed. Let \(A\) be a head on the first coin, \(B\) a head on the second, and \(C\) the event that the coins agree. Show that \(A\), \(B\), \(C\) are pairwise independent but not mutually independent.

Solution. Each of \(A\), \(B\) and \(C = \{HH, TT\}\) has probability \(\frac 12\). Each pairwise intersection is \(\{HH\}\), with probability \(\frac 14 = \frac 12\cdot \frac 12\), so each pair is independent. But \(A\cap B\cap C = \{HH\}\) has probability \(\frac 14\ne \frac 18 = P(A)P(B)P(C)\). Once \(A\) and \(B\) are known, \(C\) is certain.

1.4.3 Total probability and Bayes’ theorem

Definition 1.4.7. Events \(B_1, \dots , B_k\) form a partition of \(S\) if they are mutually exclusive and \(\bigcup _{j=1}^kB_j = S\).

Theorem 1.4.8 (Total probability). If \(B_1, \dots , B_k\) is a partition of \(S\) with each \(P(B_j)>0\), then for any event \(A\), \[P(A) = \sum _{j=1}^kP(B_j)P(A\mid B_j).\]

Proof. \(A = (A\cap B_1)\cup \cdots \cup (A\cap B_k)\), a union of mutually exclusive events. By (A3) and (1.1), \(P(A) = \sum _jP(A\cap B_j) = \sum _jP(B_j)P(A\mid B_j)\). □

In many problems the outcome depends on what happened at an intermediate stage, and we want to reason backwards from the outcome to the stage. Bayes’ theorem does this.

Theorem 1.4.9 (Bayes’ theorem). If \(B_1, \dots , B_k\) is a partition of \(S\) with each \(P(B_j)>0\), and \(P(A)>0\), then \[P(B_r\mid A) = \frac {P(B_r)P(A\mid B_r)}{\sum _{j=1}^kP(B_j)P(A\mid B_j)},\qquad r = 1, \dots , k .\]

Proof. By the definition of conditional probability and (1.1), \(P(B_r\mid A) = \frac {P(B_r\cap A)}{P(A)} = \frac {P(B_r)P(A\mid B_r)}{P(A)}\), and the denominator is given by theorem 1.4.8. □

The probabilities \(P(B_j)\) are the prior probabilities of the \(B_j\), and \(P(B_j\mid A)\) are their posterior probabilities, after \(A\) has been observed.

Example 1.4.10. In a factory, machines \(A\), \(B\) and \(C\) make \(25\%\), \(35\%\) and \(40\%\) of the car batteries, and \(5\%\), \(4\%\) and \(2\%\) of their output respectively is defective.

(i)
A battery is chosen at random. What is the probability that it is defective?
(ii)
A battery chosen at random is defective. What is the probability that it was made by machine \(B\)?
(iii)
A battery chosen at random is not defective. What is the probability that it was made by machine \(A\)?

Solution. The machines form a partition. Let \(D\) be the event that the battery is defective, so \(P(D\mid A) = 0.05\), \(P(D\mid B) = 0.04\) and \(P(D\mid C) = 0.02\). The tree diagram in figure 1.4.3 shows the two stages.

(i)
By theorem 1.4.8, \[P(D) = 0.25(0.05) + 0.35(0.04) + 0.40(0.02) = 0.0125 + 0.0140 + 0.0080 = 0.0345 .\]
(ii)
By theorem 1.4.9, \(P(B\mid D) = \frac {0.35(0.04)}{0.0345} = \frac {0.0140}{0.0345} = \frac {28}{69} = 0.406\). Machine \(B\) makes \(35\%\) of the batteries but \(40.6\%\) of the defective ones.
(iii)
\(P(D') = 1 - 0.0345 = 0.9655\), so \(P(A\mid D') = \frac {0.25(0.95)}{0.9655} = \frac {475}{1931} = 0.246\).

A0B0C0D0D0D0D0DD.......2340′90′9′55046550 0.0.298

Example 1.4.11. A disease affects \(1\%\) of a population. A test detects it in \(95\%\) of people who have it, and gives a false positive in \(10\%\) of people who do not. A person tests positive. What is the probability that they have the disease?

Solution. Let \(D\) be having the disease and \(+\) a positive test. Then \(P(D) = 0.01\), \(P(+\mid D) = 0.95\) and \(P(+\mid D') = 0.10\). By total probability, \(P(+) = 0.01(0.95) + 0.99(0.10) = 0.0095 + 0.0990 = 0.1085\), and by Bayes’ theorem \[P(D\mid +) = \frac {0.0095}{0.1085} = 0.088 .\] Fewer than \(9\%\) of those who test positive have the disease. Because the disease is rare, the false positives from the large healthy group far outnumber the true positives.

1.4.4 Contingency tables

A contingency table classifies observations by two or more categorical variables. For two variables the rows are the levels of one and the columns the levels of the other; each cell holds the joint probability (or count) of a row level and a column level occurring together. The row and column totals, the margins, give the probabilities of each level of one variable regardless of the other, and the table contains what is needed to judge whether the two variables are associated.

Example 1.4.12. Let \(A\) be the event that a person smokes and \(D\) that the person develops lung cancer. Suppose \(P(A\cap D) = 0.15\), \(P(A\cap D') = 0.25\), \(P(A'\cap D) = 0.10\) and \(P(A'\cap D') = 0.50\). Use these to examine the relationship between smoking and lung cancer.

Solution. Summing rows and columns gives the margins in table 1.1.

Table 1.1: Joint and marginal probabilities of smoking and lung cancer.
Cancer (\(D\)) No cancer (\(D'\)) Total
Smoker (\(A\)) \(0.15\) \(0.25\) \(0.40\)
Non-smoker (\(A'\)) \(0.10\) \(0.50\) \(0.60\)
Total \(0.25\) \(0.75\) \(1.00\)

If smoking and cancer were independent we would have \(P(A\cap D) = P(A)P(D) = 0.40(0.25) = 0.10\); the actual value is \(0.15\), so they are dependent. The risks of cancer in the two groups are \[P(D\mid A) = \frac {0.15}{0.40} = 0.375,\qquad P(D\mid A') = \frac {0.10}{0.60} = 0.167,\] and their ratio, the relative risk, is \(\frac {0.375}{0.167} = 2.25\): in this population a smoker is \(2.25\) times as likely as a non-smoker to develop lung cancer. Another common measure is the odds ratio, which for a \(2\times 2\) table with cells \(a\), \(b\) in the first row and \(c\), \(d\) in the second is \[\text {OR} = \frac {ad}{bc} = \frac {0.15(0.50)}{0.25(0.10)} = 3 .\] It compares the odds of cancer, \(\frac {P(D\mid A)}{P(D'\mid A)} = 0.6\) for smokers and \(\frac {P(D\mid A')}{P(D'\mid A')} = 0.2\) for non-smokers. Either measure equals \(1\) exactly when the two events are independent.

1.4.5 Practice Problems

Problems on conditional probability, independence and Bayes’ theorem. Try each one before opening the solution.

Problem 1.4.1. A manufacturer of airplane parts knows from experience that the probability is \(0.80\) that an order will be ready for shipment on time, and \(0.72\) that an order will be ready for shipment on time and will also be delivered on time. What is the probability that such an order will be delivered on time given that it was ready for shipment on time?

Show solution

Solution. Let \(R\) be the event that the order is ready on time and \(D\) that it is delivered on time. We are given \(P(R)=0.80\) and \(P(R\cap D)=0.72\). By the definition of conditional probability, \[P(D\mid R) = \frac {P(R\cap D)}{P(R)} = \frac {0.72}{0.80} = 0.9.\]

Problem 1.4.2. If we randomly pick two television sets in succession from a shipment of 240 television sets of which 15 are defective, what is the probability that they will both be defective?

Show solution

Solution. The sets are picked in succession without replacement, so the second probability is conditional on the first. Let \(D_1\) and \(D_2\) be the events that the first and second set are defective. \[P(D_1\cap D_2) = P(D_1)\,P(D_2\mid D_1) = \frac {15}{240}\times \frac {14}{239} = \frac {7}{1912} \approx 0.0037.\] After one defective set is removed, 14 of the remaining 239 are defective – that is what makes the second factor \(\tfrac {14}{239}\) and not \(\tfrac {15}{240}\).

Problem 1.4.3. Two events \(A\) and \(B\) are such that \(P(A)=\frac {1}{4}\), \(P(A\mid B)=\frac {1}{2}\) and \(P(B\mid A)=\frac {2}{3}\).

(a).
Are \(A\) and \(B\) independent events?
(b).
Find \(P(A\cap B)\).
(c).
Are \(A\) and \(B\) mutually exclusive events?
(d).
Find \(P(B)\).
(e).
Find \(P(A\cup B)\).

Show solution

Solution. It is easiest to find \(P(A\cap B)\) first and answer the rest from it.

(b). From \(P(B\mid A)=\frac {P(A\cap B)}{P(A)}\), \[P(A\cap B) = P(B\mid A)\,P(A) = \frac {2}{3}\times \frac {1}{4} = \frac {1}{6}.\]

(a). \(A\) and \(B\) are independent only if \(P(A\mid B)=P(A)\). Here \(P(A\mid B)=\tfrac 12\) but \(P(A)=\tfrac 14\), so they are not independent. (Equivalently \(P(A)P(B)=\tfrac 14\times \tfrac 13=\tfrac 1{12}\neq \tfrac 16=P(A\cap B)\).)

(c). Mutually exclusive would mean \(P(A\cap B)=0\). Here \(P(A\cap B)=\tfrac 16\neq 0\), so they are not mutually exclusive.

(d). From \(P(A\mid B)=\frac {P(A\cap B)}{P(B)}\), \[P(B) = \frac {P(A\cap B)}{P(A\mid B)} = \frac {1/6}{1/2} = \frac {1}{3}.\]

(e). By the addition rule, \[P(A\cup B) = P(A)+P(B)-P(A\cap B) = \frac {1}{4}+\frac {1}{3}-\frac {1}{6} = \frac {5}{12}.\]

Problem 1.4.4. Events \(A\) and \(B\) are such that \(P(A)=\frac {2}{5}\) and \(P(B)=\frac {1}{4}\).

(a).
If \(A\) and \(B\) are independent events, find (i) \(P(A\cap B)\), (ii) \(P(A\cap B')\), (iii) \(P(A\cup B)\).
(b).
If \(A\) and \(B\) are mutually exclusive, find (i) \(P(A\cup B)\), (ii) \(P(A'\cap B)\).

Show solution

Solution. (a). Independence gives \(P(A\cap B)=P(A)P(B)\), and \(A\) is then also independent of \(B'\). \begin {align*} \text {(i)}\quad P(A\cap B) &= \frac {2}{5}\times \frac {1}{4} = \frac {1}{10}.\\ \text {(ii)}\quad P(A\cap B') &= P(A)\,P(B') = \frac {2}{5}\times \frac {3}{4} = \frac {3}{10}.\\ \text {(iii)}\quad P(A\cup B) &= \frac {2}{5}+\frac {1}{4}-\frac {1}{10} = \frac {11}{20}. \end {align*}

(b). Mutually exclusive means \(A\cap B=\emptyset \), so \(P(A\cap B)=0\). \begin {align*} \text {(i)}\quad P(A\cup B) &= P(A)+P(B) = \frac {2}{5}+\frac {1}{4} = \frac {13}{20}.\\ \text {(ii)}\quad P(A'\cap B) &= P(B)-P(A\cap B) = \frac {1}{4}-0 = \frac {1}{4}. \end {align*}

Note that the two parts describe different situations: events cannot in general be both independent and mutually exclusive.

Problem 1.4.5. Events \(A\) and \(B\) are such that \(P(A)=\frac {4}{7}\), \(P(A\cap B')=\frac {1}{3}\) and \(P(A\mid B)=\frac {5}{14}\). Find

(a).
\(P(A\cap B)\);
(b).
\(P(B)\);
(c).
\(P(B\mid A)\).

Show solution

Solution. (a). The events \(A\cap B\) and \(A\cap B'\) partition \(A\), so \[P(A\cap B) = P(A)-P(A\cap B') = \frac {4}{7}-\frac {1}{3} = \frac {12}{21}-\frac {7}{21} = \frac {5}{21}.\]

(b). From \(P(A\mid B)=\frac {P(A\cap B)}{P(B)}\), \[P(B) = \frac {P(A\cap B)}{P(A\mid B)} = \frac {5/21}{5/14} = \frac {5}{21}\times \frac {14}{5} = \frac {2}{3}.\]

(c). \[P(B\mid A) = \frac {P(A\cap B)}{P(A)} = \frac {5/21}{4/7} = \frac {5}{21}\times \frac {7}{4} = \frac {5}{12}.\]

Problem 1.4.6. A box contains 4 black, 6 white and 2 red balls. Balls are picked out of the box without replacement. Find the probability that

(a).
the first ball picked is black and the second one is white;
(b).
the second ball picked is white;
(c).
the first ball picked is black, the second is white and the third is red;
(d).
the first three balls picked are of different colours.

Show solution

Solution. There are \(4+6+2=12\) balls, and each pick removes one.

(a). \[P(B_1\cap W_2) = \frac {4}{12}\times \frac {6}{11} = \frac {24}{132} = \frac {2}{11}.\]

(b). The second ball can be white either after a white first ball or after a non-white one: \[P(W_2) = \frac {6}{12}\cdot \frac {5}{11} + \frac {6}{12}\cdot \frac {6}{11} = \frac {30+36}{132} = \frac {66}{132} = \frac {1}{2}.\] This is just \(\tfrac {6}{12}\), the same as \(P(W_1)\) – and that is no accident. Before any ball is seen, the second position is as likely to hold any particular ball as the first is, so drawing without replacement does not favour one position over another.

(c). \[P(B_1\cap W_2\cap R_3) = \frac {4}{12}\times \frac {6}{11}\times \frac {2}{10} = \frac {48}{1320} = \frac {2}{55}.\]

(d). Three different colours means one black, one white and one red in some order. Each particular order has the same probability \(\tfrac {48}{1320}\) as in (c), and there are \(3!=6\) orders, so \[P = 6\times \frac {48}{1320} = \frac {288}{1320} = \frac {12}{55}.\]

Problem 1.4.7. The members of a consulting firm rent cars from three rental agencies: 60 percent from agency 1, 30 percent from agency 2, and 10 percent from agency 3. If 9 percent of the cars from agency 1 need an oil change, 20 percent of the cars from agency 2 need an oil change, and 6 percent of the cars from agency 3 need an oil change, what is the probability that a rental car delivered to the firm will need an oil change?

Show solution

Solution. The three agencies partition the sample space, so this is the law of total probability. Let \(A_1, A_2, A_3\) be the agencies and \(C\) the event that the car needs an oil change. \begin {align*} P(C) &= P(A_1)P(C\mid A_1)+P(A_2)P(C\mid A_2)+P(A_3)P(C\mid A_3)\\ &= (0.60)(0.09)+(0.30)(0.20)+(0.10)(0.06)\\ &= 0.054+0.060+0.006 = 0.12. \end {align*}

Problem 1.4.8. A rare but serious disease, \(D\), has been found in \(0.01\) percent of a certain population. A test has been developed that will be positive, \(p\), for 98 percent of those who have the disease and be positive for only 3 percent of those who do not have the disease. Find the probability that a person who tests positive does not have the disease.

Show solution

Solution. Note that \(0.01\) percent means \(P(D)=0.0001\), so \(P(D')=0.9999\). We are given \(P(p\mid D)=0.98\) and \(P(p\mid D')=0.03\). By the law of total probability, \begin {align*} P(p) &= P(D)P(p\mid D)+P(D')P(p\mid D')\\ &= (0.0001)(0.98)+(0.9999)(0.03)\\ &= 0.000098+0.029997 = 0.030095. \end {align*}

Then by Bayes’ theorem, \[P(D'\mid p) = \frac {P(D')P(p\mid D')}{P(p)} = \frac {0.029997}{0.030095} \approx 0.9967.\] Almost every positive result is a false alarm. This is not a fault in the test – it is what happens when a disease is rare: the 3% of a very large healthy group vastly outnumbers the 98% of a very small sick one.

Problem 1.4.9. A random sample of 200 adults is classified below according to sex and the level of education attained.

Education Male Female Total
Primary 38 45 83
Secondary 28 50 78
College 22 17 39
Total 88 112 200
Table 1.2: Level of education attained by 200 adults, by sex.

If a person is picked at random from the group, find the probability that the person is

(a).
a female;
(b).
a male, given that the person has secondary education;
(c).
not a college graduate, given that the person is a female.

Show solution

Solution. (a). There are \(45+50+17 = 112\) females out of 200: \[P(F) = \frac {112}{200} = \frac {14}{25} = 0.56.\]

(b). Conditioning on secondary education restricts attention to that row, which has \(28+50=78\) people, of whom 28 are male: \[P(M\mid S) = \frac {28}{78} = \frac {14}{39} \approx 0.359.\]

(c). Conditioning on female restricts attention to that column, which has 112 people, of whom \(45+50=95\) are not college graduates: \[P(C'\mid F) = \frac {95}{112} \approx 0.848.\]

Problem 1.4.10. A large industrial firm uses 3 local motels to provide overnight accommodation for its clients. From past experience it is known that 20% of the clients are assigned rooms at the Ramada Inn, 50% at the Sheraton, and 30% at the Lakeview Lodge. If the plumbing is faulty in 5% of the rooms at the Ramada Inn, in 4% of the rooms at the Sheraton, and in 8% of the rooms at the Lakeview Lodge, what is the probability that

(a).
a client will be assigned a room with faulty plumbing?
(b).
a person with a room having faulty plumbing was assigned accommodation at the Lakeview Lodge?

Show solution

Solution. Let \(R\), \(S\) and \(L\) be the three motels and \(F\) the event of faulty plumbing.

(a). The three motels partition the clients, so by the law of total probability \begin {align*} P(F) &= P(R)P(F\mid R)+P(S)P(F\mid S)+P(L)P(F\mid L)\\ &= (0.20)(0.05)+(0.50)(0.04)+(0.30)(0.08)\\ &= 0.010+0.020+0.024 = 0.054. \end {align*}

(b). This reverses the conditioning, so it is Bayes’ theorem: \[P(L\mid F) = \frac {P(L)P(F\mid L)}{P(F)} = \frac {0.024}{0.054} = \frac {4}{9} \approx 0.444.\] The Lakeview Lodge takes only 30% of the clients but accounts for 44% of the complaints, because its rooms are the worst.

Problem 1.4.11. Three members of a private club have been nominated for the office of president. The three members are Mr Tembo, Mr Chela and Mr Haziyu, and their probabilities of being elected are \(0.3\), \(0.4\) and \(0.3\) respectively. Should Mr Tembo be elected, the probability of an increase in membership fees is \(0.6\); should Mr Chela or Mr Haziyu be elected, the corresponding probabilities are \(0.2\) and \(0.4\).

(a).
What is the probability that there will be an increase in membership fees?
(b).
Someone considering joining the club delays the decision for several weeks, only to find that the fees have been increased. What is the probability that Mr Chela was elected president?

Show solution

Solution. Let \(T\), \(C\) and \(H\) be the events that each man is elected, and \(I\) the event of an increase in fees. Exactly one of the three is elected, so they partition the sample space – note \(0.3+0.4+0.3 = 1\).

(a). By the law of total probability, \begin {align*} P(I) &= P(T)P(I\mid T)+P(C)P(I\mid C)+P(H)P(I\mid H)\\ &= (0.3)(0.6)+(0.4)(0.2)+(0.3)(0.4)\\ &= 0.18+0.08+0.12 = 0.38. \end {align*}

(b). The fee increase is known and the election is what we are asking about, so the conditioning is reversed and this is Bayes’ theorem: \[P(C\mid I) = \frac {P(C)P(I\mid C)}{P(I)} = \frac {0.08}{0.38} = \frac {4}{19} \approx 0.211.\] Mr Chela was the most likely to be elected, at \(0.4\), yet learning that the fees went up makes him the least likely of the three to have won – he was the candidate least inclined to raise them.

Problem 1.4.12. Psychologists tend to believe that there is a relationship between aggressiveness and order of birth. The percentages of students falling into the four categories are:

Aggressiveness First born Not first born
Aggressive 25% 20%
Not aggressive 15% 40%
Table 1.3: Aggressiveness against order of birth, as percentages of all students.

A student is chosen at random.

(a).
What is the probability that the student is (i) first born, (ii) aggressive, (iii) aggressive given that the student was first born?
(b).
If \(A\) is the event that a student is aggressive and \(B\) that a student is first born, are \(A\) and \(B\) independent? Explain.
(c).
What can you say about the psychologists’ belief?

Show solution

Solution. The four cells are the joint probabilities and they sum to \(1\).

(a). Summing the first-born column and the aggressive row, \begin {align*} \text {(i)}\quad P(B) &= 0.25+0.15 = 0.40,\\ \text {(ii)}\quad P(A) &= 0.25+0.20 = 0.45,\\ \text {(iii)}\quad P(A\mid B) &= \frac {P(A\cap B)}{P(B)} = \frac {0.25}{0.40} = 0.625. \end {align*}

(b). Independence would require \(P(A\cap B) = P(A)P(B)\). Here \[P(A)P(B) = (0.45)(0.40) = 0.18, \qquad \text {but}\qquad P(A\cap B) = 0.25.\] These differ, so \(A\) and \(B\) are not independent. Equivalently, \(P(A\mid B)=0.625\) is not equal to \(P(A)=0.45\): knowing a student is first born changes the probability that they are aggressive.

(c). In these data the psychologists’ belief is supported. A first born is aggressive with probability \(0.625\), against \(0.45\) for a student chosen without regard to birth order, so the two characteristics are associated.

Association is not causation, though, and it is worth being careful here: these figures show only that aggressiveness and birth order occur together more often than chance would produce. They say nothing about which, if either, brings about the other.

Problem 1.4.13. A sociologist wants to estimate the percentage of students who have smoked cannabis. One hundred students are randomly selected. To allay any fears about confidentiality the question is asked indirectly: each student first picks a number at random between \(00\) and \(99\) without revealing it. Anyone whose number is between \(00\) and \(69\) answers the cannabis question; the others answer the non-sensitive question “does your computer number end with an even digit?”. Each student writes only the answer on a slip of paper, without indicating which question it belongs to. It turns out that \(44\%\) of the responses are “yes”. What fraction would you estimate to be answering “yes” to the cannabis question?

Show solution

Solution. The design is deliberate: because nobody knows which question a slip refers to, no individual answer reveals anything – yet the aggregate is still informative, because the two questions are mixed in known proportions.

Of the hundred numbers \(00\) to \(99\), seventy fall in \(00\)–\(69\), so a student answers the cannabis question with probability \(0.7\) and the computer-number question with probability \(0.3\). A computer number is as likely to end in an even digit as an odd one, so the non-sensitive question yields “yes” with probability \(0.5\).

Let \(p\) be the proportion who would answer “yes” to the cannabis question. By the law of total probability, \[P(\text {yes}) = 0.7p + 0.3(0.5) = 0.7p + 0.15.\] Setting this equal to the observed \(0.44\), \[0.7p = 0.44-0.15 = 0.29 \implies p = \frac {0.29}{0.7} = \frac {29}{70} \approx 0.414.\] So an estimated \(41.4\%\) of students have smoked cannabis.

Note what the randomisation costs. The \(0.3\) of responses answering the harmless question carry no information about cannabis at all, so the estimate rests on fewer usable answers than the hundred collected – the price paid for honest ones.

Problem 1.4.14. Suppose \(A\) and \(B\) are events with \(P(A) = p\), \(P(B) = 2p\) and \(P(A\cup B) = 0.75\). Evaluate \(p\) and \(P(A\setminus B)\) if

(a).
\(A\) and \(B\) are independent;
(b).
\(A\) and \(B\) are mutually exclusive.

Show solution

Solution. In both parts start from the addition rule \(P(A\cup B) = P(A)+P(B)-P(A\cap B)\). The two assumptions differ only in what they say about \(P(A\cap B)\). Also \(P(A\setminus B) = P(A\cap B') = P(A)-P(A\cap B)\).

(a). Independence gives \(P(A\cap B) = P(A)P(B) = 2p^{2}\), so \[p+2p-2p^{2} = 0.75 \implies 2p^{2}-3p+0.75 = 0 \implies p = \frac {3\pm \sqrt {3}}{4}.\] The root \(\frac {3+\sqrt 3}{4}\approx 1.18\) would make \(P(B) = 2p\) exceed 1, so \(p = \frac {3-\sqrt 3}{4}\approx 0.3170\). Then \[P(A\setminus B) = p-2p^{2} = p(1-2p) = \frac {2\sqrt 3-3}{4}\approx 0.1160.\]

(b). Mutually exclusive events have \(P(A\cap B) = 0\), so \(3p = 0.75\), giving \(p = 0.25\), and \(P(A\setminus B) = P(A) = 0.25\): when \(A\) and \(B\) cannot occur together, removing \(B\) from \(A\) removes nothing.

The two answers differ, which is the point: independent and mutually exclusive are different conditions, and two events with positive probability cannot be both. If they are mutually exclusive, the occurrence of one tells you the other did not occur, which is as far from independent as events can be.

Problem 1.4.15. Suppose that \(1\) in \(10\,000\) people (\(0.01\%\)) has a particular disease. A diagnostic test for the disease has \(99\%\) sensitivity: if a person has the disease, the test gives a positive result with probability \(0.99\). The test has \(99\%\) specificity: if a person does not have the disease, the test gives a negative result with probability \(0.99\). Let \(B\) denote the presence of the disease and \(A\) a positive test result.

(a).
Calculate \(P(A)\).
(b).
A person tests positive. What is the probability that they have the disease?

Show solution

Solution. The data are \(P(B) = 0.0001\), \(P(B') = 0.9999\) and \(P(A\mid B) = 0.99\). Specificity is about the healthy, \(P(A'\mid B') = 0.99\), so the false-positive rate is \(P(A\mid B') = 0.01\).

(a). \(B\) and \(B'\) partition the population, so by total probability \begin {align*} P(A) &= P(B)P(A\mid B)+P(B')P(A\mid B')\\ &= (0.0001)(0.99)+(0.9999)(0.01) = 0.000099+0.009999 = 0.010098. \end {align*}

(b). By Bayes’ theorem, \[P(B\mid A) = \frac {P(B)P(A\mid B)}{P(A)} = \frac {0.000099}{0.010098}\approx 0.0098.\] A test that is right 99% of the time, applied to a positive patient, leaves fewer than a 1% chance of disease. The two terms in (a) explain it: in a population of a million, about 99 sick people test positive, but about \(9\,999\) healthy people do too. When a condition is rare the false positives swamp the true ones, which is why a positive screening result is followed by a second, independent test rather than a diagnosis.

Problem 1.4.16. In a bolt factory, machines 1, 2 and 3 produce \(20\%\), \(30\%\) and \(50\%\) of the total output respectively. Of their outputs, \(5\%\), \(3\%\) and \(2\%\) respectively are defective. A bolt is selected at random.

(a).
What is the probability that it is defective?
(b).
Given that it is defective, what is the probability that it was made by machine 1?

Show solution

Solution. Let \(M_j\) be the event that the bolt came from machine \(j\) and \(D\) the event that it is defective. Every bolt comes from exactly one machine, so \(M_1, M_2, M_3\) partition the output.

(a). \begin {align*} P(D) &= \sum _{j=1}^{3}P(M_j)P(D\mid M_j) = (0.2)(0.05)+(0.3)(0.03)+(0.5)(0.02)\\ &= 0.010+0.009+0.010 = 0.029. \end {align*}

(b). \[P(M_1\mid D) = \frac {P(M_1)P(D\mid M_1)}{P(D)} = \frac {0.010}{0.029} = \frac {10}{29}\approx 0.345.\] Machine 1 makes only a fifth of the bolts but about a third of the defective ones, because its defect rate is the highest. The same reasoning gives \(P(M_2\mid D) = \frac {9}{29}\) and \(P(M_3\mid D) = \frac {10}{29}\). As a check, the three posteriors sum to 1.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.