1.3 Definitions and Axioms of Probability
Definition 1.3.1. If an experiment gives rise to a sample space \(S\) which has finite number of elements \((N)\) of equally likely outcomes, then the probability of an event \(E\) denoted by \(P(E)\) and is given by \[P(E)=\frac {n(E)}{N},\] while \(n(E)\) is the number of elements in the set \(E\). \(P(E)\) is the ratio of number of elements in the set \(E\) with the number of elements in \(S\).
Example 1.3.2. In an experiment of tossing two coins and observing the face up
- (i).
- Let the events \(A\) be a head up.
- (ii).
- Let the events \(B\) be exactly one tail.
Find the probabilities of (i) and (ii).
Solution. The sample space, \[S=\{HH, HT, TH, TT\},\quad N=4.\] Event is \[A =\{HH, HT, TH\},\quad n(A)=3\] then \[P(A) =\frac {n(A)}{N} =\frac {3}{4}.\] And \[B =\{HT,TH\},\quad n(B)=2 \] hence \[P(B) =\frac {2}{4}=\frac {1}{2}.\] □
Another approach to probability involves the conceptual repetitions of an experiment and counting the number of trials and the number of times the event in question occurs.
Definition 1.3.3. If an experiment is repeated in times and let \(M(E)\) be the number of times events \(A\) occurs in \(M\) trials \(\frac {M(E)}{M}\) relative frequency of event \(E\) and \(\frac {M(E)}{M}\) tends to stabilize as \(M\) becomes very large \[P(E)=\lim _{M\longrightarrow \infty }\frac {M(E)}{M}\] If an experiment has sample space \(S\) and an event \(E\) defined on \(S\) i.e. \(E\subset S\) then \(P(E)\) is a real number called the probability of event \(E\). \(P(\cdot )\) is a function whose domain in \(S\) and range is \([0,1]\) \[P:S\longrightarrow [0,1]\]
1.3.1 Axioms of probability
- (i).
- For any event \(E\), \(0\leq P(E)\leq 1\).
- (ii).
- \(P(S)=1\).
- (iii).
- For any finite number of mutually exclusives events \(E_1, E_2,\ldots ,E_k\) then \[P\left (\bigcup ^k_{j=1}E_j\right )=\sum _{j=1}^k P(E_j).\]
Note 1.3.4. \[\bigcup ^k_{j=1}E_j = E_1\cup E_2\cup E_3\cup \cdots \cup E_k\] \[\sum ^k_{j=1}P(E_j) = P(E_1)+ P(E_2) + \cdots +P(E_k)\]
Theorem 1.3.5. Let \(A\) and \(B\) be any set events and \(S\) be the sample space of an experiment then
- (i).
- \(P(\emptyset )=0\,\) where \(\,\emptyset \,\) is empty set.
- (ii).
- \(P(A')=1-P(A)\)
- (iii).
- \(P(A\cup B)=P(A)+P(B)-P(A\cap B)\)
- (iv).
- if \(A\subset B\) \(\implies \) \(P(A)\leq B\)
Proof. We prove (iii), we express \(A\cup B\), \(A\) and \(B\) as \begin {equation} A\cup B=(A\cap B') \cup (A\cap B) \cup (A'\cap B) \end {equation} \begin {equation} A =(A\cap B) \cup (A\cap B') \end {equation} \begin {equation} B =(A\cap B) \cup (A'\cap B) \end {equation} From the three equations we get \begin {align*} P(A\cup B) &=P(A\cap B') + P(A\cap B) + P(A'\cap B)\\ P(A) &=P(A\cap B) + P(A\cap B')\\ P(B) &=P(A\cap B) + P(A'\cap B) \end {align*}
Therefore \begin {align*} P(A\cup B) &=P(A)-P(A\cap B)+P(A\cap B)+P(B)-P(A\cap B)\\ &=P(A)+P(B)-P(A\cap B). \end {align*} □
Example 1.3.6. If event \(A\) and \(B\) are such that \(P(A)=0.3\), \(P(B)=0.6\) and \(P(A\cup B)=0.8\), evaluate the following
- (i).
- \(P(A')\)
Solution. \(P(A')=1-P(A)=1-0.3=0.7\) □
- (ii).
- \(P(A\cap B)\)
Solution. \(P(A\cap B) = P(A)+P(B)-P(A\cup B) = 0.3+0.6-0.8 = 0.1\) □
- (iii).
- \(P(A'\cup B')\)
Solution. \(P(A'\cup B') = P\left ((A\cap B)\right )' = 1-P(A\cap B) = 1-0.1 = 0.9\) □
- (iv).
- \(P(A'\cap B')\)
Solution. \(P(A'\cap B') = P\left ((A\cup B)'\right ) = 1-P(A\cup B) = 1-0.8 = 0.2\) □
Example 1.3.7. The Samsung cell phones require replacement of possible defects of the LCD and in the charging system defects. Samsung have being informed that \(2\%\) of the phones have defective LCD only and \(4\%\) have defective charging system only. If \(90\%\) of these phones have neither defective, what percentage of these have both defects?
Solution. Let \(A\) be the event of a defective LCD and \(B\) be the event of a defective charging system. We are given:
- \(P(A \cap B') = 0.02\) (Defective LCD only)
- \(P(B \cap A') = 0.04\) (Defective charging only)
- \(P((A \cup B)') = 0.90\) (Neither defective)
First, find the probability that a phone has at least one defect: \[ P(A \cup B) = 1 - P((A \cup B)') = 1 - 0.90 = 0.10 \]
We know that \(P(A \cup B)\) is the sum of the disjoint regions: \[ P(A \cup B) = P(A \cap B') + P(B \cap A') + P(A \cap B) \] \[ 0.10 = 0.02 + 0.04 + P(A \cap B) \] \[ P(A \cap B) = 0.10 - 0.06 = 0.04 \]
Conclusion: \(4\%\) of the phones have both defects. □
1.3.2 Practice problems
Past tutorial-sheet questions on sample spaces and the axioms. Try each one before opening the solution.
Problem 1.3.1. [Assignment] Consider the experiment of tossing four coins simultaneously and observing the side facing up for each coin.
- (a).
- Write down the sample space for the experiment, the event \(A\) of observing at least one head, the event \(B\) of observing at most two heads, and the event \(C\) of observing at least two tails.
- (b).
- Calculate \(P(A\cup B)\), \(P(A\cap B)\) and \(P(A^c\cup C)\).
Show solution
Solution. (a). Each coin lands two ways, so \(|S| = 2^4 = 16\) equally likely outcomes: \begin {align*} S = \{&HHHH,\ HHHT,\ HHTH,\ HTHH,\ THHH,\ HHTT,\ HTHT,\ HTTH,\\ &THHT,\ THTH,\ TTHH,\ HTTT,\ THTT,\ TTHT,\ TTTH,\ TTTT\}. \end {align*}
\(A\) excludes only \(TTTT\), so \(|A| = 15\). \(B\) collects the outcomes with \(0\), \(1\) or \(2\) heads, so \(|B| = 1+4+6 = 11\). \(C\) collects those with at least two tails, so \(|C| = 6+4+1 = 11\).
It is worth noticing that \(B\) and \(C\) are the same event. With four coins, at most two heads means at least two tails – the two descriptions pick out exactly the same 11 outcomes.
(b). \[P(A) = \frac {15}{16},\qquad P(B) = P(C) = \frac {11}{16}.\] \(A\cup B = S\), because the only outcome outside \(A\) is \(TTTT\), and \(TTTT\) is in \(B\): \[P(A\cup B) = 1.\] \(A\cap B\) is the set of outcomes with one or two heads, \(4+6 = 10\) of them: \[P(A\cap B) = \frac {10}{16} = \frac {5}{8}.\] \(A^c = \{TTTT\}\), which already lies in \(C\), so \(A^c\cup C = C\): \[P(A^c\cup C) = \frac {11}{16}.\]
Problem 1.3.2. [Assignment] For any events \(A\), \(B\) and \(C\), prove that \[P(A\cup B\cup C) = P(A)+P(B)+P(C)-P(A\cap B)-P(A\cap C)-P(B\cap C)+P(A\cap B\cap C).\]
Show solution
Solution. Apply the two-event addition rule twice. Treat \(A\cup B\) as a single event and add \(C\) to it: \[P\big ((A\cup B)\cup C\big ) = P(A\cup B) + P(C) - P\big ((A\cup B)\cap C\big ).\] The first term expands by the same rule, \(P(A\cup B) = P(A)+P(B)-P(A\cap B)\). For the last term, distribute the intersection over the union: \[(A\cup B)\cap C = (A\cap C)\cup (B\cap C),\] and apply the addition rule once more to that: \[P\big ((A\cap C)\cup (B\cap C)\big ) = P(A\cap C)+P(B\cap C)-P\big ((A\cap C)\cap (B\cap C)\big ).\] Now \((A\cap C)\cap (B\cap C) = A\cap B\cap C\). Substituting the two expansions, \begin {align*} P(A\cup B\cup C) &= \big [P(A)+P(B)-P(A\cap B)\big ] + P(C)\\ &\quad - \big [P(A\cap C)+P(B\cap C)-P(A\cap B\cap C)\big ]\\ &= P(A)+P(B)+P(C)-P(A\cap B)-P(A\cap C)-P(B\cap C)+P(A\cap B\cap C), \end {align*}
as required.
The pattern is worth remembering: adding the three single probabilities counts each pairwise overlap twice, so the pairwise terms are subtracted; that subtraction removes the triple overlap three times having added it three times, so it must be added back once.
Problem 1.3.3. [Assignment] A game is played as follows: a player selects a point at random from the square \(0\leq x\leq 1\), \(0\leq y\leq 1\). There is a number \(\beta \) such that the player wins if at least one of the coordinates of the chosen point is greater than \(\beta \). Find the value of \(\beta \) that makes the game fair.
Show solution
Solution. “At random” from the square means the probability of landing in a region equals that region’s area, since the whole square has area 1.
It is easier to work with the losing event. The player loses when both coordinates are at most \(\beta \), that is when the point falls in the smaller square \(0\leq x\leq \beta \), \(0\leq y\leq \beta \), which has area \(\beta ^{2}\). Hence \[P(\text {win}) = 1 - \beta ^{2}.\] The game is fair when the player wins half the time: \[1-\beta ^{2} = \frac {1}{2} \implies \beta ^{2} = \frac {1}{2} \implies \beta = \frac {1}{\sqrt {2}} = \frac {\sqrt {2}}{2} \approx 0.707.\] (The negative root is discarded because \(\beta \) must lie in \([0,1]\).)
Problem 1.3.4. [Tutorial Sheet 1] You wish to choose two different numbers from \(1, 2, 3, 4, 5\), and do not care about the order of selection.
- (a).
- Find the sample space for this experiment.
- (b).
- What is the probability that the numbers chosen differ by 1?
Show solution
Solution. (a). Order does not matter, so an outcome is a pair \(\{a,b\}\) with \(a\neq b\). There are \(\binom {5}{2}=10\) of them: \[S = \{12,\,13,\,14,\,15,\,23,\,24,\,25,\,34,\,35,\,45\}.\]
(b). The pairs differing by 1 are \(12, 23, 34, 45\) – four of the ten. All outcomes are equally likely, so \[P(\text {differ by }1) = \frac {4}{10} = \frac {2}{5}.\]
Problem 1.3.5. [Tutorial Sheet 1] From a set of cards numbered 1 to 20 a card is drawn at random. Find the probability that the number is
- (a).
- divisible by 4;
- (b).
- greater than 15;
- (c).
- divisible by 4 and greater than 15.
If the card is divisible by 4 and it is not replaced, find the probability that the second card drawn is
- (d).
- even;
- (e).
- odd.
Show solution
Solution. There are 20 equally likely cards.
(a). The multiples of 4 are \(4, 8, 12, 16, 20\), so \(P = \frac {5}{20} = \frac {1}{4}\).
(b). The numbers greater than 15 are \(16, 17, 18, 19, 20\), so \(P = \frac {5}{20} = \frac {1}{4}\).
(c). Both conditions hold for 16 and 20 only, so \(P = \frac {2}{20} = \frac {1}{10}\).
For parts (d) and (e), the card already drawn is divisible by 4 and so is even. It is not replaced, which leaves 19 cards: 9 even and 10 odd.
(d). \(P(\text {second is even}) = \frac {9}{19}\).
(e). \(P(\text {second is odd}) = \frac {10}{19}\).
Problem 1.3.6. [Tutorial Sheet 1] An ordinary die and two coins are thrown together. Find the probability that
- (a).
- two heads and a number less than 3 are obtained;
- (b).
- the coins show different faces and an even number is shown on the die;
- (c).
- the die shows an odd number and the coins show the same face;
- (d).
- a 6 and at least one head are obtained.
Show solution
Solution. The die and the two coins are independent, and the sample space has \(6\times 2\times 2 = 24\) equally likely outcomes.
(a). Two heads has probability \(\tfrac 14\); a number less than 3 (that is, 1 or 2) has probability \(\tfrac 26=\tfrac 13\). Hence \[P = \frac {1}{4}\times \frac {1}{3} = \frac {1}{12}.\]
(b). The coins differ in 2 of the 4 ways, probability \(\tfrac 12\); the die is even with probability \(\tfrac 36=\tfrac 12\). Hence \(P = \tfrac 12\times \tfrac 12=\tfrac 14\).
(c). The die is odd with probability \(\tfrac 12\) and the coins match with probability \(\tfrac 12\), so \(P = \tfrac 14\).
(d). The die shows 6 with probability \(\tfrac 16\); at least one head has probability \(1-\tfrac 14 = \tfrac 34\). Hence \[P = \frac {1}{6}\times \frac {3}{4} = \frac {1}{8}.\]
Problem 1.3.7. [Tutorial Sheet 1] A coin is biased so that the probability that it lands showing heads is \(\frac {2}{3}\). The coin is tossed three times. Find the probability that
- (a).
- no heads are obtained;
- (b).
- more heads than tails are obtained.
Show solution
Solution. The tosses are independent with \(P(H)=\tfrac 23\) and \(P(T)=\tfrac 13\).
(a). No heads means three tails: \[P = \left (\frac {1}{3}\right )^{3} = \frac {1}{27}.\]
(b). With three tosses, more heads than tails means two heads or three heads: \begin {align*} P &= \binom {3}{2}\left (\frac {2}{3}\right )^{2}\left (\frac {1}{3}\right ) + \left (\frac {2}{3}\right )^{3}\\ &= 3\cdot \frac {4}{9}\cdot \frac {1}{3} + \frac {8}{27} = \frac {12}{27}+\frac {8}{27} = \frac {20}{27}. \end {align*}
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.