1.3 Computing Expectations by Conditioning
\(E(X\mid Y = y)\) is a number for each \(y\). Replacing \(y\) by the random variable \(Y\) gives a random variable, written \(E(X\mid Y)\): it is the function of \(Y\) that takes the value \(E(X\mid Y = y)\) when \(Y = y\).
Lemma 1.3.1 (Law of total expectation). For random variables \(X\) and \(Y\) with \(E|X|<\infty \), \(E\left [E(X\mid Y)\right ] = E(X)\).
Proof. In the discrete case, \begin {align*} E\left [E(X\mid Y)\right ] &= \sum _yE(X\mid Y = y)P(Y = y) = \sum _y\sum _x x\,P(X = x\mid Y = y)P(Y = y)\\ &= \sum _x x\sum _yP(X = x, Y = y) = \sum _x x\,P(X = x) = E(X). \end {align*}
The continuous case is the same with integrals. □
In practice the lemma is used as \(E(X) = \sum _yE(X\mid Y = y)P(Y = y)\): condition on whatever makes \(X\) easy to describe.
Example 1.3.2 (The trapped miner). A miner is trapped in a mine with three doors. The first leads to safety after \(2\) hours of travel; the second returns him to the mine after \(3\) hours; the third returns him after \(5\) hours. Each time he chooses a door, all three are equally likely, whatever happened before. Find the expected time until he reaches safety.
Solution. Let \(X\) be the time to safety and \(Y\) the first door chosen. If \(Y = 1\) he is safe after \(2\) hours. If \(Y = 2\) he is back where he started after \(3\) hours, and because he has no memory of his previous choices the remaining time has the same distribution as \(X\); so \(E(X\mid Y = 2) = 3 + E(X)\), and similarly \(E(X\mid Y = 3) = 5 + E(X)\). By lemma 1.3.1, \[E(X) = \frac 13\left [2 + (3 + E(X)) + (5 + E(X))\right ]\quad \Longrightarrow \quad 3E(X) = 10 + 2E(X),\] so \(E(X) = 10\) hours.
Example 1.3.3 (Random sums). The number of customers entering a shop in a day is a random variable \(N\) with mean \(\mu _N\) and variance \(\sigma _N^2\). Customers spend independent amounts \(X_1, X_2, \dots \) with common mean \(\mu \) and variance \(\sigma ^2\), independent of \(N\). Find the mean of the total \(S = X_1 + \cdots + X_N\).
Solution. Given \(N = n\), \(S\) is a sum of \(n\) terms with mean \(\mu \), so \(E(S\mid N = n) = n\mu \), that is, \(E(S\mid N) = N\mu \). By lemma 1.3.1, \(E(S) = E(N\mu ) = \mu \mu _N\): the mean number of customers times the mean spend. The variance is found in example 1.6.2.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.