4.2 Computing Expectation by Conditioning

\(E(X \mid Y)\) is the expectation of \(X\) given \(Y\) which is a function of the random variable \(Y\) whose value at \(Y=y\) is \(E(X \mid Y=y)\). Therefore \(E(X \mid Y)\) is a random variable.

Result 4.2.1. Let \(X\) and \(Y\) be random variables with joint probability function \(f_{X,Y}(x,y)\) then \[E(X)=E(E(X \mid Y))\]

Proof. \(\displaystyle {\text {We show that }\hspace {0.5cm} E(X)= \begin {cases} \int \limits _yE(X \mid Y)f_Y(y)dy, &\text {if Y is continuous}\\\\ \sum \limits _yE(X \mid Y)P(Y=y), &\text {if Y is discrete}\\ \end {cases} }\)

We do for \(Y\) discrete. \begin {align*} \sum _yE(X \mid Y=y)P(Y=y) &=\sum _y\sum _xxf_{X/Y}(x/y)\,\cdot \, f_Y(y)\\ &=\sum _y\sum _xxf_{X,Y}(x,y =\sum _xx\sum _yf_{X,Y}(x,y)\\ &=\sum _xxf_X(x)\\ &=E(X)\\ \end {align*} □

Example 4.2.2. A miner is trapped in a mine containing 3 doors. The first door leads to a tunnel that will take him to safety after 3 three hours walking. The second door leads to a tunnel that will return him to the mine after five 5 hours of walking. The third door leads to a tunnel that will return him to the mine after 7 seven hours. If we assume that the miner is at all times equally to close any of the doors, what is the expected length of time until he reaches to safety?

Solution. Let \(X\) be the time until he reaches to safety.
Let \(Y\) denote the door the miner initially chooses. \begin {align*} E(X) &=E\Big (E(X \mid Y)\Big )\\ &=E(X \mid Y=1)P(Y=1) +E(X \mid Y=2)P(Y=2)+E(X \mid Y=3)P(Y=3)\\ &=\frac {1}{3}\Big (E(X \mid Y=1)+E(X \mid Y=2)+E(X \mid Y=3)\Big )\\ &=\frac {1}{3}\Big (3+5+E(X)+7+E(X)\Big )\\\\ 3E(X) &=15+2E(X)\\ \implies \hspace {0.5cm} E(X) &=15\\ \end {align*}

Example 4.2.3. An urn contains \(n\) white balls and \(m\) black balls. One ball is selected
randomly at a time until the first white ball is selected. Find the expectation of the black balls selected.

Let \(X\) be the number of black balls selected.

We want \(E(X)\) \[\text {Let}\hspace {0.5cm} Y= \begin {cases} 1, &\text {if the white ball is selected first time.}\\\\ 0, &\text {if the black ball is selected first time}\\ \end {cases} \] \[P(Y=1)=\frac {n}{n+m}\hspace {0.5cm} \text {and}\hspace {0.5cm} P(Y=0)=\frac {m}{n+m}\] Denote \(E(X)\) by \(N(n,m)=\) expected number of black balls selected. \[E(X \mid Y=1)=0,\hspace {0.5cm} E(X \mid Y=0)=1+N(n,m-1).\] \[E(X)=N(n,m)=E(X \mid Y=1)P(Y=1)+E(X \mid Y=0)P(Y=0)\] \[N(m,n)=0.\frac {n}{n+m}+\Big (1+N(n,m-1)\Big )\frac {m}{n+m}\] \[\implies \hspace {0.5cm} N(m,n)=\Big (1+N(n,m-1)\Big )\frac {m}{n+m}................... (*)\] \[\text {if}\hspace {0.4cm} m=1,\hspace {0.4cm} N(n,1)=\Big (1+N(n,0)\Big )\frac {1}{n+1}=\frac {1}{n+1}\] \[N(n,1)=\frac {1}{n+1}\] \[\text {if}\hspace {0.4cm} m=2,\hspace {0.5cm} N(n,2)=\Big (1+N(n,1)\Big )\frac {2}{n+2}=\frac {2}{n+1}\] \[\text {if}\hspace {0.4cm} m=3,\hspace {0.5cm} N(n,3)=\Big (1+N(n,2)\Big )\frac {3}{n+3}=\frac {3}{n+3}\]
\[\therefore \hspace {0.5cm} N(n,m)=\frac {m}{n+1}\]
\[\therefore \hspace {0.5cm} E(X)=N(n,m)=\frac {m}{n+1}\]

Example 4.2.4. Let \(X\) be the number of independent trials required to get first success where the probability of success is constant in each trial is \(P\). Then \(X\) has geometric distribution. Find the variance of \(X\). \[f_X(x)=P(X=x)=P(1-P)^{x-1},\hspace {0.5cm} x=1,2,........\] \[E(X)=\frac {1}{P}\] \[\text {if}\hspace {0.3cm} Y= \begin {cases} 1, &\text {if the first trial is a success}\\\\ 0, &\text {if the first trial is a failure}\\ \end {cases} \]

\[P(Y=1)=P,\hspace {0.5cm} P(Y=0)=1-P\] \[\Var (X)=E(X^2)-\Big (E(X)\Big )^2\] \begin {align*} E(X^2) &=E(X^2 \mid Y=0)P(Y=0)+E(X^2 \mid Y=1)P(Y=1)\\ &=E(1+X)^2(1-P)+(1)^2P\\ &=E\Big (1+2X+X^2\Big )(1-P)+P\\ &=\Big (1+2/P+E(X^2)\Big )(1-P)+P \end {align*}

\[E(X^2)=\frac {(P+2)}{P}\hspace {0.1cm}(1-P)+(1-P)E(X^2)+P\] \begin {align*} \implies \hspace {0.5cm} P\,E(X^2) &=\frac {(P+2)(1-P)+P^2}{P}\\ &=\frac {P-P^2+2-2P+P^2}{P} =\frac {2-P}{P}\\\\ \implies \hspace {0.5cm} E(X^2) &=\frac {2-P}{P^2} \end {align*}

\begin {align*} \therefore \hspace {0.5cm} \Var (X) &=E(X^2)-\Big (E(X)\Big )^2\\ &=\frac {2-P}{P^2}-\frac {1}{P^2}\\ &={\frac {1-P}{P^2}} \end {align*}

OR \begin {align*} E(X^2) &=\sum ^{\infty }_{x=1}x^2(1-P)^{x-2}\\ &=P(1-P)\sum ^{\infty }_{x=1}\Big (x(x-1)(1-P)^{x-2}\Big )+\sum ^{\infty }_{x=1}xP(1-P)^{x-1}\\ &=P(1-P)\sum ^{\infty }_{x=1}\frac {d^2}{dP^2}(1-P)^x-P\sum ^{\infty }_{x=1}\frac {d}{dP}(1-P)^x\\ &=P(1-P)\frac {d^2}{dP^2}\sum ^{\infty }_{x=1}(1-P)^x-P\frac {d}{dP}\sum ^{\infty }_{x=1}(1-P)^x\\ &=P(1-P)\frac {d^2}{dP^2}\Bigg (\frac {1}{1-(1-P)}-1\Bigg )-P\frac {d}{dP}\Bigg (\frac {1}{1-(1-P)}-1\Bigg )\\ &=P(1-P)\frac {d^2}{dP^2}\Bigg (\frac {1}{P}-1\Bigg )-P\frac {d}{dP}\Bigg (\frac {1}{P}-1\Bigg )\\ &=P(1-P).\frac {2}{P^3}+\frac {1}{P}=\frac {2(1-P)}{P^2}+\frac {1}{P}=\frac {2(1-P)+P}{P^2}\\ \implies \hspace {0.5cm} E(X^2) &=\frac {2-P}{P^2}\\ \end {align*}

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.