4.1 Introduction
Result 4.1.1. If \(X\) and \(Y\) have joint probability function \(f_{X,Y}(x,y)\) and \(g(x,y)\) is a real valued function then the
expectation of \(g(x,y)\) denoted as \(E(g(x,y))\) is given by \[ E(g(x,y))= \begin {cases} \int \limits _X\int \limits _Yg(x,y)f_{X,Y}(x,y)dydx &\text {if both X and Y are continuous}\\\\ \sum \limits _X\sum \limits _Yg(x,y)f_{X,Y}(x,y) &\text {if both X and Y are discrete.} \end {cases} \]
Example 4.1.2. An accident occurs at a point \(X\) that is uniformly distributed on a road of
length \(L\). At the time of the accident a police patrol car is at a location \(Y\) that is also uniformly
distributed on the same road.
Assume that \(X\) and \(Y\) are independent, find the expected distance between the police patrol car
and the point of the accident.
\[f_{X,Y}(x,y)=f_X(x)f_Y(y)=\frac {1}{L}\,\cdot \,\frac {1}{L}=\frac {1}{L^2},\hspace {0.5cm} 0<x<L\hspace {0.3cm}\& \hspace {0.3cm} 0<y<L\]
\[g(x,y)=|x-y|\]
\begin {align*} E\Big (g(x,y)\Big ) &=E(|x-y|)\\ &=\int ^L_0\int _0^L|x-y|\,\cdot \,\frac {1}{L^2} \,dx\,dy\\\\ &=\frac {1}{L^2}\int ^L_0\int _0^L|x-y|\,dx\,dy\\ \end {align*}
\[ |x-y|= \begin {cases} x-y, &\text {if}\hspace {0.3cm} x>y\\ y-x, &\text {if}\hspace {0.3cm} x<y\\ \end {cases} \]
\begin {align*} \int ^L_0|x-y|dx &=\int ^y_0(y-x)dx+\int ^L_y(x-y)dx\\\\ &=xy-\frac {x^2}{2}\Bigg |^y_0+ \frac {x^2}{2}-xy\Bigg |^L_y\\\\ &=y^2-\frac {y^2}{2}+\frac {L^2}{2}-yL-\frac {y^2}{2}+y^2\\ &=y^2+\frac {L^2}{2}-yL\\ \end {align*}
\begin {align*} \implies \hspace {0.5cm} E\Big (|x-y|\Big ) &=\frac {1}{L^2}\int ^L_0\Big (y^2+\frac {L^2}{2}-yL\Big )dy\\ &=\frac {1}{L^2}\Bigg (\frac {y^3}{3}+\frac {yL^2}{2}-\frac {y^2L}{2}\Bigg |^L_0\\\\ &=\frac {1}{L^2}\Bigg (\frac {L^3}{3}+\frac {L^3}{3}-\frac {L^3}{2}\Bigg )\\ &=\frac {L}{3}\\ \end {align*}
Theorem 4.1.3 (Boole’s Inequality). Let \(A_1, A_2,.....,A_k\) be events, then \(\displaystyle {P\Bigg (\bigcup ^k_{j=1}A_j\Bigg )\leq \sum ^k_{j=1}P(A_j)}\)
Proof. \(\text {Let}\hspace {0.5cm} X_j= \begin {cases} 1, &\text {if}\hspace {0.3cm} A_j\hspace {0.3cm} \text {ocurs}\\\\ 0, &\text {otherwise}\\ \end {cases} \hspace {1cm} \text {and}\hspace {1cm} X=\sum ^k_{j=1}X_j\\ \)
\[E(X_i)=0P\Big (A_j^c\Big )+1 P(A_j)=P(A_j)\]
\[E(X)=\sum ^k_{j=1}E(X_j)=\sum ^k_{j=1}P(A_j)\] \[\text {Let}\hspace {0.5cm} Y= \begin {cases} 1, &\text {if}\hspace {0.3cm} X\geq 1\\\\ 0, &\text {otherwise}.\\ \end {cases} \] \begin {align*} E(Y) &=0P(X<1)\text {or}\hspace {0.3cm} 0P(X=0)+1P(X\geq 1)\\ &=P(X\geq 1)\\ &=P(A_1\cup A_2\cup ........ \cup A_k)\\ &=P\Bigg (\bigcup ^k_{j=1}A_j\Bigg )\\ \end {align*}
\[\text {Thus}\hspace {0.5cm} Y\leq X\]
\[\implies \hspace {0.5cm} E(Y)\leq E(X)\]
\[\implies \hspace {0.5cm} P\Bigg (\bigcup ^k_{j=1}A_j\Bigg )\leq \sum ^k_{j=1}P(A_j)\]
□
- i
- \(\text {cov}(aX+b,cY+d)=ac\hspace {0.2cm}\text {cov}(X,Y)\).
- ii
- \(\text {cov}\Bigg (\sum \limits ^n_{i=1}X_i,\sum \limits ^k_{j=1}Y_j\Bigg )=\sum \limits ^n_{i=1}\sum \limits ^k_{j=1}\text {cov}(X_i,Y_j)\).
- iii
- \(\Var \Bigg (\sum \limits ^n_{i=1}\Bigg )=\sum \limits ^n_{i=1}\Var (X_i)+2\sum \limits _i\sum \limits _{<j}\text {cov}(X_i,X_j)\).
Proof.
- i
- you can prove this one.
- ii
- \begin {align*} \text {cov}\Bigg (\sum ^n_{i=1}X_i,\sum ^k_{j=1}Y_j\Bigg ) &= E\Bigg \{\Bigg (\sum ^n_{i=1}X_i\Bigg )\Bigg (\sum ^k_{j=1}Y_j\Bigg )\Bigg \}-E\Bigg \{\sum ^n_{i=1}X_i\Bigg \}E\Bigg \{\sum ^k_{j=1}Y_j\Bigg \}\\\\ &=E\Bigg \{\sum ^n_{i=1}\sum ^k_{j=1}X_iY_j\Bigg \}-\Bigg \{\sum ^n_{i=1}E(X_i)\Bigg \}\Bigg \{\sum ^k_{j=1}E(Y_j)\Bigg \}\\\\ &=\sum _{i=1}^n\sum ^k_{j=1}E(X_iY_j)-\sum ^n_{i=1}\sum ^k_{j=1}E(X_i)E(Y_j)\\ &=\sum ^n_{i=1}\sum ^k_{j=1}\Big \{E(X_iY_j)-E(X_i)E(Y_j)\Big \}\\ &=\sum ^n_{i=1}\sum ^k_{j=1}\text {cov}(X_i,Y_j)\\ \end {align*}
- iii
- \begin {align*} \Var \Bigg (\sum ^n_{i=1}X_i\Bigg ) &=\text {cov}\Bigg (\sum ^n_{i=1}X_i,\sum ^n_{i=1}X_i\Bigg )\\\\ &=E\Bigg \{\Bigg (\sum ^n_{i=1}X_i\Bigg )\Bigg (\sum ^n_{i=1}X_i\Bigg )\Bigg \}-E\Bigg \{\sum ^n_{i=1}X_i\Bigg \}E\Bigg \{\sum ^n_{i=1}X_i\Bigg \}\\\\ &=E\Bigg \{\sum ^n_{i=1}X_i^2\sum \sum \limits _{i<j}X_iX_j\Bigg \}-\Bigg (\sum ^n_{i=1}E(X_i)\Bigg )\Bigg (\sum ^n_{i=1}E(X_i)\Bigg )\\\\ &=\sum ^n_{i=1}E(X_i^2)+2\sum \sum \limits _{i<j}E(X_iX_j)-\Bigg (\sum ^n_{i=1}\Big (E(X_i)\Big )^2+2\sum \sum \limits _{i<j}E(X_i)E(X_j)\Bigg )\\\\ &=\Bigg (\sum ^n_{i=1}\Big (E(X_i^2)\Big )-\Big (E(X_i)^2\Big )+2\sum \sum \limits _{i<j}\Bigg (E(X_iX_j-E(X_i)E(X_j)\Bigg )\\\\ &=\sum ^n_{i=1}\Var (X_i)+2\sum \sum \limits _{i<j}\text {cov}(X_i,X_j).\\ \end {align*}
Using result [ii].
\[\text {cov}\Bigg (\sum ^n_{i=1}X_i,\sum ^n_{j=1}X_j\Bigg )=\sum ^n_{i=1}\sum ^n_{j=1}\text {cov}(X_i,X_j)\]
\[(\text {case where}\hspace {0.3cm} i=j)=\sum ^n_{i=1}\text {cov}(X_i,X_j)+2\sum \sum _{i<j}\text {cov}(X_i,X_j)\]
□
Definition 4.1.5. Let \(X\) and \(Y\) have a joint probability function \(f_{X,Y}(x,y)\), then the conditional expectation
of \(X\) given \(Y=y\) denoted by \(E(X \mid Y=y)\) is given by \[ E\Big (X/Y=y\Big )= \begin {cases} \int \limits _xxf(x/y)dx, &\text {continuous}\\\\ \sum \limits _xxf(x/y), &\text {discrete}\\ \end {cases} \]
Note: \(E(X \mid Y)\) is a function of a random variable \(Y\).
i.e \(E(X \mid Y)\) is a random variable.
Let \(E(X \mid Y=y)\) be a real number which is the value of \(E(X \mid Y)\) when \(Y=y\).
Example 4.1.6. If \(X\) and \(Y\) are independent Binomial random variables each with parameter \((n,P)\), find the conditional expectation of \(Y\) given \(X+Y=k\). \begin {align*} E(Y \mid X+Y=k) &=\sum _y yP(Y=y \mid X+Y=k)\\ &=\sum ^{\min (k,n)}_{y}y\hspace {0.1cm}\frac { \begin {pmatrix} n\\y\\ \end {pmatrix} \begin {pmatrix} n\\k-y\\ \end {pmatrix} }{ \begin {pmatrix} 2n\\y\\ \end {pmatrix} }\\ &=\text {mean of a hyper geometric}\\ &=\frac {k}{2} \end {align*}
Mean of hyper geometric random variable. If \(n\) balls are randomly selected from an urn containing \(N\)
balls of which \(m\) are white, find the expected number of white balls selected.
Let \(X=\) number of white balls selected. \begin {align*} P(X=x) &=\frac { \begin {pmatrix} m\\x\\ \end {pmatrix} \begin {pmatrix} N-m\\n-x\\ \end {pmatrix} }{ \begin {pmatrix} N\\n\\ \end {pmatrix} },\hspace {0.5cm} x=0,1,........,\min (n,m)\\ \end {align*}
\[\text {Let}\hspace {0.5cm} X_i= \begin {cases} 1, &\text {if}\hspace {0.3cm} i^{\text {th}}\hspace {0.3cm}\text {white is selected}\\\\ 0, &\text {otherwise}\\ \end {cases} \] \[\text {Let}\hspace {0.5cm} X=X_1+X_2+...........+X_n\] \begin {align*} E(X_i) &= P(i^{\text {th}}\hspace {0.3cm} \text {white ball selected})\\\\ &=\frac { \begin {pmatrix} 1\\1\\ \end {pmatrix} \begin {pmatrix} N-1\\n-1\\ \end {pmatrix} }{ \begin {pmatrix} N\\n\\ \end {pmatrix} }\\\\ &=\frac {(N-1)!(N-n)!n!}{(n-1)!(N-n)!N!}\\\\ &=\frac {1}{N}.n\\\\ &=\frac {n}{N} \end {align*}
\[\therefore \hspace {0.3cm} E(X)=\sum ^m_{i=1}E(X_i)=\sum ^m_{i=1}\frac {n}{N}=\frac {mn}{N}\]
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.