7.1 Introduction
Most important theoretical results in probability theory are limit theorems. We have laws
of large numbers and central limit theorems. The laws of large numbers are concerned
with stating the conditions under which the average of a sequence of random variables
converge while the central limit theorems are concerned with determining conditions under
which the sum of the large number of random variables has a probability distribution that
is
approximately normal.
Result 3.1.1 Markov’s Inequality
If \(X\) is a random variable that is non-negative then for any value \(b\) we have, \(P(X\geq b)\leq \frac {E(X)}{b}\)
Proof.
\(\text {Let}\hspace {0.5cm} Z= \begin {cases} 1, &\text {if}\hspace {0.3cm} X\geq b\\\\ 0, &\text {otherwise}\\ \end {cases} \) . \(\text {Then}\hspace {0.4cm}E(Z)=P(X\geq b).\)
\[Z\leq \frac {X}{b}\hspace {0.3cm}\implies \hspace {0.3cm}E(Z)\leq \frac {E(X)}{b}\]
i.e \(P(X\geq b)\leq \dfrac {E(X)}{b}\)
□
Result 7.1.1 (Chebyshev’s Inequality). If \(X\) is a random variable with finite mean \(\mu \) then for any
\(k>0\), \(P\Big \{|X-\mu |\geq k\Big \}\leq \dfrac {\Var (X)}{k^2}\)
Proof. \(|X-\mu |\geq k\hspace {0.3cm}\Longleftrightarrow \hspace {0.3cm}(X-\mu )^2\geq k^2\) \begin {align*} \implies \hspace {0.5cm} P\Big \{|X-\mu |\geq k\Big \} &=P\Big \{(X-\mu )^2\geq k^2\Big \}\\ &\leq \frac {E(X-\mu )^2}{k^2} =\frac {\Var (X)}{k^2} \end {align*} □
- a
- Suppose that it is known that the number of items produced in a factory during a week is a
random variable with mean \(60\).
- i
- what can be said about the probability that week’s production will exceed \(80\)?
- ii
- if the variance of the week’s production is known to \(16\), what can be said about the
probability that this week’s production will be between \(50\) and \(70\)?
Let \(X\) be the number of items production in a week.
- i
- \(E(X)=60\), \begin {align*} P(X>80) &\leq \frac {E(X)}{80}=\frac {60}{80}\\ &=\frac {3}{4}\\ &=0.75\\ \end {align*}
- ii
- \(E(X)=60,\hspace {0.5cm} \Var (X)=16\) \begin {align*} P(50<X<70) &=P\{50-60<X-60<70-60\}\\ &=P(-10<X-\mu <10)\\ &=P(|X-\mu |<10)\\ &=1-P(|X-\mu |\geq 10)\\ \end {align*}
But \[P\Big \{|X-\mu |\geq 10\Big \}\leq \frac {\Var (X)}{100}=\frac {16}{100}=\frac {4}{25}\]
\[\therefore \hspace {0.5cm} P\Big \{50<X<70\Big \}\geq 1-\frac {4}{25}=\frac {21}{25}\]
- b
-
- i
- If \(X\thicksim U(0,10)\), find the upper limit of \(P\{|X-5|>3\}\).
\[X\thicksim U(0,10),\hspace {0.5cm} E(X)=5,\hspace {0.5cm}\Var (x)=\frac {100}{12}=\frac {25}{3}\] By Chebyshev’s inequality \[P\{|X-\mu |>3\}\leq \frac {25/3}{9}=\frac {25}{27}=0.926\] \begin {align*} \text {But}\hspace {0.5cm} P\{|X-5|>3\} &=1-P\{|X-5|\leq 3\}\\ &=1-P\{-3\leq X-5\leq 3\}\\ &=1-P\{2\leq X\leq 8\}\\ &=1-\int ^8_2\frac {1}{10}dx\\ &=1-\Bigg (\frac {8-2}{10}\Bigg )=1-\frac {6}{10} =\frac {2}{5}\\ &=0.40\\ \end {align*} - ii
- \(X\thicksim N(\mu ,\sigma ^2)\), find the upper limit of \(P\{|X-\mu |>1.96\sigma \}\).
By Chebyshev’s inequality \[P\Big \{|X-\mu |>1.96\sigma \Big \}\leq \frac {\sigma ^2}{(1.96\sigma )^2}=\frac {1}{(1.96)^2}=0.2603\] \begin {align*} P\Big \{|X-\mu |>1.96\sigma \Big \} &=P\Big \{\Big |\frac {X-\mu }{\sigma }\Big |>1.96\Big \}\\ &=P\Big \{|Z|>1.96\Big \} \end {align*}\[\text {where}\hspace {0.5cm}Z\thicksim N(0,1)\] \[P\Big \{|X-\mu |>1.96\sigma \Big \}=0.05\]
Remark. Chebyshev’s inequality though correct, the upper bound (lower bound) is not
particularly close to the actual probability as one would expect since the inequality is valid for
all distributions (all shapes, continuous, discrete).
Proof. We take \(P\Big \{|X-\mu |>\dfrac {1}{n}\Big \}\leq \dfrac {\Var (X)}{(1/n)^2},\hspace {0.5cm} \text {by Chebyshev's}\) \begin {align*} P\Big \{|X-\mu |>\frac {1}{n}\Big \} &\leq 0\\\\ \implies \hspace {0.5cm}P\Big \{|X-\mu |>\frac {1}{n}\Big \} &=0\\\\ \lim _{n\rightarrow \infty }P\Big \{|X-\mu |>\frac {1}{n}\Big \} &=0\\\\ \implies \hspace {0.5cm} P\Big \{\lim _{n\rightarrow \infty }\Big (|X-\mu |>\frac {1}{n}\Big )\Big \} &=0\hspace {0.3cm}\text {by continuity property of probability function}\\\\ \implies \hspace {0.5cm} P\{|X-\mu |>0\} &=0\\\\ \implies \hspace {0.5cm} P\{|X-\mu |\leq 0\} &=1\\\\ P\{X=\mu \} &=1\\\\ P\{X=E(X)\} &=1. \end {align*} □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.