7.1 Introduction
The most important theoretical results in probability theory are limit theorems, of two kinds. The laws of large numbers give conditions under which the average of a sequence of random variables converges to its mean. The central limit theorems give conditions under which the sum of a large number of random variables has a probability distribution that is approximately normal. Both rest on two simple inequalities.
Result 7.1.1 (Markov’s inequality). If \(X\) is a non-negative random variable, then for any \(b > 0\), \[ P(X \ge b) \le \frac {E(X)}{b} . \]
Proof. Let \(Z = 1\) if \(X \ge b\) and \(Z = 0\) otherwise. Then \(E(Z) = P(X \ge b)\). Since \(X \ge 0\), we always have \(Z \le X/b\): if \(Z = 1\) then \(X/b \ge 1\), and if \(Z = 0\) then \(X/b \ge 0\). Taking expectations, \(E(Z) \le E(X)/b\), that is, \(P(X \ge b) \le E(X)/b\). □
Result 7.1.2 (Chebyshev’s inequality). If \(X\) is a random variable with finite mean \(\mu \) and finite variance, then for any \(k > 0\), \[ P\bigl (|X - \mu | \ge k\bigr ) \le \frac {\Var (X)}{k^2} . \]
Proof. \(|X - \mu | \ge k\) exactly when \((X - \mu )^2 \ge k^2\). Applying Markov’s inequality to the non-negative variable \((X - \mu )^2\), \[ P\bigl (|X - \mu | \ge k\bigr ) = P\bigl ((X - \mu )^2 \ge k^2\bigr ) \le \frac {E(X - \mu )^2}{k^2} = \frac {\Var (X)}{k^2} . \] □
- (a)
- The number of items a factory produces in a week is a random variable with mean
\(60\).
- (i)
- What can be said about the probability that this week’s production will be at least \(80\)?
- (ii)
- If the variance of a week’s production is \(16\), what can be said about the probability that this week’s production will be between \(50\) and \(70\)?
- (b)
-
- (i)
- If \(X \sim U(0, 10)\), find Chebyshev’s upper bound for \(P(|X - 5| > 3)\), and compare it with the exact value.
- (ii)
- If \(X \sim N(\mu , \sigma ^2)\), find Chebyshev’s upper bound for \(P(|X - \mu | > 1.96\sigma )\), and compare it with the exact value.
Solution. (a) Let \(X\) be the number of items produced in a week.
(i) By Markov’s inequality, with \(E(X) = 60\), \[ P(X \ge 80) \le \frac {E(X)}{80} = \frac {60}{80} = 0.75 . \]
(ii) With \(\mu = 60\) and \(\Var (X) = 16\), \begin {align*} P(50 < X < 70) &= P(-10 < X - \mu < 10) = P(|X - \mu | < 10) \\ &= 1 - P(|X - \mu | \ge 10) , \end {align*}
and by Chebyshev’s inequality \(P(|X - \mu | \ge 10) \le 16/100 = 4/25\). Therefore \[ P(50 < X < 70) \ge 1 - \frac {4}{25} = \frac {21}{25} = 0.84 . \]
(b)(i) For \(X \sim U(0, 10)\), \(E(X) = 5\) and \(\Var (X) = 100/12 = 25/3\). By Chebyshev’s inequality, \[ P(|X - 5| > 3) \le \frac {25/3}{9} = \frac {25}{27} = 0.926 . \] The exact value is \[ P(|X - 5| > 3) = 1 - P(2 \le X \le 8) = 1 - \int _2^8\frac {1}{10}\,dx = 1 - \frac {6}{10} = 0.40 . \]
(ii) By Chebyshev’s inequality, \[ P(|X - \mu | > 1.96\sigma ) \le \frac {\sigma ^2}{(1.96\sigma )^2} = \frac {1}{1.96^2} = 0.2603 , \] whereas, with \(Z = (X - \mu )/\sigma \sim N(0, 1)\), the exact value is \(P(|Z| > 1.96) = 0.05\).
Remark. Chebyshev’s inequality is correct, but its bound is usually far from the actual probability, as both parts of (b) show. That is the price of generality: the inequality holds for every distribution with a finite variance, continuous or discrete, whatever its shape.
Proof. Let \(\mu = E(X)\). By Chebyshev’s inequality, for every \(n \ge 1\), \[ P\Bigl (|X - \mu | > \frac 1n\Bigr ) \le \frac {\Var (X)}{(1/n)^2} = 0 , \] so each of these probabilities is \(0\). The events \(\{|X - \mu | > 1/n\}\) increase with \(n\), and their union is \(\{|X - \mu | > 0\}\), so by the continuity property of probability, \[ P\bigl (|X - \mu | > 0\bigr ) = \lim _{n\to \infty }P\Bigl (|X - \mu | > \frac 1n\Bigr ) = 0 . \] Hence \(P(X = \mu ) = P\bigl (|X - \mu | = 0\bigr ) = 1\). □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.