2.8 Markov’s and Chebyshev’s Inequality

The primary value of Markov’s and Chebyshev’s inequalities lies in their ability to provide rigorous probability bounds when our knowledge of a distribution is limited. Specifically, these inequalities allow us to estimate the likelihood of events when we know only the mean, or both the mean and variance, of a random variable. (i.e. underlying distribution is either unknown or too complex to compute directly.)

Lemma 2.8.1 (Markov’s Inequality). Let \(X\) be a non-negative random variable having a finite mean and \(t\) be a positive real number then \[P(X\geq t)\leq \frac {E(X)}{t}.\]

Proof. Define a random variable \(Y\) by \[Y = \begin {cases} 0, &\text {if}\quad X<t\\ t, &\text {if}\quad X\geq t . \end {cases}\] \(Y\) is discrete, taking the value \(0\) with probability \(P(X<t)\) and the value \(t\) with probability \(P(X\geq t)\). Since \(X\geq 0\), in both cases \[X \geq Y .\] \[E(X) \geq E(Y)=0\,P(X<t)+t\,P(X\geq t)\] \[E(X) \geq tP(X\geq t)\] \[\frac {E(X)}{t}\geq P(X\geq t)\] \[ P(X\geq t) \leq \frac {E(X)}{t}.\] □

Theorem 2.8.2 (Chebyshev’s Inequality). Let \(X\) be a random variable with mean \(\mu \) and finite variance \(\sigma ^2\), then for any real number \(t>0\) \[P(|X-\mu |\geq t)\leq \frac {\sigma ^2}{t^2}\] or, equivalently, \[P(|X-\mu |<t)\geq 1-\frac {\sigma ^2}{t^2}.\]

Proof. Since \((X - \mu )^2 \geq 0\), we apply the Markov’s inequality to obtain \[P\left ((X-\mu )^2\geq t^2\right ) \leq \frac {E(X-\mu )^2}{t^2}=\frac {\sigma ^2}{t^2}\] That is \[P\left ((X-\mu )^2\geq t^2\right ) \leq \frac {\sigma ^2}{t^2}\] Now, since \((X-\mu )^2\geq t^2 \iff |X-\mu |\geq t\). Therefore, \[P(|X-\mu |\geq t) \leq \frac {\sigma ^2}{t^2}.\] □

Remark 2.8.3.

i)

The Complementary Nature of the Bound:The event \(|X-\mu | \geq t\) represents the “tails” of the distribution—values that are far from the mean.\[ |X-\mu | \geq t \iff (X \geq \mu + t) \text { or } (X \leq \mu - t) \]

ii)

Visualising the Upper and Lower Bounds: If we set \(t = k\sigma \), Chebyshev’s inequality tells us:

The probability in the shaded tails (at distance at least \(k\sigma \) from \(\mu \)) is at most \(\frac {1}{k^2}\).
The probability in the shaded centre (within distance \(k\sigma \)) is at least \(1 - \frac {1}{k^2}\).

Case A: Probability in the Tails (Maximum \(\frac {1}{k^2}\))

Case B: Probability in the Center (Minimum \(1 - \frac {1}{k^2}\))

Example 2.8.4. From a quick analysis of records a materials control manager estimates that the mean and standard deviation of the “lead time” required in ordering a small value are 8 days and 1.5 days respectively. The manager is willing to assume the mean and variance to be absolutely correct. The manager would like to determine a time interval such that the probability is at least \(\frac {8}{9}\) that the order will be received during the time interval.

Solution. \(\mu = 8, \,\, \sigma = 1.5\) and \[P(|X-\mu |<k\sigma ) \geq 1-\frac {1}{k^2}=\frac {8}{9}\,\,\implies \,\, k=3.\] Then \[(\mu -k\sigma ,\mu +k\sigma )= (8-3(1.5),8+3(1.5)) =(3.5,12.5).\] □

Example 2.8.5. Suppose we want the number of items produced in a factory during a week is a random variable with mean 500.

1.: What can be said about the probability that the weeks production will be at least 1000?

Solution. \(\mu =500\), \(P(X\geq 1000)\) by Markov’s inequality we get \[P(X\geq 1000)\leq \frac {E(X)}{1000}=\frac {500}{1000}=\frac {1}{2}.\] □
2.: If the variance of a week’s production is known to be 100, what can be said about the probability that this week’s production will be between 420 and 580?

Solution. \(\mu =500\), \(\sigma =10\) \[P(|X-\mu |<k\sigma )\geq 1-\frac {1}{k^2}\] rewriting the expression \[P(\mu -k\sigma < X < \mu + k\sigma )\geq 1-\frac {1}{k^2}\] but we have \(P(420 < X < 580)\) therefore \[500-k(10)=420\,\,\implies k=8\] \[500+k(10)=580\,\,\implies k=8.\] Hence \[P(420<X<580)\geq 1 - \frac {1}{8^2} = \frac {63}{64} \approx 0.984 .\] Without knowing the distribution, the manager can still be sure that at least \(98\%\) of weeks produce between \(420\) and \(580\) items. □

2.8.1 Practice Problems

Problem 2.8.1. Let \(X\) be a continuous random variable with p.d.f. \(f_X(x) = 3x^{2}\), \(x\in (0,1)\).

(a).: Use Chebyshev’s inequality to obtain lower bounds on \(P\left (\frac {5}{7}<X<\frac {7}{8}\right )\) and \(P\left (\frac {1}{2}<X<1\right )\).
(b).: Compare these bounds with the exact probabilities.
(c).: Under ordinary conditions a certain machine produces bolts with a mean diameter of 2 cm and a standard deviation of \(0.0017\) cm. A bolt is produced with diameter \(2.02\) cm. Would you say conditions were ordinary? Support your answer using Chebyshev’s inequality.

Show solution

Solution. First the mean and variance: \[\mu = \int _0^{1}3x^{3}\,dx = \frac {3}{4},\qquad E(X^{2}) = \int _0^{1}3x^{4}\,dx = \frac {3}{5},\] \[\sigma ^{2} = \frac {3}{5}-\frac {9}{16} = \frac {3}{80},\qquad \sigma = \sqrt {\frac {3}{80}} \approx 0.1936.\]

(a). Chebyshev’s inequality gives \(P(|X-\mu |<k\sigma ) \geq 1-\frac {1}{k^{2}}\), so the interval must first be written as one centred on \(\mu \).

Interval \(\left (\frac 57,\frac 78\right )\). Here \(\mu = 0.75\) sits inside, but not in the middle: it is \(0.0357\) above the left endpoint and \(0.125\) below the right one. The largest symmetric interval that fits is \(\mu \pm 0.0357\), so \[k = \frac {0.0357}{0.1936} \approx 0.184.\] Since \(k<1\), the bound \(1-1/k^{2}\) is negative, and the inequality tells us only that the probability is at least \(0\) – which we knew. The bound is vacuous here.

Interval \(\left (\frac 12,1\right )\). This one is centred on \(\mu = \tfrac 34\), with half-width \(\tfrac 14\), so \[k = \frac {0.25}{0.1936} \approx 1.291 \implies P\left (\tfrac 12<X<1\right ) \geq 1-\frac {1}{1.291^{2}} = 1-0.6 = 0.4.\]

(b). The exact values are \[P\left (\tfrac 57<X<\tfrac 78\right ) = \left [x^{3}\right ]_{5/7}^{7/8} \approx 0.3055, \qquad P\left (\tfrac 12<X<1\right ) = 1-\left (\tfrac 12\right )^{3} = 0.875.\] So Chebyshev gives \(\geq 0\) against a true \(0.3055\), and \(\geq 0.4\) against a true \(0.875\). Both bounds are correct and both are weak. That is the nature of the inequality: it uses only the mean and variance and must hold for every distribution with those, including far more spread-out ones than this. When the density is known, integrate it – Chebyshev is for when it is not.

(c). The observed diameter is \[k = \frac {|2.02-2|}{0.0017} = \frac {0.02}{0.0017} \approx 11.76\] standard deviations from the mean. By Chebyshev, \[P\left (|X-\mu | \geq 11.76\,\sigma \right ) \leq \frac {1}{11.76^{2}} \approx 0.0072.\] Under ordinary conditions a deviation this large would occur at most about 7 times in a thousand – and this holds whatever the shape of the distribution, which is precisely what makes the argument convincing here: we were told nothing about it. Such an outcome is implausible enough that one should conclude conditions were not ordinary and the machine needs checking.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.