1.6 Practice problems: limit theorems
Problems on Chebyshev’s inequality, limiting distributions and the central limit theorem, written for these notes. Normal probabilities are quoted to four decimal places from the standard table.
Problem 1.6.1. [Exercise] The number of litres of fuel sold at a filling station in a day has mean \(4200\) and standard deviation \(300\). Nothing else is known about the distribution.
- (a).
- Give a lower bound for the probability that a day’s sales fall between \(3600\) and \(4800\) litres.
- (b).
- Sales are now known to be approximately normal. Compare.
Show solution
Solution. (a). The interval is \(4200\pm 600\), and \(600=k\sigma \) with \(k=\frac {600}{300}=2\). Chebyshev’s inequality gives \[P\left (\left |X-4200\right |\geq 600\right )\leq \frac {1}{2^2}=0.25,\] \[\therefore \quad P\left (3600<X<4800\right )\geq 1-0.25=0.75.\]
(b). If \(X\sim N\left (4200,300^2\right )\) then \[P\left (3600<X<4800\right )=P(-2<Z<2)=0.9772-0.0228=0.9544.\]
Note 1.6.1. \(0.75\) against \(0.9544\): the bound is correct but weak, and deliberately so. It holds for every distribution with that mean and variance, including some very lopsided ones, so it cannot be tight for the normal in particular. Use Chebyshev when the shape is unknown, and a table when it is not.
Problem 1.6.2. [Exercise] Let \(Z_n\sim BIN(n,p)\) with \(0<p<1\) fixed. Use moment generating functions to find the limiting distribution of \[Y_n=\frac {Z_n-np}{\sqrt {np(1-p)}}.\]
Show solution
Solution. Write \(q=1-p\) and \(\sigma _n=\sqrt {npq}\). Since \(M_{Z_n}(t)=\left (q+pe^t\right )^n\), \[M_{Y_n}(t)=e^{-\frac {npt}{\sigma _n}} \left (q+pe^{\frac {t}{\sigma _n}}\right )^n,\qquad \text {so}\qquad \ln M_{Y_n}(t)=-\frac {npt}{\sigma _n} +n\ln \left (q+pe^{\frac {t}{\sigma _n}}\right ).\] Expand the exponential, writing \(s=\frac {t}{\sigma _n}\to 0\): \[q+pe^{s}=q+p\left (1+s+\frac {s^2}{2}+\cdots \right ) =1+ps+\frac {ps^2}{2}+\cdots ,\] since \(q+p=1\). Now use \(\ln (1+w)=w-\frac {w^2}{2}+\cdots \) with \(w=ps+\frac {ps^2}{2}+\cdots \): \[\ln \left (q+pe^{s}\right )=ps+\frac {ps^2}{2}-\frac {p^2s^2}{2}+O\left (s^3\right ) =ps+\frac {pq\,s^2}{2}+O\left (s^3\right ),\] because \(\frac {p}{2}-\frac {p^2}{2}=\frac {p(1-p)}{2}=\frac {pq}{2}\). Therefore \[\ln M_{Y_n}(t)=-\frac {npt}{\sigma _n}+n\left [\frac {pt}{\sigma _n} +\frac {pq\,t^2}{2\sigma _n^2}+O\!\left (\frac {1}{n^{\frac 32}}\right )\right ] =\frac {npq\,t^2}{2npq}+O\!\left (\frac {1}{\sqrt n}\right ) \ \longrightarrow \ \frac {t^2}{2}.\] \[\therefore \quad M_{Y_n}(t)\longrightarrow e^{\frac {t^2}{2}}, \qquad \text {so}\qquad \frac {Z_n-np}{\sqrt {np(1-p)}}\ \overset {d}{\longrightarrow }\ N(0,1).\]
Note 1.6.2. This is the central limit theorem in the one case where it can be checked by hand from the moment generating function alone: \(Z_n\) is a sum of \(n\) independent Bernoulli variables, so the general theorem covers it. Doing it directly shows where the \(\frac {t^2}{2}\) comes from — the linear term always cancels against the centring, and the quadratic term always survives at exactly order one.
Problem 1.6.3. [Exercise] Let \(X_1,\ldots ,X_n\) be a random sample from \(\text {POI}(1)\), and let \[Y_n=\frac {\sqrt n\left (\bar X-\mu \right )}{\sigma }.\]
- (a).
- Show that \(Y_n=\sqrt n\left (\bar X-1\right )\) and that its moment generating function is \(\exp \left [-t\sqrt n+n\left (e^{\frac {t}{\sqrt n}}-1\right )\right ]\).
- (b).
- Deduce the limiting distribution of \(Y_n\).
Show solution
Solution. (a). For \(\text {POI}(1)\) both the mean and the variance are \(1\), so \(\mu =1\), \(\sigma =1\), and \(Y_n=\sqrt n\left (\bar X-1\right )\) immediately.
A sum of independent Poisson variables is Poisson with the means added, so \(S=\sum _{i=1}^nX_i\sim \text {POI}(n)\) and \(M_S(t)=e^{n\left (e^t-1\right )}\). Now \[Y_n=\sqrt n\left (\frac {S}{n}-1\right )=\frac {S-n}{\sqrt n},\] so \[M_{Y_n}(t)=E\left [e^{\frac {t(S-n)}{\sqrt n}}\right ] =e^{-t\sqrt n}\,M_S\!\left (\frac {t}{\sqrt n}\right ) =\exp \left [-t\sqrt n+n\left (e^{\frac {t}{\sqrt n}}-1\right )\right ]. \qquad \blacksquare \]
(b). This is exactly the moment generating function computed in the worked example of section 1.4, where \(Z_n\sim \text {POI}(n)\) and \(Y_n=\frac {Z_n-n}{\sqrt n}\) — and indeed \(S\) here is such a \(Z_n\). Expanding as there, \[\ln M_{Y_n}(t)=\frac {t^2}{2}+O\!\left (\frac {1}{\sqrt n}\right ) \ \longrightarrow \ \frac {t^2}{2},\] \[\therefore \quad Y_n\ \overset {d}{\longrightarrow }\ N(0,1).\]
Note 1.6.3. Part (a) is the whole point of the question. Recognising that a sample mean from \(\text {POI}(1)\) is a rescaled \(\text {POI}(n)\) turns a question about \(n\) variables into a question about one, and the limit is then something already done. Look for this reduction before starting any expansion.
Problem 1.6.4. [Exercise] A random sample of size \(20\) is taken from the distribution with density \(f(x)=2x\), \(0<x<1\). Use the central limit theorem to approximate \(P\left (0.60<\bar X<0.75\right )\).
Show solution
Solution. First the population mean and variance: \[\mu =\int _0^12x^2dx=\frac 23,\qquad E\left (X^2\right )=\int _0^12x^3dx=\frac 12,\] \[\sigma ^2=\frac 12-\left (\frac 23\right )^2=\frac 12-\frac 49=\frac {1}{18}.\] So \(\bar X\) has mean \(\frac 23=0.6667\) and variance \(\frac {1}{18\times 20}=\frac {1}{360}\), giving \[\sigma _{\bar X}=\frac {1}{\sqrt {360}}=0.0527.\] By the central limit theorem \(\bar X\) is approximately normal, so \[P\left (0.60<\bar X<0.75\right ) \approx P\!\left (\frac {0.60-0.6667}{0.0527}<Z<\frac {0.75-0.6667}{0.0527}\right ) =P(-1.26<Z<1.58),\] \[P\left (0.60<\bar X<0.75\right )\approx 0.9429-0.1038=0.8391.\]
Note 1.6.4. Simulation puts the true value at about \(0.842\), so the approximation is good to about three parts in a thousand at \(n=20\). The population here is only mildly skewed, which is why so small a sample suffices; the same \(n\) against a heavily skewed population would do far worse.
Problem 1.6.5. [Exercise] Let \(Y\sim BIN(150,0.3)\). Use a normal approximation, with a continuity correction, to find (a) \(P(Y\geq 50)\) and (b) \(P(Y=45)\).
Show solution
Solution. Here \(\mu =np=45\) and \(\sigma ^2=np(1-p)=150(0.3)(0.7)=31.5\), so \(\sigma =5.6125\).
(a). The event \(Y\geq 50\) becomes \(Y>49.5\): \[P(Y\geq 50)\approx P\!\left (Z>\frac {49.5-45}{5.6125}\right )=P(Z>0.80) =1-0.7881=0.2119.\] The exact binomial value is \(0.2101\).
(b). \[P(Y=45)\approx P\left (44.5<Y<45.5\right ) =P\!\left (\frac {-0.5}{5.6125}<Z<\frac {0.5}{5.6125}\right )=P(-0.089<Z<0.089),\] \[P(Y=45)\approx 0.5355-0.4645=0.0710,\] against an exact value of \(0.0709\).
Note 1.6.5. \(np=45\) and \(n(1-p)=105\) are both comfortably above \(5\), which is the usual condition for this approximation to be trustworthy. It is a statement about the smaller of the two expected counts, not about \(n\): a sample of \(10\,000\) with \(p=0.0002\) fails it, and no amount of extra data repairs that.
Problem 1.6.6. [Exercise] Let \(\bar X\) be the mean of a random sample of size \(50\) from a \(\chi ^2(20)\) distribution. Find an approximate value for \(P\left (19<\bar X<21\right )\).
Show solution
Solution. A \(\chi ^2(k)\) distribution has mean \(k\) and variance \(2k\), so here \(\mu =20\) and \(\sigma ^2=40\). Then \[\text {Var}\left (\bar X\right )=\frac {40}{50}=0.8,\qquad \sigma _{\bar X}=\sqrt {0.8}=0.8944.\] By the central limit theorem, \[P\left (19<\bar X<21\right ) \approx P\!\left (\frac {19-20}{0.8944}<Z<\frac {21-20}{0.8944}\right ) =P(-1.12<Z<1.12),\] \[P\left (19<\bar X<21\right )\approx 0.8686-0.1314=0.7372.\]
Note 1.6.6. No approximation was really needed: a sum of independent chi-squares is chi-square, so \(50\bar X\sim \chi ^2(1000)\) exactly, and the exact answer is \(0.7367\). The normal approximation is doing well here precisely because \(\chi ^2(1000)\) is itself nearly normal — which is the central limit theorem again, since a \(\chi ^2(1000)\) variable is a sum of a thousand independent pieces.
Problem 1.6.7. [Exercise] Let \(\bar X\) be the mean of a random sample of size \(64\) from \(\text {GAM}(2,3)\). Find an approximate value for \(P\left (5<\bar X<7\right )\).
Show solution
Solution. For \(\text {GAM}(\alpha ,\beta )\) with \(\alpha =2\) and \(\beta =3\), \[\mu =\alpha \beta =6,\qquad \sigma ^2=\alpha \beta ^2=18.\] So \(\text {Var}\left (\bar X\right )=\frac {18}{64}=0.2813\) and \(\sigma _{\bar X}=0.5303\). Then \[P\left (5<\bar X<7\right ) \approx P\!\left (\frac {5-6}{0.5303}<Z<\frac {7-6}{0.5303}\right ) =P(-1.89<Z<1.89),\] \[P\left (5<\bar X<7\right )\approx 0.9706-0.0294=0.9412.\]
Note 1.6.7. Exactly, \(\bar X\sim \text {GAM}\!\left (128,\frac {3}{64}\right )\) by the earlier exercise on the sampling distribution of a gamma mean. That gives \(0.94125\), against \(0.94124\) from the normal approximation — agreement to five decimal places. A gamma distribution with shape \(128\) is for practical purposes normal already, so there was nothing left for the approximation to get wrong.
Problem 1.6.8. [Exercise] Independent random samples of size \(36\) are taken from \(N(10,25)\) and \(N(8,16)\), with means \(\bar X\) and \(\bar Y\) respectively. Find \(P\left (\bar X>\bar Y\right )\).
Show solution
Solution. The two sample means are normal and independent: \[\bar X\sim N\!\left (10,\frac {25}{36}\right ),\qquad \bar Y\sim N\!\left (8,\frac {16}{36}\right ).\] A difference of independent normal variables is normal, with the means subtracted and the variances added: \[\bar X-\bar Y\sim N\!\left (2,\ \frac {25}{36}+\frac {16}{36}\right ) =N\!\left (2,\ \frac {41}{36}\right ),\qquad \text {sd}=\sqrt {\frac {41}{36}}=1.0672.\] Therefore \[P\left (\bar X>\bar Y\right )=P\left (\bar X-\bar Y>0\right ) =P\!\left (Z>\frac {0-2}{1.0672}\right )=P(Z>-1.87)=0.9693.\]
Note 1.6.8. No approximation is involved — both populations are normal, so the difference of the means is exactly normal at any sample size. The central limit theorem would give the same answer for non-normal populations at this \(n\), but here it is not needed, and saying so is part of the answer.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.