Proof. Write \(\overline X = (X_1 + \dots + X_n)/n\), so \(E(\overline X) = \mu \) and \(\Var (\overline X) = \sigma ^2/n\). By Chebyshev’s inequality, \[ P\bigl (|\overline X - \mu | \ge \varepsilon \bigr ) \le \frac {\Var (\overline X)}{\varepsilon ^2} = \frac {\sigma ^2}{n\varepsilon ^2} \to 0 \qquad (n \to \infty ) , \] and a probability cannot be negative, so the limit is \(0\). □

Proof. We assume that the moment generating function \(M(t)\) of the \(X_i\) exists near \(0\). Note first that \[ \frac {X_1 + \dots + X_n - n\mu }{\sigma \sqrt n} = \frac {\overline X - \mu }{\sigma /\sqrt n} = \frac {\sqrt n(\overline X - \mu )}{\sigma } . \] Suppose first that \(\mu = 0\) and \(\sigma ^2 = 1\), so that this quantity is \(Z_n = \sqrt n\,\overline X = \sum _{i=1}^nX_i/\sqrt n\). Then \begin {align*} M_{Z_n}(t) &= E\bigl (e^{tZ_n}\bigr ) = E\Bigl (\prod _{i=1}^ne^{tX_i/\sqrt n}\Bigr ) \\ &= \prod _{i=1}^nE\bigl (e^{tX_i/\sqrt n}\bigr ) \qquad \text {(independence)} \\ &= \bigl (M(t/\sqrt n)\bigr )^n \qquad \text {(identical distributions).} \end {align*}

Let \(L(t) = \ln M(t)\). Since \(M(0) = 1\), \(M'(0) = \mu = 0\) and \(M''(0) = E(X^2) = 1\), \begin {gather*} L(0) = 0 , \qquad L'(0) = \frac {M'(0)}{M(0)} = 0 , \\ L''(0) = \frac {M''(0)M(0) - \bigl (M'(0)\bigr )^2}{\bigl (M(0)\bigr )^2} = 1 . \end {gather*} By Lemma 7.2.2 it is enough to show that \(M_{Z_n}(t) \to e^{t^2/2}\), the moment generating function of \(N(0, 1)\), that is, that \(\ln M_{Z_n}(t) = nL(t/\sqrt n) \to t^2/2\). Writing \(nL(t/\sqrt n) = L(t/\sqrt n)/(1/n)\), the numerator and the denominator both tend to \(0\), so by L’Hôpital’s rule (differentiating with respect to \(n\)), \begin {align*} \lim _{n\to \infty }\frac {L(t/\sqrt n)}{1/n} &= \lim _{n\to \infty }\frac {L'(t/\sqrt n)\cdot \bigl (-\frac 12tn^{-3/2}\bigr )}{-n^{-2}} = \lim _{n\to \infty }\frac {t\,L'(t/\sqrt n)}{2n^{-1/2}} . \end {align*}

This is again of the form \(0/0\), because \(L'(0) = 0\), and applying the rule once more, \begin {align*} \lim _{n\to \infty }\frac {t\,L'(t/\sqrt n)}{2n^{-1/2}} &= \lim _{n\to \infty }\frac {t\,L''(t/\sqrt n)\cdot \bigl (-\frac 12tn^{-3/2}\bigr )}{-n^{-3/2}} \\ &= \frac {t^2}{2}\lim _{n\to \infty }L''(t/\sqrt n) = \frac {t^2}{2}L''(0) = \frac {t^2}{2} . \end {align*}

Therefore \(M_{Z_n}(t) \to e^{t^2/2}\).

For general \(\mu \) and \(\sigma ^2\), apply this to the standardised variables \(Y_i = (X_i - \mu )/\sigma \), which have mean \(0\) and variance \(1\). Since \[ \frac {1}{\sqrt n}\sum _{i=1}^nY_i = \frac {\sum _{i=1}^n(X_i - \mu )}{\sigma \sqrt n} = \frac {X_1 + \dots + X_n - n\mu }{\sigma \sqrt n} , \] the result follows. □

Solution. Let \(X\) be the number of tourists who sign up. The exact probability is \[ P(X \ge 80) = \sum _{x=80}^\infty \frac {60^xe^{-60}}{x!} = 1 - \sum _{x=0}^{79}\frac {60^xe^{-60}}{x!} , \] which is tedious to compute by hand, so we approximate it with the central limit theorem. A Poisson variable with mean \(60\) is the sum of \(60\) independent Poisson variables each with mean \(1\), \(X = \sum _{i=1}^{60}X_i\) with \(X_i \sim P(1)\), so \(X\) has mean \(60\) and variance \(60\), and \begin {align*} P(X \ge 80) &= P\Bigl (\frac {X - 60}{\sqrt {60}} \ge \frac {80 - 60}{\sqrt {60}}\Bigr ) \\ &\approx P(Z \ge 2.582) = 1 - \Phi (2.58) = 0.0049 . \end {align*}

The exact value is \(0.0078\). The approximation captures the order of magnitude, but in the far tail of a skewed distribution like the Poisson it is only rough.

Solution. Let \(X_i\) be the score on the \(i\)th die and \(X = \sum _{i=1}^{10}X_i\) the sum. For each die, \[ E(X_i) = \sum _{j=1}^6j\cdot \frac 16 = \frac 16\cdot \frac {6 \times 7}{2} = \frac 72 , \qquad E(X_i^2) = \sum _{j=1}^6j^2\cdot \frac 16 = \frac {91}{6} , \] using \(\sum _{x=1}^Nx^2 = N(N + 1)(2N + 1)/6 = 91\) for \(N = 6\). So \[ \Var (X_i) = \frac {91}{6} - \Bigl (\frac 72\Bigr )^2 = \frac {91}{6} - \frac {49}{4} = \frac {35}{12} , \] and \(E(X) = 10 \times \frac 72 = 35\), \(\Var (X) = 10 \times \frac {35}{12} = \frac {175}{6}\), so \(\operatorname {sd}(X) = \sqrt {175/6} = 5.401\). By the central limit theorem, \begin {align*} P(40 < X < 50) &\approx P\Bigl (\frac {40 - 35}{5.401} < Z < \frac {50 - 35}{5.401}\Bigr ) = \Phi (2.777) - \Phi (0.926) \\ &= 0.9973 - 0.8228 = 0.175 . \end {align*}

Proof. We prove the theorem under the extra assumption that the fourth moment is finite, \(E(X_i^4) = K < \infty \). Suppose first that \(\mu = 0\), and let \(S_n = X_1 + X_2 + \dots + X_n\). Expanding \[ E(S_n^4) = E\bigl [(X_1 + \dots + X_n)^4\bigr ] \] gives terms of five types, with \(i, j, l, k\) distinct: \[ X_i^4 , \qquad X_i^3X_j , \qquad X_i^2X_j^2 , \qquad X_i^2X_jX_l , \qquad X_iX_jX_lX_k . \] By independence and \(E(X_i) = 0\), three of these types have expectation zero: \begin {gather*} E(X_i^3X_j) = E(X_i^3)E(X_j) = 0 , \\ E(X_i^2X_jX_l) = E(X_i^2)E(X_j)E(X_l) = 0 , \\ E(X_iX_jX_lX_k) = E(X_i)E(X_j)E(X_l)E(X_k) = 0 . \end {gather*} There are \(n\) terms \(X_i^4\), and for each of the \(\binom n2\) pairs \(i < j\) there are \(\binom 42 = 6\) terms \(X_i^2X_j^2\). Therefore \[ E(S_n^4) = nK + 6\binom n2\bigl (E(X^2)\bigr )^2 = nK + 3n(n - 1)\bigl (E(X^2)\bigr )^2 . \qquad (*) \] Since \(0 \le \Var (X^2) = E(X^4) - \bigl (E(X^2)\bigr )^2 = K - \bigl (E(X^2)\bigr )^2\), we have \(\bigl (E(X^2)\bigr )^2 \le K\), and \((*)\) gives \(E(S_n^4) \le nK + 3n(n - 1)K\). Hence \[ E\Bigl (\frac {S_n^4}{n^4}\Bigr ) \le \frac {K}{n^3} + \frac {3(n - 1)K}{n^3} \le \frac {K}{n^2} + \frac {3K}{n^2} = \frac {4K}{n^2} , \] and so \[ E\Bigl (\sum _{n=1}^\infty \frac {S_n^4}{n^4}\Bigr ) = \sum _{n=1}^\infty E\Bigl (\frac {S_n^4}{n^4}\Bigr ) \le 4K\sum _{n=1}^\infty \frac {1}{n^2} < \infty . \] A non-negative random variable whose expectation is finite is itself finite with probability \(1\), so \(\sum _n S_n^4/n^4 < \infty \) with probability \(1\). The terms of a convergent series tend to zero, so with probability \(1\), \((S_n/n)^4 = S_n^4/n^4 \to 0\), and hence \(S_n/n \to 0\).

For general \(\mu \), apply this to \(Y_i = X_i - \mu \), which have mean \(0\): with probability \(1\), \[ \frac {Y_1 + \dots + Y_n}{n} = \frac {X_1 + \dots + X_n}{n} - \mu \to 0 , \] that is, \((X_1 + \dots + X_n)/n \to \mu \). □