4.5 The moment generating function of a random sum
Let \(X_1, X_2, \dots \) be independent and identically distributed, and let \(N\) be a random variable, independent of them, taking the values \(0, 1, 2, \dots \). We want the moment generating function of the random sum \[ Y = \sum _{i=1}^NX_i . \] Conditioning on \(N\): given \(N = n\), \(Y\) is a sum of \(n\) independent copies of \(X\), so \[ E\bigl (e^{tY} \mid N = n\bigr ) = E\Bigl (\prod _{i=1}^ne^{tX_i}\Bigr ) = \prod _{i=1}^nE\bigl (e^{tX_i}\bigr ) = \bigl (M_X(t)\bigr )^n , \] that is, \(E\bigl (e^{tY} \mid N\bigr ) = \bigl (M_X(t)\bigr )^N\). By Result 4.2.1, \[ M_Y(t) = E\bigl (e^{tY}\bigr ) = E\bigl [E\bigl (e^{tY} \mid N\bigr )\bigr ] = E\bigl [\bigl (M_X(t)\bigr )^N\bigr ] . \] Differentiating under the expectation, \[ M_Y'(t) = E\bigl [N\bigl (M_X(t)\bigr )^{N-1}M_X'(t)\bigr ] , \] and since \(M_X(0) = 1\) and \(M_X'(0) = E(X)\), \[ E(Y) = M_Y'(0) = E\bigl (N\,E(X)\bigr ) = E(N)E(X) . \] Differentiating again, \[ M_Y''(t) = E\Bigl [N(N - 1)\bigl (M_X(t)\bigr )^{N-2}\bigl (M_X'(t)\bigr )^2 + N\bigl (M_X(t)\bigr )^{N-1}M_X''(t)\Bigr ] , \] so \begin {align*} E(Y^2) = M_Y''(0) &= E\bigl [N(N - 1)\bigr ]\bigl (E(X)\bigr )^2 + E(N)E(X^2) \\ &= E(N^2)\bigl (E(X)\bigr )^2 - E(N)\bigl (E(X)\bigr )^2 + E(N)E(X^2) , \end {align*}
and \begin {align*} \Var (Y) &= E(Y^2) - \bigl (E(Y)\bigr )^2 \\ &= E(N^2)\bigl (E(X)\bigr )^2 - E(N)\bigl (E(X)\bigr )^2 \\ &\qquad + E(N)E(X^2) - \bigl (E(N)\bigr )^2\bigl (E(X)\bigr )^2 \\ &= \Bigl (E(N^2) - \bigl (E(N)\bigr )^2\Bigr )\bigl (E(X)\bigr )^2 \\ &\qquad + E(N)\Bigl (E(X^2) - \bigl (E(X)\bigr )^2\Bigr ) \\ &= \Var (N)\bigl (E(X)\bigr )^2 + E(N)\Var (X) . \end {align*}
The same results follow from the conditional variance formula, with \(E(Y \mid N) = N\,E(X)\) and \(\Var (Y \mid N) = N\Var (X)\).
Example 4.5.1. As in Example 4.3.1, let \(X\) be uniform on \((0, 1)\) and, given \(X = p\), let \(Y\) be binomial with parameters \(n\) and \(p\). Find the distribution of \(Y\) using moment generating functions.
Solution. Given \(X = p\), \(E\bigl (e^{tY} \mid X = p\bigr ) = \bigl (pe^t + 1 - p\bigr )^n\), the binomial moment generating function. So \begin {align*} M_Y(t) = E\bigl (e^{tY}\bigr ) &= E\bigl [E\bigl (e^{tY} \mid X\bigr )\bigr ] = \int _0^1\bigl (1 + p(e^t - 1)\bigr )^n\,dp \\ &= \Bigl [\frac {\bigl (1 + p(e^t - 1)\bigr )^{n+1}}{(n + 1)(e^t - 1)}\Bigr ]_0^1 = \frac {e^{(n+1)t} - 1}{(n + 1)(e^t - 1)} . \end {align*}
Summing the geometric series \(1 + e^t + \dots + e^{nt} = \dfrac {e^{(n+1)t} - 1}{e^t - 1}\), \[ M_Y(t) = \frac {1}{n + 1}\sum _{k=0}^ne^{kt} = \frac {1}{n + 1}\bigl (1 + e^t + e^{2t} + \dots + e^{nt}\bigr ) , \] which is the moment generating function of the discrete uniform distribution on \(\{0, 1, \dots , n\}\), each value having probability \(1/(n + 1)\). By uniqueness, this is the distribution of \(Y\), as found in Example 4.3.1.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.