3.3 The Generating Function Shortcut
Result 3.3.1. If \(X\) and \(Y\) are independent then \[G_{X+Y}(t) = G_X(t)\,G_Y(t),\qquad M_{X+Y}(t) = M_X(t)\,M_Y(t),\qquad K_{X+Y}(t) = K_X(t) + K_Y(t).\]
Combined with uniqueness, this replaces the convolution sum or integral by a multiplication. The Poisson example above is a one-line calculation this way: \[G_X(t) = e^{\mu (t-1)},\qquad G_Y(t) = e^{\lambda (t-1)},\] \[G_{X+Y}(t) = e^{\mu (t-1)}e^{\lambda (t-1)} = e^{(\mu +\lambda )(t-1)},\] which is the probability generating function of POI\((\mu +\lambda )\), so by uniqueness \(X+Y\) has that distribution. No binomial expansion is needed.
Example 3.3.2. Show that the sum of independent normal variables is normal, and identify its parameters.
Solution. Let \(X\sim N\left (\mu _1,\sigma _1^{2}\right )\) and \(Y\sim N\left (\mu _2,\sigma _2^{2}\right )\) be independent, with moment generating functions \(M_X(t)=e^{\mu _1 t + \frac 12\sigma _1^{2}t^{2}}\) and similarly for \(Y\). Then \[M_{X+Y}(t) = e^{\mu _1 t + \frac 12\sigma _1^{2}t^{2}}\; e^{\mu _2 t + \frac 12\sigma _2^{2}t^{2}} = e^{(\mu _1+\mu _2)t + \frac 12\left (\sigma _1^{2}+\sigma _2^{2}\right )t^{2}},\] which is the moment generating function of \(N\left (\mu _1+\mu _2,\ \sigma _1^{2}+\sigma _2^{2}\right )\). By uniqueness that is the distribution of \(X+Y\).
The convolution integral for this case is a completion of the square inside an exponential and takes most of a page; the generating function argument takes two lines. That contrast is the whole reason the previous chapter came first.
Remark. The departmental notes justify this step by saying that a distribution is uniquely determined by its probabilities, its moments or its cumulants. The first is a definition and the third follows from the second, but the statement about moments is false in general. There exist distinct distributions with identical moments of every order — the lognormal is the standard example, and a whole family of densities shares its moment sequence.
What is true, and what is being used here, is the uniqueness theorem of the previous chapter: if two moment generating functions agree on an interval about the origin, the distributions agree. That is a stronger hypothesis than having the same moments, because it requires the series to converge, and it is exactly the hypothesis that fails for the lognormal — whose moment generating function is infinite for every \(t>0\). The distinction is not pedantry; it is the difference between a theorem and a false statement.
Result 3.3.3 (Closure under addition). The following families are closed under the addition of independent members, each provable in one line by multiplying generating functions:
- \(-\)
- \(\text {POI}(\mu ) + \text {POI}(\lambda ) = \text {POI}(\mu +\lambda )\);
- \(-\)
- \(\text {BIN}(m,\theta ) + \text {BIN}(n,\theta ) = \text {BIN}(m+n,\theta )\), provided the success probability is the same;
- \(-\)
- \(\text {GAM}(\alpha _1,\beta ) + \text {GAM}(\alpha _2,\beta ) = \text {GAM}(\alpha _1+\alpha _2,\beta )\), provided the scale is the same;
- \(-\)
- \(N\left (\mu _1,\sigma _1^{2}\right ) + N\left (\mu _2,\sigma _2^{2}\right ) = N\left (\mu _1+\mu _2,\sigma _1^{2}+\sigma _2^{2}\right )\);
- \(-\)
- \(\chi ^{2}_{(m)} + \chi ^{2}_{(n)} = \chi ^{2}_{(m+n)}\), a special case of the gamma.
Note. The provisos matter. Two binomials with different success probabilities do not add to a binomial, and two gammas with different scales do not add to a gamma; in each case the generating functions fail to combine because the parameter that must match appears in the base rather than the exponent. It is a common error to quote these closure results without their conditions.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.