2.2 Method of Moments
Let the population p.f. or p.d.f. \(f(x;\theta _1, \dots , \theta _k)\) depend on \(k\) parameters. The method of moments equates the first \(k\) sample moments to the corresponding population moments, which are functions of the parameters, \[M'_j = \mu '_j(\theta _1,\dots ,\theta _k),\qquad j = 1,2,\dots ,k ,\] and solves for the parameters. The solutions are the method of moments estimators (MME). The method rests on the law of large numbers: each \(M_j'\) is close to \(\mu _j'\) in large samples.
Example 2.2.1. Find the method of moments estimators of \(\mu \) and \(\sigma ^2\) for a random sample from \(N(\mu ,\sigma ^2)\).
Solution. With \(k = 2\), the equations are \(M'_1 = \mu '_1\) and \(M'_2 = \mu '_2\), that is \[\bar {X} = \mu ,\qquad \frac {1}{n}\sum ^n_{i = 1} X^2_i = \sigma ^2 + \mu ^2 .\] So \(\hat {\mu } = \bar {X}\) and \[\hat \sigma ^2 = \frac {1}{n}\sum _{i=1}^n X_i^2 - \bar {X}^2 = \frac {1}{n}\sum _{i=1}^n (X_i - \bar {X})^2 .\] Note the divisor \(n\), not \(n - 1\): the method of moments estimator of the variance is slightly biased, as shown later in this chapter.
Example 2.2.2. Let \(X_1, \dots , X_n\) be a random sample from the exponential distribution with p.d.f. \(f(x;\lambda ) = \lambda e^{-\lambda x}\), \(x>0\), \(\lambda >0\). Find the method of moments estimator of \(\lambda \).
Solution. With one parameter, equate \(M'_1 = \bar X\) to \(E(X) = \frac {1}{\lambda }\): \[\bar X = \frac {1}{\lambda }\qquad \text {so}\qquad \hat {\lambda }_{\text {MME}} = \frac {1}{\bar X} = \frac {n}{\sum _{i=1}^n X_i} .\]
Example 2.2.3. Find the method of moments estimators of \(\alpha \) and \(\beta \) for a random sample from the beta distribution with p.d.f. \(f(x;\alpha ,\beta ) = \frac {x^{\alpha -1}(1-x)^{\beta -1}}{B(\alpha ,\beta )}\), \(0<x<1\).
Solution. The first two moments of the beta distribution are \[\mu '_1 = \frac {\alpha }{\alpha + \beta },\qquad \mu '_2 = \frac {\alpha (\alpha + 1)}{(\alpha + \beta )(\alpha + \beta + 1)} ,\] since \(E(X^k) = \frac {B(\alpha + k,\beta )}{B(\alpha ,\beta )}\). Write \(\bar X = M_1'\) and \(M_2' = \frac 1n\sum X_i^2\). The equations are \begin {equation} \bar X = \frac {\alpha }{\alpha + \beta } , \label {eq:mmebeta1} \end {equation} \begin {equation} M_2' = \frac {\alpha (\alpha + 1)}{(\alpha + \beta )(\alpha + \beta + 1)} . \label {eq:mmebeta2} \end {equation} By (2.1), \(\alpha + \beta = \frac {\alpha }{\bar X}\), and then (2.2) becomes \[M_2' = \bar X\,\frac {\alpha + 1}{\frac {\alpha }{\bar X} + 1} = \frac {\bar X^2(\alpha + 1)}{\alpha + \bar X} .\] Solving for \(\alpha \), \(M_2'(\alpha + \bar X) = \bar X^2(\alpha + 1)\), so \[\hat \alpha = \frac {\bar X\left (\bar X - M_2'\right )}{M_2' - \bar X^2},\qquad \hat \beta = \hat \alpha \,\frac {1 - \bar X}{\bar X} = \frac {(1 - \bar X)\left (\bar X - M_2'\right )}{M_2' - \bar X^2} .\] The denominator \(M_2' - \bar X^2\) is the sample variance with divisor \(n\).
Example 2.2.4. Find the method of moments estimator of \(\theta \) for a random sample from the uniform distribution on \((0,\theta )\), and show that it can be impossible.
Solution. Equating \(\bar X\) to \(E(X) = \frac {\theta }{2}\) gives \(\hat \theta = 2\bar X\). For the sample \(0.1, 0.2, 0.9\), \(\hat \theta = 0.8\), which is smaller than an observed value: a uniform distribution on \((0,0.8)\) could not have produced \(0.9\). The method of moments uses only the moments and ignores what the data say about the support; the maximum likelihood estimator of Example 2.4.5 does not have this defect.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.