2.2 Method of Moments
Consider a population PDF, \(f(x;\theta _1, \theta _2, \cdots , \theta _k)\), depending on one or more parameters \(\theta _1, \theta _2, \cdots , \theta _k\).
Method of moments estimators are solutions to the following system of equations \[M'_j=\mu '_j,\quad j=1,2,\cdots ,k\]
Example 2.2.1. Find the method of moments estimators for \(\mu \) and \(\sigma ^2\) if \(X_1,X_2,\cdots ,X_n\) is \(N(\mu ,\sigma ^2)\), and \( \theta = (\mu ,\sigma ^2)\).
Solution. \(k=2\), that is \(\theta _1=\mu ,\quad \theta _2=\sigma ^2\) Therefore, \[M'_j=\mu '_j,\quad j=1,2\] That is \(M'_1=\mu '_1\) and \(M'_2=\mu '_2.\) \[\implies \quad \overline {X}=\mu \quad \text {and}\quad \frac {1}{n}\sum ^n_{i = 1} X^2_j=\sigma ^2+\mu ^2\] \[\implies \quad \hat {\mu }=\overline {X}, \quad \quad \frac {1}{n}\sum _{i=1}^n X^2_1=\sigma ^2+\left (\overline {X}\right )^2\] \[\widehat {\sigma }^2=\frac {1}{n}\sum _{i=1}^n X_j^2-\left (\overline {X}\right )^2\] \[E(X)=\theta _1=\overline {X}=\frac {1}{n}\sum _{i = 1}^n X_j\] \[E(X^2) = \theta _1^2+\theta _2=\frac {1}{n}\sum _{i=1}^n X^2_j\] Therefore, the methods of moments. Estimator for: \(\mu \) is \(\overline {X}\) i.e \[\widehat {\mu }_{MME}=\overline {X}\] \(\sigma ^2\) is \(\frac {1}{n}\sum _{i = 1}^n X^2_j-(\overline {X})^2\) i.e \[\widehat {\sigma }^2_{MME}=S^{2}=\frac {1}{n}\sum _{i=1}^n (X_j-\overline {X})^2\] □
Example 2.2.2. Let \(X_1,X_2, \cdots , X_n\) be a random sample from the Exponential with PDF \(f_X(x,\lambda )=\lambda e^{\displaystyle {-\lambda x}},\quad x>0, \lambda >0\). Find the method of moments estimator.
Solution. \(\theta = \lambda \implies k=1\) \[M'_1=\mu '_1 \quad \implies \quad \frac {1}{n}\sum _{i=1}^n X_j=\mu =\frac {1}{\lambda }\] \[\frac {1}{n}\sum _{i=1}^n X_j=\frac {1}{\lambda }\quad \implies \quad \widehat {\lambda } = \frac {1}{\sum _{i=1}^n X_j}=\frac {1}{\overline {X}}\] That is \[\widehat {\lambda }_{MME} =\frac {1}{\overline {X}}.\] □
Example 2.2.3. Beta with PDF \(f_X(x,\alpha ,\beta )=\frac {x^{\alpha -1}(1-x)^{\beta -1}}{B(\alpha ,\beta )},\quad x\in (0,1)\)
Solution. \(\theta =(\alpha ,\beta )\implies k=2\). Therefore \[M'_1=\mu '_1\quad \implies \quad \frac {1}{n}\sum X_j=\mu = \frac {\alpha }{\alpha +\beta }.\] \begin {align*} M'_2=\mu '_2\quad \implies \quad \frac {1}{n}\sum X_j^2 & =\sigma ^2+\mu ^2\\ & = \frac {\alpha \beta }{(\alpha +\beta )^2(\alpha +\beta +1)}+\left (\frac {\alpha }{\alpha +\beta }\right )^2\\ & = \frac {\alpha }{(\alpha +\beta )^2}\left [\frac {\beta }{(\alpha +\beta +1)}+\alpha \right ]\\ & = \frac {\alpha }{(\alpha +\beta )^2}\left [\frac {\beta +\alpha ^2+\alpha \beta +\alpha }{\alpha +\beta +1}\right ]\\ & = \frac {\alpha }{(\alpha +\beta )^2}\left [\frac {\beta +\alpha (\alpha +\beta )+\beta }{\alpha +\beta +1}\right ]\\ & = \frac {\alpha }{(\alpha +\beta )^2}\left [\frac {\alpha +\beta +\alpha (\alpha +\beta )}{\alpha +\beta +1}\right ]\\ & = \frac {\alpha }{(\alpha +\beta )}\cdot \frac {(1+\alpha )}{(\alpha +\beta +1)} \end {align*}
\begin {equation} \frac {1}{n}\sum X_j= \overline {X}=\frac {\alpha }{\alpha +\beta } \end {equation} \begin {equation} \frac {1}{n}\sum X^2_j=\frac {\alpha (1+\alpha )}{(\alpha +\beta )(\alpha +\beta +1)} \end {equation} From equation 2.1 \[\alpha \overline {X}+\beta \overline {X}=\alpha \quad \implies \quad \beta = \frac {\alpha \left (1-\overline {X}\right )}{\overline {X}}.\] Then equation 2.2 becomes \[\frac {1}{n}\sum X^2_j = \frac {\overline {X}(\alpha +1)}{\alpha +\alpha \frac {(1-\overline {X}}{\overline {X}}+1} = \frac {\overline {X}^2(\alpha +1)}{\alpha \overline {X}+\alpha (1-\overline {X})+\overline {X}} = \frac {\overline {X}^2\alpha +\overline {X}^2}{\alpha +\overline {X}}.\] i.e \[\frac {1}{n}\sum X^2_j =\frac {\overline {X}^2\alpha +\overline {X}^2}{\alpha +\overline {X}}.\] Thus \[\frac {\alpha }{n}\sum X^2_j+\frac {\overline {X}}{n}\sum X^2_j =\overline {X}^2\alpha +\overline {X}^2\] \[\frac {\alpha }{n}\sum X^2_j-\overline {X}^2\alpha = \overline {X}^2-\frac {\overline {X}}{n}\sum X^2_j\] \[\alpha \left (\frac {1}{n}\sum X^2_j-\overline {X}^2\right ) = \overline {X}^2-\frac {\overline {X}}{n}\sum X^2_j\] \[\widehat {\alpha } = \frac {\overline {X}^2-\frac {\overline {X}}{n}\sum X^2_j}{\frac {1}{n}\sum X^2_j-\overline {X}^2}\] Taking \(S^{2}=\frac {1}{n}\sum X^2_j-\overline {X}^2\), we have that \[\widehat {\alpha }=\frac {\overline {X}^2-\frac {\overline {X}}{n}\sum X^2_j}{S^{2}}.\] Then \[\beta = \frac {\alpha (1-\overline {X})}{\overline {X}} = \frac {\overline {X}^2-\frac {\overline {X}}{n}\sum X^2_j}{\frac {1}{n}\sum X^2_j-\overline {X}^2}\cdot \frac {(1-\overline {X})}{\overline {X}} = \frac {\left (\overline {X}-\frac {1}{n}\sum X^2_j\right )(1-\overline {X})}{\frac {1}{n}\sum X^2_j-\overline {X}^2}.\] i.e \[\widehat {\beta }_{MME}=\frac {\left (\frac {1}{n}\sum X^2_j-\overline {X}\right )(\overline {X}-1)}{\frac {1}{n}\sum X^2_j-\overline {X}^2}.\] □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.