15.1 The method of moments
Definition 15.1.1. Let \(w_i\) be the data for observation \(i\) and \(\theta \) a \(k \times 1\) parameter with true value \(\theta _0\). A vector of \(\ell \) moment conditions is a function \(g(w, \theta )\) with \[ E\,g(w_i, \theta _0) = 0 . \] Its sample analogue is \(\bar g_n(\theta ) = \frac 1n\sum _{i=1}^ng(w_i, \theta )\). If \(\ell = k\), the method of moments (MM) estimator \(\hat \theta \) solves \(\bar g_n(\hat \theta ) = 0\).
- (a)
- Mean and variance. \(g = (y - \mu ,\ (y - \mu )^2 - \sigma ^2)'\) gives \(\hat \mu = \bar y\) and \(\hat \sigma ^2 = \frac 1n\sum (y_i - \bar y)^2\).
- (b)
- Least squares. In \(y = x'\beta + u\) with \(E(xu) = 0\), \(g = x(y - x'\beta )\), and \(\bar g_n(\beta ) = \frac 1nX'(Y - X\beta ) = 0\) are the normal equations.
- (c)
- Instrumental variables. With \(E(zu) = 0\) and \(\ell = k\), \(g = z(y - x'\beta )\) gives \(Z'(Y - X\beta ) = 0\), whose solution is \(b_{\text {IV}}\).
- (d)
- Maximum likelihood. The score has expectation zero at the true value (Theorem 9.1.1), so \(g = \partial \log f(w, \theta )/\partial \theta \) gives the likelihood equations, and the MLE is a method of moments estimator.
- (e)
- The gamma distribution. With shape \(\kappa \) and rate \(\lambda \), \(E(y) = \kappa /\lambda \) and \(\Var (y) = \kappa /\lambda ^2\). Matching these to \(\bar y\) and \(s^2\) gives \(\hat \kappa = \bar y^2/s^2\) and \(\hat \lambda = \bar y/s^2\).
Theorem 15.1.3. Suppose \(\ell = k\), the observations are independent and identically distributed, \(\hat \theta \xrightarrow {p} \theta _0\), \(g\) is continuously differentiable, and \[ G = E\frac {\partial g(w_i, \theta _0)}{\partial \theta '}, \qquad S = E\bigl [g(w_i, \theta _0)g(w_i, \theta _0)'\bigr ], \] with \(G\) non-singular. Then \[ \sqrt n(\hat \theta - \theta _0) \xrightarrow {d} N\bigl (0,\ G^{-1}S(G')^{-1}\bigr ) . \]
Proof. By the mean value theorem, applied row by row, \(0 = \bar g_n(\hat \theta ) = \bar g_n(\theta _0) + \bar G_n(\tilde \theta )(\hat \theta - \theta _0)\), where \(\bar G_n\) is the derivative of \(\bar g_n\) and \(\tilde \theta \) lies between \(\hat \theta \) and \(\theta _0\). Since \(\tilde \theta \xrightarrow {p} \theta _0\), the law of large numbers gives \(\bar G_n(\tilde \theta ) \xrightarrow {p} G\). Hence \[ \sqrt n(\hat \theta - \theta _0) = -\bar G_n(\tilde \theta )^{-1}\sqrt n\,\bar g_n(\theta _0), \] and \(\sqrt n\,\bar g_n(\theta _0) \xrightarrow {d} N(0, S)\) by the central limit theorem, since the \(g(w_i, \theta _0)\) are independent with mean zero and variance \(S\). Slutsky’s theorem completes the proof. □
The variance has the sandwich form of Section 10. For least squares, \(G = -E(xx')\) and \(S = E(u^2xx')\), and the theorem gives exactly White’s heteroscedasticity-robust variance of Theorem 10.2.1. For maximum likelihood, \(S\) is the information matrix and \(G = -S\) by the information equality, and the variance reduces to the inverse information of Theorem 9.1.3. The method of moments therefore contains both results.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.