4.6 Maximum likelihood for the autoregressive equation
Suppose now that the \(u_t\) are independent \(N(0, \sigma ^2)\) and the observations are \(Y_1, \dots , Y_T\). The joint density factors as \(f(Y_1)\prod _{t=2}^T f(Y_t \mid Y_{t-1})\), and given \(Y_{t-1}\), \(Y_t \sim N(\alpha + \beta Y_{t-1}, \sigma ^2)\).
- (i)
- The conditional log-likelihood, which treats \(Y_1\) as fixed, is \[ \ell _c(\alpha , \beta , \sigma ^2) = -\frac {T-1}{2}\log (2\pi \sigma ^2) - \frac {1}{2\sigma ^2}\sum _{t=2}^T (Y_t - \alpha - \beta Y_{t-1})^2 , \] and it is maximised by the least squares estimates of \(\alpha \) and \(\beta \), with \(\hat \sigma ^2 = \text {RSS}/(T - 1)\).
- (ii)
- If \(|\beta | < 1\) and the process is stationary, \(Y_1 \sim N\bigl (\alpha /(1 - \beta ),\ \sigma ^2/(1 - \beta ^2)\bigr )\), and the exact log-likelihood is \[ \ell (\alpha , \beta , \sigma ^2) = \ell _c - \frac 12\log \frac {2\pi \sigma ^2}{1 - \beta ^2} - \frac {1 - \beta ^2}{2\sigma ^2}\left (Y_1 - \frac {\alpha }{1 - \beta }\right )^2 . \] Its maximiser is not least squares and must be found numerically.
Proof. (i) The conditional density of each \(Y_t\) is normal with mean \(\alpha + \beta Y_{t-1}\) and variance \(\sigma ^2\); the log of the product of \(T - 1\) such densities is \(\ell _c\). For any \(\sigma ^2\), \(\ell _c\) is maximised over \((\alpha , \beta )\) by minimising the sum of squares, which is least squares. Setting \(\partial \ell _c/\partial \sigma ^2 = -\frac {T-1}{2\sigma ^2} + \frac {\text {RSS}}{2\sigma ^4} = 0\) gives \(\hat \sigma ^2 = \text {RSS}/(T - 1)\).
(ii) Add \(\log f(Y_1)\) for the stated normal distribution, which is Theorem 4.5.2 with normal shocks. The extra terms involve \(\beta \) non-linearly through \(1 - \beta ^2\), so the first-order conditions are no longer linear. □
The two estimators differ only through one observation’s contribution out of \(T\), so they have the same large-sample distribution. In small samples the exact likelihood uses the information in \(Y_1\), and it automatically keeps \(|\hat \beta | < 1\), since the term \(\log (1 - \beta ^2)\) tends to \(-\infty \) as \(|\beta | \to 1\). The general theory of maximum likelihood, which shows that such estimators are consistent, asymptotically normal and efficient, is developed in its own section.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.