11.1 Autocorrelated disturbances
Definition 11.1.1. The disturbances of \(Y_t = x_t'\beta + u_t\), \(t = 1, \dots , n\), follow a first-order autoregressive, or AR(1), scheme if \[ u_t = \rho u_{t-1} + \varepsilon _t, \qquad |\rho | < 1, \] where the \(\varepsilon _t\) are independent of each other and of the regressors, with mean zero and variance \(\sigma _\varepsilon ^2\). More generally \(u_t = \rho _1u_{t-1} + \dots + \rho _pu_{t-p} + \varepsilon _t\) is an AR(\(p\)) scheme, and \(u_t = \varepsilon _t + \theta \varepsilon _{t-1}\) is a first-order moving average, MA(1), scheme.
The AR(1) scheme is the AR equation of Section 4.5 with \(\alpha = 0\), \(\beta = \rho \) and \(Y\) replaced by \(u\). By Theorem 4.5.2 it is stationary, with \[ \Var (u_t) = \sigma _u^2 = \frac {\sigma _\varepsilon ^2}{1 - \rho ^2}, \qquad \Cov (u_t, u_{t-s}) = \rho ^s\sigma _u^2, \qquad \Corr (u_t, u_{t-s}) = \rho ^s . \] The whole disturbance vector therefore has \(\Var (U) = \sigma _\varepsilon ^2\Omega \), with \begin {equation} \label {eq:omegaar1} \Omega = \frac {1}{1 - \rho ^2} \begin {bmatrix} 1 & \rho & \rho ^2 & \cdots & \rho ^{n-1}\\ \rho & 1 & \rho & \cdots & \rho ^{n-2}\\ \vdots & & \ddots & & \vdots \\ \rho ^{n-1} & \rho ^{n-2} & \rho ^{n-3} & \cdots & 1 \end {bmatrix} . \end {equation} Positive \(\rho \) is by far the common case in economics. The residuals then come in runs: a positive residual tends to be followed by another positive one, and the residual plot looks smoother than white noise. Negative \(\rho \) makes the residuals alternate in sign; it can arise from over-differencing or from a misplaced lag.
Theorem 11.1.2. Let \(P\) be the \(n \times n\) matrix \[ P = \begin {bmatrix} \sqrt {1 - \rho ^2} & 0 & 0 & \cdots & 0 & 0\\ -\rho & 1 & 0 & \cdots & 0 & 0\\ 0 & -\rho & 1 & \cdots & 0 & 0\\ \vdots & & & \ddots & & \vdots \\ 0 & 0 & 0 & \cdots & -\rho & 1 \end {bmatrix} . \] Then \(P\Omega P' = I\), so that \(\Omega ^{-1} = P'P\), which is the tridiagonal matrix with diagonal \((1, 1 + \rho ^2, \dots , 1 + \rho ^2, 1)\) and \(-\rho \) on the two neighbouring diagonals. Also \(|\Omega | = 1/(1 - \rho ^2)\).
Proof. \(PU\) has first element \(\sqrt {1 - \rho ^2}\,u_1\) and \(t\)th element \(u_t - \rho u_{t-1} = \varepsilon _t\) for \(t \geq 2\). The first element has variance \((1 - \rho ^2)\sigma _\varepsilon ^2/(1 - \rho ^2) = \sigma _\varepsilon ^2\). The \(\varepsilon _t\) with \(t \geq 2\) have variance \(\sigma _\varepsilon ^2\), are uncorrelated with each other, and are uncorrelated with \(u_1\), which depends only on \(\varepsilon _1, \varepsilon _0, \varepsilon _{-1}, \dots \). Hence \(\Var (PU) = \sigma _\varepsilon ^2I\). But \(\Var (PU) = P(\sigma _\varepsilon ^2\Omega )P'\), so \(P\Omega P' = I\). Since \(P\) is lower triangular with non-zero diagonal it is non-singular, and \(\Omega = P^{-1}(P')^{-1}\), so \(\Omega ^{-1} = P'P\).
Multiplying out: column \(1\) of \(P\) has entries \(\sqrt {1 - \rho ^2}\) and \(-\rho \), so \((P'P)_{11} = 1 - \rho ^2 + \rho ^2 = 1\); for \(2 \leq t \leq n - 1\) column \(t\) has entries \(1\) and \(-\rho \), so \((P'P)_{tt} = 1 + \rho ^2\); the last column has the single entry \(1\). Columns \(t\) and \(t + 1\) overlap only in row \(t + 1\), where the entries are \(-\rho \) and \(1\), giving \((P'P)_{t,t+1} = -\rho \); columns further apart do not overlap. Finally \(|P| = \sqrt {1 - \rho ^2}\), the product of the diagonal, and \(|\Omega | = 1/|P|^2\). □
The inverse of the full matrix (11.1) is sparse: only neighbouring observations are linked, which reflects the fact that \(u_t\) depends on the past only through \(u_{t-1}\). The matrix \(P\) is the \(P\) of Aitken’s Theorem 9.4.2 for this \(\Omega \), and it will give generalised least squares by quasi-differencing.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.