8.2 The general order
Write the AR(\(p\)) process as \[ (1 - \alpha _1B - \alpha _2B^2 - \dots - \alpha _pB^p)X_t = Z_t , \qquad \text {or}\qquad \phi (B)X_t = Z_t , \] with \(\phi (B) = 1 - \alpha _1B - \dots - \alpha _pB^p\). As for the AR(1), the process can be expressed as an MA process, \[ X_t = \frac {Z_t}{\phi (B)} = f(B)Z_t , \qquad f(B) = \phi (B)^{-1} = 1 + \psi _1B + \psi _2B^2 + \dots , \] and the weights \(\psi _i\) can be found from the \(\alpha _i\) by equating coefficients of powers of \(B\) in \(\phi (B)f(B) = 1\). Since \(X_t\) is then written as an MA process, \(E(X_t) = 0\), and the variance \(\sigma _Z^2\sum _i\psi _i^2\) is finite provided \(\sum \psi _i^2\) converges. As in Theorem 6.1.2, the autocovariance is \[ \gamma (k) = \sigma _Z^2\sum _{i=0}^\infty \psi _i\psi _{i+k} , \qquad \psi _0 = 1 , \] and a sufficient condition for this to converge is that \(\sum |\psi _i|\) converges.
Theorem 8.2.1 (Yule–Walker equations). For a stationary AR(\(p\)) process, \[ \rho (k) = \alpha _1\rho (k - 1) + \alpha _2\rho (k - 2) + \dots + \alpha _p\rho (k - p) , \qquad k = 1, 2, \dots , \] and the variance is \[ \sigma _X^2 = \frac {\sigma _Z^2}{1 - \alpha _1\rho (1) - \alpha _2\rho (2) - \dots - \alpha _p\rho (p)} . \]
Proof. Multiply \(X_t = \sum _{i=1}^p\alpha _iX_{t-i} + Z_t\) by \(X_{t-k}\) and take expectations. For \(k > 0\), \(E(X_{t-k}Z_t) = 0\), because \(X_{t-k}\) is a combination of shocks up to time \(t - k\) only; so \(\gamma (k) = \sum _i\alpha _i\gamma (k - i)\), and dividing by \(\gamma (0)\) gives the equations. For \(k = 0\), \(E(X_tZ_t) = E(Z_t^2) = \sigma _Z^2\), since \(X_t = Z_t + (\text {terms in earlier shocks})\); so \(\gamma (0) = \sum _i\alpha _i\gamma (i) + \sigma _Z^2\), that is \(\gamma (0)\bigl (1 - \sum _i\alpha _i\rho (i)\bigr ) = \sigma _Z^2\). □
The Yule–Walker equations do two jobs. Taken for \(k = 1, \dots , p\), with \(\rho (-j) = \rho (j)\), they are \(p\) linear equations linking \(\rho (1), \dots , \rho (p)\) to \(\alpha _1, \dots , \alpha _p\): given the \(\alpha \)’s they give the first \(p\) autocorrelations, and given estimated autocorrelations they give estimates of the \(\alpha \)’s, one of several routes to fitting an AR model (Section 12). For \(k > p\) they are a difference equation that generates all the later autocorrelations.
Theorem 8.2.2. Let \(\pi _1, \dots , \pi _p\) be the roots, assumed distinct, of the auxiliary equation \[ y^p - \alpha _1y^{p-1} - \alpha _2y^{p-2} - \dots - \alpha _p = 0 . \] Then the general solution of the Yule–Walker difference equation is \[ \rho (k) = A_1\pi _1^{|k|} + A_2\pi _2^{|k|} + \dots + A_p\pi _p^{|k|} , \] with constants \(A_i\) fixed by the initial conditions, in particular \(\rho (0) = 1\), so \(\sum A_i = 1\). Moreover, \(\pi \) is a root of the auxiliary equation exactly when \(1/\pi \) is a root of \(\phi (B) = 0\).
Proof. Try \(\rho (k) = \pi ^k\) in \(\rho (k) = \sum _i\alpha _i\rho (k - i)\): dividing by \(\pi ^{k-p}\) gives \(\pi ^p = \alpha _1\pi ^{p-1} + \dots + \alpha _p\), the auxiliary equation. So each \(\pi _i^k\) is a solution, and since the equation is linear, so is any combination \(\sum A_i\pi _i^k\). With distinct roots, these \(p\) solutions are independent, and any solution is determined by \(p\) starting values, so every solution has this form. For the last statement, \(\phi (1/\pi ) = 1 - \alpha _1\pi ^{-1} - \dots - \alpha _p\pi ^{-p} = \pi ^{-p}(\pi ^p - \alpha _1\pi ^{p-1} - \dots - \alpha _p)\), which is zero exactly when \(\pi \) solves the auxiliary equation. □
The general form of \(\rho (k)\) tends to zero as \(k\) increases provided every \(|\pi _i| < 1\), and this is the necessary and sufficient condition for the process to be stationary. By the last part of the theorem, an equivalent way to express it is:
Result 8.2.3 (Stationarity of AR(\(p\))). The AR(\(p\)) process is stationary if and only if all the roots of \(\phi (B) = 1 - \alpha _1B - \dots - \alpha _pB^p = 0\) lie outside the unit circle.
This is the same root condition as for the invertibility of an MA process (Theorem 7.2.1), with \(\phi \) in place of \(\theta \): the duality again. An AR(\(p\)) process is always invertible, since \(Z_t = \phi (B)X_t\) is already a finite sum of observations.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.