7.1 General Order Of an AR Process

Let AR be expressed as an MA Process. Consider the equation \[X_t = \alpha _1 X_{t-1} + \alpha _2 X_{t-2} + \cdots \cdots + \alpha _p X_{t-p} + Z_t, \hspace {1cm} AR(p)\]

\[\text {or}\hspace {0.5cm}(1 - \alpha _1 B - \alpha _2 B^2 --------- \alpha _p B^p)X_t = Z_t\]

\begin {align*} \text {or}\hspace {0.5cm} X_t & = \frac {Z_t}{1 - \alpha _1 B - \alpha _2 B^2 --------- \alpha _p B^p}\\ X_t & = f(B)Z_t \end {align*}

where \begin {align*} f(B) &= (1 - \alpha _1 B - \alpha _2 B^2 ----------\alpha _p B^p)^{-1}\\ &= 1 + \beta _1 B + \beta _2 B^2 + \cdots \cdots \end {align*}

the \(\alpha _i\) and \(\beta _i\) can be found.

Since \(X_t\) is in terms of an MA, \(E(X_t)=0\) and the variance is finite provided \(\sum \limits ^{\infty }_{i=0} \beta ^2_i\) converge.

The acv.f is given by \[\varphi (k)= \sigma ^2_Z \,\sum ^{\infty }_{i=0} \beta _i \beta _{i+k}\hspace {0.3cm},\hspace {0.3cm} \text {where}\hspace {0.3cm} \beta _0=1\] the sufficient condition for this to converge , \(\Sigma |\beta _i|\) converge

To find the ac.f, we assume as stationality we multiply \[X_t= \alpha _1 X_{t-1} + \alpha _2 X_{t-2} + \cdots \cdots + \alpha _p X_{t-p}\] by \(X_{t-k}\) and take expectations and divide by \(\sigma ^2_X\). Assuming \(var(X_t)\) is finite. We have \[\rho (k)= \alpha _1 \rho (k-1) + \cdots \cdots + \alpha _p \rho (k-p)\hspace {0.3cm},\hspace {0.3cm} \forall k\geq 0\] This is called Yule Walker equations and this has a general solution \[\rho (k)= A_1\pi ^{|k|}_1 + A_2\pi ^{|k|}_2 + \cdots \cdots + A_p\pi ^{|k|}_p\] and the \(\{\pi _i\}\) are the roots of the auxiliary equation. \[Y^p - \alpha _1 Y^{p-1} - \alpha _2 Y^{p-2} -------- \alpha _p = 0\] Then the \(A_{i's}\) are chosen to satisfy the initial conditions depending on \(\rho (0)=1\), i.e \(\sum A_i=1\).
This first Yule Walker equations provide several routes to estimating the parameters of AR(p) model, by replacing the theoretical covariances with estimated values. Further restrictions on the \(\{A_{i's}\}\) using \(\rho (0)=1\) and \(\rho (-k) = \rho (k)\).The general form of \(\rho (k)\) tends to zero as \(k\rightarrow \infty \) (increases) provided \(|\pi _i|<1\), this is a necessary and sufficient condition for the process to be stationary.

An equivalent way to express stationality of AR(p) is to express the roots as \(\phi (B) = 1 -\alpha _1 B -\alpha _2 B^2 --------- \alpha _p B^p=0\) must lie outside the unit circle.

When the roots are real \(A_1\) and \(A_2\) can be found as follows, since \(\rho (0)=1\)
from \(\rho (k)= A_1\pi ^{|k|}_1 +A_2\pi ^{|k|}_2 +...............+ A_p\pi ^{|k|}_p\)

\begin {align*} \rho (0) &= A_1\pi ^{|0|}_1 + A_2\pi ^{|0|}_2\\ &= A_1 + A_2\\ 1&=A_1 + A_2 \end {align*}

from the Yule Walker equation
\(\rho (k)=\alpha _1 \rho (k-1) +\alpha _2 \rho (k-2) + ..........+ \alpha _p \rho (k-p), \forall k\geq 0\)

we have
\begin {align*} \rho (1) &=\alpha _1 \rho (1-1) + \alpha _2 \rho (1-2)\\ &=\alpha _1 \rho (0) +\alpha _2 \rho (-1)\\ &=\alpha _1 \rho (0) +\alpha _2 \rho (1)\\\\ \implies \rho (1) &=\alpha _1 +\alpha _2 \rho (1)\\\\ \implies \rho (1) &=\frac {\alpha _1}{1-\alpha _2}\\ \end {align*}

\(\rho (1)= A_1\pi _1 + A_2\pi _2\)
But \(1= A_1+A_2\) or \(1-A_1=A_2\) , Therefore \begin {align*} \rho (1) &= A_1\pi _1 +A_2\pi _2\\ &= A_1\pi _1 +(1-A_1)\pi _2\\\\ \implies \frac {\alpha _1}{1-\alpha _2} &=A_1\pi _1 +(1-A_1)\pi _2 \\\\ \implies A_1 &= \frac {\Big [\dfrac {\alpha _1}{1-\alpha _2}-\pi _2\Big ]}{\pi _1 -\pi _2}\\\\ \implies A_2 &= 1-A_1\\ &= \frac {\pi _1 -\pi _2 -\Big [\dfrac {\alpha _1}{1-\alpha _2}-\pi _2\Big ]}{\pi _1 -\pi _2}\\ \end {align*}

for \(k\geq 2\),
\(\rho (k) =\rho (k-1)-\dfrac {1}{2}\rho (k-2)\) and \(\rho (2), \rho (3)\) can be found by successive substitution.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.