11.1 The partial autocorrelation function
The autocorrelation function alone cannot identify an autoregressive model. For an AR(1) process the correlation between \(X_t\) and \(X_{t+2}\) is \(\rho (2) = \alpha ^2\), which is not zero, but none of it is direct: it is entirely inherited through \(X_{t+1}\). The partial autocorrelation is what removes that inheritance.
Definition 11.1.1. The partial autocorrelation at lag \(k\), written \(\phi _{kk}\), is the correlation between \(X_t\) and \(X_{t+k}\) after removing the linear effect of the intervening values \(X_{t+1}, \dots , X_{t+k-1}\): \[ \phi _{kk} = \Corr \left (X_t, \ X_{t+k} \;\middle |\; X_{t+1}, \dots , X_{t+k-1}\right ) . \] Equivalently, \(\phi _{kk}\) is the coefficient of \(X_{t-k}\) in the best linear predictor of \(X_t\) from the previous \(k\) values, \[ \hat X_t = \phi _{k1}X_{t-1} + \phi _{k2}X_{t-2} + \dots + \phi _{kk}X_{t-k} . \]
“Best” means minimising the mean squared error \(E(X_t - \hat X_t)^2\), for a stationary process with mean \(0\). The coefficient of \(X_{t-k}\) measures what \(X_{t-k}\) adds to the prediction when the values \(X_{t-1}, \dots , X_{t-k+1}\) are already used, which is the direct link the definition asks for.
Theorem 11.1.2 (Yule–Walker form of the partial autocorrelations). The coefficients of the best linear predictor from \(k\) past values solve \[ \rho (j) = \sum _{i=1}^{k}\phi _{ki}\,\rho (j - i) , \qquad j = 1, \dots , k , \] and \(\phi _{kk}\) is the last coefficient of that system. In matrix form, with \(\mathbf {R}_k\) the \(k \times k\) matrix of \(\rho (i - j)\), \[ \mathbf {R}_k\boldsymbol \phi _k = \boldsymbol \rho _k , \qquad \boldsymbol \phi _k = (\phi _{k1}, \dots , \phi _{kk})^\top , \quad \boldsymbol \rho _k = (\rho (1), \dots , \rho (k))^\top . \]
Proof. The mean squared error \(Q = E\bigl (X_t - \sum _i\phi _{ki}X_{t-i}\bigr )^2\) is a quadratic in the \(\phi _{ki}\). Setting \(\partial Q/\partial \phi _{kj} = 0\), \[ -2E\Bigl [\Bigl (X_t - \sum _i\phi _{ki}X_{t-i}\Bigr )X_{t-j}\Bigr ] = 0 , \] that is, \[ \gamma (j) = \sum _i\phi _{ki}\gamma (j - i) , \qquad j = 1, \dots , k : \] the prediction error is uncorrelated with each predictor, exactly like the normal equations of least squares. Dividing by \(\gamma (0)\) gives the system. The matrix \(\mathbf {R}_k\) is non-negative definite (Theorem 3.3.1), and positive definite for any process that is not perfectly predictable, so the solution is unique and minimises \(Q\). □
Solution. At \(k = 1\) the system is \(\rho (1) = \phi _{11}\), so \[ \phi _{11} = \rho (1) . \] At \(k = 2\), \[ \rho (1) = \phi _{21} + \phi _{22}\rho (1) , \qquad \rho (2) = \phi _{21}\rho (1) + \phi _{22} . \] Eliminating \(\phi _{21}\) (multiply the first by \(\rho (1)\) and subtract it from the second), \[ \phi _{22} = \frac {\rho (2) - \rho (1)^2}{1 - \rho (1)^2} . \] For an AR(1) process \(\rho (k) = \alpha ^k\), so \[ \phi _{22} = \frac {\alpha ^2 - \alpha ^2}{1 - \alpha ^2} = 0 , \] exactly as the definition promises: once \(X_{t+1}\) is accounted for, nothing direct connects \(X_t\) to \(X_{t+2}\).
Theorem 11.1.4 (AR and MA partial autocorrelations).
- (i)
- If the process is a stationary AR(\(p\)), then \(\phi _{pp} = \alpha _p\), and \(\phi _{kk} = 0\) for every \(k > p\). That is, the partial autocorrelation function cuts off after lag \(p\).
- (ii)
- For the MA(1) process \(X_t = Z_t + \beta Z_{t-1}\) with \(\beta \ne 0\), \(\phi _{22} = -\rho (1)^2/(1 - \rho (1)^2) \ne 0\); the partial autocorrelations do not cut off.
Proof. (i) Let \(k \ge p\) and try \(\phi _{ki} = \alpha _i\) for \(i \le p\) and \(\phi _{ki} = 0\) for \(p < i \le k\). Then \(\sum _i\phi _{ki}\rho (j - i) = \sum _{i=1}^p\alpha _i\rho (j - i)\), which equals \(\rho (j)\) for every \(j \ge 1\) by the Yule–Walker equations (Theorem 8.2.1). So this choice solves the system of Theorem 11.1.2, and since the solution is unique, it is the solution. Its last coefficient is \(\alpha _p\) when \(k = p\), and \(0\) when \(k > p\).
(ii) By Example 11.1.3 with \(\rho (2) = 0\), \(\phi _{22} = -\rho (1)^2/(1 - \rho (1)^2)\), which is non-zero because \(\rho (1) = \beta /(1 + \beta ^2) \ne 0\). □
Part (i) is intuitive: an AR(\(p\)) process is a regression on its own last \(p\) values with an error uncorrelated with the past, so once those \(p\) values are known, earlier ones add nothing. For the MA(1), more generally, the partial autocorrelations decay geometrically in size and never vanish, because an invertible MA(1) is an AR(\(\infty \)) (Section 7): every past value has some direct effect. With \(\beta = 0.5\), for example, \(\rho (1) = 0.4\) and \[ \phi _{11} = 0.4 , \quad \phi _{22} = -0.190 , \quad \phi _{33} = 0.094 , \quad \phi _{44} = -0.047 , \dots , \] alternating in sign and halving at each step.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.