12.1 Estimating the mean
The natural estimate of the mean \(\mu \) of a stationary process is the sample mean \(\bar x\). It is unbiased, but its variance is not the familiar \(\sigma ^2/n\), because the observations are correlated.
Theorem 12.1.1. For a stationary process with variance \(\sigma ^2\) and ac.f. \(\rho (k)\), \[ \Var (\bar X) = \frac {\sigma ^2}{n}\Bigl [1 + 2\sum _{k=1}^{n-1}\Bigl (1 - \frac kn\Bigr )\rho (k)\Bigr ] . \] For an AR(1) process with parameter \(\alpha \) and large \(n\), this is approximately \(\dfrac {\sigma ^2}{n}\cdot \dfrac {1 + \alpha }{1 - \alpha }\).
Proof. \(\Var (\bar X) = \frac 1{n^2}\sum _{s=1}^n\sum _{t=1}^n\Cov (X_s, X_t) = \frac {\sigma ^2}{n^2}\sum _s\sum _t\rho (t - s)\). The \(n\) diagonal terms give \(n\), and for each lag \(k \ge 1\) there are \(n - k\) pairs with \(t - s = k\) and another \(n - k\) with \(s - t = k\), so the double sum is \(n + 2\sum _{k=1}^{n-1}(n - k)\rho (k)\); dividing by \(n^2\) gives the formula. For an AR(1), \(\rho (k) = \alpha ^k\), and for large \(n\) the factors \(1 - k/n\) are close to \(1\) wherever \(\alpha ^k\) is not negligible, so the bracket is close to \(1 + 2\sum _{k\ge 1}\alpha ^k = 1 + 2\alpha /(1 - \alpha ) = (1 + \alpha )/(1 - \alpha )\). □
With positive autocorrelation, the sample mean is much less precise than it would be with independent observations. For an AR(1) with \(\alpha = 0.6\) the factor is \(1.6/0.4 = 4\) (with \(n = 100\) the exact bracket is \(3.92\)): \(100\) correlated observations give the precision of only about \(25\) independent ones, an effective sample size of \(25\). A confidence interval for \(\mu \) computed as if the data were independent would be half as wide as it should be.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.