13.2 The Kalman Filter
Result 13.2.1. Write \(\widehat {\textbf {x}}_{t\mid s}\) for the expectation of \(\textbf {x}_t\) given observations up to time \(s\), and \(P_{t\mid s}\) for its error covariance. The Kalman filter alternates two steps.
Predict
\[\widehat {\textbf {x}}_{t\mid t-1} = F\,\widehat {\textbf {x}}_{t-1\mid t-1}, \qquad P_{t\mid t-1} = F\,P_{t-1\mid t-1}F^{t} + Q .\]
Update
With the innovation \(e_t = \textbf {y}_t - H\widehat {\textbf {x}}_{t\mid t-1}\) and the Kalman gain \[K_t = P_{t\mid t-1}H^{t}\left (H P_{t\mid t-1}H^{t} + R\right )^{-1},\] \[\widehat {\textbf {x}}_{t\mid t} = \widehat {\textbf {x}}_{t\mid t-1} + K_t e_t, \qquad P_{t\mid t} = \left (I - K_tH\right )P_{t\mid t-1} .\]
Remark. Read the update as a weighted compromise. The innovation \(e_t\) is the part of the new observation the model did not anticipate, and the gain \(K_t\) decides how much of it to believe. If the observation noise \(R\) is large the gain is small and the filter trusts its own prediction; if the state noise \(Q\) is large the prediction is uncertain, the gain is near one, and the filter follows the data. Everything the filter does is that trade, applied recursively.
The recursion is what makes it usable. Each step needs only the previous estimate and the current observation, not the whole history, so the cost per observation is constant and the filter runs in real time. That is why it is used in navigation and tracking as much as in statistics.
Note. The filter is optimal in two distinct senses that are worth keeping apart. If the noise is Gaussian, \(\widehat {\textbf {x}}_{t\mid t}\) is the conditional expectation and therefore the best estimator of any kind. Without normality it is still the best linear estimator — the same distinction as between conditional expectation and linear regression.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.