2.2 Markovian Stochastic Process

Consider a stochastic process \(\, \{X(t)\, , \, t\in T\}\,\) which has a countable state space.
The index set \(T\) can be discrete or continuous. Hence we have the following two cases

1.

\(T\) is continuous.
Let \(\, t_0 < t_1 < \, \cdots \cdots \, < t_n < t\, \) where \(\, t_0\, , \, t_1\, \cdots \cdots \, t_n \in T\). Then the process \(\{X(t)\}\) is Markovian if \[P\left (X(t) = x\, |\, X(t_n) = x_n \, , \, x(t_{n - 1}) = x_{n - 1}\, , \cdots \cdots \, \, X(t_0) = x_0\right ) = P\left (X(t) = x\, |\, X(t_n) = x_n\right )\] for all states \(x\).

2.

\(T\) is discrete.
Then the process \(\, \{X_n\, , \, n = 0\, , \, 1\, , \, 2\, , \, \cdots \cdots \}\) is Markovian if \[P\left (X_{n + 1} = j\, |\, X_n = i_0\, ,\, X_{n - 1} = i_1\, , \, \cdots \cdots \,,\, X_0 = i_n\right ) = P\left (X_{n + 1} = j\, |\, X_n = i_0\right )\] for all states \(j\).

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.