3.3 Unconditional Distributions
\[P_{ij}^n\left (X_n = j\, \big /\, X_0 = i\right )\]
Let \(P_{(0)}\) be the initial distribution of Markov chain \(\, \{X_n\, , \, n = 0\, , \, 1\, , \, 2\, , \, \cdots \cdots \cdots \}\) i.e we take \(\, P_{(0)} = \left (\alpha _0\, , \, \alpha _1\, , \, \cdots \cdots \right )\,\) where \(\, \alpha _i = P\left (X_0 = i\right )\hspace {0.3cm} \forall _i = 0\, , \, 1\, , \, 2\, , \, \cdots \cdots \)
In general, we denote \(P_{(n)}\) as the \(n^{\text {th}}\) step unconditional distribution \begin {align*} P\left (X_1 = j\right )\, & = \sum ^{\infty }_{i = 0} P\left (X_1 = j\, \big /\, X_0 = i\right )\, P\left (X_0 = i\right )\\ & = \, \sum ^{\infty }_{i = 0} P_{ij}\, \alpha _i\\ & = \, \sum ^{\infty }_{i = 0} \alpha _i\, P_{ij}\\ & = \, \text {this is the } \, (j)^{\text {th}}\, \text {element of }\, P_{(0)}P. \end {align*}
Hence \(\hspace {0.2cm} P_{(1)} = P_{(0)}P\)
\[P_{(2)} = P_{(1)}P = P_{(0)}PP = P_{(0)}P^2.\]
In general \(\hspace {0.2cm} P_{(n)}\, = \, P_0\, P^n\hspace {0.2cm} \forall \, n = 1\, , \, 2\, , \, \cdots \)
Example 3.3.1. Suppose that coin 1 has probability 0.7 of coming up heads and coin 2 has
probability 0.6 of coming up heads.
If the coin flipped today comes up heads then we select coin 1 to flip tomorrow and if it
comes up tails, then we select coin 2 to flip tomorrow.
If the coin initially flipped is equally likely to be coin 1 or coin 2 then what is the probability
that the coin flipped on the third day after the initial flip is coin 1?
Solution. \(X_n \, = \, \) the coin flipped on the \(n^{\text {th}}\) day
\(S = \{\)coin 1 \(\, , \, \) coin 2\(\}\) \[P = \bordermatrix { & C_1 & C_2\cr C_1 & 0.7 & 0.3 \cr C_2 & 0.6 & 0.4\cr }\] □
Theorem 3.3.2 (Path Analysis). \begin {align*} P\left (X_{n + 1} = j, \, X_n = i\, |\, X_{n - 1} = k\right ) & = P\left (X_{n + 1} = j/X_n = i,\, X_{n -1} = k\right )\, P\left (X_n = i/X_{n - 1} = k\right )\\ & = P\left (X_{n + 1} = j/X_n = i\right )\, P\left (X_n = i/X_{n - 1} = k\right )\\ & = P_{ij}\, P_{ki}\\ & = P_{ki}\, P_{ij}. \end {align*}
We note that \(P^{(2)}_{kj}\) is the sum of probabilities of all routes from state \(k\) to state \(j\) in two steps.
Example 3.3.3. Consider a Markov chain with the following transition matrix \[\bordermatrix {~ & A & B & C & D\cr A & .1 & .3 & .2 & .4\cr B & .2 & .3 & .2 & .3\cr C & .3 & .3 & .1 & .3\cr D & .2 & .1 & .4 & .3\cr }\] Find \(\, P\left (X_5 = A, \, X_4 =B, \, X_3 =C, \, X_2 = B, \, X_1 = A/ X_0 = B\right )\)
Solution. \[ = P_{BA}\, P_{AB}\, P_{BC}\, P_{CB}\, P_{BA} = (.2)(.3)(.2)(.3)(.2) = 0.00072.\] □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.