3.3 Unconditional Distributions
Let \(p^{(0)} = (\alpha _0, \alpha _1, \dots )\), with \(\alpha _i = P(X_0 = i)\), be the initial distribution, written as a row vector, and \(p^{(n)}\) the distribution of \(X_n\). Conditioning on \(X_0\), \[P(X_n = j) = \sum _iP(X_n = j\mid X_0 = i)P(X_0 = i) = \sum _i\alpha _iP_{ij}^{(n)},\] which is the \(j\)th entry of the row vector \(p^{(0)}P^n\). So \[p^{(n)} = p^{(0)}P^n,\qquad \text {and equivalently}\qquad p^{(n + 1)} = p^{(n)}P .\] The initial distribution and the transition matrix together determine the whole process: by repeated conditioning, \[P(X_0 = i_0, X_1 = i_1, \dots , X_n = i_n) = \alpha _{i_0}P_{i_0i_1}P_{i_1i_2}\cdots P_{i_{n-1}i_n} .\]
Example 3.3.1. Coin \(1\) shows heads with probability \(0.7\) and coin \(2\) with probability \(0.6\). If the coin flipped today shows heads, coin \(1\) is flipped tomorrow; if tails, coin \(2\). The coin flipped initially is equally likely to be either. Find the probability that the coin flipped on the third day after the initial flip is coin \(1\).
Solution. Let \(X_n\) be the coin flipped on day \(n\). Tomorrow’s coin is coin \(1\) exactly when today’s flip shows heads, so \[P = \bordermatrix {~ & 1 & 2\cr 1 & 0.7 & 0.3\cr 2 & 0.6 & 0.4\cr },\qquad P^2 = \begin {pmatrix}0.67 & 0.33\\ 0.66 & 0.34\end {pmatrix},\qquad P^3 = \begin {pmatrix}0.667 & 0.333\\ 0.666 & 0.334\end {pmatrix}.\] With \(p^{(0)} = (0.5, 0.5)\), \[P(X_3 = 1) = 0.5(0.667) + 0.5(0.666) = 0.6665 .\]
Example 3.3.2 (A path probability). A chain on states \(A, B, C, D\) has transition matrix \[P = \bordermatrix {~ & A & B & C & D\cr A & 0.1 & 0.3 & 0.2 & 0.4\cr B & 0.2 & 0.3 & 0.2 & 0.3\cr C & 0.3 & 0.3 & 0.1 & 0.3\cr D & 0.2 & 0.1 & 0.4 & 0.3\cr } .\] Find \(P(X_1 = A, X_2 = B, X_3 = C, X_4 = B, X_5 = A\mid X_0 = B)\) and \(P_{BA}^{(2)}\).
Solution. By the product formula above, the path \(B\to A\to B\to C\to B\to A\) has probability \[P_{BA}P_{AB}P_{BC}P_{CB}P_{BA} = (0.2)(0.3)(0.2)(0.3)(0.2) = 0.00072 .\] A two-step probability is the sum over all intermediate states, that is, over all routes: \[P_{BA}^{(2)} = \sum _kP_{Bk}P_{kA} = (0.2)(0.1) + (0.3)(0.2) + (0.2)(0.3) + (0.3)(0.2) = 0.20 .\]
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.