Solution. By Chapman–Kolmogorov, \(P^{(2)} = P^{2}\): \[P^{2} = \begin {pmatrix} 0.7(0.7)+0.3(0.4) & 0.7(0.3)+0.3(0.6)\\ 0.4(0.7)+0.6(0.4) & 0.4(0.3)+0.6(0.6)\end {pmatrix} = \begin {pmatrix} 0.61 & 0.39\\ 0.52 & 0.48\end {pmatrix},\] and each row still sums to one, which is the check.

For the stationary distribution solve \(\pi P = \pi \) with \(\pi _1+\pi _2=1\): \[0.7\pi _1 + 0.4\pi _2 = \pi _1 \implies 0.4\pi _2 = 0.3\pi _1 \implies \pi _2 = \tfrac 34\pi _1 .\] With \(\pi _1+\pi _2=1\) this gives \[\pi = \left (\tfrac 47,\ \tfrac 37\right ) \approx (0.571,\ 0.429).\] The rows of \(P^{2}\) are already closer to \(\pi \) than those of \(P\), which is the convergence the chain is heading towards.

Solution. Return to the origin is possible only after an even number of steps, and it requires equal numbers of left and right moves: \[p_{00}^{(2n)} = \binom {2n}{n}\left (\tfrac 12\right )^{2n}, \qquad p_{00}^{(2n+1)} = 0 .\] By Stirling’s approximation \(\binom {2n}{n} \sim 4^{n}/\sqrt {\pi n}\), so \[p_{00}^{(2n)} \sim \frac {1}{\sqrt {\pi n}} .\] Hence \(\sum _n p_{00}^{(n)}\) behaves like \(\sum _n n^{-1/2}\), which diverges. The origin is recurrent, and since the walk is irreducible every state is.

The exponent is what decides it. In three dimensions the corresponding probability decays like \(n^{-3/2}\), whose sum converges, so the walk is transient — the walk returns in one and two dimensions and escapes in three.

Solution. The offspring generating function is \[G(s) = \tfrac 14 + \tfrac 14 s + \tfrac 12 s^{2},\] and the mean offspring number is \(G'(1) = \tfrac 14 + 1 = \tfrac 54 > 1\), so extinction is not certain and the extinction probability is the smallest root of \(G(s)=s\) in \([0,1]\). \[\tfrac 12 s^{2} - \tfrac 34 s + \tfrac 14 = 0 \quad \Longrightarrow \quad 2s^{2} - 3s + 1 = 0 \quad \Longrightarrow \quad (2s-1)(s-1) = 0 .\] The roots are \(s=\tfrac 12\) and \(s=1\), so the extinction probability is \[q = \tfrac 12 .\]

The two roots are the two possibilities, and which one is taken is decided by the mean: with \(\mu \leq 1\) the only root in \([0,1)\) disappears and extinction is certain, while with \(\mu >1\) the smaller root is the answer. Here a population growing on average by a quarter each generation still dies out half the time.

Solution. The number in an interval of length \(t\) is POI\((\lambda t)\), so with \(t=1\), \[P(N_1 = 2) = \frac {e^{-3}3^{2}}{2!} = \frac {9e^{-3}}{2} \approx 0.224 .\] The waiting time to the first arrival is exponential with rate \(\lambda \), so \[E(T_1) = \frac {1}{\lambda } = \frac 13 \text { hour} = 20 \text { minutes}.\]

Worth noting: the expected time to the next arrival is \(20\) minutes however long one has already waited, by the memorylessness of the exponential. That is the property which makes the Poisson process the only counting process with independent stationary increments.