Chapter 3
Markov Chains
3.1 Introduction
3.2 The Chapman–Kolmogorov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.4.1 Recurrence and transience
3.5 Random Walk Models
3.5.1 The one-dimensional random walk
3.5.2 Symmetric random walks in two and three dimensions
3.6 Limiting Probabilities
3.7 Expected Numbers of Visits
3.7.1 Visits to transient states
3.8 Branching Processes
3.2 The Chapman–Kolmogorov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.4.1 Recurrence and transience
3.5 Random Walk Models
3.5.1 The one-dimensional random walk
3.5.2 Symmetric random walks in two and three dimensions
3.6 Limiting Probabilities
3.7 Expected Numbers of Visits
3.7.1 Visits to transient states
3.8 Branching Processes