Chapter 3
Markov Chains
A discrete time homogeneous Markov process is called a Markov chain.
3.1 Introduction
3.2 Chapman Kolmogrov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.5 Random Walk Models
3.5.1 One Dimensional Random Walk
3.5.2 Two Dimensional Symmetrical Random Walk
3.5.3 Three Dimensional Symmetric Random Walk
3.6 Limiting probabilities
3.7 Expected Number of Visits
3.8 Expected Number of Visists to a Transient State
3.9 Branching Process
3.2 Chapman Kolmogrov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.5 Random Walk Models
3.5.1 One Dimensional Random Walk
3.5.2 Two Dimensional Symmetrical Random Walk
3.5.3 Three Dimensional Symmetric Random Walk
3.6 Limiting probabilities
3.7 Expected Number of Visits
3.8 Expected Number of Visists to a Transient State
3.9 Branching Process