Stochastic Processes

Lecture Notes

_________________________________________

\[p_{ij}^{(m+n)} = \sum _{k} p_{ik}^{(m)}\,p_{kj}^{(n)}\]

TABLE OF CONTENTS
1 Warming Up
1.1 Expectations of Sums
1.2 Conditional Expectation
1.3 Computing Expectations by Conditioning
1.4 Computing Probabilities by Conditioning
1.5 Markov’s Inequality
1.6 Computing Variances by Conditioning
2 Stochastic Processes
2.1 Definitions
2.2 Markov Processes
2.3 Homogeneous Markov Processes
3 Markov Chains
3.1 Introduction
3.2 The Chapman–Kolmogorov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.5 Random Walk Models
3.6 Limiting Probabilities
3.7 Expected Numbers of Visits
3.8 Branching Processes
4 The Poisson Process
4.1 Counting Processes
4.2 The Poisson Process
4.3 Interarrival and Waiting Times
4.4 Further Properties of the Poisson Process
5 Birth and Death Processes
5.1 The Simple Birth Process
5.2 The Simple Death Process
5.3 The Simple Birth and Death Process
5.4 Immigration
5.5 Queues
6 Brownian Motion
6.1 Practice Problems