Stochastic Processes

Lecture Notes

_________________________________________

\[p_{ij}^{(m+n)} = \sum _{k} p_{ik}^{(m)}\,p_{kj}^{(n)}\]

TABLE OF CONTENTS
TABLE OF CONTENTS
1 Warming Up
1.1 Examples on Probability
1.2 Conditional Expectation
1.3 Computing Expectations by Conditioning
1.4 Computing Probability by Conditioning
1.5 Markov’s Inequality
1.6 Computing Variance by Conditioning
2 Stochastic Process
2.1 Definitions
2.2 Markovian Stochastic Process
2.3 Homogeneous Markov Processes
3 Markov Chains
3.1 Introduction
3.2 Chapman Kolmogrov Equations
3.3 Unconditional Distributions
3.4 Classification of States
3.5 Random Walk Models
3.6 Limiting probabilities
3.7 Expected Number of Visits
3.8 Expected Number of Visists to a Transient State
3.9 Branching Process
4 Stochastic Process with Continuous Time
4.1 Poisson Process
4.2 Interarrival and Waiting Time Distributions
4.3 Further Properties of Poisson Process
5 Birth and Death Process
5.1 The Simple Birth Process
5.2 Simple Death Process
5.3 Simple Birth and Death Process
5.4 Effects of Immigration
5.5 Queues (Consider a Markov Chain with continuous time)
6 Brownian Motion
REFERENCES
REFERENCES
6.1 Practice Problems