Contents
1 Functions of random variables
1.1 Cumulative distribution technique
1.2 Sums of independent random variables
1.3 Sums of independent binomial variables
1.4 Distribution of the range of a random sample
1.5 Joint distributions
1.6 Practice problems
2 Generating functions
2.1 The probability generating function
2.2 Properties of the probability generating function
2.3 Sums and linear functions
2.4 The moment generating function
2.5 The cumulant generating function
2.6 Uniqueness and continuity
2.7 Practice problems
3 Convolutions
3.1 The discrete case
3.2 The continuous case
3.3 The generating function shortcut
3.4 Practice problems
4 Conditional expectation
4.1 Introduction
4.2 Computing expectations by conditioning
4.3 Computing probabilities by conditioning
4.4 Conditional variance
4.5 The moment generating function of a random sum
4.6 Practice problems
5 Compound distributions
5.1 Mean, variance and generating function
5.2 The compound Poisson distribution
5.3 Practice problems
6 Modes of stochastic convergence
6.1 The four definitions
6.2 The hierarchy
6.3 Every converse fails
6.4 Practice problems
7 Limit theorems
7.1 Introduction
7.2 The weak law of large numbers and the central limit theorem
7.3 Slutsky’s theorem
7.4 Other inequalities
7.5 Practice problems
8 Additional topics in probability
8.1 Stochastic processes
8.2 The Poisson process
8.3 Markov chains
8.4 Transient and recurrent states
8.5 Practice problems
9 Martingales
9.1 Definition and first examples
9.2 Martingale differences and Hoeffding’s inequality
9.3 Convergence
9.4 Practice problems
10 Simulation
10.1 The probability integral transform
10.2 The inverse transform method
10.3 Rejection sampling
10.4 Simulating the normal distribution
10.5 Practice problems
1.1 Cumulative distribution technique
1.2 Sums of independent random variables
1.3 Sums of independent binomial variables
1.4 Distribution of the range of a random sample
1.5 Joint distributions
1.6 Practice problems
2 Generating functions
2.1 The probability generating function
2.2 Properties of the probability generating function
2.3 Sums and linear functions
2.4 The moment generating function
2.5 The cumulant generating function
2.6 Uniqueness and continuity
2.7 Practice problems
3 Convolutions
3.1 The discrete case
3.2 The continuous case
3.3 The generating function shortcut
3.4 Practice problems
4 Conditional expectation
4.1 Introduction
4.2 Computing expectations by conditioning
4.3 Computing probabilities by conditioning
4.4 Conditional variance
4.5 The moment generating function of a random sum
4.6 Practice problems
5 Compound distributions
5.1 Mean, variance and generating function
5.2 The compound Poisson distribution
5.3 Practice problems
6 Modes of stochastic convergence
6.1 The four definitions
6.2 The hierarchy
6.3 Every converse fails
6.4 Practice problems
7 Limit theorems
7.1 Introduction
7.2 The weak law of large numbers and the central limit theorem
7.3 Slutsky’s theorem
7.4 Other inequalities
7.5 Practice problems
8 Additional topics in probability
8.1 Stochastic processes
8.2 The Poisson process
8.3 Markov chains
8.4 Transient and recurrent states
8.5 Practice problems
9 Martingales
9.1 Definition and first examples
9.2 Martingale differences and Hoeffding’s inequality
9.3 Convergence
9.4 Practice problems
10 Simulation
10.1 The probability integral transform
10.2 The inverse transform method
10.3 Rejection sampling
10.4 Simulating the normal distribution
10.5 Practice problems
Rationale
This course in the mathematical theory of probability gives the techniques and skills a student needs to “think probabilistically”, which is essential both for understanding probability theory and for applying it. It gives the reasoning and proofs behind many of the concepts used in statistical inference, and it grounds the student in the mathematics of probability. It prepares the student for continuous-time Markov chains, renewal theory, queueing theory, reliability theory and Brownian motion.
Objectives
At the end of this course, the student should be able to:
- 1.
- find the distributions of functions of random variables;
- 2.
- compute expectations, probabilities and variances by conditioning;
- 3.
- determine the limits of sequences of random variables;
- 4.
- describe convergence in probability, stochastic convergence and asymptotic normality;
- 5.
- describe the Poisson process and Markov chains;
- 6.
- generate values of a random variable with an arbitrary distribution.
Course content
- 1.
- Functions of random variables. The cumulative distribution function technique; transformation methods; the probability integral transformation; transformations that are one to one; joint transformations; sums of random variables and the convolution formula; the method of moment generating functions; probability generating functions and characteristic functions.
- 2.
- Conditional expectation. Computing expectations and probabilities by conditioning; conditional variance; conditional expectation and prediction.
- 3.
- Limiting distributions. Sequences of random variables; the weak and strong laws of large numbers; the central limit theorem; asymptotic normal distributions; properties of stochastic convergence; further limit theorems; convergence in probability; Slutsky’s theorem.
- 4.
- Additional topics in probability. The one-sided Chebyshev inequality; Chernoff bounds; bounding the error when a sum of independent Bernoulli random variables is approximated by a Poisson random variable; introductions to the Poisson process and to Markov chains.
- 5.
- Martingale theory. Definition and examples of martingales; martingale differences and Hoeffding’s inequality; convergence of martingales; stopping times; optional stopping; the maximal inequality.
- 6.
- Simulation. General techniques for simulating continuous random variables; simulation from discrete distributions; variance reduction techniques.