Contents
1 FUNCTIONS OF RANDOM VARIABLES
1.1 Cumulative distribution technique
1.2 Sums of Independent Random Variables
1.3 Sum of Independent Binomial
1.4 Distribution of the Range of a Random Sample
1.5 Joint Distributions
1.6 Practice Problems
2 GENERATING FUNCTIONS
2.1 The Probability Generating Function
2.2 Properties of the Probability Generating Function
2.3 Sums and Linear Functions
2.4 The Moment Generating Function
2.5 The Cumulant Generating Function
2.6 Uniqueness and Continuity
2.7 Practice Problems
3 CONVOLUTIONS
3.1 The Discrete Case
3.2 The Continuous Case
3.3 The Generating Function Shortcut
3.4 Practice Problems
4 CONDITIONAL EXPECTATION
4.1 Introduction
4.2 Computing Expectation by Conditioning
4.3 Computing Probabilities by Condition
4.4 Conditional Variance
4.5 Moment Generating Function of the Sum of a Random Number of Random Variables
4.6 Practice Problems
5 COMPOUND DISTRIBUTIONS
5.1 Mean, Variance and Generating Function
5.2 The Compound Poisson
5.3 Practice Problems
6 MODES OF STOCHASTIC CONVERGENCE
6.1 The Four Definitions
6.2 The Hierarchy
6.3 Every Converse Fails
6.4 Practice Problems
7 LIMIT THEOREMS
7.1 Introduction
7.2 Weak Law of Large Numbers and Central Limit Theorem
7.3 Slutsky’s Theorem
7.4 Other Inequalities
7.5 Practice Problems
8 ADDITIONAL TOPICS IN PROBABILITY
8.1 Stochastic Processes
8.2 Poisson Process
8.3 Markov Chains (Sequence of Dependent Trials)
8.4 Transient and Recurrent States
9 MARTINGALES
9.1 Definition and First Examples
9.2 Martingale Differences and Hoeffding’s Inequality
9.3 Convergence
9.4 Practice Problems
10 SIMULATION
10.1 The Probability Integral Transform
10.2 The Inverse Transform Method
10.3 Rejection Sampling
10.4 Simulating the Normal Distribution
10.5 Practice Problems
1.1 Cumulative distribution technique
1.2 Sums of Independent Random Variables
1.3 Sum of Independent Binomial
1.4 Distribution of the Range of a Random Sample
1.5 Joint Distributions
1.6 Practice Problems
2 GENERATING FUNCTIONS
2.1 The Probability Generating Function
2.2 Properties of the Probability Generating Function
2.3 Sums and Linear Functions
2.4 The Moment Generating Function
2.5 The Cumulant Generating Function
2.6 Uniqueness and Continuity
2.7 Practice Problems
3 CONVOLUTIONS
3.1 The Discrete Case
3.2 The Continuous Case
3.3 The Generating Function Shortcut
3.4 Practice Problems
4 CONDITIONAL EXPECTATION
4.1 Introduction
4.2 Computing Expectation by Conditioning
4.3 Computing Probabilities by Condition
4.4 Conditional Variance
4.5 Moment Generating Function of the Sum of a Random Number of Random Variables
4.6 Practice Problems
5 COMPOUND DISTRIBUTIONS
5.1 Mean, Variance and Generating Function
5.2 The Compound Poisson
5.3 Practice Problems
6 MODES OF STOCHASTIC CONVERGENCE
6.1 The Four Definitions
6.2 The Hierarchy
6.3 Every Converse Fails
6.4 Practice Problems
7 LIMIT THEOREMS
7.1 Introduction
7.2 Weak Law of Large Numbers and Central Limit Theorem
7.3 Slutsky’s Theorem
7.4 Other Inequalities
7.5 Practice Problems
8 ADDITIONAL TOPICS IN PROBABILITY
8.1 Stochastic Processes
8.2 Poisson Process
8.3 Markov Chains (Sequence of Dependent Trials)
8.4 Transient and Recurrent States
9 MARTINGALES
9.1 Definition and First Examples
9.2 Martingale Differences and Hoeffding’s Inequality
9.3 Convergence
9.4 Practice Problems
10 SIMULATION
10.1 The Probability Integral Transform
10.2 The Inverse Transform Method
10.3 Rejection Sampling
10.4 Simulating the Normal Distribution
10.5 Practice Problems
Rationale _______________________________________________________________________________________________
The course, mathematical theory of probability gives techniques and skills for a student to ”think
probabilistically”. It is extremely important in both understanding and applying probability theory to
be able to ”think
probabilistically”. The course gives the reasoning and proofs of a number of concepts used in
statistical inference. It grounds the student in mathematics in probability. This prepares a student for
continuous-time Markov Chains, Renewal Theory, Queuing Theory as well as Reliability Theory
and
Brownian Motion.
Objectives
At the end of this course, the student should be able to:-
- 1.
- Find distributions of functions of variables.
- 2.
- Compute expectation, probability and variance by conditioning.
- 3.
- Determine the limits of sequences of random variables.
- 4.
- Describe convergence in probability, stochastic convergence and
asymptotic normal distribution. - 5.
- Describe the Poisson process and Markov chain.
- 6.
- Generate values of an arbitrary distributed random variables.
Course Content ______________________________________________________________________________________
- 1.
- Functions of Random Variables
Cumulative distribution function technique, Transformation methods, Probability integral transformation; transformation that are
one-to-one; joint transformations, Sums of random variables,
Convolution formula; moment generating method, Probability
generating and characteristic functions. - 2.
- Condition Expectation
Computing expectation by conditioning, Computing probability by conditioning, Condition variance, Condition expectation and
prediction. - 3.
- Limiting Distributions
Sequences of random random variables, Weak law of large numbers, Central limit theorem, Strong law of large numbers, Asymptotic normal distributions, Properties of stochastic convergence, Additional limit theorems, Convergence in probability; Slutsky’s theorem. - 4.
- Addition topics in probability
One sided Chebyshev inequality; Chernoff bounds; bounding the
error probability when approximating a sum of independent Bernoulli random variables by a Poisson random variable, Introduction to the Poisson process, Introduction to Markov chains. - 5.
- Martingale theory
Definition and examples of Martingale; Martingale differences and
Hoehding’s inequality; Convergence of Martingale; Stopping times;
Optional stopping; the maximal inequality. - 6.
- Simulation
General techniques for simulating continuous random variables,
Simulation from discrete distributions, Variance reduction techniques.