10.5 Practice Problems

Problem 10.5.1. Give an algorithm, based on a single uniform variate, for simulating a random variable with the Pareto density \(f(x) = \theta x^{-(\theta +1)}\), \(x>1\).

Show solution

Solution. The distribution function is \[F(x) = \int _1^{x}\theta t^{-(\theta +1)}\,dt = 1 - x^{-\theta },\qquad x>1 .\] Setting \(u = 1-x^{-\theta }\) and solving, \[x^{-\theta } = 1-u \quad \Longrightarrow \quad x = (1-u)^{-1/\theta }.\] So generate \(U\sim \) UNIF\((0,1)\) and return \(X = (1-U)^{-1/\theta }\), or equivalently \(X = U^{-1/\theta }\) since \(1-U\) is uniform as well.

Problem 10.5.2. A discrete random variable takes the values \(1,2,3,4\) with probabilities \(0.2,\ 0.3,\ 0.4,\ 0.1\). Describe how to simulate it from one uniform variate, and state which value is returned when \(U = 0.63\).

Show solution

Solution. Form the cumulative probabilities \(0.2,\ 0.5,\ 0.9,\ 1.0\) and return the smallest value whose cumulative probability is at least \(U\): \[X = \begin {cases} 1, & 0<U\leq 0.2,\\ 2, & 0.2<U\leq 0.5,\\ 3, & 0.5<U\leq 0.9,\\ 4, & 0.9<U<1 . \end {cases}\] For \(U=0.63\) we have \(0.5<0.63\leq 0.9\), so \(X=3\).

Ordering the values by decreasing probability first — here \(3,2,1,4\) — reduces the expected number of comparisons, which is worth doing when the routine is called many times.

Problem 10.5.3. Show that if \(U\sim \) UNIF\((0,1)\) then \(-\log U \sim \) EXP\((1)\), and hence give a method for simulating a GAM\((n,1)\) variable for integer \(n\).

Show solution

Solution. For \(x>0\), \[P\left (-\log U > x\right ) = P\left (U < e^{-x}\right ) = e^{-x},\] so \(-\log U\) has the exponential survival function with rate \(1\).

A GAM\((n,1)\) variable is the sum of \(n\) independent EXP\((1)\) variables — which is the closure result of the convolutions chapter. So generate \(U_1,\dots ,U_n\) independently and return \[X = -\sum _{i=1}^{n}\log U_i = -\log \left (\prod _{i=1}^{n}U_i\right ),\] the product form saving \(n-1\) logarithms.

Problem 10.5.4. A rejection scheme uses the UNIF\((0,1)\) density as envelope for the target \(f(x) = 6x(1-x)\) on \((0,1)\). Find the smallest valid constant \(c\) and the expected number of draws per accepted value.

Show solution

Solution. Here \(g(x)=1\) on \((0,1)\), so \(c\) must satisfy \(f(x)\leq c\) for all \(x\), that is \(c \geq \max _{0<x<1} 6x(1-x)\). Differentiating, the maximum is at \(x=\tfrac 12\) with value \(6\cdot \tfrac 12\cdot \tfrac 12 = 1.5\). Hence \(c = 1.5\).

The acceptance probability is \(1/c = 2/3\), so the expected number of draws per accepted value is \(c = 1.5\). Two thirds of the proposals are kept, which for a target this close in shape to its envelope is about as efficient as rejection sampling gets.

Problem 10.5.5. Explain why the inverse transform method cannot be used directly to simulate a standard normal variable, and name two methods that can.

Show solution

Solution. The inverse transform requires \(F^{-1}\) in closed form. The standard normal distribution function \[\Phi (x) = \int _{-\infty }^{x}\frac {1}{\sqrt {2\pi }}e^{-t^{2}/2}\,dt\] has no elementary antiderivative, so \(\Phi ^{-1}\) cannot be written in elementary functions; it can only be evaluated numerically, which defeats the purpose of a cheap generator.

Two methods avoid this. Box–Muller converts two uniforms into two independent normals through the polar representation, using only a logarithm, a square root and two trigonometric calls. Rejection sampling with a heavier-tailed envelope — the Cauchy or a double exponential — also works, since both dominate the normal density after scaling. Numerical inversion of \(\Phi \) is a third route and is what most software actually uses, but it is an approximation rather than an exact transformation.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.