1.9 Practice Problems
Problem 1.9.1. Let \(X_1,\dots ,X_n\) be a random sample from a continuous distribution with density \(p(x)\) and distribution function \(P(x)\). Derive the density of the \(r\)th order statistic \(X_{(r)}\), explaining the combinatorial factor.
Show solution
Solution. Partition the sample by position relative to a small interval about \(x\). For \(X_{(r)}\) to lie in \((x, x+dx)\) we need exactly one observation in that interval, exactly \(r-1\) below it, and the remaining \(n-r\) above. The number of ways of allocating labelled observations to these three groups is the multinomial coefficient \(\dfrac {n!}{(r-1)!\,1!\,(n-r)!}\), and by independence each such allocation has probability \(\left [P(x)\right ]^{r-1}p(x)\,dx\left [1-P(x)\right ]^{n-r}\). Hence \[p_{(r)}(x) = \frac {n!}{(r-1)!\,(n-r)!}\left [P(x)\right ]^{r-1} \left [1-P(x)\right ]^{n-r}p(x).\]
Problem 1.9.2. Let \(X_1,\dots ,X_n\) be a random sample from the uniform distribution on \((0,\theta )\). Find \(E\left (X_{(r)}\right )\) and \(\operatorname {var}\left (X_{(r)}\right )\) for \(r=1,\dots ,n\).
Show solution
Solution. Writing \(X_{(r)} = \theta U_{(r)}\) with \(U_{(r)}\) the corresponding uniform order statistic on \((0,1)\), the density of \(U_{(r)}\) is \(\dfrac {n!}{(r-1)!(n-r)!}u^{r-1}(1-u)^{n-r}\), which is \(\operatorname {Beta}(r,\,n-r+1)\). Therefore \[E\left (X_{(r)}\right ) = \frac {r\theta }{n+1},\qquad \operatorname {var}\left (X_{(r)}\right ) = \frac {r(n-r+1)\theta ^{2}}{(n+1)^{2}(n+2)} .\] Note the mean divides \((0,\theta )\) into \(n+1\) equal parts, and the variance is largest in the middle of the sample and smallest at the extremes.
Problem 1.9.3. For the same uniform sample, obtain the product moment \(E\left (X_{(1)}X_{(n)}\right )\) and hence \(\operatorname {cov}\left (X_{(1)},X_{(n)}\right )\). Comment on the sign of the covariance and on its behaviour as \(n\to \infty \). Where to start: the joint density of \(X_{(1)}\) and \(X_{(n)}\) from Section 1.4, integrated over the region \(x<y\).
Problem 1.9.4. Let \(X_1,\dots ,X_n\) be a random sample from a continuous distribution. Derive the joint density of \(X_{(r)}\) and \(X_{(s)}\) for \(r<s\), and explain which of the four groups of observations each factor of your expression counts. Where to start: partition the sample into the four groups below \(x\), at \(x\), between \(x\) and \(y\), at \(y\), and above \(y\), and count the allocations as in Problem 1.9.1.
Problem 1.9.5. The range of a sample is \(W = X_{(n)}-X_{(1)}\). Derive the density of \(W\) for a sample of size \(n\) from the exponential distribution with mean \(\lambda \), and show that \(W\) has the same distribution as the largest of \(n-1\) observations from the same distribution. Where to start: the joint density of the extremes, with the change of variable \(w = y-x\); Section 1.5.
Problem 1.9.6. The midrange is \(M = \tfrac 12\left (X_{(1)}+X_{(n)}\right )\). Show that if the parent density is symmetric about \(\mu \), then \(M\) is an unbiased estimator of \(\mu \). Explain why \(M\) is nevertheless a poor estimator for a heavy-tailed parent, and a good one for a uniform parent. Where to start: symmetry gives \(X_{(1)}-\mu \overset {d}{=} \mu -X_{(n)}\). For the second part compare the variance of the extremes under a uniform parent with their behaviour under heavy tails.
Problem 1.9.7. Show that the conditional distribution of \(X_{(r)}\) given \(X_{(s)} = y\), for \(r<s\), is that of the \(r\)th order statistic in a sample of size \(s-1\) from the parent distribution truncated to \((-\infty , y]\). Explain intuitively why the observations above \(X_{(s)}\) contribute nothing. Where to start: divide the joint density of Section 1.6 by the marginal of \(X_{(s)}\) and recognise the result as a density of the truncated parent.
Problem 1.9.8. Establish that \(E\left (X_{(r)}\right )\) exists for every \(r\) whenever \(E(X)\) exists, and give an example of a distribution for which \(E\left (X_{(n)}\right )\) fails to exist for some \(n\) although each \(X_i\) has a finite median. Where to start: bound \(|X_{(r)}|\) by \(\max _i|X_i|\leq \sum _i|X_i|\). For the counterexample take a Cauchy parent, whose median exists and whose mean does not.
Problem 1.9.9. A system consists of \(n\) components with independent lifetimes. Express the lifetime of the system as an order statistic when the components are arranged (a) in series, (b) in parallel, and (c) so that the system survives while at least \(k\) components survive. State which order statistic governs each case. Where to start: a series system fails with the first failure and a parallel system with the last; the general case is a \(k\)-out-of-\(n\) system.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.