1.7 Expected values and Moments of Order Statistics

Let \(X_1, \, X_2, \, \cdots \, , \, X_n\) be a random sample from a continuous distribution with cdf \(P(x)\) and pdf \(p(x)\). Let \(f_r(x)\) be the pdf of \(X_{(r)}\) (\(r^{\text {th}}\) order statistics). \[\mu _r = E\left (X_{(r)}\right ) = \int _x x\, f_r(x)\, dx\,, \] \[f_r(x) = \frac {n!}{(r - 1)!\, (n - r)!}\, p(x)\, \left (P(x)\right )^{r - 1}\, \left (1 - P(x)\right )^{n - r}.\] \begin {align*} \mu _r & = \int _x \frac {n!}{(r - 1)!\, (n - r)!}p(x)\, \left (P(x)\right )^{r - 1}\, \left (1 - P(x)\right )^{n - r } \, dx\\ & = \frac {n!}{(r - 1)!\, (n - r)!}\int x\, \left (P(x)\right )\, \left (1 - P(x)\right )^{n - r}\, dP(x)\, , \hspace {0.5cm} \boxed {p(x)dx = dP(x)}\\ & = \frac {n\, (n - 1)!}{(r - 1)!\, (n - r)!}\int ^{\infty }_{-\infty } x\, \left (P(x)\right )^{r - 1}\, \left (1 - P(x)\right )^{n - r}\, dP(x)\\ & = n\, \binom {n - 1}{r - 1}\, \int ^{\infty }_{-\infty } x\, \left (P(x)\right )^{r - 1}\, \left (1 - P(x)\right )^{n - r}\, dP(x)\hspace {1cm} \cdots \hspace {0.3cm} 1.8.1. \end {align*}

Since \(0 \leq P(x) \leq 1\) \[|\mu _r| \leq n \, \binom {n - 1}{r - 1}\, \int |x|\, dP(x)\hspace {1cm} \cdots \hspace {0.3cm} 1.8.2\] 1.8.2 shows that \(\mu _r\) exists provided \(E(X)\) exists.

The converse is not necessary true i.e \(\mu _r\) may exists even when \(E(X)\) doesn’t exist.

For equation 1.8.1, let \(u = P(x)\), then \(du = dP(x)\) \[P^{-1}(u) = P^{-1}\left (P(x)\right ) = x\] \[\mu _r = n\, \binom {n - 1}{r - 1}\int _0^1P^{-1}(u)\, u^{r - 1}\, \left (1 - u\right )^{n - r}\, du\hspace {1cm} \cdots \hspace {0.5cm} 1.8.3\]

If the mean \(E(X) = \int ^1_0P^{-1}(u)\, du\) doesn’t exist because of singularities at \(u = 0\), or \(u = 1\), \(\mu _r\) may never the less exist for certain values of \(r\). For examples the Cauchy distribution \(\mu _r\) exists unless \(r = 1\) or \(r = n\) \[p(x) = \frac {1}{\pi \left (1 + (x - \theta )^2\right )}.\] If \(E(g(x))\) exists where \(g(x)\) is some function of \(X\) then \(E(g(X_{(r)})\) will also exist.

The special cases of the \(g(\cdot )\) function \[g(x) = x^k, \, E\left (X^k_{(r)}\right ) = \text {the}\, K^{\text {th}}\, \text {raw moment of}\, X_{(r)}.\] \[g(x) = \left (x - \mu \right )^{k}\, , \, \hspace {0.3cm} E\left (X_{(r)}^k - \mu ^k_{r}\right )^k = \text {the}\, K^{\text {th}}\, \text {central moment of}\, X_{(r)}.\] \[g(x) = e^{tx}\, , \, \hspace {0.3cm} E\left (e^{t X_{(r)}}\right ) = \text {the mgf of the }\, r^{\text {th}}\, \text {order statistics}.\]

The product moments may be defined as usual \[\mu _{rs} = E\left (X_{(r)}\, X_{(s)}\right )\hspace {1cm} \cdots \cdots \hspace {1cm} 1.8.4\] \[\text {Covariance}\hspace {0.3cm} \sigma _{rs} = E\left [\left (X_{(r)} - \mu _r\right )\left (X_{(s)} - \mu _s\right )\right ]\hspace {1cm}\cdots \cdots \hspace {1cm} 1.8.5\] \[\text {Variance}\hspace {0.5cm} \sigma ^{2}_{r} = E\left [\left (X_{(r)} - \mu _r\right )^2\right ]\hspace {1cm}\cdots \cdots \hspace {1cm}1.8.6\]

Example 1.7.1. Let \(X\) be a uniform random variable on \((0,1)\). Find the following given a random sample of size \(n\).

(a): mean and variance of \(X_{(r)}\),

Solution. \(X\thicksim U(0,1)\,\) i.e \(\, p(x) = 1, \hspace {0.5cm} P(x) = x, \hspace {0.3cm} x \in (0, 1)\) \begin {align*} \mu _r & = n \, \binom {n - 1}{n - r}\int ^1_0x\cdot 1\cdot \cdot x^{r - 1}\cdot (1 - x)^{n - r}\, dx\\ & = n\, \binom {n - 1}{n - r}\underbrace {\int ^1_0x^r\, (1 - x)^{n - r}\, dx}_{B(r + 1, n - r+ 1}\\ & = \frac {n\, (n - 1)!}{(n - r)!\, (r - 1)!}\cdot \frac {r!\, (n - r)!}{(n + 1)!}\\ \implies \hspace {0.3cm} \mu _r & = \frac {r}{n + 1}. \end {align*}
If \(r = 1, \hspace {0.3cm} \mu _1 = \dfrac {1}{n + 1}.\) \[E\left (X_{(r)}^2\right ) = \frac {r\, (r + 1)}{(n + 1)\, (n + 2)}\] Hence \begin {align*} \sigma ^2_r & = E\left (X_{(r)}\right )^2 - \left (E(X_{(r)}\right )^2\\ & = \frac {r\, (r + 1)}{(n + 1)\, (n + 2)} - \left (\frac {r}{n + 1}\right )^2\\ & = \frac {(r^2 + r)}{(n + 1)\, (n + 2)} - \frac {r^2}{(n + 1)^2}\\ & = \frac {(n + 1)\, (r^2 + r) - r^2\, (n + 2)}{(n + 1)^2 \, (n + 2)} = \frac {nr^2 + nr + r^2 + r - nr^2 - 2r^2}{(n + 1)^2\, (n + 2)}\\ & = \frac {r\, (n + 1 - r)}{(n + 1)^2 \, (n + 2)}. \end {align*} □
(b): product moment of \(X_{(r)}\) and \(X_{(s)}\), \(\, s>r\)

Solution. \begin {align*} \mu _{rs} = & \iint \frac {n!\, xy\, x^{r - 1}\, (y - x)^{s - r - 1}}{(r - 1)!\, (s - r - 1)!\, (n - s)!}(1 - y)^{n -s}\, dx\, dy\\ = & \frac {n!}{(r - 1)!\, (s - r - 1)!\, (n - s)!}\int _0^1y\, (1 - y)^{n - s}\int ^y_0 x^r\, (y - x)^{s - r - 1}\, dx \, dy\\ = & C\, \int ^1_0y\, (1 - y)^{n - s}\, y^{s - r - 1}\, \int ^y_0x^r\, \left (1 - \frac {x}{y}\right )^{s-r-1}\, dx\, dy\\ & (\text {let}\, z = \frac {x}{y}\, \text {then}\, y\, dz = dx)\\ = & \frac {n!}{(r - 1)!\, (s - r - 1)!\, (n - s)!}\int ^1_0y\, (1 - y)^{n - s}\, y^{s - 1}\, dy\, \int ^1_0z^r\, (1 - z)^{s-r-1}\, dz\\ E\left (X_{(r)}X_{(s)}\right ) & = \frac {r \, (s + 1)}{(n + 1)\, (n + 2)}. \end {align*} □
(c): covariance of \(X_{(r)}\) and \(X_{(s)}\).

Solution. From (a) and (b) \begin {align*} Cov\left (X_{(r)}, X_{(s)}\right ) & = E\left (X_{(r)}X_{(s)}\right ) - E\left (X_{(r)}\right )\, E\left (X_{(s)}\right )\\ & = \frac {r\, (s + 1)}{(n + 1)(n + 2)} - \frac {r}{(n + 1)}\cdot \frac {s}{(n+1)} = \frac {(n + 1)\, r\, (s + 1) - (n + 2)\, rs}{(n + 1)^2(n + 2)}\\ & = \frac {r\, (n - s + 1)}{(n + 1)^2\, (n + 2)}. \end {align*} □

Let us consider the product moment for four order statistics \(1 \leq r < s < t < u \leq n\) \begin {align*} f_{r,s,t,u}(x_1, x_2, x_3, x_4) = & \frac {n!\, x_1^{r - 1}\, (x_2 - x_1)^{s - r - 1}\, (x_3 -x_2)^{t-s-1}\, (x_4 - x_3)^{u -t - 1}\, (1 - x_4)^{n - u}}{(r - 1)!(s - r - 1)!\, (t - s - 1)!\, (u - t - 1)!\, (n - u)!}\\ & 0 < x_1 < x_2 < x_3 < x_4 < 1. \end {align*}

making the following transformation \(\,(x_1,\, x_2,\, x_3,\, x_4)\, \, \longrightarrow \, \, (y_1,\, y_2,\, y_3, \, y_4)\) \[x_1 = y_1y_2y_3y_4\, , \,\,\, x_2 = y_2y_3y_4\, , \,\,\, x_3 = y_3y_4\, , \,\,\, x_4 = y_4\] \[J = y_2\, y_3^2\, y_4^3\] \begin {align*} f_{Y_1,Y_2,Y_3,Y_4}(y_1,y_2,y_3,y_4) & =A\, y_1^{r - 1}(1 - y_1)^{s - r - 1}\, y^{s - 1}_2(1 - y_2)^{t - s - 1}\,y^{t - 1}_3(1 - y_3)^{u - t - 1}\, y_4^{u - 1}(1 - y_4)^{n - u} \end {align*}

\[A = \frac {1}{B(r,s - r)}\cdot \frac {1}{B(u, n - u + 1)}\]

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.