1.8 More on Moments

An alternative formula for \(\mu _r = E(X_{(r)})\) may be found by integrating by parts. \[\mu _r = \int ^{\infty }_{-\infty } x\, dF_r(x)\hspace {1cm} \cdots \hspace {1cm} *,\] where \(F_r(x)\) is the cdf of \(X_{(r)}\).

Note 1.8.1. For any cdf \(P(X)\) , the existence of \(E(X)\) implies that

(i): \(\lim \limits _{x \rightarrow - \infty }x\, P(x) = 0\)
(ii): \(\lim \limits _{x \rightarrow + \infty }x\, \left (1 - P(x)\right ) = 0\).

Therefore \(\lim \limits _{x \rightarrow - \infty } x\, F_r(x) = 0\) and \(\lim \limits _{x \rightarrow + \infty }x\, \left (1 - F_r(x)\right ) = 0\,\) since \(\mu _r\) exists.

\(*\) can be written as \[\mu _r = \int ^0_{-\infty }x\, dF_r(x) - \int _0^{\infty }x\, d\left (1 - F_r(x)\right )\hspace {1cm} \cdots \hspace {1cm} 1.8.1\] integrating by parts \begin {align*} \mu _r & = x\, F_r(x)\Big |^0_{-\infty } - \int ^0_{-\infty } F_r(x)\, dx - x(1 - F_r(x)\Big |^{\infty }_0 + \int _0^{\infty }\left (1 - F_r(x)\right )\, dx\\ & = \int ^{\infty }_0 \left (1 - F_r(x)\right )\, dx - \underbrace {\int _{-\infty }^0F_r(x)\, dx}_{\text {let}\, y = -x\implies dy = -dx}\\ & = \int ^{\infty }_0\left (1 - F_r(x)\right )\, dx - \int ^0_{\infty }F_r(-y)\, \left (-dy\right )\\ & = \int ^{\infty }_0\left (1 - F_r(x) - F_r(-x)\right )\, dx\\ \therefore \, \mu _r & = \int ^{\infty }_0\left (1 - F_r(x) - F_r(-x)\right )\, dx\hspace {1cm} \cdots \hspace {1cm} 1.8.2 \end {align*}

The range \(W = X_{(n)} - X_{(1)}\) we have \[E(W) = E(X_{(n)}) - E(X_{(1)}) = \mu _n - \mu _1\] Therefore \begin {align*} E(W) & = \int ^{\infty }_0\left (1 - F_n(x) - F_n(-x) - 1 + F_1(x) + F_1(-x)\right )\, dx\\ & = \int ^{\infty }_0\left (F_1(x) - F_n(x) + F_1(-x) - F_n(-x)\right )\, dx\\ & = \int ^{\infty }_0\left (F_1(x) - F_n(x)\right )\, dx + \underbrace {\int ^{\infty }_0\left (F_1(-x) - F_n(-x)\right )\, dx}_{\text {let}\, y = -x\implies dy = -dx}\\ & = \int ^{\infty }_0\left (F_1(x) - F_n(x)\right )\, dx + \int ^{-\infty }_0\left (F_1(y) - F_n(y)\right )\, (-dy)\\ & = \int ^{\infty }_0\left (F_1(x) - F_n(x)\right )\, dx + \int ^0_{-\infty }\left (F_1(y) - F_n(y)\right )\, dy\\ \therefore \, E(W) & = \int ^{\infty }_{-\infty }\left (F_1(x) - F_n(x)\right )\, dx\hspace {1cm} \cdots \hspace {1cm} 1.8.3 \end {align*}

Recall that \[ F_r(x) = \sum ^n_{j = r} \binom {n}{j}\, \left (P(x)\right )^j\, \left (1 - P(x)\right )^{n - j}.\] \begin {align*} F_1(x) & = \sum ^n_{j = 1} \binom {n}{j}\, \left (P(x)\right )^j\, \left (1 - P(x)\right )^{n - j} = 1 - \left (1 - P(x)\right )^{n}.\\\\ F_n(x) & = \left (P(x)\right )^n. \end {align*}

Therefore 1.8.3 becomes \[E(W) = \int ^{\infty }_{-\infty } 1 - \left (1 - P(x)\right )^n - \left (P(x)\right )^n\, dx\hspace {1.5cm}\cdots \hspace {1cm} 1.8.4\]

Useful checks on computation of the raw moments are provided by noting that \[\left (\sum _{r = 1}^nX^k_{(r)}\right )^m = \left (\sum _{r = 1}^nX^k_r\right )^m\hspace {1.5cm} \cdots \hspace {1cm} 1.8.5\] For every \(k\) and \(m\).

If \(k = 1\), and \(m = 1\) then \[\sum ^n_{r =1}X_{(r)} = \sum ^n_{r = 1}X_r.\] Taking expectation both sides we get \[\sum ^n_{r = 1} \mu _r = n\, \mu \hspace {1.5cm} \cdots \hspace {1cm} 1.8.6\]

If \(k = 2\) and \(m = 1\) \[\left (\sum ^n_{r = 1} X^2_{(r)}\right ) = \left (\sum ^n_{r = 1}X_r\right )\hspace {0.4cm}\implies \hspace {0.5cm} \sum ^n_{r = 1} X_{(r)}^2 = \sum ^n_{r = 1}X_r^2.\] Taking expectation both sides \begin {align*} \sum ^n_{r = 1}E\left (X^2_{(r)}\right ) & = \sum ^n_{r = 1} E\left (X^2_r\right ) = n\, (\sigma ^2 + \mu ^2)\\ \sum ^n_{r = 1}E\left (X^2_{(r)}\right ) & = n\, (\sigma ^2 + \mu ^2)\hspace {1.5cm} \cdots \hspace {1cm} 1.8.7 \end {align*}

If \(k = 1\), and \(m = 2\) \[\left (\sum ^n_{r = 1} X_{(r)}\right )^2 = \left (\sum ^n_{r = 1}X_r\right )^2\]

\[\sum ^n_{r = 1}X_{(r)}\cdot \sum ^n_{r = 1} X_{(r)} = \sum ^n_{r = 1} X_r \, \sum ^n_{r = 1} X_r\]

\[\sum ^n_{i = 1}\sum ^n_{j = 1} X_{(i)}\, X_{(j)} = \sum ^n_{i = r}\sum ^n_{j = 1}X_i\, X_j\]

\[\sum ^n_{i = 1} X_{(i)}^2 + 2\underset {i < j}{\sum \sum } X_{(i)} \, X_{(j)} = \sum ^n_{i = 1} X_i^2 + 2\underset {i<j}{\sum \sum }X_i\, X_j\] Taking expectations both sides we get \begin {align*} \sum ^n_{i = 1} E\left (X^2_{(i)}\right ) + 2\, \underset {i< j}{\sum \sum }E\left (X_{(i)}\, X_{(j)}\right ) & = n(\sigma ^2 + \mu ^2) + 2\, \binom {n}{2}\, \mu ^2\\ & = n\sigma ^2 + n\mu ^2 + n(n - 1)\mu ^2\\ & = n\sigma ^2 + n\mu ^2 + n^2\mu ^2 - n\mu ^2\\ & = n\sigma ^2 + n^2\sigma ^2\\ & = n(\sigma ^2 + n\mu ^2). \end {align*}

\[\sum ^n_{r = 1} E\left (X_{(i)}^2\right ) + 2\, \underset {r<s}{\sum \sum }E\left (X_{(r)}\, X_{(s)}\right )\hspace {1.5cm} \cdots \hspace {1cm} 1.8.8\]

\[\sum ^n_{r = 1} X_{(r)} = \sum ^n_{r = 1} X_{(r)}\]

\[\sum ^n_{r = 1} \left (X_{(r)} - \mu _r\right ) = \sum ^n_{r = 1} (X_r - \mu )\] squaring both sides \[\sum ^n_{r = 1}\sum ^n_{s = 1}\left (X_{(r)} - \mu _r\right )\,\left (X_{(s)} - \mu _s\right ) = \sum ^n_{r = 1}\sum ^n_{s = 1}\left (X_r - \mu \right )\, \left (X_s - \mu \right ).\]

Taking expectation both sides \[\sum ^n_{r =1}\sum ^{n}_{s = 1}Cov\left (X_{(r)},X_{(s)}\right ) = \sum ^n_{r = 1}\sum ^n_{s = 1} Cov\left (X_r, X_s\right )\]

\[\sum ^n_{r = 1}\sigma ^2_r + 2\, \underset {r<s}{\sum \sum }Cov\left (X_{(r)}, X_{(s)}\right ) = n\sigma ^2\hspace {1.5cm} \cdots \hspace {1cm}1.8.9\] (This is true for any distribution)

Read!!! Discrete case of order statistics.

Example 1.8.2. Let \(X_1, \, X_2, \, \cdots \, , \, X_n\) be random sample from exponential with mean \(1/\lambda \). \[P(x) = 1 - e^{-\lambda \, x}\]

\[E(W) = \int ^{\infty }_{-\infty }\left (1 - \left (1 - P(x)\right )^n - \left (P(x)\right )^n\right )\, dx\]

\[E(W) = \int ^{\infty }_01 - e^{-\lambda nx} - \left (1 - e^{-\lambda x}\right )^n\, dx\] Let \(u = 1 - e^{-\lambda x} \,\, \implies \, \, du = \lambda e^{-\lambda x}\, dx\) \[e^{-\lambda x} = 1 - u\hspace {1cm} \frac {du}{\lambda (1 - u)} = dx\] \begin {align*} E(W) & = \int ^1_0 1 - (1 - u)^n - u^n \, \frac {du}{\lambda (1 - u)}\\ & = \frac {1}{\lambda }\int ^1_0-(1 - \mu )^{n - 1}\, du + \frac {1}{\lambda }\int ^1_0\frac {1 - u^n}{1 - u}\, du\\ & = \frac {1}{\lambda }\cdot \frac {(1 - u)^n}{n}\Big |^1_0 + \frac {1}{\lambda }\int ^1_0\sum _{k = 0}^{n - 1} u^k\, du\\ & = \frac {1}{\lambda n} + \frac {1}{\lambda }\sum ^{n - 1}_{k = 0}\frac {u^{k + 1}}{k + 1}\Big |^1_0\\ & = \frac {1}{\lambda n} + \frac {1}{\lambda }\sum ^{n - 1}_{k = 0}\frac {1}{k + 1}. \end {align*}

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.