2.3 Order Statistics
Definition 2.18 (Order statistics). Let \(X_1,X_2,\dots ,X_n\) be a random sample from a population with continuous cdf \(F_X\), and let \[X_{(1)}<X_{(2)}<\dots <X_{(n)}\] be the same \(n\) observations rearranged in ascending order of magnitude. These are the order statistics of the sample, and \(X_{(r)}\), the \(r^{th}\) smallest, \(1\leq r\leq n\), is the \(r^{th}\) order statistic.
Two pieces of notation are in play and they must not be confused. \(X_i\) is the \(i^{th}\) observation as recorded; \(X_{(i)}\) is the \(i^{th}\) smallest. In the sample \(5,\ 2,\ 9\) we have \(X_1=5\) but \(X_{(1)}=2\).
Continuity of \(F_X\) is what makes the ordering unique. For a continuous distribution \(P\left (X_i=X_j\right )=0\) whenever \(i\neq j\), so with probability one no two observations are equal and the inequalities above are strict. Real data recorded to a fixed number of decimal places do produce ties, and those are handled by the mid-rank convention of a later section, but the theory is built on the continuous case.
Example 2.19. The seven observations \(12,\ 5,\ 19,\ 5.5,\ 8,\ 21,\ 14\) have order statistics \[X_{(1)}=5,\ X_{(2)}=5.5,\ X_{(3)}=8,\ X_{(4)}=12,\ X_{(5)}=14,\ X_{(6)}=19,\ X_{(7)}=21 .\] The sample median is \(X_{(4)}=12\), the range is \(X_{(7)}-X_{(1)}=16\), and the largest observation \(21\) is \(X_{(7)}\) regardless of where it appeared in the recording.
Note 2.20. Order statistics are the reason non-parametric methods work at all. Every test in these notes uses the data only through order or rank, never through the raw magnitudes, and that is precisely why no distributional form has to be assumed. The confidence interval for the median of the previous section is the clearest case: its endpoints are order statistics, \(\left (X_{(k)},X_{(n-k+1)}\right )\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.