4.5 Question 5

Problem 4.5.1. [2013 examination]

(a).
Define the following:
(i).
the size of a test;
(ii).
a uniformly minimum variance unbiased estimator of \(\tau (\theta )\).
(b).
Let \(X_1,X_2,\ldots ,X_n\) be a random sample from a distribution with probability density function \(f(x)\).
(i).
Derive the probability density function of \(X_{(1)}=\min (X_1,X_2,\ldots ,X_n)\).
(ii).
If \(f(x)=\frac {1}{x^2}\), \(x>1\), find the probability density function of the order statistics \(Y_1,Y_2,\ldots ,Y_n\).
(iii).
If \(f(x)=\frac {1}{x^2}\), \(x>1\), find the probability density function of \(X_{(n)}=\max (X_1,X_2,\ldots ,X_n)\).
(c).
Let \(X_1,X_2,\ldots ,X_n\) be a random sample from the \(\text {POI}(\theta )\) distribution with probability function \(f(x,\theta )=\frac {e^{-\theta }\theta ^x}{x!}\), \(x=0,1,2,\ldots \)
(i).
Find the maximum likelihood estimator of \(\theta \).
(ii).
Show that \(\bar X\) is a UMVUE of \(\theta \).
(iii).
Show that \(T=\left (\frac {n-1}{n}\right )^{\sum _{i=1}^nX_i}\) is an unbiased estimator of \(\tau (\theta )=e^{-\theta }\).

Show solution

Solution. (a)(i). The size of a test with power function \(\pi (\theta )\) is \[\alpha =\sup _{\theta \in \Theta _0}\pi (\theta ),\] the largest probability of rejecting \(H_0\) when \(H_0\) is true. For a simple null this is just \(\pi (\theta _0)\).

(ii). \(T\) is a uniformly minimum variance unbiased estimator of \(\tau (\theta )\) if \(E_\theta (T)=\tau (\theta )\) for all \(\theta \), and for every other unbiased estimator \(T^*\), \[\text {Var}_\theta (T)\leq \text {Var}_\theta \left (T^*\right ) \qquad \text {for all }\theta .\] The word uniformly carries the “for all \(\theta \)”: one estimator must win everywhere, not merely at some convenient parameter value.

(b)(i). The minimum exceeds \(y\) exactly when every observation does, so \[1-F_{X_{(1)}}(y)=P\left (X_{(1)}>y\right )=\prod _{i=1}^nP(X_i>y) =\left [1-F(y)\right ]^n.\] Differentiating \(F_{X_{(1)}}(y)=1-\left [1-F(y)\right ]^n\), \[f_{X_{(1)}}(y)=n\left [1-F(y)\right ]^{n-1}f(y).\]

(ii). With \(f(x)=\frac {1}{x^2}\) on \(x>1\), \[F(x)=\int _1^x\frac {1}{u^2}\,du=1-\frac {1}{x},\qquad x>1.\] The joint density of the order statistics is \(n!\) times the joint density of the sample, restricted to the ordered region — there are \(n!\) orderings of the sample producing the same ordered values. Hence \[f_{Y_1,\ldots ,Y_n}(y_1,\ldots ,y_n)=n!\prod _{i=1}^nf(y_i) =\frac {n!}{\prod _{i=1}^ny_i^2},\qquad 1<y_1<y_2<\cdots <y_n.\]

(iii). The maximum is at most \(y\) exactly when every observation is, so \[F_{X_{(n)}}(y)=\left [F(y)\right ]^n=\left (1-\frac {1}{y}\right )^n,\] \[f_{X_{(n)}}(y)=n\left [F(y)\right ]^{n-1}f(y) =\frac {n}{y^2}\left (1-\frac {1}{y}\right )^{n-1},\qquad y>1.\] (By way of comparison, part (i) gives \(f_{X_{(1)}}(y)=\frac {n}{y^{n+1}}\) for the same \(f\).)

(c)(i). \[L(\theta )=\frac {e^{-n\theta }\theta ^{\sum x_i}}{\prod x_i!},\qquad \ell (\theta )=-n\theta +\left (\sum x_i\right )\ln \theta -\ln \prod x_i!,\] \[\ell '(\theta )=-n+\frac {\sum x_i}{\theta }=0 \quad \implies \quad \hat \theta =\frac {\sum x_i}{n}=\bar X,\] with \(\ell ''(\theta )=-\frac {\sum x_i}{\theta ^2}<0\), so this is a maximum.

(ii). The Poisson family is a one-parameter exponential family, since \[f(x,\theta )=\frac {1}{x!}\exp \left [x\ln \theta -\theta \right ],\] so \(S=\sum X_i\) is a complete sufficient statistic for \(\theta \). Now \(\bar X=S/n\) is a function of \(S\) alone and is unbiased, \[E\left (\bar X\right )=\frac {1}{n}\sum E(X_i)=\frac {n\theta }{n}=\theta .\] By the Lehmann–Scheffé theorem an unbiased function of a complete sufficient statistic is the unique UMVUE, so \(\bar X\) is a UMVUE of \(\theta \). \(\blacksquare \)

(iii). \(S=\sum X_i\sim \text {POI}(n\theta )\), so for any constant \(a>0\) \[E\left (a^S\right )=\sum _{s=0}^\infty a^s\frac {e^{-n\theta }(n\theta )^s}{s!} =e^{-n\theta }\sum _{s=0}^\infty \frac {(an\theta )^s}{s!} =e^{-n\theta }e^{an\theta }=e^{n\theta (a-1)}.\] Putting \(a=\frac {n-1}{n}\) gives \(a-1=-\frac {1}{n}\), and therefore \[E(T)=e^{n\theta \left (-\frac {1}{n}\right )}=e^{-\theta }=\tau (\theta ). \qquad \blacksquare \]

Note 4.5.1. It is tempting to estimate \(e^{-\theta }\) by \(e^{-\bar X}\), which is the maximum likelihood estimator by invariance. That estimator is biased. Part (iii) produces an unbiased one instead, and since \(T\) is a function of the complete sufficient statistic \(S\), Lehmann–Scheffé makes it the UMVUE of \(e^{-\theta }\) as well.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.