4.2 Question 2

Problem 4.2.1. [2013 examination]

(a).
Define the following:
(i).
the \(t\)-distribution with \(n\) degrees of freedom (no density expression);
(ii).
a most powerful test of size \(\alpha \).
(b).
Suppose that \(X\) and \(Y\) have joint probability density function \[f(x,y)=\frac {2}{3}(x+1),\qquad 0<x<1,\quad 0<y<1.\]
(i).
Find the joint c.d.f. of \(X\) and \(Y\).
(ii).
Find the marginal p.d.f.s \(f_X(x)\) and \(f_Y(y)\) using the joint c.d.f. in (i).
(iii).
Determine whether \(X\) and \(Y\) are independent.
(c).
Let \(X_1,\ldots ,X_n\) be a random sample from \(f(x;\theta )=\theta e^{-\theta x}\), \(x>0\).
(i).
Find the method of moments estimator of \(\theta \).
(ii).
Find the maximum likelihood estimator of \(\theta \).
(iii).
Determine whether \(T=\bar X^2\) is a consistent estimator of \(\tau (\theta )=\frac {1}{\theta ^2}\).

Show solution

Solution. (a)(i). If \(Z\sim N(0,1)\) and \(V\sim \chi ^2(n)\) are independent, then \[T=\frac {Z}{\sqrt {V/n}}\] has the \(t\)-distribution with \(n\) degrees of freedom.

(ii). A test of \(H_0\) against a simple \(H_1\) with critical region \(C\) is most powerful of size \(\alpha \) if its size is \(\alpha \) and, for every other test with critical region \(A\) of size at most \(\alpha \), \[P\left (\underline {X}\in C\mid H_1\right )\geq P\left (\underline {X}\in A\mid H_1\right ).\]

(b)(i). For \(0<x<1\) and \(0<y<1\), \[F(x,y)=\int _0^x\int _0^y\frac {2}{3}(u+1)\,dv\,du =\frac {2}{3}\,y\int _0^x(u+1)\,du =\frac {2}{3}\,y\left [\frac {u^2}{2}+u\right ]_0^x =\frac {y\left (x^2+2x\right )}{3}.\] Completing the definition over the plane, \(F(x,y)=0\) if \(x\leq 0\) or \(y\leq 0\); \(F(x,y)=\frac {x^2+2x}{3}\) for \(0<x<1,\ y\geq 1\); \(F(x,y)=y\) for \(x\geq 1,\ 0<y<1\); and \(F(x,y)=1\) for \(x\geq 1,\ y\geq 1\).

(ii). The marginal c.d.f.s are the limits of \(F\) in the other variable: \[F_X(x)=F(x,1)=\frac {x^2+2x}{3},\qquad F_Y(y)=F(1,y)=\frac {y(1+2)}{3}=y.\] Differentiating, \[f_X(x)=\frac {2x+2}{3}=\frac {2}{3}(x+1),\quad 0<x<1;\qquad f_Y(y)=1,\quad 0<y<1.\] So \(Y\) is uniform on \((0,1)\).

(iii). \[F_X(x)\,F_Y(y)=\frac {x^2+2x}{3}\cdot y=F(x,y)\] for every \((x,y)\), so \(X\) and \(Y\) are independent. The same conclusion follows from the density: \(f(x,y)=\frac {2}{3}(x+1)\cdot 1=f_X(x)f_Y(y)\), and the support is a rectangle.

(c)(i). Here \(E(X)=\frac {1}{\theta }\). Equating the first population moment to the first sample moment, \[\frac {1}{\theta }=\bar X\qquad \implies \qquad \tilde \theta =\frac {1}{\bar X}.\]

(ii). \(L(\theta )=\theta ^ne^{-\theta \sum x_i}\), so \[\ell (\theta )=n\ln \theta -\theta \sum x_i,\qquad \ell '(\theta )=\frac {n}{\theta }-\sum x_i=0 \quad \implies \quad \hat \theta =\frac {n}{\sum x_i}=\frac {1}{\bar X},\] and \(\ell ''(\theta )=-\frac {n}{\theta ^2}<0\) confirms a maximum. Here the two methods agree.

(iii). Write \(S=\sum X_i\sim \text {GAM}\left (n,\frac {1}{\theta }\right )\), for which \(E\left (S^j\right )=\frac {\Gamma (n+j)}{\Gamma (n)\,\theta ^j}\). With \(\bar X=S/n\), \[E\left (\bar X^2\right )=\frac {(n+1)n}{n^2\theta ^2}=\frac {n+1}{n\theta ^2} \ \longrightarrow \ \frac {1}{\theta ^2},\] so \(T\) is asymptotically unbiased, and \[\text {Var}\left (\bar X^2\right ) =\frac {(n+3)(n+2)(n+1)n-n^2(n+1)^2}{n^4\theta ^4} =\frac {(n+1)(4n+6)}{n^3\theta ^4}\ \longrightarrow \ 0.\] Mean square error therefore tends to zero, so by Chebyshev’s inequality \(T\stackrel {p}{\longrightarrow }\frac {1}{\theta ^2}\): \[\therefore \quad T=\bar X^2\ \text {\textbf {is a consistent estimator of}}\ \frac {1}{\theta ^2}.\]

Note 4.2.1. A shorter argument: \(\bar X\stackrel {p}{\to }\frac {1}{\theta }\) by the weak law of large numbers, and squaring is continuous, so \(\bar X^2\stackrel {p}{\to } \frac {1}{\theta ^2}\). The longer computation is worth doing once because it shows \(T\) is biased at every finite \(n\) — consistency and unbiasedness are different properties, and this estimator has one without the other.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.