Solution. (a)(i). Invariance: if \(\hat \theta \) is the maximum likelihood estimator of \(\theta \) and \(\tau \) is any function of \(\theta \), then \(\tau \left (\hat \theta \right )\) is the maximum likelihood estimator of \(\tau (\theta )\). No re-maximisation is needed.

(ii). Newton’s method: when the likelihood equation \(\ell '(\theta )=0\) cannot be solved in closed form, start from an initial value \(\theta _0\) and iterate \[\theta _{k+1}=\theta _k-\frac {\ell '(\theta _k)}{\ell ''(\theta _k)}, \qquad k=0,1,2,\ldots \] until successive values agree to the accuracy required. The limit is the maximum likelihood estimate.

(b)(i). Neyman–Pearson lemma. Let \(X_1,\ldots ,X_n\) have joint density \(f(\underline {x};\theta )\) and consider the simple hypotheses \(H_0:\theta =\theta _0\) against \(H_1:\theta =\theta _1\). If \(k>0\) and \(C\) is a region of the sample space such that \[\text {(1)}\quad \frac {f\left (\underline {x};\theta _0\right )} {f\left (\underline {x};\theta _1\right )}\leq k\ \text { for }\underline {x}\in C, \qquad \text {(2)}\quad \frac {f\left (\underline {x};\theta _0\right )} {f\left (\underline {x};\theta _1\right )}\geq k\ \text { for }\underline {x}\notin C,\] \[\text {(3)}\quad P\left (\underline {X}\in C\mid \theta _0\right )=\alpha ,\] then \(C\) is the critical region of a most powerful test of size \(\alpha \).

(ii). Proof. Write \(f_0=f(\underline {x};\theta _0)\) and \(f_1=f(\underline {x};\theta _1)\). On \(C\) condition (1) says \(f_1\geq \frac {1}{k}f_0\), and off \(C\) condition (2) says \(f_1\leq \frac {1}{k}f_0\); put \(k'=\frac {1}{k}\).

Let \(A\) be the critical region of any other test with size at most \(\alpha \), so \(\int _Af_0\leq \alpha =\int _Cf_0\). Split both regions on their common part: \[\int _Cf_1-\int _Af_1 =\left (\int _{C\cap A}f_1+\int _{C\setminus A}f_1\right ) -\left (\int _{C\cap A}f_1+\int _{A\setminus C}f_1\right ) =\int _{C\setminus A}f_1-\int _{A\setminus C}f_1.\] Now \(C\setminus A\subseteq C\), where \(f_1\geq k'f_0\), and \(A\setminus C\subseteq C^c\), where \(f_1\leq k'f_0\). Therefore \[\int _Cf_1-\int _Af_1 \ \geq \ k'\int _{C\setminus A}f_0-k'\int _{A\setminus C}f_0 \ =\ k'\left (\int _Cf_0-\int _Af_0\right )\ \geq \ 0,\] the last step because \(\int _Cf_0=\alpha \geq \int _Af_0\) and \(k'>0\). Hence \[P\left (\underline {X}\in C\mid \theta _1\right )\geq P\left (\underline {X}\in A\mid \theta _1\right ),\] so no test of size \(\alpha \) or less has greater power than \(C\): the test with critical region \(C\) is most powerful of size \(\alpha \). \(\blacksquare \)

(c)(i). The transformation \(u=x\), \(v=xy\) is one-to-one from \(\{x>0,y>0\}\) onto \(\{u>0,v>0\}\), with inverse \[x=u,\qquad y=\frac {v}{u}.\] The Jacobian is \[J=\begin {vmatrix} \frac {\partial x}{\partial u} & \frac {\partial x}{\partial v}\\[4pt] \frac {\partial y}{\partial u} & \frac {\partial y}{\partial v} \end {vmatrix} =\begin {vmatrix} 1 & 0\\[4pt] -\frac {v}{u^2} & \frac {1}{u} \end {vmatrix} =\frac {1}{u},\] so \[f_{U,V}(u,v)=f\left (u,\frac {v}{u}\right )\left |J\right | =u\,e^{-u\left (\frac {v}{u}+1\right )}\cdot \frac {1}{u} =e^{-(u+v)},\qquad u>0,\ v>0.\]

(ii). The joint density factorises as \(e^{-u}\cdot e^{-v}\) over a rectangular support, so \(U\) and \(V\) are independent, each \(\text {EXP}(1)\). Integrating out \(u\), \[f_V(v)=\int _0^\infty e^{-(u+v)}\,du=e^{-v}\int _0^\infty e^{-u}\,du=e^{-v}, \qquad v>0.\] \[\therefore \quad V=XY\sim \text {EXP}(1).\]