Solution. Recognise the density first: it is \(\text {GAM}(\alpha =2,\ \text {rate }\theta )\), which saves deriving moments from scratch.

(a) Method of moments. The first population moment is \[E(X)=\int _0^\infty x\cdot \theta ^2xe^{-\theta x}\,dx =\theta ^2\int _0^\infty x^2e^{-\theta x}\,dx=\theta ^2\cdot \frac {2}{\theta ^3} =\frac {2}{\theta }.\] Equating it to the first sample moment \(\bar X\), \[\frac {2}{\theta }=\bar X\quad \implies \quad \tilde \theta =\frac {2}{\bar X} =\frac {2n}{\sum _{i=1}^n X_i}.\]

(b) Sufficient statistic. The likelihood is \[L(\theta )=\prod _{i=1}^n\theta ^2x_ie^{-\theta x_i} =\theta ^{2n}e^{-\theta \sum x_i}\cdot \prod _{i=1}^n x_i.\] This is already in the factorisation form \(g\!\left (\sum x_i;\theta \right )h(\mathbf {x})\), with \[g\!\left (\textstyle \sum x_i;\theta \right )=\theta ^{2n}e^{-\theta \sum x_i}, \qquad h(\mathbf {x})=\prod x_i,\] and \(h\) free of \(\theta \). By the factorisation criterion \[T=\sum _{i=1}^n X_i\ \text { is sufficient for }\theta .\]

(c) Maximum likelihood. Take logarithms: \[\ell (\theta )=2n\ln \theta +\sum \ln x_i-\theta \sum x_i.\] \[\frac {d\ell }{d\theta }=\frac {2n}{\theta }-\sum x_i=0 \quad \implies \quad \hat \theta =\frac {2n}{\sum x_i}=\frac {2}{\bar X}.\] It is a maximum because \[\frac {d^2\ell }{d\theta ^2}=-\frac {2n}{\theta ^2}<0\quad \text {for every }\theta >0.\]

(d) Information function. For a single observation, \[\ln f=2\ln \theta +\ln x-\theta x,\qquad \frac {\partial \ln f}{\partial \theta }=\frac {2}{\theta }-x,\qquad \frac {\partial ^2\ln f}{\partial \theta ^2}=-\frac {2}{\theta ^2}.\] The second derivative is already free of \(x\), so taking the expectation changes nothing: \[I_1(\theta )=-E\!\left [\frac {\partial ^2\ln f}{\partial \theta ^2}\right ] =\frac {2}{\theta ^2},\qquad \text {and for the sample}\qquad I_n(\theta )=\frac {2n}{\theta ^2}.\]

Solution. This is the negative binomial in its “trials until the third success” form: \(X\) counts the trials needed to obtain \(3\) successes, each of probability \(\theta \). Its mean is therefore \[E(X)=\frac {3}{\theta }.\]

(a) Method of moments. \[\frac {3}{\theta }=\bar X\quad \implies \quad \tilde \theta =\frac {3}{\bar X} =\frac {3n}{\sum X_i}.\]

(b) Sufficient statistic. The likelihood is \[L(\theta )=\prod _{i=1}^n\binom {x_i-1}{2}\theta ^3(1-\theta )^{x_i-3} =\underbrace {\theta ^{3n}(1-\theta )^{\sum x_i-3n}}_{g\left (\sum x_i;\theta \right )} \cdot \underbrace {\prod _{i=1}^n\binom {x_i-1}{2}}_{h(\mathbf {x})}.\] By the factorisation criterion, \[T=\sum _{i=1}^n X_i\ \text { is sufficient for }\theta .\]

(c) Maximum likelihood. \[\ell (\theta )=\sum \ln \binom {x_i-1}{2}+3n\ln \theta +\left (\sum x_i-3n\right )\ln (1-\theta ),\] \[\frac {d\ell }{d\theta }=\frac {3n}{\theta }-\frac {\sum x_i-3n}{1-\theta }=0.\] Multiplying through by \(\theta (1-\theta )\), \[3n(1-\theta )=\theta \left (\sum x_i-3n\right ) \ \implies \ 3n-3n\theta =\theta \sum x_i-3n\theta \ \implies \ 3n=\theta \sum x_i,\] the \(3n\theta \) terms cancelling on both sides, so \[\hat \theta =\frac {3n}{\sum x_i}=\frac {3}{\bar X}.\]

(d) Fisher information. For one observation the terms in \(\theta \) are \[\ln f=\text {constant}+3\ln \theta +(x-3)\ln (1-\theta ),\] \[\frac {\partial \ln f}{\partial \theta }=\frac {3}{\theta }-\frac {x-3}{1-\theta }, \qquad \frac {\partial ^2\ln f}{\partial \theta ^2}=-\frac {3}{\theta ^2}-\frac {x-3}{(1-\theta )^2}.\] Now the second derivative does involve \(x\), so the expectation must be taken. Since \(E(X)=\frac {3}{\theta }\), \[E(X-3)=\frac {3}{\theta }-3=\frac {3(1-\theta )}{\theta },\] and therefore \[I_1(\theta )=-E\!\left [\frac {\partial ^2\ln f}{\partial \theta ^2}\right ] =\frac {3}{\theta ^2}+\frac {1}{(1-\theta )^2}\cdot \frac {3(1-\theta )}{\theta } =\frac {3}{\theta ^2}+\frac {3}{\theta (1-\theta )}.\] Over the common denominator \(\theta ^2(1-\theta )\), \[I_1(\theta )=\frac {3(1-\theta )+3\theta }{\theta ^2(1-\theta )} =\frac {3}{\theta ^2(1-\theta )},\qquad I_n(\theta )=\frac {3n}{\theta ^2(1-\theta )}.\]

Solution. This is the lognormal density. The one observation worth making before any algebra is that \(W=\log Y\sim N(\mu ,\sigma ^2)\): everything then follows from the normal case.

(a) Sufficient statistic. Writing \(w_i=\log y_i\), \[L(\theta )=\left (2\pi \sigma ^2\right )^{-n/2}\left (\prod y_i\right )^{-1} \exp \!\left (-\frac {1}{2\sigma ^2}\sum (w_i-\mu )^2\right ).\] Expanding the square, \(\sum (w_i-\mu )^2=\sum w_i^2-2\mu \sum w_i+n\mu ^2\), so \[L(\theta )=\underbrace {\left (2\pi \sigma ^2\right )^{-n/2} \exp \!\left (-\frac {\sum w_i^2-2\mu \sum w_i+n\mu ^2}{2\sigma ^2}\right )} _{g\left (\sum w_i,\ \sum w_i^2;\ \theta \right )} \cdot \underbrace {\left (\prod y_i\right )^{-1}}_{h(\mathbf {y})}.\] \[\therefore \quad T=\left (\sum _{i=1}^n\log Y_i,\ \sum _{i=1}^n(\log Y_i)^2\right ) \text { is jointly sufficient for }\theta .\]

(b) Maximum likelihood. These are the normal estimators applied to the \(w_i\): \[\hat \mu =\frac {1}{n}\sum _{i=1}^n\log Y_i,\qquad \hat \sigma ^2=\frac {1}{n}\sum _{i=1}^n\left (\log Y_i-\hat \mu \right )^2.\]

(c) Information matrix. Again as for the normal in \(w\), \[I(\theta )=n\begin {pmatrix}\frac {1}{\sigma ^2} & 0\\[3pt] 0 & \frac {1}{2\sigma ^4}\end {pmatrix}.\] The off-diagonal entries are zero, so \(\hat \mu \) and \(\hat \sigma ^2\) are asymptotically independent.

(d) MLE of \(E(Y)\). For the lognormal, \[E(Y)=e^{\mu +\frac {\sigma ^2}{2}}.\] By the invariance property of maximum likelihood — the MLE of \(\tau (\theta )\) is \(\tau \) evaluated at the MLE — \[\widehat {E(Y)}=e^{\hat \mu +\frac {\hat \sigma ^2}{2}}.\]

Solution. (a). \(E(X)=\frac {\theta }{2}\), so equating to \(\bar X\) gives \[T_2=2\bar X,\qquad E(T_2)=2\cdot \frac {\theta }{2}=\theta .\] So \(T_2\) is unbiased.

(b). The largest order statistic has density \[g(x)=\frac {nx^{n-1}}{\theta ^n},\qquad 0<x<\theta ,\] so \[E(T_1)=\int _0^\theta x\cdot \frac {nx^{n-1}}{\theta ^n}\,dx=\frac {n\theta }{n+1} \ \longrightarrow \ \theta \quad \text {as }n\to \infty .\] It is biased for every finite \(n\) — always under-estimating, as it must, since the largest observation cannot exceed \(\theta \) — but asymptotically unbiased.

(c). Also \[E\left (T_1^2\right )=\int _0^\theta x^2\cdot \frac {nx^{n-1}}{\theta ^n}\,dx =\frac {n\theta ^2}{n+2},\] so, using the result of (d) below, \[\text {MSE}(T_1)=\frac {2\theta ^2}{(n+1)(n+2)}\ \longrightarrow \ 0 \quad \text {as }n\to \infty .\] An estimator whose mean squared error tends to zero converges in probability to \(\theta \), so \(T_1\) is consistent.

(d). Since \(T_1\) is biased, use \(\text {MSE}=E\left (T_1^2\right )-2\theta E(T_1)+\theta ^2\): \[\text {MSE}(T_1)=\frac {n\theta ^2}{n+2}-\frac {2n\theta ^2}{n+1}+\theta ^2 =\theta ^2\cdot \frac {n(n+1)-2n(n+2)+(n+1)(n+2)}{(n+1)(n+2)}.\] The numerator is \[\left (n^2+n\right )-\left (2n^2+4n\right )+\left (n^2+3n+2\right )=2,\] the \(n^2\) and \(n\) terms cancelling completely, so \[\text {MSE}(T_1)=\frac {2\theta ^2}{(n+1)(n+2)}.\]

(e). \(T_2\) is unbiased, so its MSE is its variance. With \(\text {Var}(X)=\frac {\theta ^2}{12}\), \[\text {MSE}(T_2)=\text {Var}\left (2\bar X\right )=4\cdot \frac {\theta ^2}{12n} =\frac {\theta ^2}{3n}.\]

(f). \[e(T_1,T_2)=\frac {\text {MSE}(T_2)}{\text {MSE}(T_1)} =\frac {\frac {\theta ^2}{3n}}{\frac {2\theta ^2}{(n+1)(n+2)}} =\frac {(n+1)(n+2)}{6n}.\]

Solution. (a). \[E(X)=\int _\theta ^\infty x\cdot \frac {2\theta ^2}{x^3}\,dx =2\theta ^2\int _\theta ^\infty x^{-2}dx=2\theta ^2\cdot \frac {1}{\theta }=2\theta .\] Setting \(2\theta =\bar X\), \[T_1=\frac {\bar X}{2},\qquad E(T_1)=\frac {2\theta }{2}=\theta ,\] so \(T_1\) is unbiased.

(b). The likelihood is \[L(\theta )=\frac {2^n\theta ^{2n}}{\prod x_i^3},\qquad \text {valid only while } \theta <x_i\text { for every }i.\] Differentiating is useless here: \(L\) is increasing in \(\theta \), so it is made as large as the constraint permits. The binding constraint is \(\theta \leq \min _i x_i\), hence \[T_2=X_{(1)}=\min _i X_i.\]

(c). The smallest order statistic has density \[g(x)=n\left [1-F(x)\right ]^{n-1}f(x) =n\left (\frac {\theta ^2}{x^2}\right )^{n-1}\cdot \frac {2\theta ^2}{x^3} =\frac {2n\theta ^{2n}}{x^{2n+1}},\qquad x>\theta ,\] using \(F(x)=1-\frac {\theta ^2}{x^2}\). Then \[E(T_2)=\int _\theta ^\infty x\cdot \frac {2n\theta ^{2n}}{x^{2n+1}}\,dx =2n\theta ^{2n}\int _\theta ^\infty x^{-2n}dx =2n\theta ^{2n}\cdot \frac {\theta ^{1-2n}}{2n-1}=\frac {2n\theta }{2n-1}.\] As \(n\to \infty \) this tends to \(\theta \), so \(T_2\) is asymptotically unbiased. Note it over-estimates, the mirror image of the uniform case, because the smallest observation must exceed \(\theta \).

(d). By the same integral with \(x^2\), \[E\left (T_2^2\right )=2n\theta ^{2n}\int _\theta ^\infty x^{1-2n}dx =\frac {2n\theta ^2}{2n-2}=\frac {n\theta ^2}{n-1}.\] Hence \[\text {MSE}(T_2)=\frac {n\theta ^2}{n-1}-\frac {4n\theta ^2}{2n-1}+\theta ^2 =\frac {\theta ^2}{(n-1)(2n-1)}\ \longrightarrow \ 0,\] so \(T_2\) is consistent.

Solution. (a). \(\ell (\theta )=n\ln \theta -\theta \sum x_i\), so \[\frac {d\ell }{d\theta }=\frac {n}{\theta }-\sum x_i=0\quad \implies \quad \hat \theta =\frac {n}{\sum X_i}=\frac {1}{\bar X},\] a maximum since \(\frac {d^2\ell }{d\theta ^2}=-\frac {n}{\theta ^2}<0\).

(b). By invariance, the MLE of \(\frac {1}{\theta }\) is \(\frac {1}{\hat \theta }=\bar X\). Since \(E(X)=\frac {1}{\theta }\), \[E\left (\bar X\right )=\frac {1}{\theta },\] so it is unbiased.

(c). The exponential family here has \(T=\sum X_i\) as a complete sufficient statistic for \(\theta \). Now \(\bar X=\frac {T}{n}\) is a function of \(T\) alone and is unbiased for \(\frac {1}{\theta }\). By the Lehmann–Scheffé theorem an unbiased function of a complete sufficient statistic is the unique UMVUE, so \[\bar X\ \text { is the U.M.V.U.E. of }\ \frac {1}{\theta }.\]

(d) and (e). By the weak law of large numbers \(\bar X\xrightarrow {P} E(X)=\frac {1}{\theta }\), so the estimator in (b) is consistent. Since \(g(u)=\frac {1}{u}\) is continuous at \(u=\frac {1}{\theta }\neq 0\), the continuous mapping theorem gives \[\hat \theta =\frac {1}{\bar X}\xrightarrow {P}\theta ,\] so the estimator in (a) is consistent as well.

Solution. (a). \(\ell (\theta )=-n\theta +\ln \theta \sum x_i-\sum \ln x_i!\), so \[\frac {d\ell }{d\theta }=-n+\frac {\sum x_i}{\theta }=0\quad \implies \quad \hat \theta =\bar X.\]

(b). From \(\ln f=-\theta +x\ln \theta -\ln x!\), \[\frac {\partial ^2\ln f}{\partial \theta ^2}=-\frac {x}{\theta ^2},\qquad I_1(\theta )=-E\!\left [-\frac {X}{\theta ^2}\right ]=\frac {\theta }{\theta ^2} =\frac {1}{\theta }.\] \[\therefore \quad \text {CRLB}=\frac {1}{nI_1(\theta )}=\frac {\theta }{n}.\]

(c). \(\bar X\) is unbiased for \(\theta \) and \(\text {Var}\left (\bar X\right )=\frac {\theta }{n}\), which equals the bound. An unbiased estimator attaining the CRLB has the smallest possible variance, so \(\bar X\) is the U.M.V.U.E. of \(\theta \).

(d). For \(\tau (\theta )=e^{-\theta }\) we have \(\tau '(\theta )=-e^{-\theta }\), and the bound for a function of \(\theta \) is \[\text {CRLB}=\frac {\left [\tau '(\theta )\right ]^2}{nI_1(\theta )} =\frac {e^{-2\theta }}{\frac {n}{\theta }}=\frac {\theta e^{-2\theta }}{n}.\]

(e). Since \(\sum X_i\sim \text {POI}(n\theta )\), its probability generating function is \(E\left (s^{\sum X_i}\right )=e^{n\theta (s-1)}\). Putting \(s=\frac {n-1}{n}\), \[E(T)=e^{n\theta \left (\frac {n-1}{n}-1\right )} =e^{n\theta \left (-\frac {1}{n}\right )}=e^{-\theta },\] so \(T\) is unbiased for \(e^{-\theta }\).

(f). Using the same generating function with \(s^2\), \[E\left (T^2\right )=e^{n\theta \left (\left (\frac {n-1}{n}\right )^2-1\right )} =e^{n\theta \cdot \frac {1-2n}{n^2}}=e^{-2\theta +\frac {\theta }{n}},\] \[\text {Var}(T)=e^{-2\theta +\frac {\theta }{n}}-e^{-2\theta } =e^{-2\theta }\left (e^{\theta /n}-1\right ).\] Therefore \[\text {eff}(T)=\frac {\text {CRLB}}{\text {Var}(T)} =\frac {\frac {\theta e^{-2\theta }}{n}}{e^{-2\theta }\left (e^{\theta /n}-1\right )} =\frac {\theta }{n\left (e^{\theta /n}-1\right )}.\]

Solution. This is \(\text {GAM}(\alpha =\theta ,\ \beta =2)\): shape unknown, scale known and equal to \(2\). So \(E(X)=2\theta \) and \(\text {Var}(X)=4\theta \).

(a). Equating \(E(X)=2\theta \) to \(\bar X\), \[T_1=\frac {\bar X}{2}.\]

(b). The log-likelihood is \[\ell (\theta )=-n\theta \ln 2-n\ln \Gamma (\theta )+(\theta -1)\sum \ln x_i -\frac {1}{2}\sum x_i,\] \[\frac {d\ell }{d\theta }=-n\ln 2-n\psi (\theta )+\sum \ln x_i=0,\] where \(\psi =\frac {\Gamma '}{\Gamma }\) is the digamma function. So \(T_2\) solves \[\psi (\theta )=\frac {1}{n}\sum _{i=1}^n\ln x_i-\ln 2.\] There is no closed form: \(\psi \) cannot be inverted in elementary functions, so \(T_2\) must be obtained numerically — by Newton’s method, as in Section 2.4.

(c). \(T_1\) is unbiased, since \(E\left (\frac {\bar X}{2}\right )=\frac {2\theta }{2}=\theta \), so its MSE is its variance: \[\text {MSE}(T_1)=\text {Var}\!\left (\frac {\bar X}{2}\right ) =\frac {1}{4}\cdot \frac {\text {Var}(X)}{n}=\frac {1}{4}\cdot \frac {4\theta }{n} =\frac {\theta }{n}.\]

Solution. (i). The likelihood is \[L=\frac {\lambda ^{n\alpha }}{\Gamma (\alpha )^n} \left (\prod x_i\right )^{\alpha -1}e^{-\lambda \sum x_i}.\] Both \(\prod x_i\) and \(\sum x_i\) appear, and neither can be dispensed with while \(\alpha \) is unknown, so \[T=\left (\sum _{i=1}^nX_i,\ \prod _{i=1}^nX_i\right ) \quad \text {--- equivalently }\left (\sum X_i,\ \sum \ln X_i\right )\text { ---}\] is jointly sufficient for \((\alpha ,\lambda )\).

(ii). With \(\alpha =4\) the density is \(f(x)=\frac {\lambda ^4x^3e^{-\lambda x}}{6}\), so \[\ell (\lambda )=4n\ln \lambda +3\sum \ln x_i-\lambda \sum x_i-n\ln 6,\] \[\frac {d\ell }{d\lambda }=\frac {4n}{\lambda }-\sum x_i=0 \quad \implies \quad \hat \lambda =\frac {4n}{\sum X_i}=\frac {4}{\bar X}.\]

(iii). From \(\ln f=4\ln \lambda +3\ln x-\lambda x-\ln 6\), \[\frac {\partial ^2\ln f}{\partial \lambda ^2}=-\frac {4}{\lambda ^2},\qquad I_1(\lambda )=\frac {4}{\lambda ^2},\qquad \text {CRLB}=\frac {1}{nI_1(\lambda )}=\frac {\lambda ^2}{4n}.\]

(iv). With \(\alpha \) known, this is a one-parameter exponential family and \(T=\sum X_i\) is a complete sufficient statistic. Since each \(X_i\sim \text {GAM}(4,\lambda )\), we have \(T\sim \text {GAM}(4n,\lambda )\), and \[E\!\left (\frac {1}{T}\right ) =\int _0^\infty \frac {1}{t}\cdot \frac {\lambda ^{4n}t^{4n-1}e^{-\lambda t}} {\Gamma (4n)}\,dt=\frac {\lambda \,\Gamma (4n-1)}{\Gamma (4n)}=\frac {\lambda }{4n-1}.\] So the MLE \(\frac {4n}{T}\) is biased, with \(E\!\left (\frac {4n}{T}\right )=\frac {4n\lambda }{4n-1}\). Correcting the constant, \[E\!\left (\frac {4n-1}{T}\right )=\lambda ,\] and \(\frac {4n-1}{T}\) is a function of the complete sufficient statistic alone. By the Lehmann–Scheffé theorem, \[\text {UMVUE of }\lambda \ =\ \frac {4n-1}{\sum _{i=1}^nX_i}.\]

Solution. (i). The likelihood is \[L=\frac {\lambda ^{\sum x_i}e^{-n\lambda }}{\prod x_i!} =\underbrace {\lambda ^{\sum x_i}e^{-n\lambda }}_{g\left (\sum x_i;\lambda \right )} \cdot \underbrace {\frac {1}{\prod x_i!}}_{h(\mathbf {x})},\] so \(T=\sum _{i=1}^nX_i\) is sufficient for \(\lambda \).

(ii). The MLE is \(\hat \lambda =\bar X\), with \(\text {Var}\left (\bar X\right )=\frac {\lambda }{n}\). From \(\frac {\partial ^2\ln f}{\partial \lambda ^2}=-\frac {x}{\lambda ^2}\) we get \(I_1(\lambda )=\frac {1}{\lambda }\) and \(\text {CRLB}=\frac {\lambda }{n}\). Hence \[\text {eff}\left (\hat \lambda \right ) =\frac {\text {CRLB}}{\text {Var}\left (\bar X\right )} =\frac {\lambda /n}{\lambda /n}=1.\] The MLE is efficient — it attains the bound exactly.

(iii). \(W=X_1\) is also unbiased for \(\lambda \), since \(E(X_1)=\lambda \), but it throws away \(n-1\) of the observations. With \(\text {Var}(W)=\lambda \), \[\text {RE}\left (\hat \lambda ,W\right ) =\frac {\text {Var}(W)}{\text {Var}\left (\hat \lambda \right )} =\frac {\lambda }{\lambda /n}=n.\] The sample mean is \(n\) times as efficient as a single observation.

Solution. (i). Write the likelihood with the product in the exponent: \[L=\frac {\theta ^n}{\prod (1+x_i)^{\theta +1}} =\theta ^n\exp \!\left (-(\theta +1)\sum \ln (1+x_i)\right ).\] This depends on the sample only through \(\sum \ln (1+x_i)\), so \[T=\sum _{i=1}^n\ln \left (1+X_i\right )\ \text { is sufficient for }\theta .\]

(ii). From \(\ln f=\ln \theta -(\theta +1)\ln (1+x)\), \[\frac {\partial \ln f}{\partial \theta }=\frac {1}{\theta }-\ln (1+x),\qquad \frac {\partial ^2\ln f}{\partial \theta ^2}=-\frac {1}{\theta ^2}.\] The second derivative is free of \(x\), so \[I_1(\theta )=\frac {1}{\theta ^2},\qquad \text {CRLB}=\frac {1}{nI_1(\theta )}=\frac {\theta ^2}{n}.\]

Solution. This is the Bernoulli distribution, \(E(X)=\theta \) and \(\text {Var}(X)=\theta (1-\theta )\).

(a). The likelihood factorises as \[L(\theta )=\prod _{i=1}^n\theta ^{x_i}(1-\theta )^{1-x_i} =\theta ^{\sum x_i}(1-\theta )^{n-\sum x_i} =\underbrace {\left (\frac {\theta }{1-\theta }\right )^{\sum x_i} (1-\theta )^n}_{g\left (\sum x_i,\ \theta \right )}\cdot \underbrace {1}_{h(\underline {x})},\] so by the factorisation criterion \[S=\sum _{i=1}^nX_i\] is sufficient for \(\theta \).

(b). Taking logarithms, \[\ell (\theta )=\left (\sum x_i\right )\ln \theta +\left (n-\sum x_i\right )\ln (1-\theta ),\] \[\ell '(\theta )=\frac {\sum x_i}{\theta }-\frac {n-\sum x_i}{1-\theta }=0 \quad \implies \quad (1-\theta )\sum x_i=\theta \left (n-\sum x_i\right ),\] which gives \(\sum x_i=n\theta \) and hence \[\hat \theta =\frac {\sum x_i}{n}=\bar X,\] with \(\ell ''(\theta )<0\) throughout, confirming a maximum.

(c). Written in exponential family form, \[f(x,\theta )=(1-\theta )\exp \left [x\ln \frac {\theta }{1-\theta }\right ],\] so \(S=\sum X_i\) is a complete sufficient statistic. Now \(\bar X=S/n\) is a function of \(S\) alone and is unbiased, \(E\left (\bar X\right )=\frac {1}{n}\cdot n\theta =\theta \). By the Lehmann–Scheffé theorem, \[\therefore \quad T=\bar X\ \textbf {is the UMVUE of }\theta .\]

(d). \(\bar X\stackrel {p}{\longrightarrow }\theta \) by the weak law of large numbers, and squaring is a continuous function, so \(\bar X^2\stackrel {p}{\longrightarrow }\theta ^2\). Directly: \[E\left (\bar X^2\right )=\text {Var}\left (\bar X\right )+\left [E\bar X\right ]^2 =\frac {\theta (1-\theta )}{n}+\theta ^2\ \longrightarrow \ \theta ^2,\] and the variance of \(\bar X^2\) tends to \(0\) as well, so the mean square error vanishes. \[\therefore \quad T_1=\bar X^2\ \textbf {is consistent for }\theta ^2.\]

Solution. This is the exponential distribution shifted to start at \(\theta \): if \(Z\sim \text {EXP}(1)\) then \(X=\theta +Z\).

(a). \(E(X)=\theta +E(Z)=\theta +1\). Equating to the first sample moment, \[\bar X=\theta +1\qquad \implies \qquad \tilde \theta =\bar X-1.\]

(b). The likelihood needs care, because \(\theta \) appears in the support: \[L(\theta )=\prod _{i=1}^ne^{-(x_i-\theta )}=e^{-\sum x_i}e^{n\theta } \qquad \text {provided }x_i>\theta \ \text {for every }i.\] As a function of \(\theta \) this is strictly increasing, so it is made as large as possible by taking \(\theta \) as large as the constraint allows. The constraint is \(\theta <\min _ix_i\), hence \[\hat \theta =X_{(1)}=\min \left (X_1,\ldots ,X_n\right ).\] Differentiating would have given \(\ell '(\theta )=n\neq 0\) and no solution at all — the maximum is at the boundary, not at a turning point.

(c). First find \(E\left (X_{(1)}\right )\). The minimum exceeds \(m\) only if every observation does, so for \(m>\theta \) \[P\left (X_{(1)}>m\right )=\left [e^{-(m-\theta )}\right ]^n=e^{-n(m-\theta )}, \qquad f_{X_{(1)}}(m)=ne^{-n(m-\theta )}.\] So \(X_{(1)}\) is itself shifted exponential, with mean \(\frac 1n\) above \(\theta \): \[E\left (X_{(1)}\right )=\int _\theta ^\infty m\,ne^{-n(m-\theta )}dm =\theta +\frac {1}{n}.\] Therefore \[E(T)=E\left (X_{(1)}-\frac {2}{n}\right )=\theta +\frac {1}{n}-\frac {2}{n} =\theta -\frac {1}{n},\] \[\lim _{n\to \infty }E(T)=\theta ,\] \[\therefore \quad T\ \textbf {is asymptotically unbiased for }\theta ,\] though it is biased by \(-\frac 1n\) at every finite \(n\).