Proof. \(E(T) = \tau (\theta )\)

\[\int _A T(X)\, f_{\theta }(x)\, dx = \tau (\theta )\]

\[\frac {\partial }{\partial \theta }\int _A T(X)\, f_{\theta }(x)\, dx = \tau '(\theta )\]

\[\int _A T(X)\, \frac {\partial }{\partial \theta }f_{\theta }(x)\, dx = \tau '(\theta )\]

\[\int _A T(X)\, \frac {\frac {\partial }{\partial \theta }f_{\theta }(x)}{f_{\theta }(x)}\, \cdot \, f_{\theta }(x)\, dx = \tau '(\theta )\]

\[\int _A T(X)\, S(\theta , X)\, f_{\theta }(x) \, dx = \tau '(\theta )\] \[E\left [T(X)\, S(\theta ,X)\right ] = \tau '(\theta )\] \[\cov \left [T(X),S(\theta ,X)\right ] = E\left [T(X)\, S(\theta , X)\right ] - E\left [T(X)\right ]\, \underbrace {E\left [S(\theta ,X)\right ]}_0\] Therefore \(\, \cov \left [T(X), S(\theta , X)\right ] = \tau '(\theta )\).

By the Cauchy-Schwartz inequality \(\, \left [\cov (X,Y)\right ]^2 \leq \var (X)\, \var (Y).\) \[-1 \, \leq \, \frac {\cov (X,Y)}{\sqrt {\var (X)\, \var (Y)}}\, \leq 1.\] Therefore \[\left (\cov \left [T(X),S(\theta ,X)\right ]\right )^2 \leq \var (T(X))\, \cdot \, \var \left [S(\theta ,X)\right ]\hspace {0.3cm}\cdots \hspace {0.3cm} (2.1)\] \begin {align*} \left [\tau '(\theta )\right ]^2 & \geq \var (T(x))\, J(\theta )\\ \var (T(X))\, & \geq \, \frac {\left [\tau '(\theta )\right ]^2}{J(\theta )} \end {align*}

Equality in \((2.1)\) holds iff \(T\) and \(S(\theta ,X)\) are linear functions of each other. i.e \[S(\theta , x) = A_1(\theta ) \, T(x) + A_2(\theta )\] \[\int S(\theta , x)\, d\theta = \int \left (A_1(\theta )\, T(X) + A_2(\theta )\right )\, d\theta \] \[\int \frac {\partial }{\partial \theta }\log \, f_{\theta }(x)\, dx = C_1(\theta )\, T(X) + C_2(\theta ) + \underbrace {C_3(X)}_{\text {constant}}\] \[\log \, f_{\theta }(x) = C_1(\theta ) \, T(x) + C_2(\theta ) + C_3(x)\] \begin {align*} f_{\theta }(x) & = e^{C_1(\theta ) \, T(x) + C_2(\theta ) + C_3(x)} = \underbrace {e^{C_2(\theta )}}_{C(\theta )} \, \exp \Big \{\underbrace {C_1(\theta )}_{q(\theta )}\, \underbrace {T(x)}_{T(X)}\Big \}\, \underbrace {e^{C_3(x)}}_{h(x)} \end {align*}

which is an exponential family with natural sufficient statistic \(T(X)\). □

Solution. \(\tau (\theta ) = \theta \, \implies \, \tau '(\theta ) = 1\).
\(T(X) = \sum ^n_{i = 1}X_i\,\) is a complete sufficient statistic. Now \(E\left (\overline {X}\right ) = \theta \), therefore \(\overline {X}\) is the \(U.M.V.U.E\) of \(\theta \). \[L(\theta ) = \frac {e^{-\theta }\, \theta ^x}{x!}.\]

\[\mathcal {L}(\theta ) = -\theta + x\, \log \theta - \log x!.\]

\[S(\theta ) = -1 + \frac {x}{\theta }.\]

\[I(\theta ) = -\left (-\frac {x}{\theta ^2}\right ) = \frac {x}{\theta ^2}.\]

\[J_1(\theta ) = E\left [I(\theta ,X)\right ] = E\left (\frac {X}{\theta ^2}\right ) = \frac {\theta }{\theta ^2} = \frac {1}{\theta }.\] \[\therefore \hspace {0.3cm} J(\theta ) = \frac {n}{\theta }.\]

\[C.R.L.B = \frac {1}{J(\theta )} = \frac {\theta }{n}.\]

\[\var \left (\overline {X}\right ) = \frac {\var (X)}{n} = \frac {\theta }{n} = CRLB.\] Therefore the \(UMVUE\) attains the \(CRLB\).

OR
\(X\thicksim POI(\theta )\) is the \(REF\) distribution with natural sufficient statistic \(\sum ^n_{i = 1} X_i\). \(\overline {X}\) is also a natural sufficient statistic. Therefore \(\var (\overline {X}) = CRLB\). □

Solution.

(a)

\(f_{\theta }(x) = \theta \, x^{\theta - 1}\, = \underbrace {\theta }_{C(\theta }\, \exp \Big \{\underbrace {\theta }_{q(\theta )}\, \underbrace {x}_{T(X)}\Big \}\, \underbrace {\frac {1}{x}}_{h(x)}.\)
Let \(\eta = \theta \) \[f_{\eta }(x) = \eta \, \exp \left \{\eta \, \log x\right \}\, \frac {1}{x}.\] Therefore \(f_{\theta }(x)\) is a \(REF\) distribution and hence \(\sum ^n_{i = 1}\log x_i\,\) is a compete sufficient statistic. \[-\log X_i \thicksim EXP\left (\frac {1}{\theta }\right )\]

\[T = - \sum ^n_{i = 1} \log X_i \thicksim GAM\left (n, \frac {1}{\theta }\right ).\] by problem 1.2.4 \[E\left (X^p\right ) = \frac {\beta ^p\, \Gamma \left (\alpha + p\right )}{\Gamma (\alpha )}\]

\[E\left (T^{-1}\right ) = \frac {\left (\frac {1}{\theta }\right )^{-1}\, \Gamma (n - 1)}{\Gamma (n)} = \frac {\theta \, \Gamma (n - 1)}{(n - 1)\, \Gamma (n - 1)} = \frac {\theta }{n - 1}.\]

\[\therefore \, \, E\left [\frac {n - 1}{T} \right ] = \theta .\] Therefore \(\,\frac {n - 1}{T}\,\) is the \(UMVUE\) of \(\theta \).

Now \[L_1(\theta ) = \theta \, x^{\theta - 1}\]

\[\mathcal {L}_1(\theta ) = \log \theta + (\theta - 1)\, \log x\]

\[S_1(\theta ) = \frac {1}{\theta } + \log x\]

\[I_1(\theta ) = -\left (-\frac {1}{\theta ^2}\right ) = \frac {1}{\theta ^2}\]

\[J_1(\theta ) = \frac {1}{\theta ^2}\]

\[\therefore \hspace {0.3cm} J(\theta ) = n\, J_1(\theta ) = \frac {n}{\theta ^2}.\]

\[CRLB = \frac {\left [\tau '(\theta )\right ]^2}{J(\theta )} = \frac {1}{J(\theta )} = \frac {\theta ^2}{n}.\]

Now \[\var \left [\frac {n - 1}{T}\right ] = (n - 1)^2\, \var \left (\frac {1}{T}\right )\]

\[\var \left (\frac {1}{T}\right ) = E\left (\frac {1}{T}\right )^2 - \left (\left (\frac {1}{T}\right )\right )^2.\]

\[E\left (\frac {1}{T}\right )^2 = E\left (T^{-2}\right ) = \frac {\left (\frac {1}{\theta }\right )^{-2}\, \Gamma (n - 2)}{\Gamma (n)} = \frac {\theta ^2\, \Gamma (n - 2)}{(n - 1)\, (n - 2)\, \Gamma (n - 2)} = \frac {\theta ^2}{(n - 1)\, (n - 2)}.\]

\[\var \left (\frac {1}{T}\right ) = \frac {\theta ^2}{(n - 1)(n - 2)} - \frac {\theta ^2}{(n - 1)^2}.\] \begin {align*} \therefore \,\, \var \left (\frac {n - 1}{T}\right ) & = (n - 1)^2\, \theta ^2\, \left [\frac {1}{(n - 1)(n - 2)} - \frac {1}{(n - 1)^2}\right ] = \left (\frac {n - 1}{n - 2} - 1\right )\, \theta ^2\\ & = \left (\frac {n - 1 - n + 2}{n - 2}\right )\, \theta ^2\\ & = \frac {\theta ^2}{n - 2} > \frac {\theta ^2}{n}. \end {align*}

Therefore the variance of the \(UMVUE\) of \(\theta \) does not achieve the \(CRLB\).

OR
\(T = -\sum ^n_{i = 1} \log X_i\) is a natural sufficient statistic. But \(g(T) = \frac {n - 1}{T}\) is not a natural sufficient statistic. Therefore \(g(T) = \frac {n - 1}{T}\) does not achieve the \(CRLB\). \[\textit {efficiency} = \frac {CRLB}{\var [g(T)]} = \frac {\theta ^2/n}{\theta ^2/n - 2} = \frac {n - 2}{n}.\] i.e \(\, \frac {n - 2}{n}\longrightarrow 1, \, g(T)\) is an asymptotically efficient estimator.

(b)

\(\tau (\theta ) = \theta (1 - \theta ) = \theta - \theta ^2\) \[CRLB = \frac {\left [\tau '(\theta )\right ]^2}{J(\theta )} = \frac {\left (1 - 2\theta \right )^2}{n/\theta ^2} = \frac {\theta ^2\, (1 - 2\theta )^2}{n}.\]

Proof. Let \(S(\theta ,X)\) be the score vector, so that \(E_{\theta }\left [S\right ]=0\) and \(\var _{\theta }(S)=J(\theta )\). Differentiating the unbiasedness identity \(E_{\theta }\left [T(X)\right ]=\tau (\theta )\) with respect to \(\theta _j\) and interchanging derivative and integral, which regularity permits, \[\frac {\partial \tau }{\partial \theta _j} = \int T(x)\,\frac {\partial }{\partial \theta _j} f_{\theta }(x)\,dx = \int T(x)\, S_j(\theta ,x)\, f_{\theta }(x)\,dx = \cov _{\theta }\!\left (T, S_j\right ),\] the last equality because \(E_{\theta }\left [S_j\right ]=0\). Collecting the \(k\) components, \(\cov _{\theta }(T,S) = D(\theta )\).

Now take any fixed vector \(c\in \mathbb {R}^{k}\) and apply the Cauchy–Schwarz inequality to the scalar random variables \(T\) and \(c^{t}S\): \[\left [\cov _{\theta }\!\left (T,\, c^{t}S\right )\right ]^{2} \;\leq \; \var _{\theta }(T)\ \var _{\theta }\!\left (c^{t}S\right ),\] that is \[\left (c^{t}D\right )^{2} \;\leq \; \var _{\theta }(T)\ c^{t}J(\theta )\,c .\] Choose \(c = J^{-1}(\theta )\,D(\theta )\), which is legitimate because \(J(\theta )\) is positive definite in a regular model. Then \(c^{t}D = D^{t}J^{-1}D\) and \(c^{t}Jc = D^{t}J^{-1}JJ^{-1}D = D^{t}J^{-1}D\), so the inequality reads \[\left (D^{t}J^{-1}D\right )^{2} \;\leq \; \var _{\theta }(T)\ D^{t}J^{-1}D .\] Since \(J^{-1}\) is positive definite, \(D^{t}J^{-1}D>0\) unless \(D=0\), and dividing through gives \[\var _{\theta }(T) \;\geq \; D(\theta )^{t}\,J^{-1}(\theta )\,D(\theta ).\] If \(D=0\) the inequality is trivial. □