Proof. Differentiate \(\int T(x)f_\theta (x)\,dx = \tau (\theta )\) under the integral sign: \[\tau '(\theta ) = \int T(x)\frac {\partial f_\theta (x)}{\partial \theta }\,dx = \int T(x)\,S(\theta ; x)\,f_\theta (x)\,dx = E_\theta \left [T\,S(\theta ; X)\right ].\] Since \(E_\theta (S) = 0\) (theorem 2.2.1), this is \(\cov _\theta (T, S) = \tau '(\theta )\). By the Cauchy–Schwarz inequality, \begin {equation} \label {eq:cauchyschwarz} \left [\cov (T, S)\right ]^2\le \var (T)\var (S), \end {equation} that is, \(\left [\tau '(\theta )\right ]^2\le \var _\theta (T)J(\theta )\), which is the inequality.

Equality in (2.3) holds if and only if \(T\) and \(S\) are linearly related with probability one: \(S(\theta ; x) = A_1(\theta )T(x) + A_2(\theta )\). Integrating with respect to \(\theta \), \[\ln f_\theta (x) = C_1(\theta )T(x) + C_2(\theta ) + C_3(x),\] where the “constant” of integration \(C_3\) may depend on \(x\). Hence \[f_\theta (x) = e^{C_2(\theta )}\exp \left \{C_1(\theta )T(x)\right \}e^{C_3(x)},\] an exponential family with natural sufficient statistic \(T\). Conversely, in such a family the score is linear in \(T\), and equality holds. □

Solution. \(T = \sum X_i\) is complete and sufficient (example 1.6.5) and \(E(\bar X) = \theta \), so \(\bar X\) is the UMVUE. From example 2.1.3, \(J(\theta ) = \frac n\theta \), so the CRLB is \(\frac \theta n = \var (\bar X)\). Alternatively, the Poisson is an exponential family with natural sufficient statistic \(\sum X_i\), of which \(\bar X\) is a linear function, so equality holds by theorem 2.3.1.

Solution.

(a)

\(f_\theta (x) = \theta \exp \{\theta \ln x\}\cdot \frac 1x\) is a REF with \(\eta = \theta >0\) and natural statistic \(\ln x\). So \(\sum \ln X_i\), or equivalently \(T = -\sum \ln X_i\), is complete and sufficient. Since \(P(-\ln X>y) = P(X<e^{-y}) = e^{-\theta y}\), each \(-\ln X_i\) is exponential with mean \(\frac 1\theta \), and \(T\sim \text {GAM}\left (n, \frac 1\theta \right )\). By problem 1.2.1, \[E(T^{-1}) = \frac {\theta \,\Gamma (n - 1)}{\Gamma (n)} = \frac {\theta }{n - 1},\qquad E(T^{-2}) = \frac {\theta ^2\Gamma (n - 2)}{\Gamma (n)} = \frac {\theta ^2}{(n - 1)(n - 2)} .\] So \(\frac {n - 1}{T}\) is unbiased, and being a function of \(T\) it is the UMVUE. Its variance is \[(n - 1)^2\left [\frac {\theta ^2}{(n - 1)(n - 2)} - \frac {\theta ^2}{(n - 1)^2}\right ] = \theta ^2\left [\frac {n - 1}{n - 2} - 1\right ] = \frac {\theta ^2}{n - 2} .\] For one observation \(\mathcal {L}_1 = \ln \theta + (\theta - 1)\ln x\), \(I_1 = \frac {1}{\theta ^2}\), so \(J(\theta ) = \frac {n}{\theta ^2}\) and the CRLB is \(\frac {\theta ^2}{n}<\frac {\theta ^2}{n - 2}\). The bound is not attained, as theorem 2.3.1 predicts, since \(\frac {n - 1}{T}\) is not a linear function of \(T\). The efficiency is \[\frac {\theta ^2/n}{\theta ^2/(n - 2)} = \frac {n - 2}{n}\longrightarrow 1,\] so the UMVUE is asymptotically efficient.

(b)

\(\tau '(\theta ) = 1 - 2\theta \), so the CRLB is \(\frac {(1 - 2\theta )^2}{n/\theta ^2} = \frac {\theta ^2(1 - 2\theta )^2}{n}\).

Solution. By example 2.2.3, \(J(\theta ) = \frac {n}{\theta ^2}\). For \(\tau (\theta ) = \theta \) the CRLB is \(\frac {\theta ^2}{n} = \var (\bar X)\), so \(\bar X\) is efficient; it is a linear function of the natural sufficient statistic \(\sum X_i\). For \(\tau (\theta ) = \frac 1\theta \), \(\tau '(\theta ) = -\frac {1}{\theta ^2}\) and the CRLB is \(\frac {1}{n\theta ^2}\). The UMVUE is \(\frac {n - 1}{T}\) (example 1.6.16), and by the calculation of example 2.3.4 with \(\theta \) replaced by \(\frac 1\theta \) its variance is \(\frac {1}{(n - 2)\theta ^2}\), larger than the bound. Only \(\theta \) (and linear functions of it) can be estimated efficiently.

Proof. Let \(S(\theta ,X)\) be the score vector, so that \(E_{\theta }\left [S\right ]=0\) and \(\var _{\theta }(S)=J(\theta )\). Differentiating the unbiasedness identity \(E_{\theta }\left [T(X)\right ]=\tau (\theta )\) with respect to \(\theta _j\) and interchanging derivative and integral, which regularity permits, \[\frac {\partial \tau }{\partial \theta _j} = \int T(x)\,\frac {\partial }{\partial \theta _j} f_{\theta }(x)\,dx = \int T(x)\, S_j(\theta ,x)\, f_{\theta }(x)\,dx = \cov _{\theta }\!\left (T, S_j\right ),\] the last equality because \(E_{\theta }\left [S_j\right ]=0\). Collecting the \(k\) components, \(\cov _{\theta }(T,S) = D(\theta )\).

Now take any fixed vector \(c\in \mathbb {R}^{k}\) and apply the Cauchy–Schwarz inequality to the scalar random variables \(T\) and \(c^{t}S\): \[\left [\cov _{\theta }\!\left (T,\, c^{t}S\right )\right ]^{2} \;\leq \; \var _{\theta }(T)\ \var _{\theta }\!\left (c^{t}S\right ),\] that is \[\left (c^{t}D\right )^{2} \;\leq \; \var _{\theta }(T)\ c^{t}J(\theta )\,c .\] Choose \(c = J^{-1}(\theta )\,D(\theta )\), which is legitimate because \(J(\theta )\) is positive definite in a regular model. Then \(c^{t}D = D^{t}J^{-1}D\) and \(c^{t}Jc = D^{t}J^{-1}JJ^{-1}D = D^{t}J^{-1}D\), so the inequality reads \[\left (D^{t}J^{-1}D\right )^{2} \;\leq \; \var _{\theta }(T)\ D^{t}J^{-1}D .\] Since \(J^{-1}\) is positive definite, \(D^{t}J^{-1}D>0\) unless \(D=0\), and dividing through gives \[\var _{\theta }(T) \;\geq \; D(\theta )^{t}\,J^{-1}(\theta )\,D(\theta ).\] If \(D=0\) the inequality is trivial. □