Solution. The joint p.f. is \(\prod \theta ^{x_i}(1 - \theta )^{1 - x_i} = \theta ^t(1 - \theta )^{n - t}\), where \(t = \sum x_i\), and \(T\sim \text {BIN}(n,\theta )\). So, for any \(\mathbf x\) with \(\sum x_i = t\), \[P(\mathbf X = \mathbf x\mid T = t) = \frac {\theta ^t\, (1 - \theta )^{n - t}}{\binom {n}{t}\theta ^t\, (1 - \theta )^{n - t}} = \frac {1}{\binom {n}{t}} ,\] which does not depend on \(\theta \). Given the number of successes, every arrangement of them is equally likely, whatever the success probability.

Proof in the discrete case, \(k = 1\). If \(T\) is sufficient, then \(P(\mathbf X = \mathbf x) = P(T = t)\,P(\mathbf X = \mathbf x\mid T = t)\), the first factor depending on \(\theta \) and \(t\) only and the second on \(\mathbf x\) only; this is the factorisation. Conversely, if the factorisation holds, then \(P(T = t) = \sum _{\mathbf y : T(\mathbf y) = t}g(t;\theta )h(\mathbf y) = g(t;\theta )\sum _{\mathbf y : T(\mathbf y) = t}h(\mathbf y)\), so \[P(\mathbf X = \mathbf x\mid T = t) = \frac {g(t;\theta )h(\mathbf x)}{g(t;\theta )\sum _{T(\mathbf y) = t}h(\mathbf y)} = \frac {h(\mathbf x)}{\sum _{T(\mathbf y) = t}h(\mathbf y)} ,\] which is free of \(\theta \). □

Solution. The likelihood is \[L(\theta ) = \underbrace {\theta ^n \left (\prod ^n_{i = 1}x_i\right )^{\theta - 1}}_{g(t;\theta )}\cdot \underbrace {1}_{h(\mathbf x)} ,\] so \(T = \prod ^n_{i = 1} X_i\) is sufficient, and so is the one-to-one function \(\sum _{i=1}^n\ln X_i\), of which the M.L. estimator of Example 2.4.4 is a function.

Solution. As in Example 2.4.6, the likelihood is \[L(\theta ) = I_{\left (\theta - \frac 12, \infty \right )}(y_1) \cdot I_{\left (-\infty , \theta + \frac 12\right )}(y_n),\] a function of \(\theta \) and the pair \((y_1,y_n)\) only, with \(h = 1\). So \((Y_1, Y_n)\) is jointly sufficient for \(\theta \): a single parameter can need two statistics.

Solution. The likelihood is \begin {align*} L(\mu , \sigma ^2) &= \left (2\pi \sigma ^2\right )^{-n/2} \exp \left \{-\frac {1}{2\sigma ^2} \sum ^n_{i = 1} (x_i - \mu )^2\right \}\\ &= \left (2\pi \sigma ^2\right )^{-n/2} \exp \left \{-\frac {1}{2\sigma ^2}\left (\sum x^2_i - 2\mu \sum x_i + n\mu ^2\right )\right \}, \end {align*}

which depends on the data only through \(\sum x_i\) and \(\sum x_i^2\); with \(h = 1\), Theorem 2.5.3 shows \(T\) is sufficient. The log likelihood is \[l(\mu , \sigma ^2) = -\frac {n}{2}\ln (2\pi ) - \frac {n}{2}\ln \sigma ^2 - \frac {1}{2\sigma ^2}\sum ^n_{i = 1}(x_i - \mu )^2 ,\] so the score vector has components \[\frac {\partial l}{\partial \mu } = \frac {1}{\sigma ^2}\sum ^n_{i = 1}(x_i - \mu ) ,\qquad \frac {\partial l}{\partial \sigma ^2} = -\frac {n}{2\sigma ^2} + \frac {1}{2\sigma ^4}\sum ^n_{i = 1}(x_i - \mu )^2 .\] Differentiating again, the information matrix is \[I(\mu ,\sigma ^2) = \begin {pmatrix} \frac {n}{\sigma ^2} & \frac {1}{\sigma ^4}\sum (x_i - \mu )\\[1ex] \frac {1}{\sigma ^4}\sum (x_i - \mu ) & -\frac {n}{2\sigma ^4} + \frac {1}{\sigma ^6}\sum (x_i - \mu )^2 \end {pmatrix}.\] Taking expectations, with \(E(X_i - \mu ) = 0\) and \(E(X_i - \mu )^2 = \sigma ^2\), \[J(\mu ,\sigma ^2) = \begin {pmatrix} \frac {n}{\sigma ^2} & 0\\[1ex] 0 & -\frac {n}{2\sigma ^4} + \frac {n\sigma ^2}{\sigma ^6} \end {pmatrix} = \begin {pmatrix} \frac {n}{\sigma ^2} & 0\\[1ex] 0 & \frac {n}{2\sigma ^4} \end {pmatrix}.\] Setting the score to zero, the first equation gives \(\hat {\mu } = \bar {x}\), and then the second gives \(\hat {\sigma }^2 = \frac {1}{n}\sum (x_i - \bar {x})^2\). So the M.L. estimator is \(\hat {\theta } = \left (\bar {X},\ \frac {1}{n}\sum (X_i - \bar {X})^2\right )^{\mathsf T}\), the same as the method of moments estimator here.

Solution. First, \(E(\bar {X}) = \frac {1}{n}\sum E(X_i) = \mu \). For \(S^2\), write \(X_i - \bar X = (X_i - \mu ) - (\bar X - \mu )\); since \(\sum (X_i - \mu ) = n(\bar X - \mu )\), \[\sum ^n_{i = 1}(X_i - \bar {X})^2 = \sum ^n_{i = 1}(X_i - \mu )^2 - n(\bar {X} - \mu )^2 .\] Taking expectations, \[E\left [\sum ^n_{i = 1}(X_i - \bar {X})^2\right ] = n\sigma ^2 - n\cdot \frac {\sigma ^2}{n} = (n - 1)\sigma ^2 ,\] so \(E(S^2) = \sigma ^2\). The divisor \(n - 1\) is exactly what removes the bias.

Proof. Write \(T - \tau (\theta ) = (T - E(T)) + (E(T) - \tau (\theta ))\) and square. The cross term has expectation \(2(E(T) - \tau (\theta ))E(T - E(T)) = 0\), leaving \(E(T - E(T))^2 + (E(T) - \tau (\theta ))^2\). □

Solution.

(a)

Since \(E(X) = \int _0^\theta \frac {x}{\theta }\,dx = \frac {\theta }{2}\), \(E(T_1) = 2E(\bar X) = \theta \): unbiased.

(b)

By Example 1.2.4, \(T_2\) has p.d.f. \(\frac {ny^{n-1}}{\theta ^n}\) on \((0,\theta )\) and \(E(T_2) = \frac {n\theta }{n + 1}\): biased, with \(\text {bias}(T_2) = -\frac {\theta }{n + 1}\).

(c)

Since \(E(X^2) = \frac {\theta ^2}{3}\), \(\text {Var}(X) = \frac {\theta ^2}{12}\), and \[\text {MSE}(T_1) = \text {Var}(2\bar X) = \frac {4}{n}\cdot \frac {\theta ^2}{12} = \frac {\theta ^2}{3n} .\] For \(T_2\), \(E(T_2^2) = \int _0^\theta y^2\frac {ny^{n-1}}{\theta ^n}\,dy = \frac {n\theta ^2}{n + 2}\), so \[\text {Var}(T_2) = \frac {n\theta ^2}{n + 2} - \frac {n^2\theta ^2}{(n+1)^2} = \frac {n\theta ^2}{(n + 1)^2(n + 2)} ,\] and by Theorem 2.5.11 \[\text {MSE}(T_2) = \frac {n\theta ^2}{(n + 1)^2(n + 2)} + \frac {\theta ^2}{(n + 1)^2} = \frac {\theta ^2}{(n + 1)^2}\cdot \frac {2n + 2}{n + 2} = \frac {2\theta ^2}{(n + 1)(n + 2)} .\]

The two are equal for \(n = 1\) and \(n = 2\), and for \(n\geq 3\) the biased estimator \(T_2\) has the smaller mean squared error; its MSE falls like \(\frac {1}{n^2}\) against \(\frac 1n\) for \(T_1\).

Proof. Let \(S = S(\theta ;\mathbf X) = \frac {\partial }{\partial \theta }\ln L(\theta ;\mathbf X)\). Differentiating \(\int L\,d\mathbf x = 1\) gives \(E(S) = \int \frac {\partial L}{\partial \theta }\,d\mathbf x = 0\), and differentiating once more gives \(\text {Var}(S) = E(S^2) = J(\theta )\). Differentiating \(\int T L\,d\mathbf x = \tau (\theta )\) gives \(E(TS) = \tau '(\theta )\), so \(\text {Cov}(T,S) = \tau '(\theta )\). By the Cauchy–Schwarz inequality, \(\left [\text {Cov}(T,S)\right ]^2\leq \text {Var}(T)\text {Var}(S)\), that is \([\tau '(\theta )]^2\leq \text {Var}(T)J(\theta )\). □

Solution.

(a)

\(E(T_1) = E(X) = \theta \). By Theorem 1.2.3, with \(F(y) = 1 - e^{-y/\theta }\), the minimum has p.d.f. \(n\left (e^{-y/\theta }\right )^{n-1}\frac 1\theta e^{-y/\theta } = \frac {n}{\theta }e^{-ny/\theta }\): it is exponential with mean \(\frac {\theta }{n}\). So \(E(T_2) = n\cdot \frac {\theta }{n} = \theta \).

(b)

With \(\ln f(x;\theta ) = -\ln \theta - \frac {x}{\theta }\), \(\frac {\partial ^2}{\partial \theta ^2}\ln f = \frac {1}{\theta ^2} - \frac {2x}{\theta ^3}\), so \[J(\theta ) = n\,E\left [\frac {2X}{\theta ^3} - \frac {1}{\theta ^2}\right ] = \frac {n}{\theta ^2},\qquad \text {CRLB} = \frac {1}{J(\theta )} = \frac {\theta ^2}{n},\] in agreement with Example 2.4.11.

(c)

\(\text {Var}(T_1) = \frac {\text {Var}(X)}{n} = \frac {\theta ^2}{n}\), using \(E(X^2) = 2\theta ^2\): \(T_1\) attains the bound and is the UMVUE. Since \(X_{(1)}\) is exponential with mean \(\frac \theta n\), its variance is \(\frac {\theta ^2}{n^2}\), and \(\text {Var}(T_2) = n^2\cdot \frac {\theta ^2}{n^2} = \theta ^2\): \(T_2\) does not attain the bound.

(d)

\(e(T_1) = \frac {\theta ^2/n}{\theta ^2/n} = 1\) and \(e(T_2) = \frac {\theta ^2/n}{\theta ^2} = \frac {1}{n}\). The estimator based on the smallest observation alone wastes almost all the information.

(e)

With \(\tau (\theta ) = \frac 1\theta \), \(\tau '(\theta ) = -\frac {1}{\theta ^2}\) and \[\text {CRLB} = \frac {\theta ^{-4}}{n/\theta ^2} = \frac {1}{n\theta ^2} .\]

Proof. By Theorem 2.5.11, \(\text {MSE}(T_n) = \text {Var}(T_n) + [\text {bias}(T_n)]^2\to 0\), and by Chebyshev’s inequality in the form \(P(|T_n - \tau |\geq \varepsilon )\leq \frac {E(T_n - \tau )^2}{\varepsilon ^2}\), the probability tends to \(0\). □

Solution. Since \(T = \frac {n-1}{n}S^2\) and \(E(S^2) = \sigma ^2\), \[E(T) = \frac {n-1}{n}\sigma ^2,\qquad \text {bias}(T) = -\frac {\sigma ^2}{n}\to 0 .\] For the variance, \(\frac {(n-1)S^2}{\sigma ^2}\sim \chi ^2(n - 1)\), which has variance \(2(n - 1)\), so \(\text {Var}(S^2) = \frac {2\sigma ^4}{n - 1}\) and \[\text {Var}(T) = \left (\frac {n - 1}{n}\right )^2\frac {2\sigma ^4}{n - 1} = \frac {2(n-1)\sigma ^4}{n^2}\to 0 .\] By Theorem 2.5.20, \(T\) is consistent. So is \(S^2\), for the same reason.

Solution. For \(0<\varepsilon <\theta \), since \(Y_n<\theta \), \[P(|Y_n - \theta |\geq \varepsilon ) = P(Y_n\leq \theta - \varepsilon ) = \left (\frac {\theta - \varepsilon }{\theta }\right )^n\to 0 .\]