Proof. Take the scalar case. The score \(S(\theta _0; X) = \sum _{i=1}^nS_1(\theta _0; X_i)\) is a sum of independent, identically distributed terms which, under \(H_0\), have mean \(0\) and variance \(J_1(\theta _0)\) by theorem 2.2.1. So \(E(S) = 0\) and \(\var (S) = nJ_1(\theta _0) = J(\theta _0)\), and by the central limit theorem \(S(\theta _0; X)/\sqrt {J(\theta _0)}\underset {D}{\longrightarrow }Z\sim N(0, 1)\). The map \(z\mapsto z^2\) is continuous, so by part (2) of the limit theorems \(R(\theta _0; X)\underset {D}{\longrightarrow }Z^2\sim \chi ^2(1)\).

For \(\theta \in \mathbb {R}^k\) the multivariate central limit theorem gives \(J(\theta _0)^{-1/2}S(\theta _0; X)\underset {D}{\longrightarrow }Z\sim \text {MVN}(0, I_k)\), and \(Z^{\mathsf T}Z\sim \chi ^2(k)\). □

Proof. For \(k = 1\), theorem 2.5.3 gives \(\sqrt {J(\theta _0)}(\hat \theta - \theta _0)\underset {D}{\longrightarrow }N(0, 1)\), so its square tends to \(\chi ^2(1)\). Since \(\hat \theta \) is consistent and \(J_1\) is continuous, \(\frac {J(\hat \theta )}{J(\theta _0)} = \frac {J_1(\hat \theta )}{J_1(\theta _0)}\underset {P}{\longrightarrow }1\), and by Slutsky’s theorem \(w(\theta _0; X) = \frac {J(\hat \theta )}{J(\theta _0)}\cdot J(\theta _0)(\hat \theta - \theta _0)^2\) has the same limit. The vector case uses the multivariate normal limit of section 2.5 in the same way. □

Solution. \(\mathcal {L}(\theta ) = \ln \binom nx + x\ln \theta + (n - x)\ln (1 - \theta )\), so \[S(\theta ) = \frac x\theta - \frac {n - x}{1 - \theta } = \frac {x - n\theta }{\theta (1 - \theta )},\qquad I(\theta ) = \frac {x}{\theta ^2} + \frac {n - x}{(1 - \theta )^2} .\] The ML estimate is \(\hat \theta = \frac xn\), and since \(E(X) = n\theta \), \[J(\theta ) = \frac {n}{\theta } + \frac {n}{1 - \theta } = \frac {n}{\theta (1 - \theta )} .\]

Score test

\[R(\theta _0; x) = \frac {S(\theta _0)^2}{J(\theta _0)} = \frac {(x - n\theta _0)^2}{\theta _0^2(1 - \theta _0)^2}\cdot \frac {\theta _0(1 - \theta _0)}{n} = \frac {(x - n\theta _0)^2}{n\theta _0(1 - \theta _0)},\] the square of the familiar statistic \(\frac {\hat \theta - \theta _0}{\sqrt {\theta _0(1 - \theta _0)/n}}\).

Wald test

\[w(\theta _0; x) = (\hat \theta - \theta _0)^2J(\hat \theta ) = \frac {(x - n\theta _0)^2}{n\hat \theta (1 - \hat \theta )} .\] The two differ only in where the variance is evaluated: at \(\theta _0\) for the score test, at \(\hat \theta \) for the Wald test. If \(J(\theta _0)\) is used in the Wald statistic the two coincide.

Comparison

With \(n = 100\), \(x = 60\) and \(\theta _0 = 0.5\): \begin {gather*} R = \frac {100}{25} = 4.000,\qquad w = \frac {100}{100(0.6)(0.4)} = 4.167,\\ 2\ln \Lambda = 2\left [60\ln \frac {0.6}{0.5} + 40\ln \frac {0.4}{0.5}\right ] = 4.027 . \end {gather*} All three exceed \(\chi ^2_1(0.05) = 3.841\), so \(H_0\) is rejected at the \(5\%\) level by each. The three statistics are asymptotically equivalent under \(H_0\), and here they agree to within about \(4\%\); for small samples, or estimates near the boundary of \(\Omega \), they can disagree much more.