Solution. Here \(\Omega = \mathbb {R}\) and \(\Omega _0 = \{\mu _0\}\). The unrestricted MLE is \(\hat \mu = \bar x\) and the restricted one is \(\mu _0\), so \[\Lambda (x) = \frac {(2\pi \sigma ^2)^{-n/2}\exp \left (-\frac {1}{2\sigma ^2}\sum (x_i - \bar x)^2\right )} {(2\pi \sigma ^2)^{-n/2}\exp \left (-\frac {1}{2\sigma ^2}\sum (x_i - \mu _0)^2\right )} = \exp \left (\frac {1}{2\sigma ^2}\left [\sum (x_i - \mu _0)^2 - \sum (x_i - \bar x)^2\right ]\right ).\] Writing \(x_i - \mu _0 = (x_i - \bar x) + (\bar x - \mu _0)\) and using \(\sum (x_i - \bar x) = 0\), \begin {equation} \label {eq:sumsquaressplit} \sum (x_i - \mu _0)^2 = \sum (x_i - \bar x)^2 + n(\bar x - \mu _0)^2 . \end {equation} So \[\Lambda (x) = \exp \left (\frac {n(\bar x - \mu _0)^2}{2\sigma ^2}\right ) = \exp \left (\frac {z^2}{2}\right ),\qquad z = \frac {\bar x - \mu _0}{\sigma /\sqrt n} .\] \(\Lambda >k\) is equivalent to \(|z|>c\). Under \(H_0\), \(Z\sim N(0,1)\), so \(c = z_{\alpha /2}\) and the GLRT is the familiar two-sided \(z\) test: reject \(H_0\) if \[\left |\frac {\bar x - \mu _0}{\sigma /\sqrt n}\right |>z_{\alpha /2} .\] Note that \(2\ln \Lambda = z^2\sim \chi ^2(1)\) exactly under \(H_0\).

Solution.

(a)

Here \(\theta = (\mu ,\sigma ^2)\), \(\Omega = \{(\mu ,\sigma ^2) : \mu \in \mathbb {R}, \sigma ^2>0\}\) and \(\Omega _0 = \{(0,\sigma ^2) : \sigma ^2>0\}\). The log likelihood is \[l(\mu ,\sigma ^2) = -\frac n2\ln (2\pi \sigma ^2) - \frac {1}{2\sigma ^2}\sum (x_i - \mu )^2 .\] Over \(\Omega \) the MLEs are \(\hat \mu = \bar x\) and \(\hat \sigma ^2 = \frac 1n\sum (x_i - \bar x)^2\) (example 2.5.7). Substituting, the exponent becomes \(-\frac {n\hat \sigma ^2}{2\hat \sigma ^2} = -\frac n2\), so \[L(\hat \theta ;x) = (2\pi \hat \sigma ^2)^{-n/2}e^{-n/2} .\] Under \(H_0\), \(\mu = 0\) and the score equation \(\frac {\partial l}{\partial \sigma ^2} = -\frac {n}{2\sigma ^2} + \frac {1}{2\sigma ^4}\sum x_i^2 = 0\) gives \(\hat \sigma _0^2 = \frac 1n\sum x_i^2\), and in the same way \[L(\hat \theta _0;x) = (2\pi \hat \sigma _0^2)^{-n/2}e^{-n/2} .\] Hence, using (3.4) with \(\mu _0 = 0\), \begin {align*} \Lambda (x) &= \left (\frac {\hat \sigma _0^2}{\hat \sigma ^2}\right )^{n/2} = \left (\frac {\sum x_i^2}{\sum (x_i - \bar x)^2}\right )^{n/2}\\ &= \left (1 + \frac {n\bar x^2}{\sum (x_i - \bar x)^2}\right )^{n/2} = \left (1 + \frac {1}{n-1}\cdot \frac {n\bar x^2}{s^2}\right )^{n/2} . \end {align*}

This is an increasing function of \(\frac {n\bar x^2}{s^2}\), so \(\Lambda >k\) is equivalent to \(\frac {n\bar x^2}{s^2}>c\).

(b)

Write \[T = \frac {n\bar X^2}{S^2} = \frac {\left (\frac {\bar X}{\sigma /\sqrt n}\right )^2/1} {\frac {(n-1)S^2}{\sigma ^2}\Big /(n-1)} .\] Under \(H_0\), \(\frac {\bar X}{\sigma /\sqrt n}\sim N(0,1)\), so its square is \(\chi ^2(1)\); \(\frac {(n-1)S^2}{\sigma ^2}\sim \chi ^2(n-1)\); and \(\bar X\) and \(S^2\) are independent. By definition 1.3.41, \(T\sim F(1,n-1)\). For \(n = 20\) and size \(0.05\), \[P(F(1,19)>c) = 0.05\quad \Longrightarrow \quad c = 4.38 ,\] so the test rejects \(H_0\) if \(\frac {20\bar x^2}{s^2}>4.38\). Since \(T = \left (\frac {\bar X}{S/\sqrt n}\right )^2\) is the square of the one-sample \(t\) statistic and \(t_{19,0.025}^2 = 2.093^2 = 4.38\), this is the two-sided \(t\) test.

Solution. Since \(W = \frac {(n-1)S^2}{\sigma ^2}\sim \chi ^2(n-1)\) has variance \(2(n-1)\), \[\operatorname {Var}(S^2) = \operatorname {Var}\left (\frac {\sigma ^2}{n-1}W\right ) = \frac {\sigma ^4}{(n-1)^2}\cdot 2(n-1) = \frac {2\sigma ^4}{n-1} .\]

Solution.

(a)

Write \(s_x = \sum x_i = n\bar x\), \(s_y = \sum y_j = m\bar y\) and \(s = s_x + s_y\). The log likelihood is \[l(\theta ,\mu ) = -n\ln \theta - \frac {s_x}{\theta } - m\ln \mu - \frac {s_y}{\mu } .\] Setting \(\frac {\partial l}{\partial \theta } = -\frac n\theta + \frac {s_x}{\theta ^2}\) and \(\frac {\partial l}{\partial \mu }\) to zero gives \(\hat \theta = \bar x\) and \(\hat \mu = \bar y\), and then \[L(\hat \theta ,\hat \mu ) = \bar x^{-n}\bar y^{-m}e^{-(n+m)} .\] Under \(H_0\) the two samples share the mean \(\theta \), \(l(\theta ) = -(n+m)\ln \theta - \frac {s}{\theta }\), the restricted MLE is \(\hat \theta _0 = \frac {s}{n+m}\), and \[L(\hat \theta _0,\hat \theta _0) = \left (\frac {s}{n+m}\right )^{-(n+m)}e^{-(n+m)} .\] Hence \[\Lambda = \frac {\left (\frac {s}{n+m}\right )^{n+m}}{\bar x^n\bar y^m} .\] With \(t = \frac {s_x}{s}\) we have \(\bar x = \frac {ts}{n}\) and \(\bar y = \frac {(1-t)s}{m}\), so the powers of \(s\) cancel and \[\Lambda = \frac {n^n m^m}{(n+m)^{n+m}}\cdot \frac {1}{t^n(1-t)^m} .\] This depends on the data only through \(t\). The GLRT rejects \(H_0\) when \(t^n(1-t)^m<c\), that is, when \(t\) is too close to \(0\) or too close to \(1\): one sample has a much larger total than the other would suggest.

(b)

Under \(H_0\), divide numerator and denominator by \(\theta \): \(T = \frac {U}{U+V}\) with \(U = \frac {\sum X_i}{\theta }\) and \(V = \frac {\sum Y_j}{\theta }\). Each \(\frac {X_i}{\theta }\sim \text {EXP}(1)\) with m.g.f. \((1-u)^{-1}\), so \(U\) has m.g.f. \((1-u)^{-n}\) and \(U\sim \text {GAM}(n,1)\); likewise \(V\sim \text {GAM}(m,1)\), independent of \(U\). By example 1.3.12, \(T = \frac {U}{U+V}\sim \text {BETA}(n,m)\). The critical values are therefore beta quantiles, chosen so that the two tails of \(t^n(1-t)^m<c\) have total probability \(\alpha \).

Sketch for one parameter. Take \(p = r = 1\) and \(\Omega _0 = \{\theta _0\}\), so \(2\ln \Lambda = 2\left [l(\hat \theta ) - l(\theta _0)\right ]\). Expand \(l(\theta _0)\) about \(\hat \theta \), where \(l'(\hat \theta ) = 0\): \[l(\theta _0) = l(\hat \theta ) + \frac 12 l''(\theta ^*)(\theta _0 - \hat \theta )^2\] for some \(\theta ^*\) between \(\theta _0\) and \(\hat \theta \). So \[2\ln \Lambda = -l''(\theta ^*)(\hat \theta - \theta _0)^2 = \frac {-l''(\theta ^*)}{nI(\theta _0)}\left [\sqrt {nI(\theta _0)}\,(\hat \theta - \theta _0)\right ]^2 ,\] where \(I(\theta )\) is the Fisher information in one observation. Under \(H_0\), \(\hat \theta \to \theta _0\), the law of large numbers gives \(-\frac 1n l''(\theta ^*)\to I(\theta _0)\), and \(\sqrt {nI(\theta _0)}(\hat \theta - \theta _0)\xrightarrow {d}N(0,1)\). So the first factor tends to \(1\) and the second to the square of a standard normal, which is \(\chi ^2(1)\). □

Solution. The log likelihood is \(l(\lambda ) = n\bar x\ln \lambda - n\lambda - \sum \ln x_i!\), with unrestricted MLE \(\hat \lambda = \bar x\). So \[2\ln \Lambda = 2\left [l(\bar x) - l(\lambda _0)\right ] = 2n\left [\bar x\ln \frac {\bar x}{\lambda _0} - (\bar x - \lambda _0)\right ].\] With \(n = 50\), \(\bar x = 2.4\) and \(\lambda _0 = 2\), \[2\ln \Lambda = 100\left [2.4\ln 1.2 - 0.4\right ] = 100\left [0.43757 - 0.4\right ] = 3.757 .\] Here \(r = 1\) and \(\chi ^2_{1,0.05} = 3.841\). Since \(3.757<3.841\) we do not reject \(H_0\); the approximate \(p\)-value is \(P(\chi ^2(1)>3.757) = 0.053\).

For comparison, the Wald statistic \(\frac {\bar x - \lambda _0}{\sqrt {\lambda _0/n}} = \frac {0.4}{0.2} = 2.0\) has square \(4.0>3.841\) and would just reject. The two statistics agree to first order in large samples, but at the edge of the critical region they can lead to different decisions; the likelihood ratio uses the shape of the whole log likelihood and is usually preferred.