Example 3.3.3. Let \(X_1, X_2, \cdots , X_n\) be a random sample from \(X\thicksim N(\mu , \sigma ^2)\), where \(\sigma ^2\) is known. Find the generalized likelihood ratio test of the hypothesis \(H_0: \, \mu = \mu _0\) against \(H_1:\, \mu \neq \mu _0\).

Example 3.3.4. Let \(X_1, X_2, \cdots , X_n\) be a random sample from the \(N(\mu ,\sigma ^2)\) distribution where \(\mu \) and \(\sigma ^2\) are unknown. Consider a test of \(H_0: \mu = 0\) against \(H_1: \mu \neq 0\).

(i).

Show that the generalized likelihood ratio test of \(H_0\) against \(H_1\) has critical region \[R = \left \{(x_1, \cdots , x_n) \, \Big | \, \frac {n\overline {x}^2}{s^2} > c\right \}.\]

Solution. \(\Omega = \{(\mu , \sigma ^2)| -\infty < \mu < \infty , \sigma ^2>0\}\) and \(\Omega _0= \{(\mu , \sigma ^2) |\mu = 0, \sigma ^2 > 0\}\) \[\Lambda (x) = \frac {L(\hat {\theta }; x)}{L(\hat {\theta }_R; x)} \quad \text {where}\quad \theta = (\mu , \sigma ^2)\] \[L(\theta ;x) = \prod ^n_{i = 1}\frac {1}{\sqrt {2\pi \sigma ^2}}e^{-\frac {1}{2\sigma ^2}\sum ^n_{i = 1} (x_i - \mu )^2} = (2\pi \sigma ^2)^{-\frac {n}{2}}\, e^{-\frac {1}{2\sigma ^2} (x_i - \mu )^2}\] \[l(\theta ) = -\frac {n}{2}\ln (2\pi \sigma ^2) - \frac {1}{2\sigma ^2}\sum ^n_{i = 1}(x_i - \mu )^2\]

\[S(\theta ) = \begin {pmatrix} \frac {\partial l}{\partial \mu }\\ \frac {\partial l}{\partial \sigma ^2}\\ \end {pmatrix}\] \[\frac {\partial l}{\partial \mu } = -\frac {1}{2\sigma ^2}(-2)\sum ^n_{i = 1}(x_i -\mu ) = \frac {1}{\sigma ^2}\sum ^n_{i = 1}(x_i -\mu ).\] \(\frac {\partial l}{\partial \mu } = 0\) \[\implies \quad \sum ^n_{i = 1} x_i - n\mu = 0\] \[\therefore \quad \hat {\mu } = \overline {x}.\]

\[\frac {\partial l}{\partial \sigma ^2} = -\frac {n}{2}\cdot \frac {2\pi }{2\pi \sigma ^2} + \frac {1}{2(\sigma ^2)^2}\sum ^n_{i =1}(x_i - \mu )^2 = -\frac {n}{2\sigma ^2}+ \frac {1}{2(\sigma ^2)^2}\sum ^n_{i = 1}(x_i - \mu )^2.\] \(\frac {\partial l}{\partial \sigma ^2} = 0\) \[\implies \quad \hat {\sigma }^2 = \frac {1}{n}\sum ^n_{i = 1}(x_i - \mu )^2\] Therefore \begin {align*} L(\hat {\theta }) & = L(\hat {\mu }, \hat {\sigma }^2)\\ & = (2\pi \hat {\sigma }^2)^{-\frac {n}{2}}e^{-\frac {1}{2\hat {\sigma }^2}\sum ^n_{i = 1}(x_i - \hat {\mu })^2}\\ & = (2\pi \hat {\sigma }^2)^{-\frac {n}{2}}e^{-\frac {1}{2\hat {\sigma }^2}\sum ^n_{i = 1}(x_i - \overline {x})^2}\\ & = \left (\frac {2\pi }{n}\sum ^n_{i = 1} (x_i - \overline {x})^2\right )^{-\frac {n}{2}}e^{-\frac {n}{2}} \end {align*}

Under \(H_0\): \[L(\theta _R;x) = \prod ^n_{i = 1}\frac {1}{\sqrt {2\pi \sigma ^2}}e^{-\frac {1}{2\sigma ^2}x_i^2}\] \[L(\theta ) = (2\pi \sigma ^2)^{-\frac {n}{2}}e^{-\frac {1}{2\sigma ^2}\sum ^n_{i = 1} x_i^2}\] \[l(\theta ) = -\frac {n}{2}\ln (2\pi \sigma ^2) - \frac {1}{2\sigma ^2}\sum ^n_{i = 1}x_i^2\] \[S(\theta ) = \frac {\partial l}{\partial \sigma ^2}\] \[\frac {\partial l}{\partial \sigma ^2} = -\frac {n}{2}\cdot \frac {2\sigma }{2\sigma \, \sigma ^2} + \frac {1}{2(\sigma ^2)^2}\sum ^n_{i = 1}x_i^2=-\frac {n}{2\sigma ^2}+ \frac {1}{2(\sigma ^2)^2}\sum ^n_{i = 1} x_i^2\] \[\frac {\partial l}{\partial \sigma ^2} = 0\quad \implies \quad \hat {\sigma }^2_R = \frac {1}{n}\sum ^n_{i = 1} x_i^2\] Therefore \begin {align*} L(\hat {\theta }_R) & = L(\hat {\theta }^2_R)\\ & = (2\pi \hat {\theta }^2_R)^{-\frac {n}{2}}\, e^{-\frac {1}{2\hat {\sigma }^2_R}\sum ^n_{i = 1}x_i^2}\\ & = \left (\frac {2\pi }{n}\sum ^n_{i = 1} x_i^2\right )^{-\frac {n}{2}} e^{-\frac {n}{2}}. \end {align*}

Therefore \[\frac {L(\hat {\theta })}{L(\hat {\theta }_R)} > k\] \[\frac {\left (\frac {2\pi }{n}\sum ^n_{i = 1} (x_i - \overline {x})^2\right )^{-\frac {n}{2}}e^{-\frac {n}{2}}}{\left (\frac {2\pi }{n}\sum ^n_{i = 1} x_i^2\right )^{-\frac {n}{2}}e^{-\frac {n}{2}}} > k\] \[\left (\frac {\sum ^n_{i = 1} (x_i - \overline {x})^2}{\sum ^n_{i = 1} x_i^2}\right )^{-\frac {n}{2}} > k\] \[\left (\frac {\sum ^n_{i = 1}(x_i - \overline {x} + \overline {x})^2}{\sum ^n_{i = 1}(x_i - \overline {x})^2}\right )^{\frac {n}{2}} > k\] \[\left (\frac {\sum ^n_{i = 1} \left [(x_i - \overline {x})^2 + 2\overline {x}(x_i - \overline {x})+ \overline {x}^2\right ]}{\sum ^n_{i = 1} (x_i - \overline {x})^2}\right )^{\frac {n}{2}} > k\] \[\left (\frac {\sum ^n_{i = 1} (x_i - \overline {x})^2 + n\overline {x^2}}{\sum ^n_{i = 1}(x_i - \overline {x})^2}\right )^{\frac {n}{2}}\] \[\implies \quad \left (1 + \frac {n\overline {x}^2}{\sum ^n_{i = 1}(x_i - \overline {x})^2}\right )^{\frac {n}{2}} > k\] \[\frac {n\overline {x}^2}{(n-1)S^2}> k_1\] \[\frac {n\overline {x}^2}{S^2} > c\] Therefore \[R = \left \{(x_1, \cdots , x_n)\, \Big |\, \frac {n\overline {x}^2}{S^2} > c\right \}.\] □

(ii).

Show that the statistic \(T = \frac {n\overline {X}^2}{S^2}\) has a \(F(1,n-1)\) distribution under \(H_0\) and thus find a size \(\alpha = 0.05\) test for \(n = 20\).

Solution. \begin {align*} T & = \frac {n\overline {X}^2}{S^2} = \frac {n\left (\frac {\overline {X}-0}{\sigma }\right )^2}{\frac {S^2}{\sigma ^2}} = \frac {\left (\frac {\overline {X}-0}{\frac {\sigma }{\sqrt {n}}}\right )^2}{\frac {(n-1)\frac {S^2}{\sigma ^2}}{n-1}} \end {align*}

\[\frac {\overline {X} - 0}{\frac {\sigma }{\sqrt {n}}}\, \thicksim \, N(0,1)\quad \implies \quad \left (\frac {\overline {X}-0}{\frac {\sigma }{\sqrt {n}}}\right )^2\, \thicksim \, \frac {\chi ^2(1)}{1}\]

\[\frac {(n-1)S^2}{\sigma ^2}\, \thicksim \, \chi ^2(n-1)\] \[\frac {(n-1)\frac {S^2}{\sigma ^2}}{n-1}\, \thicksim \, \frac {\chi ^2(n-1)}{n-1}\] \(\overline {X}\) and \(S^2\) are independent.

Therefore, \(\chi ^2(1)\) and \(\chi ^2(n-1)\) are independent. \[\therefore \quad \frac {n\, \overline {X}^2}{S^2}\, \thicksim \, F(1,n-1).\] \(P_{H_0}(X\in R) = \alpha \) \[P\left (\frac {n\overline {X}^2}{S^2} > c\right ) = \alpha \] \[P(F_{1,n-1} > c) = 0.05\] \[P(F_{1,19} > c) = 0.05\] \(c = 4.35\)

Therefore \[R = \left \{(x_1, \cdots , x_n)\, \Big |\, \frac {20\overline {x}^2}{S^2} > 4.35\right \}\] is the critical region. □

Example 3.3.6. Let \(X_1, X_2, \cdots , X_n\) and \(Y_1, Y_2, \cdots , Y_m\) be two independent random samples from \[f(x;\theta ) = \frac {1}{\theta }e^{-\frac {x}{\theta }}\, , \quad x >0\] and \[f(y;\mu ) = \frac {1}{\mu }e^{-\frac {x}{\mu }}\, , \quad y >0\] respectively.

(a).

Show that the generalized likelihood ratio for testing the hypothesis \(H_0:\theta = \mu \) against \(H_1: \theta \neq \mu \) is a function of the statistic \[T = \frac {\sum ^n_{i = 1} X_i}{\sum ^n_{i = 1} X_i + \sum ^m_{i = j} Y_j}.\]

Solution. \(\Omega = \{(\theta ,\mu ):\, \theta > 0,\, \mu > 0\}\quad \) and \(\quad \Omega _0 = \{(\theta ,\mu ):\, \theta = \mu \}\) \[\Lambda (x,y) = \frac {L(\hat {\theta })}{L(\hat {\theta }_R)}\] \begin {align*} L(\theta , \mu ) & = \prod ^n_{i = 1} f(x_i;\theta )\, \prod ^m_{j = 1} f(y_j;\mu )\\ & = \prod ^n_{i = 1} \frac {1}{\theta }\, e^{-\frac {x_i}{\theta }}\, \prod ^m_{j=1}\frac {1}{\mu }\, e^{-\frac {y_j}{\mu }}\\ & = \theta ^{-n}\, e^{-\frac {1}{\theta }\sum ^n_{i =1} x_i}\, \mu ^{-m}\, e^{-\frac {1}{\mu }\sum ^m_{j=1}y_j}. \end {align*}

\[l(\theta ,\mu ) = -n\ln \theta - \frac {1}{\theta }\sum ^n_{i = 1} x_i - m\ln \mu - \frac {1}{\mu }\sum ^m_{j = 1} y_j.\]

The score vector \[S(\theta ,\mu ) = \begin {pmatrix} \frac {\partial l}{\partial \theta }\\ \frac {\partial l}{\partial \mu }\\ \end {pmatrix}\]

\[\frac {\partial l}{\partial \theta } = -\frac {n}{\theta } + \frac {1}{\theta ^2}\, \sum ^n_{i = 1} x_i\] \(\frac {\partial l}{\partial \theta }= 0\) \[\frac {n}{\theta } = \frac {1}{\theta ^2}\, \sum ^n_{i = 1} x_i\quad \implies \quad \hat {\theta } = \frac {1}{n}\sum ^n_{i = 1} x_i = \overline {x}.\] Similarly \[\hat {\mu } = \frac {1}{m}\, \sum ^m_{j = 1} y_j = \overline {y}.\]

\begin {align*} L(\hat {\theta }) & = L(\hat {\theta },\hat {\mu })\\ & = \hat {\theta }^{-n} \, e^{-\frac {1}{\hat {\theta }}\sum ^n_{i = 1} x_i}\, \hat {\mu }^{-m}\, e^{-\frac {1}{\hat {\mu }}\sum ^m_{j = 1}y_j}\\ & = \overline {x}^{-n} \, e^{-\frac {1}{\hat {\theta }}\, n\hat {\theta }}\,\, \overline {y}^{-m}\, e^{-\frac {1}{\hat {\mu }} \, m\hat {\mu }}\\ & = \frac {e^{-(n + m)}}{\overline {x}^n\, \overline {y}^m}. \end {align*}

Under \(H_0: \theta = \mu \) \begin {align*} L(\theta ) & = \prod ^n_{i =1} f(x_i;\theta )\, \prod ^m_{j = 1} f(y_j;\theta )\\ & = \prod ^n_{i = 1} \frac {1}{\theta }\, e^{-\frac {x_i}{\theta }}\, \prod ^m_{j = 1}\frac {1}{\theta }e^{-\frac {y_j}{\theta }}\\ & = \theta ^{-n}\, e^{-\frac {1}{\theta }\sum ^n_{i = 1} x_i}\, \theta ^{-m}\, e^{-\frac {1}{\theta }\sum ^m_{j = 1} y_j}\\ & = \theta ^{-(n + m)}\, e^{-\frac {1}{\theta }\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j\right )}. \end {align*}

\[l(\theta ) = -(n+m)\log \theta - \frac {1}{\theta }\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1}y_j\right )\]

\[S(\theta ) = \frac {-(n+ m)}{\theta } + \frac {1}{\theta ^2}\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j\right ).\] \(S(\theta ) = 0\) \[\frac {n+m}{\theta }=\frac {1}{\theta ^2}\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j\right )\]

\[\hat {\theta }_R = \frac {1}{n + m}\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j\right ) = \frac {n\overline {x}+ m\overline {y}}{n + m}.\]

\begin {align*} L(\hat {\theta }_R & = \hat {\theta }_R^{-(n + m)}\, e{-\frac {1}{\hat {\theta }_R}\left (\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j\right )}\\ & = \left (\frac {n\overline {x} + m\overline {y}}{n + m}\right )^{-(n + m)} \, e^{-\frac {1}{\hat {\theta }_R}(n + m)\hat {\theta }_R}\\ & = \left (\frac {n\overline {x} + m\overline {y}}{n + m}\right )^{-(n+m)}\, e^{-(n+m)}. \end {align*}

Thus \[\Omega (x,y) = \frac {L(\hat {\theta })}{L(\hat {\theta }_R)} \, > \, k\]

\[\frac {\frac {e^{-(n + m)}}{\overline {x}^n \,\overline {y}^m}}{\frac {e^{-(n +m)}}{\left (\frac {n\overline {x} + m\overline {y}}{n + m}\right )^{n+m}}} > k\]

\[\frac {\left (\frac {n\overline {x} +m\overline {y}}{n + m}\right )^{n + m}}{\overline {x}^n\,\overline {y}^m} \, >\, k\]

\[\left (\frac {1}{n + m}\right )^{n + m}\, \left (\frac {n \overline {x} + m\overline {y}}{\overline {x}^n}\right )^n\left (\frac {n\overline {x}+m\overline {y}}{\overline {y}}\right )^m > k\]

\[\frac {n^n\, m^m}{(n + m)^{n+m}}\left (\frac {n\overline {x} + m\overline {y}}{n\overline {x}}\right )^n\left (\frac {n\overline {x}+m\overline {y}}{m\overline {y}}\right )^m > k\]

\[\left (\frac {n\overline {x}}{n\overline {x}+m\overline {y}}\right )^{-n}\left (\frac {m\overline {y} + n\overline {x} - n\overline {x}}{n\overline {x}+ m\overline {y}}\right )^{-m} > k_1\]

\[\left (\frac {n\overline {x}}{n\overline {x} + m\overline {y}}\right )^{-n}\left (1 - \frac {n\overline {x}}{n\overline {x}+m\overline {y}}\right )^{-m} > k_1\]

\[\implies \quad \left (\frac {\sum ^n_{i = 1} x_i}{\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j}\right )^{-n}\, \left (1 - \frac {\sum ^n_{i = 1} x_i}{\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j}\right )^{-m} > k_1\] which is a function of \(t = \frac {\sum ^n_{i = 1} x_i}{\sum ^n_{i = 1} x_i + \sum ^m_{j = 1} y_j}\) \[t^n(1-t)^{-m} > k_1\] \[\implies \quad \frac {1}{t^n}\left (\frac {1}{1-t}\right )^m > k_1\] \[t^n(1 - t)^m < c\] i.e. \[R = \left \{(x,y)\, |\, t^n(1-t)^m < c\right \}\] is the critical region. □

(b).

Find the distribution of \(T\) under \(H_0\).

Solution. \(T = \frac {\sum ^n_{i = 1} X_i}{\sum ^n_{i = 1} X_i + \sum ^m_{j}y_j}\)

Under \(H_0\) \[T = \frac {\frac {\sum ^n_{i = 1}X_i}{\theta }}{\frac {\sum ^n_{i = 1}X_i}{\theta } + \frac {\sum ^m_{j=1}Y_j}{\theta }} = \frac {U}{U + V}.\] \[X_i \thicksim \operatorname {EXP}(\theta )\] \[\frac {X_i}{\theta }\thicksim \operatorname {EXP}(1)\] \[M_{\frac {X_i}{\theta }}(t) = \frac {1}{1-t}\] \[M_U(t) = \left (\frac {1}{1-t}\right )^n\] \(U \thicksim \operatorname {GAM}(n,1)\), similarly \(\, V\thicksim \operatorname {GAM}(m,1)\) then \(\, U + V \, \thicksim \, \operatorname {GAM}(n+m,1)\).

Therefore \[T = \frac {U}{U + V}\, \thicksim \, \operatorname {BETA}(n,m).\] □