3.4 Uniformly Most Powerful Tests

Definition 3.4.1. A test \(\phi ^*\) of \(H_0: \theta \in \Omega _0\) versus \(H_1: \theta \in \Omega _1\) is a uniformly most powerful (UMP) test of size \(\alpha \) if

(i).: \(\sup _{\theta \in \Omega _0}\pi _{\phi ^*}(\theta ) = \alpha \)
(ii).: \(\pi _{\phi ^*}(\theta ) \geq \pi _{\phi }(\theta )\), for all \(\theta \in \Omega _1\) and for any other test \(\phi \) with size \(\alpha \).

i.e. a test \(\phi ^*\) is UMP of size \(\alpha \) if it has size \(\alpha \) and if among all other tests of size \(\alpha \) it has the larges power function for alternative values of \(\theta \) (i.e \(\theta \in \Omega _1\)).

Example 3.4.2. Let \(X_1, X_2, \cdots , X_n\) be a random sample from \(X\thicksim N(\theta , 1)\).

(a).: Find the most powerful test of size \(\alpha = 0.05\) of \(H_0: \theta = 0\) against \(H_1: \theta = \theta _1\) where \(\theta _1 > 0\).

Solution. \( \frac {L(\theta _1)}{L(\theta _0)} > k\) \[\frac {\prod ^n_{i = 1}\frac {1}{\sqrt {2\pi }}e^{-\frac {1}{2}(x_i - \theta _1)^2}}{\prod ^n_{i = 1} \frac {1}{\sqrt {2\pi }}e^{-\frac {1}{2}x_i^2}}\, > \, k\] \[\frac {(2\pi )^{-\frac {n}{2}}e^{-\frac {1}{2}\sum ^n_{i = 1} (x_i - \theta _1)^2}}{(2\pi )^{-\frac {n}{2}}e^{-\frac {1}{2}\sum ^n_{i = 1}x_i^2}}\, >\, k\] \[\implies \quad e^{\frac {1}{2}\sum ^n_{i = 1}x_i^2 - \frac {1}{2}\sum _{i = 1}^n(x_i - \theta _1)^2} >k\] \[e^{\frac {1}{2}\sum ^n_{i = 1}(x_i^2 + 2\theta _1x_i - \theta _1^2- x_i)} > k\] \[\sum ^n_{i = 1}(-\theta _1^2 + 2\theta _1x_i) > k_1\] \[-n\theta ^2_1 + 2\theta _1\sum ^n_{i = 1} > k_1\] \[2\theta _1 \sum ^n_{i = 1} > k_2\] \[\sum ^n_{i = 1} x_i > k_3\] \[\overline {x}> c\] i.e \[R = \left \{(x_1, \cdots , x_n)\, \Big |\, \overline {x} > c\right \}\] is the critical region. \[P_{H_0}(X\in R) = \alpha \] \[P_{H_0}(\overline {X} > c )= 0.05\] \[P\left (\frac {\overline {X} - \mu _0}{\frac {\sigma }{\sqrt {n}}}>\frac {c - 0}{\frac {1}{\sqrt {n}}}\right ) = 0.05\] \[P(Z > c\sqrt {n}) = 0.05\quad \implies \quad c = \frac {1.645}{\sqrt {n}}.\] Therefore, \[R = \left \{(x_1, x_2, \cdots , x_n)\, \Big |\, \overline {x} > \frac {1.645}{\sqrt {n}}\right \}\] is the critical region of the MP test of size \(\alpha = 0.05\) of \(H_0:\theta = 0\) vs \(H_1: \theta = \theta _1\) (where \(\theta _1 > 0\)). □
(b).: Find the uniformly most powerful test of size \(\alpha = 0.05\) of \(H_0: \theta = 0\) against \(H_1: \theta > 0\).

Solution. The construction of the MP test in (a) only used the fact that that \(\theta _1 > 0\) i.e we would obtain the same critical region \[R = \left \{(x_1, x_2, \cdots , x_n)\, \Big |\, \overline {x} > \frac {1.645}{\sqrt {n}}\right \}\] for each \(\theta _1 > 0\).
Therefore, the test with this critical region is the UMP test of: \(H_0: \theta = 0\) against \(H_1: \theta > 0\). □
(c).: Sketch the power function of the test in (b).

Solution. \begin {align*} \pi (\theta ) & = P_{\theta }(X \in R)\\ & = P_{\theta }\left (\overline {X} > \frac {1.645}{\sqrt {n}}\right )\\ & = P\left (\frac {\overline {X} - \theta }{\frac {\sigma }{\sqrt {n}}}> \frac {\frac {1.645}{\sqrt {n}} - \theta }{\frac {1}{\sqrt {n}}}\right )\\ & = P(Z > 1.645 -\theta \sqrt {n}) \end {align*}
i.e. when \begin {align*} \theta = 0,& \quad \pi (\theta ) = P(Z > 1.645) = 0.05\\ \theta = 2,& \quad \pi (\theta ) = P(Z > 2\sqrt {n}) > 0.05 \end {align*} □

Example 3.4.3. Suppose that \(X_1, X_2, \cdots , X_{20}\) is a random sample from a normal distribution, \(X_i \thicksim N(0,\sigma ^2)\).

(a).: Derive the UMP size \(\alpha = 0.05\) test of \(H_0: \sigma = \sigma _0\) against \(H_1: \sigma < \sigma _0\).

Solution. Consider \(\, H_0: \sigma = \sigma _0\) versus \(H_1: \sigma = \sigma _1\quad (\sigma _1 < \sigma _0)\) \[\frac {\prod ^n_{i = 1} f(x_i;\theta _1)}{\prod ^n_{i = 1} f(x_i;\theta _0)}\, > \, k\] \[\frac {\prod ^n_{i = 1} \frac {1}{\sqrt {2\pi \sigma ^2_1}}e^{-\frac {1}{2\sigma ^2_1}x_i^2}}{\prod ^n_{i = 1} \frac {1}{\sqrt {2\pi \sigma ^2_0}}e^{-\frac {1}{2\sigma ^2_0}x_i^2}}\, > \, k\] \[\frac {(2\pi \sigma ^2_1)^{-\frac {n}{2}}e^{-\frac {1}{2\sigma ^2_1}\sum ^n_{i=1}x_i^2}}{(2\pi \sigma ^2_0)^{-\frac {n}{2}}e^{-\frac {1}{2\sigma ^2_0}\sum ^n_{i = 1}x_i^2}}\, > \, k\] \[\left (\frac {\sigma ^2_1}{\sigma ^2_0}\right )^{-\frac {n}{n}}e^{\frac {1}{2\sigma _0^2}\sum ^n_{i = 1}x_i^2 - \frac {1}{2\sigma ^2_1}\sum ^n_{i = 1} x_i^2}\, > \, k\] \[\left (\frac {1}{2\sigma ^2_0}-\frac {1}{2\sigma ^2_1}\right )\sum ^n_{i = 1} x_i^2\, > \, k_1\] \[\implies \quad \sum ^n_{i=1} x_i^2 < c\] i.e. \[R = \left \{(x_1, \cdots , x_n)\, \Big |\, \sum ^{20}_{i =1} x_i^2 < c\right \}\] is the critical region \[P_{H_0}(X\in R) = \alpha \] \[P_{H_0}\left (\sum ^{20}_{i = 1} X_i^2 < c\right ) = 0.05\] \[P\left (\sum ^{20}_{i = 1}\left (\frac {X_i - 0}{\sigma _0}\right )^2 < \frac {c}{\sigma ^2_0}\right ) = 0.05\] \[P\left (\chi ^2_{20} < \frac {c}{\sigma ^2_0}\right ) = 0.05\] \[\frac {c}{\sigma ^2_0}=\chi ^2_{20,0.95}\] \[\frac {c}{\sigma ^2_0} = 10.85\quad \implies \quad c = 10.85\sigma ^2_0.\] Therefore \[R = \left \{(x_1, \cdots , x_n) \, \Big |\, \sum ^{20}_{i = 1} x_i^2 < 10.85\sigma ^2_0\right \}\] is the critical region of the MP size \(\alpha = 0.05\) test of \(H_0: \sigma = \sigma _0\) vs \(H_1: \sigma = \sigma _1\quad (\sigma _1 < \sigma _0)\).
The construction of this test may use the fact that \(\sigma _1 < \sigma _0\). Therefore, the same critical region is obtained for all \(\sigma < \sigma _0\).
Therefore, \[R = \left \{(x_1, \cdots , x_n)\, \Big |\, \sum ^{20}_{i = 1} x^2_i < 10.85\sigma ^2_0\right \}\] is the critical region of the UMP size \(\alpha = 0.05\) test of \(H_0: \sigma = \sigma _0\) against \(H_1: \sigma < \sigma _0\). □
(b).: Sketch the power function of this test.

Solution. \begin {align*} \pi (\theta ) & = P_{\theta }(X\in R)\\ & = P_{\theta }\left (\sum ^{20}_{i = 1} X_i^2< 10.85\sigma ^2_0\right )\\ & = P\left (\sum ^{20}_{i = 1}\left [\frac {X_i - 0}{\sigma }\right ]^2 < \frac {10.85\sigma ^2_0}{\sigma ^2}\right )\\ & = P\left (\chi ^2_{20} < \frac {10.85\sigma ^2_0}{\sigma ^2}\right ) \end {align*}
When \begin {align*} \sigma ^2 = \sigma ^2_0, &\quad P(\chi ^2_{20} < 10.85) = 0.05, \quad \quad & (\sigma ^2_0 < \sigma ^2)\\ \sigma ^2 > \sigma ^2_0,&\quad P(\chi ^2_{20} < \underbrace {10.85 \left (\frac {\sigma ^2_0}{\sigma ^2}\right )}_{< 10.85}\, < \, 0.05, \quad \quad & \left (\frac {\sigma ^2_0}{\sigma ^2}< 1\right )\\ \sigma ^2 < \sigma ^2_0,&\quad P(\chi ^2_{20} < \underbrace {10.85 \left (\frac {\sigma ^2_0}{\sigma ^2}\right )}_{> 10.85}\, > 0.05\, \quad \quad & (\frac {\sigma _0^2}{\sigma ^2}>1) \end {align*} □

Example 3.4.4. Let \(X_1, X_2, \cdots , X_n\) be a random sample from the \(N(\theta ,1)\) distribution. We wish to test the hypothesis \(H_0: \theta = 0\) against \(H_1: \theta \neq 0\). Consider the critical region \[\left \{(x_1, x_2, \cdots , x_n)\, \Big |\, |\overline {x}| > \frac {1.96}{\sqrt {n}}\right \}.\]

(a).: Graph the power function of this test.

Solution. \begin {align*} \pi (\theta ) & = P_{\theta }(X\in R)\\ & = P_{\theta }\left (|\overline {X}|>\frac {1.96}{\sqrt {n}}\right )\\ & = P_{\theta }\left (\overline {X} > \frac {1.96}{\sqrt {n}}\right ) + P_{\theta }\left (\overline {X}< \frac {-1.96}{\sqrt {n}}\right )\\ & = P\left (\frac {\overline {X}-\theta }{\frac {\sigma }{\sqrt {n}}} > \frac {\frac {1.96}{\sqrt {n}}-\theta }{\frac {1}{\sqrt {n}}}\right ) + P\left (\frac {\overline {X}-\theta }{\frac {\sigma }{\sqrt {n}}}< \frac {\frac {-1.96}{\sqrt {n}}-\theta }{\frac {1}{\sqrt {n}}}\right )\\ & =P(Z < -1.96-\theta \sqrt {n}) + P(Z>1.96- \theta \sqrt {n}) \end {align*}
when \begin {align*} \theta = 0, & \quad P(Z < -1.96) + P(Z > 1.96) = 0.05\\ \theta = 1(n=1), & \quad P(Z < -2.96) + P(Z>0.96) = 0.1701\\ \theta = -1 (n=1), & \quad P(Z < -0.96) + P(Z> 2.96) = 0.1701 \end {align*} □
(b).: Is it uniformly most powerful?

Solution. \[R = \left \{(x_1, \cdots , x_n) \, \Big |\, \overline {x} > \frac {1.645}{\sqrt {n}}\right \}\] is the critical region of the UMP test of size \(\alpha = 0.05\) of \(H_0: \theta = 0\) against \(H_1: \theta = \theta _1\) if \(\theta _1 > 0\).
Similarly \[R = \left \{(x_1, \cdots , x_n) \, \Big |\, \overline {x} < \frac {1.645}{\sqrt {n}}\right \}\] is the critical region of the UMP test of size \(\alpha = 0.05\) of \(H_0: \theta = 0\) against \(H_1: \theta = \theta _1\) if \(\theta _1 < 0\).
Therefore, \[R = \left \{(x_1, \cdots , x_n) \, \Big |\, |\overline {x}| > \frac {1.96}{\sqrt {n}}\right \}\] is not a uniformly most powerful test of size \(\alpha = 0.05\) of \(\, H_0: \theta = 0\) against \(H_1: \theta \neq 0\).
i.e. There is no UMP test of \(H_0: \theta = 0\) against \(H_1: \theta \neq 0\). □

Definition 3.4.5. A joint probability (density) function \(f(x_1, \cdots , x_n;\theta )\) is said to have a monotone likelihood ratio (MLR) in the statistic \(T(X)\) if for any two values of the parameter, \(\theta _1 < \theta _2\), the ratio \[\frac {f(x_1, \cdots , x_n;\theta _2)}{f(x_1, \cdots , x_n;\theta _1)}\] depends on \(x\) only through the function \(t(x)\) and this ratio is a nondecreasing (or nonincreasing) function of \(t(x)\).

Theorem 3.4.6. If a joint probability (density) function \(f(x_1, \cdots , x_n; \theta )\) has a monotone likelihood ratio (nondecreasing) in the statistic \(T(X)\), then a UMP test of size \(\alpha \) for \(H_0: \theta = \theta _0\) versus \(H_1: \theta > \theta _0\) is to reject \(H_0\) if \(t(x) \geq k\), where \(P_{H_0}(T(X) \geq k) = \alpha \).

Note 3.4.7.

1.: If the MLR is nondecreasing then a UMP test of \(H_0: \theta = \theta _0\) versus \(H_1: \theta < \theta _0\) has critical region \(t(x) \leq k\).
2.: If the MLR is nonincreasing then a UMP test of \(H_0: \theta = \theta _0\) versus \(H_1: \theta > \theta _0\) has critical region \(t(x) \leq k\).
3.: Sometimes the hypotheses for the above UMP test are written as \[H_0:\, \theta \leq \theta _0\quad \text {versus}\quad H_1: \theta > \theta _0\]

Example 3.4.8. Let \(X_1, \cdots , X_n\) be a random sample from the distribution \[f(x;\theta ) = \theta ^x\, (1 - \theta )^{1 - x}\, , \quad 0 < \theta < 1.\] Find a UMP size \(\alpha \) test of \(H_0:\theta = \theta _0\) versus \(H_1: \theta > \theta _0\). Sketch the power function.

Solution. \[f(x_1, \cdots , x_n; \theta ) = \prod ^n_{i = 1} \theta ^{x_i}\, (1 - \theta )^{1 - x_i}\, = \, \theta ^{\sum ^n_{i = 1}} (1 - \theta )^{n - \sum ^n_{i = 1} x_i}.\] \begin {align*} \frac {f(x_1, \cdots , x_n; \theta _2)}{f(x_1, \cdots , x_n; \theta _1)} & = \frac {\theta _2^{\sum ^n_{i = 1}} (1 - \theta _2)^{n - \sum ^n_{i = 1} x_i}}{\theta _1^{\sum ^n_{i = 1}} (1 - \theta _1)^{n - \sum ^n_{i = 1} x_i}}\quad \text {for}\quad \theta _1 < \theta _2\\ & = \left (\frac {\theta _2}{\theta _1}\right )^{\sum x_i}\left (\frac {1 - \theta _2}{1 - \theta _1}\right )^n\left (\frac {1 - \theta _1}{1 - \theta _2}\right )^{\sum x_i}\\ & = \left (\frac {\theta _2(1 - \theta _1)}{\theta _1(1 - \theta _2)}\right )^{\sum x_i}\left (\frac {1 - \theta _2}{1 - \theta _1}\right )^n \end {align*}

Now, from \(\theta _1 < \theta _2\) we get \(\frac {\theta _2}{\theta _1} > 1\) and from \(-\theta _1 > - \theta _2 \implies 1 - \theta _1 > 1 - \theta _2\) \[\frac {1 - \theta _1}{1 - \theta _2}> 1\] i.e \[\frac {\theta _2(1 - \theta _1)}{\theta _1(1 - \theta _2)} >1\] Therefore, the ratio is a nondecreasing function of \(t(x) = \sum ^n_{i = 1} x_i\) i.e.a MLR in \(t(x)\).

Therefore, a UMP test has critical region \[\left \{(x_1, \cdots , x_n)\, \Big |\, \sum ^n_{i = 1} x_i \geq k\right \}\] where \[P_{H_0}\left (\sum ^n_{i = 1} X_i \geq k\right ) = \alpha .\] Let \(\, Y = \sum ^n_{i = 1} X_i\, \thicksim \, \operatorname {BIN}(n,\theta )\) \[P_{H_0}(Y \geq k) = \alpha \] \[\implies \quad \sum ^n_{y = k}\binom {n}{y}\theta ^y_0(1 - \theta _0)^{n-y} = \alpha \]

\(\pi (\theta ) = P_{\theta }(X\in R) = P_{\theta }(Y \geq k)\).

i.e. \(P_{\theta _0}(Y \geq k) = \alpha \) \[P_{\theta _0}(Y\geq k) < P_{\theta }(Y\geq k)\quad \text {for}\quad \theta _0 < \theta \] □

Example 3.4.9. Let \(X_1, \cdots , X_n\) be a random sample from the \(EXP(\theta )\) distribution with p.d.f \[f(x;\theta ) = \frac {1}{\theta }e^{-\frac {x}{\theta }}\, , \quad x > 0.\] Find a UMP size \(\alpha \) test of: \(H_0: \theta \geq \theta _0\) versus \(H_1: \theta < \theta _0\). Sketch the power function.

Solution. \[f(x_1, \cdots , x_n ; \theta ) = \prod ^n_{i = 1} \frac {1}{\theta }e^{-\frac {x_i}{\theta }} = \theta ^{-n}e^{-\frac {1}{\theta }\sum ^n_{i = 1} x_i}\] \begin {align*} \frac {f(x_1, \cdots , x_n ; \theta _2)}{f(x_1, \cdots , x_n ; \theta _1)} & = \frac {\theta _2^{-n}e^{-\frac {1}{\theta _2}\sum ^n_{i = 1} x_i}}{\theta _1^{-n}e^{-\frac {1}{\theta _1}\sum ^n_{i = 1} x_i}}\quad \quad \text {for}\quad \theta _1 < \theta _2\\ & = \left (\frac {\theta _1}{\theta _2}\right )^n\, e^{\frac {1}{\theta _1}\sum x_i - \frac {1}{\theta _2}\sum x_i}\\ & = \left (\frac {\theta _1}{\theta _2}\right )^n\, e^{\left (\frac {1}{\theta _1} - \frac {1}{\theta _2}\right )\sum x_i} \end {align*}

Therefore, the ratio is a nondecreasing function of \(t(x) = \sum ^n_{i = 1} x_i\). i.e a monotone likelihood ratio (MLR) in \(t(x)\).

Therefore, a UMP test of \(H_0: \theta \geq \theta _0\) versus \(H_1: \theta < \theta _0\) has critical region of the form \[R = \left \{(x_1, \cdots , x_n)\, : \, \sum ^n_{i = 1} x_i \leq c\right \}\] where \(c\) is determined from \[P_{H_0}\left (\sum ^n_{i = 1} X_i \leq c\right ) = \alpha \] Let \(Y = \sum ^n_{i = 1} \thicksim \operatorname {GAM}(n,\theta ).\) \[P_{H_0}(Y \leq c) = \alpha \] \[\int ^c_0\frac {1}{\theta ^n_0\, \Gamma (n)}\, y^{n- 1}\, e^{-\frac {x}{\theta _0}}\, dx = \alpha .\]

\(H_0: \, \theta \geq \theta _0\)

\(H_1:\, \theta < \theta _0\) \[\pi (\theta ) = P_{\theta }(X \in R) = P_{\theta }(Y \leq c).\] \[U = \frac {2Y}{\theta } \thicksim \chi ^2_{(2n)}\] \[P_{H_0}(Y \leq c) = \alpha \] \[P\left (\frac {2Y}{\theta _0} \leq \frac {2c}{\theta _0}\right ) = \alpha \] \[P\left (\chi ^2_{(2n)} \leq \frac {2c}{\theta _0}\right ) = \alpha \] \[\frac {2c}{\theta _0} = \chi ^2_{2n,1-\alpha }\quad \implies \quad c = \frac {\theta _0}{2}\, \chi ^2_{2n,1-\alpha }\] Therefore \[R = \left \{(x_1, \cdots , x_n):\, \sum ^n_{i = 1} x_i \leq \frac {\theta _0}{2}\, \chi ^2_{2n,1-\alpha }\right \}\] the critical region of the UMP test of size \(\alpha \) \[\pi (\theta ) = P_{\theta }\left (\chi ^2_{(2n)} \leq \frac {2c}{\theta }\right ).\] □

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.