Simultaneous Confidence Intervals

The first of the three methods. Bonferroni’s inequality requires no distributional theory beyond what is already available: if \(m\) statements are each made at level \(\alpha /m\), then the probability that any one of them fails is at most \(\alpha \), whatever the dependence between them.

Using Bonferroni’s Inequality

For any events \(A_1, A_2,\ldots , A_k\) \[P\Bigg (\bigcup ^k_{j=1}A_j\Bigg )\leq \sum ^k_{j=1}P\big (A_j\big ).\] This inequality is known as Bonferroni’s inequality. Applying this inequality to confidence intervals we have the next Lemma.

Step 1: Check if it is true for \(n\geq a\), here \(n=2\).
Step 2: Assume it is true for \(n=k\).
Step 3: Show that if it is true for \(n=k\) then it is true for \(n=k+1\)

Proof.

1.: \(P\big (A_1\cup A_2\big )=P(A_1)+P(A_2)-P(A_1\cap A_2)\)
2.: \(P\Bigg (\bigcup \limits ^k_{j=1}A_j\Bigg )\leq \displaystyle {\sum ^k_{j=1}P(A_j)}\)
3.: We show that \(\displaystyle {P\Bigg (\bigcup ^{k+1}_{j=1}A_j\Bigg )\leq \sum ^{k+1}_{j=1}P(A_j)}\) \begin {align*} \text {L.H.S}\quad P\Bigg (\bigcup ^{k+1}_{j=1}A_j\Bigg ) & = P\Bigg (\bigg (\bigcup ^k_{j=1}A_j\big )\cup A_{k+1}\Bigg )\\ & = P(D)+P(A_{k+1})-P(D\cup A_{k+1}),\quad D=\bigcup ^k_{j=1}A_j\\ &\leq \sum ^k_{j=1}P(A_j)+P(A_{k+1})-P(D\cap A_{k+1})\\ & \leq \sum ^{k+1}_{j=1}P(A_j)\\ \end {align*}

□

Lemma 3.7.2. Let \(I_j(Y)\) be a \(100(1-\alpha )\%\) confidence interval for the parameter \(\gamma _j,\, j=1,2,\ldots ,k\). The probability that all \(k\) intervals simultaneously contain their parameter is greater than or equal to \[1-\sum ^k_{j=1}\alpha _j\quad \text {i.e}\quad P\Bigg \{\bigcap ^k_{j=1}\Big (\gamma _j\in I_j(Y)\Big )\Bigg \}\geq 1-\sum ^k_{j=1}\alpha _j\]

Proof. Apply the Bonferroni’s Inequality. \begin {align*} P\Big \{\bigcap ^k_{j=1}\big (\gamma _j\in I_j(Y)\big )\Big \} & = 1- P\Big \{\bigcap ^k_{j=1}\big (\gamma _j\in I_j(Y)\big )\Big \}^c\\ & = 1 -P\Big \{\bigcup ^k_{j=1}\big (\gamma _j\not \in I_j(Y)\big )\Big \}\\ \therefore P\Bigg (\bigcap ^k_{j=1}\big (\gamma _j\in I_j(Y)\big )\Bigg ) & \geq 1-\sum ^k_{j=1}\alpha _j \end {align*}

Usually \(\alpha _{j'}\)s are all equal. Thus if we want simultaneous \(100(1-\alpha )\%\) C.Is of \(\beta _{j'}\)s we set \[\alpha _j=\frac {\alpha }{k}\] □

Definition 3.7.3.

1.: A contrast of parameters \(\beta _1,\beta _2,\ldots ,\beta _k\) is a linear combination of the parameter with coefficients adding up to zero \[\text {i.e}\quad \sum ^k_{j=1}C_j\beta _j=C^tB\] such that \(\sum \limits ^k_{j=1}C_j=0\).
2.: A simple contrast is a contrast such that there are only two non-zero elements of the vector \(C\), a one (1) and a minus one \((-1)\). \[\text {i.e}\quad \beta _1-\beta _2=C^tB,\quad C^t= \begin {pmatrix} 1 & -1 & 0 & \cdots & 0\\ \end {pmatrix} \] \(\beta _j-\beta _{j'}=C^tB,\quad C\) has one(1) in the \(j^{\text {th}}\) position and minus one \((-1)\) in the \(j'^{\text {th}}\) position.

Example 3.7.4. Suppose in a linear model \[Y=XB+\varepsilon \] \(Y\) is \(15\times 1\), \(\quad X\) is \(15\times 3\), \(\quad B\) is \(3\times 1\), \(\quad \varepsilon \) is \(15\times 1\), \[X= \begin {pmatrix} 1_5 & 0 & 0\\ 0 & 1_5 & 0\\ 0 & 0 & 1_5\\ \end {pmatrix} \,,\quad \widehat {B}= \begin {pmatrix} 29.4 & 29.6 & 28.0\\ \end {pmatrix}^t \,, \quad \text {MSE} = 9.7\]

1.

(a): Find \(95\%\) C.I for \(\beta _2\).
(b): Find \(95\%\) C.I for \(\beta _3-\beta _1\).
(c): Find \(95\%\) C.I for \(\frac {\beta _1+\beta _2+\beta _3}{3}\).

2.

(a): Simultaneous C.Is for \(\beta _1,\beta _2,\beta _3\).
(b): Simultaneous C.I for all the simple contrasts.

Solution.

\[\widehat {B}= \begin {pmatrix} 29.4 & 29.6 & 28.0\\ \end {pmatrix}^t \,, \quad \text {MSE} = 9.7\, ,\quad X^tX= \begin {pmatrix} 5 & 0 & 0\\ 0 & 5 & 0\\ 0 & 0 & 5\\ \end {pmatrix} \]

1.

(a): \(95\%\) C.I for \(\beta _2\) \begin {align*} \widehat {\beta }_2 & \pm t^{0.05}_{12}\,\sqrt {\text {MSE}\,\big (X^tX\big )^{-1}_{22}}\\ 29.6 & \pm 2.179\sqrt {9.7\times \frac {1}{5}}\\ \end {align*}
(b): \(95\%\) C.I for \(\beta _3-\beta _1\) \begin {align*} \widehat {\beta }_3-\widehat {\beta }_1 & \pm t^{0.05}_{12}\sqrt {\text {MSE}\,C^t\big (X^tX\big )^{-1}C}\quad ,\quad C= \begin {pmatrix} -1 & 0 & 1\\ \end {pmatrix}^t\\ 28.0-29.4 & \pm 2.179\sqrt {9.7\times \big (\frac {1}{5}+\frac {1}{5}\big )}\\ \end {align*}
(c): \(95\%\) C.I for \(\frac {\beta _1+\beta _2+\beta _3}{3}\) \begin {align*} \frac {\widehat {\beta }_1+\widehat {\beta }_2+\widehat {\beta }_3}{3} & \pm t^{0.05}_{12}\sqrt {\text {MSE}\, C^t\big (X^tX\big )^{-1}C}\qquad C= \begin {pmatrix} \frac {1}{3} & \frac {1}{3}&\frac {1}{3}\\ \end {pmatrix}^t\\ \frac {29.4+29.6+28.0}{3} & \pm 2.179\sqrt {9.7\times \frac {3}{9\times 5}}\\ \end {align*}

2.

(a)

Simultaneous \(94\%\) C.I for \[\widehat {\beta }_1,\quad \widehat {\beta }_2,\quad \widehat {\beta }_3,\qquad \frac {\alpha }{3}=\frac {0.06}{3}=0.02\] \begin {align*} \beta _1 :\quad & \widehat {\beta }_1 \pm t^{0.01}_{12}\sqrt {\text {MSE}\,\big (X^tX\big )^{-1}_{11}}\\ & 29.4\pm 2.68\sqrt {9.7\times \frac {1}{5}} \end {align*}

\begin {align*} \beta _2 :\quad & \widehat {\beta }_2 \pm t^{0.01}_{12}\sqrt {\text {MSE}\,\big (X^tX\big )^{-1}_{22}}\\ & 29.6 \pm 2.68 \sqrt {9.7\times \frac {1}{5}} \end {align*}

\begin {align*} \beta _3 :\quad & \widehat {\beta }_3 \pm t^{0.01}_{12}\sqrt {\text {MSE}\,\big (X^tX\big )^{-1}_{33}}\\ & 28.0 \pm 2.681\sqrt {9.7\times \frac {1}{5}}\\ & 28.0 \pm 3.696\\ \end {align*}

(b)

\(94\%\) simultaneous for all the simple contrast \[\widehat {\beta }_1-\widehat {\beta }_2,\quad \widehat {\beta }_1-\widehat {\beta }_3,\quad \widehat {\beta }_2-\widehat {\beta }_3\] \[\frac {\alpha }{2}=\frac {0.06}{3}=0.02\] \begin {align*} \beta _1-\beta _2:\quad & \widehat {\beta }_1-\widehat {\beta }_2 \pm t^{0.01}_{12}\sqrt {\text {MSE}\,C^t\big (X^tX\big )^{-1}C}\\ & 29.4-29.6 \pm 2.68\sqrt {9.7\times \frac {2}{5}}\,,\quad C= \begin {pmatrix} 1 & -1 & 0\\ \end {pmatrix}^t \end {align*}

\begin {align*} \beta _1-\beta _3:\quad & \widehat {\beta }_1-\widehat {\beta }_3 \pm t^{0.01}_{12}\sqrt {\text {MSE}\,C^t\big (X^tX\big )^{-1}C}\\ & 29.4-28.0 \pm 2.681\sqrt {9.7\times \frac {2}{5}}\,,\quad C= \begin {pmatrix} 1 & 0 & -1\\ \end {pmatrix}^t \end {align*}

\begin {align*} \beta _2-\beta _3 :\quad & 29.6-28.0 \pm 2.681\sqrt {9.7\times \frac {2}{5}}\\ & (29.6-28.0)\pm 5.281\\ \end {align*}

[next] [prev] [prev-tail] [front] [up]