3.2 Equivariant Estimators

Some models have a symmetry, and it is natural to ask that the estimator have the same symmetry.

Definition 3.2.1. A model \(\{f_\theta (x) : \theta \in \mathbb {R}\}\) with \(f_\theta (x) = f_0(x - \theta )\) for a known density \(f_0\) is a location family, and \(\theta \) is a location parameter.

Example 3.2.2. Show that \(N(\theta , 1)\), the Cauchy distribution with location \(\theta \), and the shifted exponential \(f_\theta (x) = e^{-(x - \theta )}\), \(x>\theta \), are location families.

Solution. In each case \(f_\theta (x) = f_0(x - \theta )\), with respectively \[f_0(z) = \frac {1}{\sqrt {2\pi }}e^{-z^2/2},\qquad f_0(z) = \frac {1}{\pi (1 + z^2)},\qquad f_0(z) = e^{-z}I(z>0) .\] In each, \(X = \theta + Z\) where \(Z\) has the fixed density \(f_0\).

If \(X_i = \theta + Z_i\), shifting every observation by \(a\) is the same as shifting the parameter by \(a\). A sensible estimator should therefore shift by \(a\) too.

Definition 3.2.3. An estimator \(\tilde \theta (X_1, \dots , X_n)\) is location equivariant if \[\tilde \theta (x_1 + a, \dots , x_n + a) = \tilde \theta (x_1, \dots , x_n) + a\] for every constant \(a\) and every sample \((x_1, \dots , x_n)\).

Example 3.2.4. Show that the sample mean, the sample median and \(X_{(1)}\) are location equivariant.

Solution. \(\frac 1n\sum (x_i + a) = \frac 1n\sum x_i + \frac 1n\cdot na = \bar x + a\). Adding \(a\) to every observation does not change their order, so every order statistic, and in particular the median and \(X_{(1)}\), increases by \(a\). By contrast, \(\frac 1n\sum x_i^2\) is not equivariant.

Theorem 3.2.5. In a location family, the bias, variance and mean square error of a location equivariant estimator do not depend on \(\theta \).

Proof. Write \(X_i = \theta + Z_i\), where \(Z_1, \dots , Z_n\) are independent with the known density \(f_0\). By equivariance with \(a = \theta \), \[\tilde \theta (X_1, \dots , X_n) - \theta = \tilde \theta (Z_1, \dots , Z_n),\] whose distribution involves \(f_0\) only. Hence the distribution of the error \(\tilde \theta - \theta \), and every moment of it, is the same for all \(\theta \). □

Since every equivariant estimator has constant risk, they can be compared by a single number, and there is a best one. It is called the Pitman estimator.

Example 3.2.6. For a random sample from the shifted exponential \(f_\theta (x) = e^{-(x - \theta )}\), \(x>\theta \), find the best estimator, in mean square error, of the form \(X_{(1)} - c\).

Solution. Each such estimator is equivariant. By example 2.5.7, \(X_{(1)} - \theta \) is exponential with mean \(\frac 1n\) and variance \(\frac 1{n^2}\), so \[\MSE \left (X_{(1)} - c\right ) = E\left (X_{(1)} - \theta - c\right )^2 = \frac {1}{n^2} + \left (\frac 1n - c\right )^2 ,\] which, as theorem 3.2.5 says, is free of \(\theta \). It is minimised at \(c = \frac 1n\), giving the unbiased estimator \(X_{(1)} - \frac 1n\) with mean square error \(\frac {1}{n^2}\), half that of the ML estimator \(X_{(1)}\).

Definition 3.2.7. A model \(\{f_\theta (x) : \theta >0\}\) with \(f_\theta (x) = \frac 1\theta f_1\left (\frac x\theta \right )\) for a known density \(f_1\) is a scale family, and \(\theta \) is a scale parameter. An estimator \(\tilde \theta _k\) of \(\theta ^k\) is scale equivariant if \[\tilde \theta _k(cx_1, \dots , cx_n) = c^k\,\tilde \theta _k(x_1, \dots , x_n)\] for every \(c>0\) and every sample.

In a scale family \(X = \theta Z\) with \(Z\) having density \(f_1\), so changing the unit of measurement multiplies \(\theta \) by \(c\) and \(\theta ^k\) by \(c^k\).

Example 3.2.8. Let \(X_1, \dots , X_n\) be a random sample from \(N(0, \theta ^2)\), \(\theta >0\). Show that

(a)
this is a scale family;
(b)
the UMVUE of \(\theta ^2\) is scale equivariant.

Solution.

(a)
\(f_\theta (x) = \frac {1}{\sqrt {2\pi }\,\theta }e^{-\frac 12(x/\theta )^2} = \frac 1\theta f_1\left (\frac x\theta \right )\) with \(f_1(z) = \frac {1}{\sqrt {2\pi }}e^{-z^2/2}\).
(b)
The family is a REF with natural sufficient statistic \(\sum X_i^2\), which is complete. Since \(E(X_i^2) = \theta ^2\), the UMVUE of \(\theta ^2\) is \(\tilde \theta _2 = \frac 1n\sum X_i^2\), and \[\tilde \theta _2(cx_1, \dots , cx_n) = \frac 1n\sum (cx_i)^2 = c^2\,\tilde \theta _2(x_1, \dots , x_n),\] so it is scale equivariant with \(k = 2\).

Problem 3.2.1. Show that the \(UNIF(\theta - 1, \theta + 1)\) distribution is a location invariant family.

Show solution

Solution. The UNIF\((\theta -1,\theta +1)\) density is \[f_{\theta }(x) = \frac 12\,I\left (\theta -1<x<\theta +1\right ) = \frac 12\,I\left (-1<x-\theta <1\right ) = f_0(x-\theta ),\] where \(f_0\) is the UNIF\((-1,1)\) density. Since every member of the family is obtained from the single density \(f_0\) by a shift, the family is location invariant with location parameter \(\theta \).

Equivalently: if \(X\sim \) UNIF\((\theta -1,\theta +1)\) then \(X+c\sim \) UNIF\((\theta +c-1,\ \theta +c+1)\), so translating the data by \(c\) moves the parameter by \(c\) and leaves the family as a whole unchanged.

Problem 3.2.2. Let \(X_1, \, \cdots \, , \, X_n\) be a random sample the \(UNIF(0,\theta )\) distribution. Show that then

(a)
\(UNIF(0,\theta )\) densities is a scale invariant family.
(b)
\(UMVUE\) of \(\theta \) is a scale equivariant estimator.

Show solution

Solution.

(a)

The UNIF\((0,\theta )\) density is \[f_{\theta }(x) = \frac {1}{\theta }\,I\left (0<x<\theta \right ) = \frac {1}{\theta }\,I\left (0<\frac {x}{\theta }<1\right ) = \frac {1}{\theta }f_1\!\left (\frac {x}{\theta }\right ),\] with \(f_1\) the UNIF\((0,1)\) density. Every member is obtained from \(f_1\) by the scaling \(x\mapsto x/\theta \), so the family is scale invariant. Equivalently, if \(X\sim \) UNIF\((0,\theta )\) and \(c>0\) then \(cX\sim \) UNIF\((0,c\theta )\).

(b)

Since \(E\left (X_{(n)}\right )=n\theta /(n+1)\) and \(X_{(n)}\) is complete and sufficient, the UMVUE is \[\widehat {\theta } = \frac {n+1}{n}X_{(n)} .\] Scale equivariance asks that \(\widehat {\theta }(cX_1,\dots ,cX_n) = c\,\widehat {\theta }(X_1,\dots ,X_n)\) for every \(c>0\). The maximum of the rescaled sample is \(c\,X_{(n)}\), so \[\widehat {\theta }(cX) = \frac {n+1}{n}\,cX_{(n)} = c\,\widehat {\theta }(X),\] and the estimator is scale equivariant.

Note. Equivariance is a consistency requirement, not an optimality one: it says only that measuring the data in different units should change the estimate in the same units. It is worth checking because an estimator that fails it is answering a different question depending on whether the lifetimes were recorded in days or in hours.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.