1.1 Introduction

Assume that we have designed and collected data \(\, X_1\, , \, X_2\, , \, \cdots \cdots \cdots \, , \, X_n\,,\,\) where \(n\) is the sample size, is fixed and known.
Statistical inference deals with methods of using the outcomes to obtain information on the “true distribution function” (or “true parameter”) underlying the experiment.
The two main topics in statistical inference are estimation and hypothesis testing.

: In estimation, the observations are used to find an approximate value (estimate) for some numerical characteristic of the true distribution function. This can take the form of one numerical value (point estimation) or a set of possible values (interval estimation).
: In hypothesis testing, the observations are used to conclude whether or not the true distribution or true parameter belongs to a smaller subset of distribution functions or parameter space respectively.

Note. We will sometimes denote the random sample \(X_1\, , \, X_2, \, \cdots \,, \, X_n\) drawn from a probability (density) function (p.d.f) \(f_{\theta }(x)\) compactly by a random vector \( X = (X_1\, ,\, X_2, \, \cdots \, X_n)\) i.e \[f_{\theta }(x) = f_{\theta }(x_1\, , \, x_2\, , \, \cdots \, , \, x_n)\]

Example 1.1.1. Suppose \(\, X = (X_1\, ,\, X_2 , \, \cdots , \, X_n)\,\) is a random sample from a \(N(\mu \, ,\, \sigma ^2)\) distribution. Then \(\theta = (\mu \, , \, \sigma ^2)\) \[f_{\theta }(x) = \prod ^n_{i = 1}\, \dfrac {1}{\sqrt {2\pi \sigma ^2}}e^{-\dfrac {1}{2\sigma ^2}(x_i -\mu )^2}\] and \(\Omega = \big \{(\mu \, , \, \sigma ^2):\, -\infty < \mu < \infty \, , \, \sigma ^2 >0\big \}\,\) is the parameter space.

Note. Models are written in the form \(\big \{f_{\theta }(x)\, :\, \theta \in \Omega \big \}\,\) where \(\Omega \) is the parameter space and \(f_{\theta }(x)\) is the p.d.f.

Definition 1.1.2. A statistic, \(T(X)\), is a function of the data which does not depend on the unknown parameter \(\theta \).

e.g \(\hspace {0.5cm}\, \, \overline {X} = {\frac {1}{n}\sum ^n_{i = 1} x_i } = \) sample mean, \(S^2 = {\frac {1}{n - 1}\sum ^n_{i=1}(X_i - \overline {X})^2} = \) Sample variance,
\(X_{(1)} = \min (X_1\, ,\, X_2 , \, \cdots , \, X_n)\,\), \(X_{(n)} = \max (X_1\, ,\, X_2 , \,\cdots , \, X_n)\,\),
\((X_{(1)}\, ,\, X_{(2)}, \, \cdots , \, X_{(n)}) = \) order statistics.

\(\dfrac {\overline {X} - \mu }{\sigma /\sqrt {n}}\,\) is not a statistic if \(\mu \) and \(\sigma ^2\) are unknown.

Note.

1.: Although a statistic \(T(X)\) , is not a function of \(\theta \), its distribution can depend on \(\theta \).
2.: An estimator is a statistic used to estimate an unknown parameter \(\theta \).

i.e \(\,\) An estimator is a random variable. e.g \(\,\overline {X}\)

An estimate is an observed value of a random variable e.g \(\,\overline {x}\).

[next] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.