2.3 Population Moments

(A).

EXPECTATION: If \(\underline {X}\) is a random variable with pdf \(f\big (\underline {X}\big )\) then the expectation or mean of a scalar-valued function \(g(\underline {X})\) is defined as \begin {equation} \tag {1} E\Big (g(\underline {X})\Big ) = \int ^{\infty }_{-\infty } g(\underline {X})f\big (\underline {X}\big ) d\underline {X} \end {equation}

(B).

Properties of the expectation

(a)

Linearity \begin {equation} \tag {2} E[ag_1(\underline {X}) + b g_2(\underline {X})]= aE\Big (g_1(\underline {X})\Big ) + bE\Big (g_2(\underline {X})\Big ) \end {equation}

(b)

Partition \(\underline {X}' = \begin {pmatrix} \underline {X}_1', & \underline {X}_2'\\ \end {pmatrix}\) . The expectation of a function of \(\underline {X}_1\) may be written in terms of marginal distributions \(\underline {X}_1\) as follows \begin {align*} E\big \{g(\underline {X}_1)\big \} & = \int ^{\infty }_{-\infty } g(\underline {X}_1)\cdot f(\underline {X}) d\underline {X}\\ & = \int ^{\infty }_{-\infty } g(\underline {X}_1)\cdot f_1(x_1)dx_1 \end {align*}

where \(f_1(x_1)\) is known, the second expression is useful for computation.

(c)

If \(\underline {X}_1\) and \(\underline {X}_2\) are independent and \(g_i(\underline {X}_i)\) is a function of \(\underline {x}_i\) alone \(i=1(1,2)\) then \[E\big [g_1(\underline {x}_1)g_2(\underline {x}_2)\big ] = E\big [g_1(\underline {x}_1)\big ]E\big [g_2(\underline {x}_2)\big ]\] more generally the expectation of the matrix valued (or vector valued) functions of \(X\).
\(G(\underline {X}) = \Big (g_{ij}(\underline {X})\Big )\) is defined to be the matrix \begin {align*} E\big [G(\underline {X})\big ] & = \Bigg (E\big [g_{ij}(\underline {X})\big ]\Bigg )\\\\ G(\underline {X}) & = \begin {pmatrix} g_{11}(\underline {X}) & g_{12}(\underline {X})\\\\ g_{21}(\underline {X}) & g_{22}(\underline {X})\\ \end {pmatrix}\\\\ E\big [G(\underline {X})\big ] & = \begin {pmatrix} E\, g_{11}(\underline {X}) & E\, g_{12}(\underline {X})\\ & \\ E\, g_{21}(\underline {X}) & E\, g_{22} (\underline {X})\\ \end {pmatrix}\\ \end {align*}

(d)

The vector \(E(\underline {X}) = \underline {\mu }\) is called the population mean vector of \(\underline {X}\). Thus \[\mu _i = \int ^{\infty } _{\infty } x_i f(\underline {x}) d\underline {X}, i = 1,2,\ldots , p\] The population mean vector possesses the linearity property \[E\big (A\underline {X} + \underline {b}\big ) = AE(\underline {X}) + \underline {b}\] where \(A_{q\times p}\) and \(b_{q\times 1}\) are constants.

Note 2.4. \(\displaystyle {E(\underline {X}) = \begin {pmatrix} E(X_1)\\ E(X_2)\\ \vdots \\ \vdots \\ E(X_p)\\ \end {pmatrix} }\)

(e)

The matrix \[E\big \{(\underline {X}-\underline {\mu })(\underline {X}-\underline {\mu })'\big \} = \Sigma = var(\underline {X})\] The matrix that equal to \(\displaystyle {\Sigma }\) is called covariance of \(\underline {X}\) also known as variance-covariance matrix or dispersion matrix.
More generally when can desire the covariance between two vectors \(\underline {X}_{p\times 1}\) and \(\underline {Y}_{q\times 1}\) by the \(p \times q\) matrix. \[cov(\underline {X}, \underline {Y}) = E\big \{(\underline {X}-\underline {\mu }_X)(\underline {X}-\underline {\mu }_Y)'\big \}\] where \(E(\underline {X}) = \underline {\mu }_X\) and \(E(\underline {Y}) = \underline {\mu }_Y\).

E.g let \(\underline {X} = \begin {pmatrix} x_1, & x_2, & x_3\\ \end {pmatrix}'\) , \(\underline {Y}= \begin {pmatrix} y_1, & y_2\\ \end {pmatrix}'\)

\begin {align*} cov ( \underline {X},\underline {Y}) & = E\Bigg \{\begin {pmatrix} x_1 - \mu _1\\ x_2 - \mu _2\\ x_3 - \mu _3\\ \end {pmatrix} \begin {pmatrix} y_1 - \mu ^*_1, & y_2 - \mu ^*_2\\ \end {pmatrix} \Bigg \}\\\\ &= \begin {bmatrix} E ( x_1-\mu _1) ( y_1 - \mu ^*_1) & E (x_1-\mu _1) (y_2 - \mu ^*_2)\\\\ E (x_2 - \mu _2) (y_1 - \mu ^*_1) & E (x_2-\mu _2) (y_2 - \mu ^*_2)\\\\ E (x_3 - \mu _3) (y_1 - \mu ^*_1) & E (x_3-\mu _3) (y_2- \mu ^*_2)\\ \end {bmatrix}_{3\times 2}\\ \end {align*}

(C).

Properties of Covariances

let \(var(\underline {X})=\displaystyle {\Sigma }= \sigma _{ij}\)

(a)

\(\sigma _{ij} = Cov(X_i,X_j), i\neq j, \sigma _{ii} = var(X_i) = \sigma _i^2\)

(b)

\(\displaystyle {\Sigma } = E(\underline {X}\cdot \underline {X}')-\underline {\mu }\underline {\mu }'\) ( extension of \(var(X) = E(X^2)- \mu ^2\))

(c)

\(\displaystyle {var(\underline {a}'\underline {X})'= \underline {a}' var(\underline {X} \underline {a} = \sum ^p_{ij=1}a_ia_j\sigma _{ij}}\)

(d)

\(\displaystyle {\Sigma \geq 0}\) (ps.d)

(e)

\(var(A\underline {X} + \underline {b}) = A var(\underline {X}) A'\)

(f)

\( Cov(\underline {X}, \underline {X}) = var(\underline {X})\)

(g)

\(Cov(\underline {X}, \underline {Y}) = Cov( \underline {Y}, \underline {X})'\) see previous example

(h)

\(Cov(\underline {X}_1 + \underline {X}_2, \underline {Y}) = Cov(\underline {X}_1, \underline {Y}) + Cov(\underline {X}_2,\underline {Y})\)

(i)

If \(p=q\) \[var(\underline {X}+ \underline {Y}) = var(\underline {X}) + Cov(\underline {X},\underline {Y}) + Cov(\underline {Y}, \underline {X}) + var(\underline {Y})\]

(j)

\(Cov(A\underline {X},B\underline {Y}) = A Cov ( \underline {X},\underline {Y})B'\)

(k)

If \(\underline {X}\) and \(\underline {Y}\) are independent then \(Cov(\underline {X},\underline {Y}) = 0_{p\times q}\)

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.