3.5 Choosing the functional form
Economic theory sometimes dictates the form: a Cobb–Douglas technology is log-log by assumption. More often it only restricts it (demand falls with price) and the data must decide. Three tools help.
Plots and residuals
Plot \(Y\) against \(X\), and plot the residuals of a trial model against \(X\) and against the fitted values. Systematic curvature in the residuals, as in the linear trend of Example 3.2.2, means the form is wrong. A spread that fans out as the fitted values grow suggests that a logarithmic dependent variable would stabilise the variance.
\(R^2\) is comparable only for the same dependent variable
\(R^2\) measures the share of the variance of the dependent variable explained. The models \(Y = \alpha + \beta X\) and \(Y = \alpha + \beta \log X\) explain the same variable, so their \(R^2\) values can be compared. The models \(Y = \alpha + \beta X\) and \(\log Y = \alpha + \beta X\) do not: one explains the variation in \(Y\) and the other the variation in \(\log Y\), and a higher \(R^2\) for one says nothing about which fits the data better.
The Box–Cox transformation
The linear and logarithmic forms are both members of one family.
Definition 3.5.1. For \(Y > 0\), the Box–Cox transformation with parameter \(\lambda \) is \[ Y^{(\lambda )} = \begin {cases} \dfrac {Y^{\lambda } - 1}{\lambda }, & \lambda \neq 0,\\[6pt] \log Y, & \lambda = 0. \end {cases} \]
- (i)
- \(Y^{(\lambda )}\) is continuous in \(\lambda \) at \(\lambda = 0\), and \(\lambda = 1\) gives \(Y - 1\), a linear model in \(Y\).
- (ii)
- Suppose \(Y_i^{(\lambda )} = \alpha + \beta X_i + u_i\) with \(u_i\) independent \(N(0, \sigma ^2)\). After maximising over \(\alpha \), \(\beta \) and \(\sigma ^2\), the log-likelihood of \(\lambda \), as a density for the observed \(Y\)’s, is \[ \ell (\lambda ) = \text {const} - \frac n2\log \frac {\text {RSS}(\lambda )}{n} + (\lambda - 1)\sum \log Y_i , \] where \(\text {RSS}(\lambda )\) is the residual sum of squares from regressing \(Y^{(\lambda )}\) on \(X\).
- (iii)
- If the data are first divided by their geometric mean \(\tilde Y = \exp \bigl (\frac 1n\sum \log Y_i\bigr )\), the last term vanishes, and models with different \(\lambda \) can be compared by their residual sums of squares alone.
Proof. (i) \(Y^{\lambda } = e^{\lambda \log Y}\), so by l’Hôpital’s rule \(\lim _{\lambda \to 0}(e^{\lambda \log Y} - 1)/\lambda = \lim _{\lambda \to 0}\log Y\,e^{\lambda \log Y} = \log Y\).
(ii) The normal model is a density for \(Z_i = Y_i^{(\lambda )}\). To obtain the density of \(Y_i\), multiply by the Jacobian \(dZ_i/dY_i = Y_i^{\lambda - 1}\), which contributes \((\lambda - 1)\sum \log Y_i\) to the log-likelihood. The normal part, maximised over \(\alpha \), \(\beta \) and \(\sigma ^2\), is \(-\frac n2\log (2\pi \hat \sigma ^2) - \frac n2\) with \(\hat \sigma ^2 = \text {RSS}(\lambda )/n\); this is shown for the general linear model in the section on maximum likelihood.
(iii) For \(Y_i^* = Y_i/\tilde Y\), \(\sum \log Y_i^* = \sum \log Y_i - n\log \tilde Y = 0\). □
The Jacobian term is the reason \(R^2\) and RSS cannot be compared across a linear and a logarithmic dependent variable: the two models put probability on different scales. Scaling by the geometric mean puts them on the same scale. The model with the smaller RSS then has the higher likelihood, and the difference in log-likelihoods is \(\frac n2\log (\text {RSS}_1/\text {RSS}_2)\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.