12 Non linear estimation

Every model so far has been linear in its parameters, even when, like a polynomial, it was curved in \(x\). Many models from science and engineering are not. A drug’s concentration in the blood decays as \(\theta _1e^{-\theta _2x}\); the rate of an enzyme reaction follows \(\theta _1x/(\theta _2 + x)\); a population grows towards a ceiling along an S-shaped curve. The parameters have physical meanings: a half-life, a maximum rate, a ceiling. This section treats the non-linear regression model \[ y_i = f(\mathbf {x}_i, \boldsymbol \theta ) + \varepsilon _i , \qquad i = 1, \dots , n , \] where \(f\) is a known function, non-linear in the \(p\) parameters \(\boldsymbol \theta = (\theta _1, \dots , \theta _p)^\top \), and the \(\varepsilon _i\) are independent with mean \(0\) and variance \(\sigma ^2\). Least squares still defines the estimates, but they can no longer be written down: they must be found by iteration, and their properties hold only approximately.

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.