Bias in Nonlinear Estimation

Abstract
Although it is widely recognized that maximum‐likelihood estimates of the parameters in non‐linear models are generally biased, little work appears to have been done on quantitatively assessing these biases. In this paper the difficulties of exact calculation of the bias for a simple example are first illustrated, after which a general method of calculating the biases in a class of nonlinear least‐squares problems is presented. The bias in Bayesian estimation is also considered, although the illustrative examples are all for the case of a uniform prior, i.e. the estimation is maximum likelihood. In the most important of the subsidiary results, a generalized ratio of the bias to the variance–covariance matrix of the parameter estimates is defined, and shown to be closely related to Beale's (1960) measures of nonlinearity. Finally the philosophy is applied to a somewhat different problem, namely determining the biases in the maximum‐likelihood estimates of the parameters of the gamma distribution.