Asymptotic Properties of Nonlinear Least Squares Estimates in Stochastic Regression Models

Stochastic regression models of the form $y_i = f_i(\theta) + \varepsilon_i$, where the random disturbances $\varepsilon_i$ form a martingale difference sequence with respect to an increasing sequence of $\sigma$-fields $\{\mathcal{G}_i\}$ and $f_i$ is a random $\mathcal{G}_{i - 1}$-measurable function of an unknown parameter $\theta$, cover a broad range of nonlinear (and linear) time series and stochastic process models. Herein strong consistency and asymptotic normality of the least squares estimate of $\theta$ in these stochastic regression models are established. In the linear case $f_i(\theta) = \theta^T\psi_i$, they reduce to known results on the linear least squares estimate $(\sum^n_1\psi_i\psi^T_i)^{-1}\sum^n_1\psi_i y_i$ with stochastic $\mathcal{G}_{i - 1}$-measurable regressors $\psi_i$.