Comparative Aspects of Neural Network Algorithms for On-Line Modelling of Dynamic Processes

Abstract
This paper reviews the model structures and learning rules of four commonly used artificial neural networks: the cerebellar model articulation controller (CMAC), B-splines, radial basis functions (RBF) and multi-layered perceptron (MLP) networks. Their dynamic modelling abilities are compared using a two-dimensional non-linear noisy time series. The network performances are evaluated based on their network surface plots, phase/time history plots, learning curves, prediction error autocorrelation functions and finally their short-range prediction error variances. The modelling results suggest that all four networks were able to capture the underlying dynamics of the time series. Also, specific prior knowledge about the time series was incorporated into the B-splines model, and is used to highlight an important trade-off between the model flexibility and high-dimensional modelling ability in the B-splines and CM AC networks. In general, when the network model is well conditioned and linear with respect to its adaptable parameters, simpler on-line learning rules often provide adequate convergence properties. Alternatively, when the model is highly non-linear, complicated learning rules which utilize high-order gradient information are generally required at the expense of increased computational complexity.