An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer
- 1 January 1995
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 6 (1) , 31-42
- https://doi.org/10.1109/72.363452
Abstract
Multilayer perceptrons are successfully used in an increasing number of nonlinear signal processing applications. The backpropagation learning algorithm, or variations hereof, is the standard method applied to the nonlinear optimization problem of adjusting the weights in the network in order to minimize a given cost function. However, backpropagation as a steepest descent approach is too slow for many applications. In this paper a new learning procedure is presented which is based on a linearization of the nonlinear processing elements and the optimization of the multilayer perceptron layer by layer. In order to limit the introduced linearization error a penalty term is added to the cost function. The new learning algorithm is applied to the problem of nonlinear prediction of chaotic time series. The proposed algorithm yields results in both accuracy and convergence rates which are orders of magnitude superior compared to conventional backpropagation learning.Keywords
This publication has 18 references indexed in Scilit:
- Continuous-time temporal back-propagation with adaptable time delaysIEEE Transactions on Neural Networks, 1993
- On a class of efficient learning algorithms for neural networksNeural Networks, 1992
- A tree-structured adaptive network for function approximation in high-dimensional spacesIEEE Transactions on Neural Networks, 1991
- Networks for approximation and learningProceedings of the IEEE, 1990
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988
- Predicting chaotic time seriesPhysical Review Letters, 1987
- Ergodic theory of chaos and strange attractorsReviews of Modern Physics, 1985
- Measuring the strangeness of strange attractorsPhysica D: Nonlinear Phenomena, 1983
- Geometry from a Time SeriesPhysical Review Letters, 1980
- Oscillation and Chaos in Physiological Control SystemsScience, 1977