Fast parallel off-line training of multilayer perceptrons
- 1 May 1997
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 8 (3) , 646-653
- https://doi.org/10.1109/72.572103
Abstract
Various approaches to the parallel implementation of second-order gradient-based multilayer perceptron training algorithms are described. Two main classes of algorithm are defined involving Hessian and conjugate gradient-based methods. The limited- and full-memory Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithms are selected as representative examples and used to show that the step size and gradient calculations are critical components. For larger problems the matrix calculations in the full-memory algorithm are also significant. Various strategies are considered for parallelization, the best of which is implemented on parallel virtual machine (PVM) and transputer-based architectures. Results from a range of problems are used to demonstrate the performance achievable with each architecture. The transputer implementation is found to give excellent speed-ups but the problem size is limited by memory constraints. The speed-ups achievable with the PVM implementation are much poorer because of inefficient communication, but memory is not a difficulty.Keywords
This publication has 10 references indexed in Scilit:
- Neural network modelling of a 200 MW boiler systemIEE Proceedings - Control Theory and Applications, 1995
- Online neural control applied to a bank-to-turn missile autopilotPublished by American Institute of Aeronautics and Astronautics (AIAA) ,1995
- Fast Gradient Based Off-Line Training of Multilayer PerceptronsPublished by Springer Nature ,1995
- Steepest descent algorithms for neural network controllers and filtersIEEE Transactions on Neural Networks, 1994
- Comparative Aspects of Neural Network Algorithms for On-Line Modelling of Dynamic ProcessesProceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 1993
- SUPERVISED LEARNING ON LARGE REDUNDANT TRAINING SETSInternational Journal of Neural Systems, 1993
- A scaled conjugate gradient algorithm for fast supervised learningNeural Networks, 1993
- Nonlinear internal model control strategy for neural network modelsComputers & Chemical Engineering, 1992
- Neural networks for control systems—A surveyAutomatica, 1992
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988