Efficient classification for multiclass problems using modular neural networks
- 1 January 1995
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 6 (1) , 117-124
- https://doi.org/10.1109/72.363444
Abstract
The rate of convergence of net output error is very low when training feedforward neural networks for multiclass problems using the backpropagation algorithm. While backpropagation will reduce the Euclidean distance between the actual and desired output vectors, the differences between some of the components of these vectors increase in the first iteration. Furthermore, the magnitudes of subsequent weight changes in each iteration are very small, so that many iterations are required to compensate for the increased error in some components in the initial iterations. Our approach is to use a modular network architecture, reducing a K-class problem to a set of K two-class problems, with a separately trained network for each of the simpler problems. Speedups of one order of magnitude have been obtained experimentally, and in some cases convergence was possible using the modular approach but not using a nonmodular network.<>Keywords
This publication has 10 references indexed in Scilit:
- An improved algorithm for neural network classification of imbalanced training setsIEEE Transactions on Neural Networks, 1993
- Multiple Network Systems (Minos) Modules: Task Division and Module DiscriminationPublished by Springer Nature ,1991
- SuperSAB: Fast adaptive back propagation with good scaling propertiesNeural Networks, 1990
- Why are “What” and “Where” Processed by Separate Cortical Visual Systems? A Computational InvestigationJournal of Cognitive Neuroscience, 1989
- Modularity and scaling in large phonemic neural networksIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- Accelerating the convergence of the back-propagation methodBiological Cybernetics, 1988
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988
- Statistical pattern recognition with neural networks: benchmarking studiesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1988
- Learning sets of filters using back-propagationComputer Speech & Language, 1987
- The Modularity of MindPublished by MIT Press ,1983