Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training
- 1 January 1994
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 5 (5) , 792-802
- https://doi.org/10.1109/72.317730
Abstract
We analyze the effects of analog noise on the synaptic arithmetic during multilayer perceptron training, by expanding the cost function to include noise-mediated terms. Predictions are made in the light of these calculations that suggest that fault tolerance, training quality and training trajectory should be improved by such noise-injection. Extensive simulation experiments on two distinct classification problems substantiate the claims. The results appear to be perfectly general for all training schemes where weights are adjusted incrementally, and have wide-ranging implications for all applications, particularly those involving "inaccurate" analog neural VLSI.Keywords
This publication has 13 references indexed in Scilit:
- Synaptic weight noise during multilayer perceptron training: fault tolerance and training improvementsIEEE Transactions on Neural Networks, 1993
- Multilayer Perceptron Learning Optimized for On-Chip Implementation: A Noise-Robust SystemNeural Computation, 1992
- Noise injection into inputs in back-propagation learningIEEE Transactions on Systems, Man, and Cybernetics, 1992
- Using additive noise in back-propagation trainingIEEE Transactions on Neural Networks, 1992
- Fault tolerance and redundancy of neural nets for the classification of acoustic dataPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1991
- A stochastic version of the delta rulePhysica D: Nonlinear Phenomena, 1990
- Sensitivity of feedforward neural networks to weight errorsIEEE Transactions on Neural Networks, 1990
- Curvature-Driven Smoothing in Backpropagation Neural NetworksPublished by Springer Nature ,1990
- Optimization by Simulated AnnealingScience, 1983
- Spontaneous behaviour in neural networksJournal of Theoretical Biology, 1972