Reduction of required precision bits for back-propagation applied to pattern recognition
- 1 March 1993
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 4 (2) , 270-275
- https://doi.org/10.1109/72.207614
Abstract
The number of precision bits for operations and data are limited in the hardware implementations of backpropagation (BP). Reduction of rounding error due to this limited precision is crucial in the implementation. The new learning algorithm is based on overestimation of significant error in order to alleviate underflow and omission of weight updating for correctly recognized patterns. While the conventional BP algorithm minimizes the squared error between output signals and supervising data, the new learning algorithm minimizes the weighted error function. In the learning simulation of multifont capital recognition, this algorithm converged recognition accuracy to 100% with only 8-b precision. In addition, the recognition accuracy for characters that did not appear in the training data reached 94.9%. This performance is equivalent to that of a conventional BP with 12-b precision. Moreover, it is found that the performance of the weighted error function is high even when only a small number of hidden neurons is used. Consequently, the algorithm reduces the required amount of weight memory.Keywords
This publication has 8 references indexed in Scilit:
- Characterization of artificial neural network algorithmsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Back propagation simulations using limited precision calculationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Finite precision error analysis of neural network electronic hardware implementationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Back propagation learning with trinary quantization of weight updatesNeural Networks, 1991
- Training layered perceptrons using low accuracy computationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1991
- A digital multilayer neural network with limited binary expressionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- On the approximate realization of continuous mappings by neural networksNeural Networks, 1989
- Asynchronous VLSI neural networks using pulse-stream arithmeticIEEE Journal of Solid-State Circuits, 1988