Improving generalization performance using double backpropagation
- 1 January 1992
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 3 (6) , 991-997
- https://doi.org/10.1109/72.165600
Abstract
In order to generalize from a training set to a test set, it is desirable that small changes in the input space of a pattern do not change the output components. This can be done by forcing this behavior as part of the training algorithm. This is done in double backpropagation by forming an energy function that is the sum of the normal energy term found in backpropagation and an additional term that is a function of the Jacobian. Significant improvement is shown with different architectures and different test sets, especially with architectures that had previously been shown to have very good performance when trained using backpropagation. It is shown that double backpropagation, as compared to backpropagation, creates weights that are smaller, thereby causing the output of the neurons to spend more time in the linear region.Keywords
This publication has 4 references indexed in Scilit:
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989
- Learning in feedforward layered networks: the tiling algorithmJournal of Physics A: General Physics, 1989
- Factors influencing learning by backpropagationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1988
- Neural net pruning-why and howPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1988