On Langevin Updating in Multilayer Perceptrons

1 September 1994

journal article
Published by MIT Press in Neural Computation

Vol. 6 (5) , 916-926
https://doi.org/10.1162/neco.1994.6.5.916

Abstract

The Langevin updating rule, in which noise is added to the weights during learning, is presented and shown to improve learning on problems with initially ill-conditioned Hessians. This is particularly important for multilayer perceptrons with many hidden layers, that often have ill-conditioned Hessians. In addition, Manhattan updating is shown to have a similar effect.

Keywords

This publication has 10 references indexed in Scilit:

Cooling schedules for learning in neural networks
Physical Review E, 1993
Pattern Discrimination Using Feedforward Networks: A Benchmark Study of Scaling Behavior
Neural Computation, 1993
Multilayer Perceptron Learning Optimized for On-Chip Implementation: A Noise-Robust System
Neural Computation, 1992
First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method
Neural Computation, 1992
Mass reconstruction with a neural network
Physics Letters B, 1992
Using additive noise in back-propagation training
IEEE Transactions on Neural Networks, 1992
Predicting the Future: Advantages of Semilocal Units
Neural Computation, 1991
Creating artificial neural networks that generalize
Neural Networks, 1991
Explorations of the mean field theory learning algorithm
Neural Networks, 1989
On the complex Langevin equation
Nuclear Physics B, 1988