Globally Optimal Parameters for On-Line Learning in Multilayer Neural Networks
- 29 September 1997
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review Letters
- Vol. 79 (13) , 2578-2581
- https://doi.org/10.1103/physrevlett.79.2578
Abstract
We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.Keywords
This publication has 9 references indexed in Scilit:
- On-line learning with adaptive back-propagation in two-layer networksPhysical Review E, 1997
- Neural Networks for Pattern RecognitionPublished by Oxford University Press (OUP) ,1995
- On-line backpropagation in two-layered neural networksJournal of Physics A: General Physics, 1995
- On-line learning in soft committee machinesPhysical Review E, 1995
- Exact Solution for On-Line Learning in Multilayer Neural NetworksPhysical Review Letters, 1995
- Learning by on-line gradient descentJournal of Physics A: General Physics, 1995
- Optimal generalization in perceptionsJournal of Physics A: General Physics, 1992
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Learning in Artificial Neural Networks: A Statistical PerspectiveNeural Computation, 1989