On-line Gibbs Learning
- 15 April 1996
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review Letters
- Vol. 76 (16) , 3021-3024
- https://doi.org/10.1103/physrevlett.76.3021
Abstract
We propose a new model of on-line learning which is appropriate for learning realizable and unrealizable, smooth as well as threshold, functions. Following each presentation of an example the new weights are chosen from a Gibbs distribution with an on-line energy that balances the need to minimize the instantaneous error against the need to minimize the change in the weights. We show that this algorithm finds the weights that minimize the generalization error in the limit of an infinite number of examples. The asymptotic rate of convergence is similar to that of batch learning.Keywords
This publication has 5 references indexed in Scilit:
- Learning from noisy data: An exactly solvable modelPhysical Review E, 1995
- Local and Global Convergence of On-Line LearningPhysical Review Letters, 1995
- On-Line Learning with a PerceptronEurophysics Letters, 1994
- Statistical mechanics of learning from examplesPhysical Review A, 1992
- Learning processes in neural networksPhysical Review A, 1991