Mean Field Approach to Bayes Learning in Feed-Forward Neural Networks
- 11 March 1996
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review Letters
- Vol. 76 (11) , 1964-1967
- https://doi.org/10.1103/physrevlett.76.1964
Abstract
We propose an algorithm to realize Bayes optimal predictions for feed-forward networks which is based on the Thouless-Anderson-Palmer mean field method developed for the statistical mechanics of disordered systems. We conjecture that our approach will be exact in the thermodynamic limit. The algorithm results in a simple built-in leave-one-out cross validation of the predictions. Simulations for the case of the simple perceptron and the committee machine are in excellent agreement with the results of replica theory.Keywords
This publication has 11 references indexed in Scilit:
- The statistical mechanics of learning a ruleReviews of Modern Physics, 1993
- Optimal Learning with a Neural NetworkEurophysics Letters, 1993
- Generalization in a Large Committee MachineEurophysics Letters, 1992
- A Practical Bayesian Framework for Backpropagation NetworksNeural Computation, 1992
- Statistical mechanics of learning from examplesPhysical Review A, 1992
- Generalization performance of Bayes optimal classification algorithm for learning a perceptronPhysical Review Letters, 1991
- The space of interactions in neural networks: Gardner's computation with the cavity methodJournal of Physics A: General Physics, 1989
- The space of interactions in neural network modelsJournal of Physics A: General Physics, 1988
- Statistical Decision Theory and Bayesian AnalysisPublished by Springer Nature ,1985
- Solution of 'Solvable model of a spin glass'Philosophical Magazine, 1977