A review of Bayesian neural networks with an application to near infrared spectroscopy
- 1 January 1996
- journal article
- review article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 7 (1) , 56-72
- https://doi.org/10.1109/72.478392
Abstract
MacKay's (1992) Bayesian framework for backpropagation is a practical and powerful means to improve the generalization ability of neural networks. It is based on a Gaussian approximation to the posterior weight distribution. The framework is extended, reviewed, and demonstrated in a pedagogical way. The notation is simplified using the ordinary weight decay parameter, and a detailed and explicit procedure for adjusting several weight decay parameters is given. Bayesian backprop is applied in the prediction of fat content in minced meat from near infrared spectra. It outperforms "early stopping" as well as quadratic regression. The evidence of a committee of differently trained networks is computed, and the corresponding improved generalization is verified. The error bars on the predictions of the fat content are computed. There are three contributors: The random noise, the uncertainty in the weights, and the deviation among the committee members. The Bayesian framework is compared to Moody's GPE (1992). Finally, MacKay and Neal's automatic relevance determination, in which the weight decay parameters depend on the input number, is applied to the data with improved results.Keywords
This publication has 11 references indexed in Scilit:
- The Evidence Framework Applied to Classification NetworksNeural Computation, 1992
- Information-Based Objective Functions for Active Data SelectionNeural Computation, 1992
- Exact Calculation of the Hessian Matrix for the Multilayer PerceptronNeural Computation, 1992
- A Practical Bayesian Framework for Backpropagation NetworksNeural Computation, 1992
- Bayesian InterpolationNeural Computation, 1992
- Optimal minimal neural interpretation of spectraAnalytical Chemistry, 1992
- IMPROVING GENERALIZATION OF NEURAL NETWORKS THROUGH PRUNINGInternational Journal of Neural Systems, 1991
- Neural network ensemblesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- Bayesian Inductive Inference and Maximum EntropyPublished by Springer Nature ,1988
- Statistical Decision Theory and Bayesian AnalysisPublished by Springer Nature ,1985