Schwarz, Wallace, and Rissanen: Intertwining Themes in Theories of Model Selection
- 1 August 2001
- journal article
- Published by Wiley in International Statistical Review
- Vol. 69 (2) , 185-212
- https://doi.org/10.1111/j.1751-5823.2001.tb00456.x
Abstract
Summary: Investigators interested in model order estimation have tended to divide themselves into widely separated camps; this survey of the contributions of Schwarz, Wallace, Rissanen, and their coworkers attempts to build bridges between the various viewpoints, illuminating connections which may have previously gone unnoticed and clarifying misconceptions which seem to have propagated in the applied literature. Our tour begins with Schwarz's approximation of Bayesian integrals via Laplace's method. We then introduce the concepts underlying Rissanen's minimum description length principle via a Bayesian scenario with a known prior; this provides the groundwork for understanding his more complex non‐Bayesian MDL which employs a “universal” encoding of the integers. Rissanen's method of parameter truncation is contrasted with that employed in various versions of Wallace's minimum message length criteria. Rissanen's more recent notion of stochastic complexity is outlined in terms of Bernardo's information‐theoretic derivation of the Jeffreys prior.Keywords
This publication has 83 references indexed in Scilit:
- Bayesian Model Selection and Model AveragingJournal of Mathematical Psychology, 2000
- Akaike's Information Criterion and Recent Developments in Information ComplexityJournal of Mathematical Psychology, 2000
- Improved model selection criterionCommunications in Statistics - Simulation and Computation, 1999
- Segmentation and Classification of Edges Using Minimum Description Length Approximation and Complementary Junction CuesComputer Vision and Image Understanding, 1997
- Reversible jump Markov chain Monte Carlo computation and Bayesian model determinationBiometrika, 1995
- Learning Bayesian networks: The combination of knowledge and statistical dataMachine Learning, 1995
- Density estimation by stochastic complexityIEEE Transactions on Information Theory, 1992
- Information-theoretic asymptotics of Bayes methodsIEEE Transactions on Information Theory, 1990
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974