Counting probability distributions: Differential geometry and model selection
- 26 September 2000
- journal article
- Published by Proceedings of the National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 97 (21) , 11170-11175
- https://doi.org/10.1073/pnas.170283897
Abstract
A central problem in science is deciding among competing explanations of data containing random errors. We argue that assessing the "complexity" of explanations is essential to a theoretically well-founded model selection procedure. We formulate model complexity in terms of the geometry of the space of probability distributions. Geometric complexity provides a clear intuitive understanding of several extant notions of model complexity. This approach allows us to reconceptualize the model selection problem as one of counting explanations that lie close to the "truth." We demonstrate the usefulness of the approach by applying it to the recovery of models in psychophysics.Keywords
This publication has 22 references indexed in Scilit:
- The Importance of Complexity in Model SelectionJournal of Mathematical Psychology, 2000
- Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability DistributionsNeural Computation, 1997
- Fisher information and stochastic complexityIEEE Transactions on Information Theory, 1996
- Minimum complexity density estimationIEEE Transactions on Information Theory, 1991
- Information-theoretic asymptotics of Bayes methodsIEEE Transactions on Information Theory, 1990
- Stochastic complexity and the mdl principleEconometric Reviews, 1987
- Stochastic Complexity and ModelingThe Annals of Statistics, 1986
- Universal coding, information, prediction, and estimationIEEE Transactions on Information Theory, 1984
- A foundation of information geometryElectronics and Communications in Japan (Part I: Communications), 1983
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978