Variational inference for Dirichlet process mixtures
Top Cited Papers
Open Access
- 1 March 2006
- journal article
- Published by Institute of Mathematical Statistics in Bayesian Analysis
- Vol. 1 (1) , 121-143
- https://doi.org/10.1214/06-ba104
Abstract
Dirichlet process (DP) mixture models are the cornerstone of nonparametric Bayesian statistics, and the development of Monte-Carlo Markov chain (MCMC) sampling methods for DP mixtures has enabled the application of nonparametric Bayesian methods to a variety of practical data analysis problems. However, MCMC sampling can be prohibitively slow, and it is important to explore alternatives. One class of alternatives is provided by variational methods, a class of deterministic algorithms that convert inference problems into optimization problems (Opper and Saad 2001; Wainwright and Jordan 2003). Thus far, variational methods have mainly been explored in the parametric setting, in particular within the formalism of the exponential family (Attias 2000; Ghahramani and Beal 2001; Blei et al. 2003). In this paper, we present a variational inference algorithm for DP mixtures. We present experiments that compare the algorithm to Gibbs sampling algorithms for DP mixtures of Gaussians and present an application to a large-scale image analysis problem.Keywords
This publication has 14 references indexed in Scilit:
- Graphical ModelsStatistical Science, 2004
- Gibbs Sampling Methods for Stick-Breaking PriorsJournal of the American Statistical Association, 2001
- Computational Methods for Mixture of Dirichlet Process ModelsPublished by Springer Nature ,1998
- Bayesian Density Estimation and Inference Using MixturesJournal of the American Statistical Association, 1995
- Estimating normal means with a conjugate style dirichlet process priorCommunications in Statistics - Simulation and Computation, 1994
- [Practical Markov Chain Monte Carlo]: Comment: One Long Run with Diagnostics: Implementation Strategies for Markov Chain Monte CarloStatistical Science, 1992
- Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric ProblemsThe Annals of Statistics, 1974
- Ferguson Distributions Via Polya Urn SchemesThe Annals of Statistics, 1973
- A Bayesian Analysis of Some Nonparametric ProblemsThe Annals of Statistics, 1973
- Concepts of Independence for Proportions with a Generalization of the Dirichlet DistributionJournal of the American Statistical Association, 1969