A hierarchical Dirichlet language model
- 12 September 1995
- journal article
- research article
- Published by Cambridge University Press (CUP) in Natural Language Engineering
- Vol. 1 (3) , 289-308
- https://doi.org/10.1017/s1351324900000218
Abstract
We discuss a hierarchical probabilistic model whose predictions are similar to those of the popular language modelling procedure known as ‘smoothing’. A number of interesting differences from smoothing emerge. The insights gained from a probabilistic view of this problem point towards new directions for language modelling. The ideas of this paper are also applicable to other problems such as the modelling of triphomes in speech, and DNA and protein sequences in molecular biology. The new algorithm is compared with smoothing on a two million word corpus. The methods prove to be about equally accurate, with the hierarchical model using fewer computational resources.Keywords
This publication has 10 references indexed in Scilit:
- Bayesian neural networks and density networksNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 1995
- Learning classification treesStatistics and Computing, 1992
- Bayesian Mixture ModelingPublished by Springer Nature ,1992
- Developments in Maximum Entropy Data AnalysisPublished by Springer Nature ,1989
- Maximum Entropy and Bayesian MethodsPublished by Springer Nature ,1989
- Estimation of probabilities from sparse data for the language model component of a speech recognizerIEEE Transactions on Acoustics, Speech, and Signal Processing, 1987
- Estimation of probabilities in the language model of the IBM speech recognition systemIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984
- A Maximum Likelihood Approach to Continuous Speech RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1983
- Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric ProblemsThe Annals of Statistics, 1974
- Probability, Frequency and Reasonable ExpectationAmerican Journal of Physics, 1946