A whole sentence maximum entropy language model
- 23 November 2002
- proceedings article
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 230-237
- https://doi.org/10.1109/asru.1997.659010
Abstract
We introduce a new kind of language model, which modelswhole sentences or utterances directly using the Maximum Entropyparadigm. The new model is conceptually simpler, and more naturallysuited to modeling whole-sentence phenomena, than the conditional MEmodels proposed to date. By avoiding the chain rule, the model treatseach sentence or utterance as a "bag of features", where features arearbitrary computable properties of the sentence. The model is unnormalizable,but this does not...Keywords
This publication has 7 references indexed in Scilit:
- Inducing features of random fieldsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1997
- A maximum entropy approach to adaptive statistical language modellingComputer Speech & Language, 1996
- A maximum entropy model for prepositional phrase attachmentPublished by Association for Computational Linguistics (ACL) ,1994
- Trigger-based language models: a maximum entropy approachPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1993
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1984
- Generalized Iterative Scaling for Log-Linear ModelsThe Annals of Mathematical Statistics, 1972
- Information Theory and Statistical MechanicsPhysical Review B, 1957