Precise n-gram Probabilities from Stochastic Context-free Grammars

Preprint

10 May 1994

preprint
Published by arXiv in arXiv

https://doi.org/10.48550/arXiv.cmp-lg/9405016

Abstract

We present an algorithm for computing n-gram probabilities from stochastic context-free grammars, a procedure that can alleviate some of the standard problems associated with n-grams (estimation from sparse data, lack of linguistic structure, among others). The method operates via the computation of substring expectations, which in turn is accomplished by solving systems of linear equations derived from the grammar. We discuss efficient implementation of the algorithm and report our practical experience with it.

Keywords

PROBABILITIES FROM STOCHASTIC
PRACTICAL EXPERIENCE
GRAM PROBABILITIES
STOCHASTIC CONTEXT
SYSTEMS OF LINEAR
FREE GRAMMARS
CONTEXT FREE
ALGORITHM
GRAMMARS

All Related Versions

Version 1, 1994-05-10, ArXiv

This publication has 0 references indexed in Scilit: