Neural networks, maximum mutual information training, and maximum likelihood training (speech recognition)
- 4 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 493-496 vol.1
- https://doi.org/10.1109/icassp.1990.115757
Abstract
A Gaussian-model classifier trained by maximum mutual information estimation (MMIE) is compared to one trained by maximum-likelihood estimation (MLE) and to an artificial neural network (ANN) on several classification tasks. Similarity of MMIE and ANN results for uniformly distributed data confirm that the ANN is better than the MLE in some cases due to the ANNs use of an error-correcting training algorithm. When the probability model fits the data well, MLE is better than MMIE if the training data are limited, but they are equal if there are enough data. When the model is a poor fit, MMIE is better than MLE. Training dynamics of MMIE and ANN are shown to be similar under certain assumptions. MMIE seems more susceptible to overtraining and computational difficulties than the ANN. Overall, ANN is the most robust of the classifiers.Keywords
This publication has 4 references indexed in Scilit:
- Maximum mutual information estimation of hidden Markov model parameters for speech recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- How limited training data can allow a neural network to outperform an 'optimal' statistical classifierPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- The Acoustic-Modeling Problem in Automatic Speech Recognition.Published by Defense Technical Information Center (DTIC) ,1987
- Learning representations by back-propagating errorsNature, 1986