Towards understanding spontaneous speech: word accuracy vs. concept accuracy
- 24 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2, 1009-1012
- https://doi.org/10.1109/icslp.1996.607774
Abstract
Describes an approach to the automatic evaluation of both the speech recognition and understanding capabilities of a spoken dialogue system for train timetable information. For performance judgement, we use word accuracy for recognition and concept accuracy for understanding. Both measures are calculated by comparing these modules' outputs with a correct reference answer. We report evaluation results for a spontaneous speech corpus with about 10,000 utterances. We observed a nearly linear relationship between word accuracy and concept accuracy.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Parsing word graphs using a linguistic grammar and a statistical language modelPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Black box and glass box evaluation of the SUNDIAL systemPublished by International Speech Communication Association ,1993
- A spoken dialogue system for German intercity train timetable inquiriesPublished by International Speech Communication Association ,1993
- Procedure for quantitatively comparing the syntactic coverage of English grammarsPublished by Association for Computational Linguistics (ACL) ,1991
- Combining Categorial Grammar and UnificationPublished by Springer Nature ,1988