Automatic Evaluation of Information Ordering: Kendall's Tau
Open Access
- 1 December 2006
- journal article
- Published by MIT Press in Computational Linguistics
- Vol. 32 (4) , 471-484
- https://doi.org/10.1162/coli.2006.32.4.471
Abstract
This article considers the automatic evaluation of information ordering, a task underlying many text-based applications such as concept-to-text generation and multidocument summarization. We propose an evaluation method based on Kendall's τ, a metric of rank correlation. The method is inexpensive, robust, and representation independent. We show that Kendall's τ correlates reliably with human ratings and reading times.Keywords
This publication has 10 references indexed in Scilit:
- A Probabilistic Account of Logical MetonymyComputational Linguistics, 2003
- Inferring Strategies for Sentence Ordering in Multidocument News SummarizationJournal of Artificial Intelligence Research, 2002
- Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural LanguageJournal of Artificial Intelligence Research, 1999
- Correlating and predicting psychiatric symptom ratings: Spearmans r versus Kendalls tau correlationJournal of Psychiatric Research, 1999
- Inference during reading.Psychological Review, 1992
- Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures.Journal of Educational Psychology, 1991
- Regression analyses of repeated measures data in cognitive research.Journal of Experimental Psychology: Learning, Memory, and Cognition, 1990
- The Interpretation of Rank CorrelationsJournal of the Royal Statistical Society Series C: Applied Statistics, 1975
- A NEW MEASURE OF RANK CORRELATIONBiometrika, 1938
- A New Measure of Rank CorrelationBiometrika, 1938