Apples and Oranges: An Empirical Comparison of Commonly Used Indices of Interrater Agreement.

Publisher Website
Google Scholar
Abstract
Empirical comparison of common indices of interrater agreement indicated that the indices are not directly comparable in regard to the magnitude of the estimates produced or the information conveyed. Some of the indices may be severely distorted if applied to homogeneous sets of ratings. Although analyses of ratings at an item level might suggest inadequate agreement, composites of those items may yield satisfactory indications of interrater agreement.

This publication has 0 references indexed in Scilit: