A Generalized Kappa Coefficient

Publisher Website
Google Scholar
Abstract
Previously proposed methods for calculating the kappa measure of nominal rating agreement among multiple raters are not applicable in many situations. This paper presents a more general computational method which can be used across a broader range of rating designs, including those in which raters vary with respect to their base rates and pairs of raters vary with respect to how many subjects they rate in common. A Monte Carlo method for determining the statistical significance of this generalized kappa coefficient is discussed.

This publication has 4 references indexed in Scilit: