Interjudge Agreement and the Maximum Value of Kappa
- 1 December 1989
- journal article
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 49 (4) , 835-850
- https://doi.org/10.1177/001316448904900407
Abstract
The observed degree of agreement between judges is commonly summarized using Cohen's (1960) kappa. Previous research has related values of kappa to the marginal distributions of the agreement matrix. This manuscript provides an approach for calculating maximum values of kappa as a function of observed agreement proportions between judges. Solutions are provided separately for matrices of size 2 x 2, 3 x 3, 4 x 4, and k X k; plots are provided for the 2 x 2, 3 x 3, and 4 x 4 matrices.Keywords
This publication has 6 references indexed in Scilit:
- Standard error of the kappa statistic.Psychological Bulletin, 1987
- Kappa, Measures of Marginal Symmetry and Intraclass CorrelationsEducational and Psychological Measurement, 1985
- Coefficient Kappa: Some Uses, Misuses, and AlternativesEducational and Psychological Measurement, 1981
- The Measurement of Observer Agreement for Categorical DataPublished by JSTOR ,1977
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960