Original article
High agreement but low Kappa: I. the problems of two paradoxes

https://doi.org/10.1016/0895-4356(90)90158-LGet rights and content

Abstract

In a fourfold table showing binary agreement of two observers, the observed proportion of agreement, P0 can be paradoxically altered by the chance-corrected ratio that creates κ as an index of concordance. In one paradox, a high value of P0 can be drastically lowered by a substantial imbalance in the table's marginal totals either vertically or horizontally. In the second pardox, κ will be higher with an asymmetrical rather than symmetrical imbalance in marginal totals, and with imperfect rather than perfect symmetry in the imbalance. An adjustment that substitutes Kmax for κ does not repair either problem, and seems to make the second one worse.

References (9)

There are more references available in the full text version of this article.

Cited by (2259)

  • The Kappa Paradox Explained

    2024, Journal of Hand Surgery
View all citing articles on Scopus

Supported in part by a Grant from the Andrew W. Mellon Foundation and by Veterans Administration Funds.

Professor of Medicine and Epidemiology; Director, Clinical Epidemiology Unit and The Robert Wood Johnsson Clinical Scholars Program, Yale University School of Medicine, New Haven, Connecticut. Senior Biostatistician, Cooperative Studies Program Coordinating Center, Veterans Administration Medical Center West Haven, Connecticut.

View full text