Handling over-dispersion of performance indicators

Open Access
Publisher Website
Google Scholar
Abstract
Objectives: A problem can arise when a performance indicator shows substantially more variability than would be expected by chance alone, since ignoring such “over-dispersion” could lead to a large number of institutions being inappropriately classified as “abnormal”. A number of options for handling this phenomenon are investigated, ranging from improved risk stratification to fitting a statistical model that robustly estimates the degree of over-dispersion. Design: Retrospective analysis of publicly available data on survival following coronary artery bypass grafts, emergency readmission rates, and teenage pregnancies. Setting: NHS trusts in England. Results: Funnel plots clearly show the influence of the method chosen for dealing with over-dispersion on the “banding” a trust receives. Both multiplicative and additive approaches are feasible and give intuitively reasonable results, but the additive random effects formulation appears to have a stronger conceptual foundation. Conclusion: A random effects model may offer a reasonable solution. This method has now been adopted by the UK Healthcare Commission in their derivation of star ratings.