Abstract
Purpose: Monte Carlo methods were used to assess how the value of outcome comparisons depends on the number of patients per provider. Methods: We simulated two patient data sets that have been used for well-known studies of outcome comparisons: mortality rates for coronary artery bypass surgeons from New York and Pennsylvania, and 30-day hospital mortality rates of Medicare patients from a national data set. In the simulated data sets, each surgeon or hospital provider was assigned a true or underlying probability of mortality. Results: For the simulated CABG surgery data set, the underlying probability of mortality explained 30% of the variation in the observed mortality rate when there were 100 patients per physician, and 63% when there were 400 patients. The positive predictive value of using an observed mortality rate in the bottom 10% to identify a surgeon whose underlying probability of mortality was in the bottom 10% was 31% for 100 patients and 59% for 400 patients. The relationship between underlying and observed rates was weaker in the simulated Medicare data set with the same number of patients per provider. For a given data set, the amount of random variation in the observed rates of adverse outcomes among providers can be estimated with a simple equation. Conclusions: The results show that the assessment of provider outcomes may be greatly affected by random variation. An indication of the amount of random variation in a given data set can be obtained from the examples in this study and an equation for estimating random variation. © 1997 Elsevier Science Ltd.