Reliability of a Utilization Review Instrument in a Large Field Study

Publisher Website
Google Scholar
Abstract
One important question for a utilization manage ment program is whether the utilization review in strument is consistent or stable when used on many occasions by the same abstractor (intrarater reliabil ity) or by several abstractors (inter-rater reliability). As part of a nationwide study of inappropriate utili zation of inpatient services by the Department of Veterans Affairs, we conducted a thorough investi gation of the inter-rater reliability of a widely used utilization review instrument by 27 nurse abstrac tors. All abstractors were extensively trained, both by the developers of the instrument and by use of practice medical records. A standard protocol for re solving questions was implemented, with immediate communication of decisions to abstractors. The re sults of three reliability assessments, conducted im mediately after formal training, after several weeks of reviewing practice records, and midway through review of the study records, demonstrated good to excellent reliability, both when comparing the nurse abstractors with a physician gold standard and among themselves. Therefore, with appropriate training and monitoring, utilization management programs in large hospitals, multihospital systems, and other health care organizations needing to examine inpa tient utilization should feel confident that they can achieve reviews that would be in close agreement with physician and other nurse abstractors. Such con fidence should increase the acceptability of utilization management programs.