Understanding and limiting observer variability: Lessons from liver scans

Open Access
Publisher Website
Google Scholar
Abstract
Two specialists in nuclear medicine and two trainees independently read 106 liver scans for abnormalities suggestive of metastases. The four observers made a positive diagnosis with a significantly different frequency. The overall agreement between pairs of observers was 0.83-0.87. After adjusting for the expected chance agreement, K-values between 0.66 and 0.74 were obtained. No difference was found in the level of agreement between the two specialists or between the two trainees. After a consensus conference where the observers tried to unify their diagnostic interpretation, another 106 scans were read for abnormalities suggestive of metastases. The conference did not increase the level of agreement. Scintigraphic lesions that especially imply a risk of disagreement are pointed out.