Study Design for Calibration of Clinical Examiners Measuring Periodontal Parameters

Publisher Website
Google Scholar
Abstract
We present an approach to examiner calibration study design where the number of calibration subjects is based on a specified margin of error (half-width of the 95% confidence interval [CI]) of the percentage of agreement (exact and within 1 mm) for both intra- and interexaminer reliability assessments. An experienced standard examiner (S) trained three dental hygienists (A, B, and C) in correct procedures for obtaining a variety of periodontal measures. Duplicate measurements of probing depth (PD [mm]) and the free gingival margin to the cemento-enamel junction (CEJ-GM [mm]) were obtained in a pilot study to design a formal examiner calibration study, where sample sizes were adjusted for the effects of within-subject clustering of binary indices of agreement. Within-subject clustering of agreement indices resulted in an approximate four-fold increase in the variance of the estimates of percentage of agreement with the standard. PD and CEJ-GM percentage of exact agreement measurements (95% CI) for each examiner-standard pair, respectively, were as follows: AS=55% (48%, 61%) and 70% (62%, 78%); BS=52% (45%, 59%) and 73% (63%, 82%); and CS=55% (50%, 61%) and 72% (65%, 79%). The corresponding 95% CIs unadjusted for the effects of clustering underestimated the margin of error associated with the estimates of exact agreement by as much as 57% for PD and 68% for CEJ-GM. Failure to account for dependence among site-level agreement indices results in a false sense of precision in the resulting reliability estimates and can lead to faulty inference.
Funding Information
  • National Institutes of Health (P20 RR 17696)
  • National Institutes of Health (U24 DE 16508)