Measuring Operative Performance after Laparoscopic Skills Training: Edited Videotape versus Direct Observation

Publisher Website
Google Scholar
Abstract
Background and Purpose: Global assessment by direct observation has been validated for evaluating operative performance of surgery residents after formal skills training but is time-consuming. The purpose of this study was to compare global assessment performed from edited videotape with scores from direct observation. Materials and Methods: Junior surgery residents (N = 22) were randomized to 2 weeks of formal videotrainer skills training or a control group. Laparoscopic cholecystectomy was performed at the beginning and end of the rotation, and global assessment scores were compared for the training and control groups. Laparoscopic videotapes were edited: initial (2 minutes), cystic duct/artery (6 minutes), and fossa dissection (2 minutes). Two independent raters performed both direct observation and videotape assessments, and scores were compared for each rater and for interrater reliability using a Spearman correlation. Results: Correlation coefficients for videotape versus direct observation for five global assessment criteria were < 0.33 for both raters (NS for all values). The correlation coefficient for interrater reliability for the overall score was 0.57 (P = 0.01) for direct observation v 0.28 (NS) for videotape. The trained group had significantly better overall performance than the control group according to the assessment by direct observation (P = 0.02) but not by videotape assessment (NS). Conclusions: Direct observation demonstrated improved overall performance of junior residents after formal skills training on a videotrainer. Global assessment from an edited 10-minute videotape did not correlate with direct observation and had poor interrater reliability. Efficient and valid methods of evaluating operative performance await development.

This publication has 26 references indexed in Scilit: