Assessment of psychotherapy skills: the problem of interrater agreement

Publisher Website
Google Scholar
Abstract
Thirteen experienced supervising psychiatrists independently rated the psychotherapy interviews of 6 residents, using a detailed assessment instrument. Although interrater agreement was significantly greater than would be expected by chance, the extent of agreement was uniformly low. Agreement was significantly poorer for judgments about technique skills than about communication skills and for 2nd yr than for 1st yr residents. Other factors, such as the faculty rank and training of the rater, did not covary significantly with observed ratings. Rating methods which purport to assess psychotherapy skills should probably undergo careful methodological evaluation and testing before they are accepted as valid assessment techniques.

This publication has 14 references indexed in Scilit: