It May Not Be Worth the Effort!

Publisher Website
Google Scholar
Abstract
Within the social skills research area, the ratings of trained judges are presumed to be of better reliability compared to those of untrained peers, but possibly at a cost in social validity since the latter directly represents the criterion. To investigate these issues, videotapes were obtained of 12 males who interacted with a female confederate in a typical four-minute simulated heterosocial situation. These were exhibited to a group of judges who had been trained to rate social skills and anxiety, and to a group that had received no training in this task. Judge types did not differ in mean levels of social skills and anxiety ratings, suggesting that trained judges' impressions are socially valid. However, the trained judges' interrater reliability was only slightly better than that of the peer judges. The latter finding was used to argue that untrained peer judges possibly can be used just as well as trained assistants to provide criterion ratings in social skills research.