Role of Behavior Level, Behavioral Variability, and Ratee Order in the Formation of Appraisal Ratings

Publisher Website
Google Scholar
Abstract
Two laboratory studies were conducted to assess the role of behavioral data in the formation of appraisal ratings. In the first study, which included 200 participants, significant main effects using a split-plot factorial analysis of variance design were found for performance variability, performance level, and ratee order. A central drift phenomenon also was discovered with ratees whose performance profiles were both similar and extreme (extremely high or extremely low), receiving ratings that drifted toward average as a function of rating queue position. A second study, which included 126 participants, replicated this drift phenomenon and isolated the cause as profile similarity rather than profile extremity. Implications of these findings for rating accuracy and future "truescore" research designs are discussed.