FURTHER WITHIN‐SETTING EMPIRICAL TESTS OF THE SITUATIONAL SPECIFICITY HYPOTHESIS IN PERSONNEL SELECTION

Publisher Website
Google Scholar
Abstract
The situational specificity hypothesis in personnel selection holds that variation in observed validity coefficients across studies for the same test and job is due to subtle variations from setting to setting in what constitutes job performance. This hypothesis therefore predicts that, if the setting does not vary, validity will not vary. Using data from a single large‐sample validity study (N= 1,455), this research generated numerous small‐sample studies for which the setting (organization, job, test, criterion measure, applicant pool, time period, and sample size) was held constant. It was found that even under these circumstances there was substantial variability across studies in (a) observed validity coefficients, (b) significance levels, and (c) (using traditional data analytic methods) conclusions about the presence or absence of validity. These findings disconfirm the situational specificity hypothesis and argue strongly against traditional data‐analytic procedures and the practice of reliance on single small‐sample studies. In contrast to the erroneous conclusions produced by traditional data‐analytic procedures, meta‐analytic methods correctly estimated the population observed validity at .22 and correctly indicated that all between‐study variance in observed validities was due to sampling error alone.