Reliability: Paradigm or Paradox, with Especial Reference to Personality Tests

Publisher Website
Google Scholar
Abstract
Conventional reliability theory is inapplicable to many domains, including personality and projective tests. Repeat reliability, or temporal stability, is a matter to be investigated, not assumed; it is obvious that measures of some personality characteristics, e.g., mood, if temporally stable cannot be valid. As for internal consistency, classical mental test theory holds that an unreliable test cannot be valid. It is demonstrated mathematically that the maximum validity of a test with zero internal consistency (reliability) is one. The supposed limitation of validity follows only from the assumption of a random error component, uncorrelated with anything, to test scores. This often useful assumption leads us astray. A physical example is given as well as that of projective tests. Some consideration is given to the nature of these measurements. The largest validity squared is a conservative reliability estimate from classical test theory which may be used without being hurt by the theory's inappropriateness, if one insists on a reliability estimate. However, it is preferable to simply present the validity coefficients.

This publication has 1 reference indexed in Scilit: