Evaluating Reading Diagnostic Tests: An Application of Confirmatory Factor Analysis To Multitrait-Multimethod Data

Publisher Website
Google Scholar
Abstract
Diagnostic reading tests, in contrast to achievement tests, claim to measure specific components of ability hypothesized to be important for diagnosis or remedia tion. A minimal condition for demonstrating the con struct validity of such tests is that they are able to dif ferentiate validly between the reading traits that they claim to measure (e.g., comprehension, sound dis crimination, blending). This condition is rarely tested, but multitrait-multimethod (MTMM) designs are ideally suited for this purpose. This is demonstrated in two studies based on the 1966 version of the Stanford Diagnostic Reading Test (SDRT). In each study, the application of the Campbell-Fiske guidelines and con firmatory factor analysis (CFA) to the MTMM data in dicated that the SDRT subscales could be explained in terms of a method/halo effect and a general reading factor that was not specific to any of the subscales; this refutes the construct validity of the 1966 version of the SDRT as a diagnostic test. Other diagnostic tests probably suffer the same weakness and should also be evaluated in MTMM studies.