Computerized Mastery Testing With Nonequivalent Testlets

Publisher Website
Google Scholar
Abstract
A procedure for determining the effect of testlet nonequivalence on the operating characteristics of a testlet-based computerized mastery test (CMT) is introduced. The procedure involves estimating the CMT decision rule twice—once with testlets treated as equivalent and once with testlets treated as nonequivalent. In the equivalent testlet mode, the likelihood functions (LFS) estimated for specific number-correct (NC) scores are assumed to be con stant across testlets and a single set of cutscores is estimated for all testlets. In the nonequivalent testlet mode, the LFS estimated for specific NC scores are allowed to vary from one testlet to another and a different set of cutscores is estimated for each permutation of testlet presentation order. Small differences between the estimated operating characteristics of the equivalent testlet decision rule and the nonequivalent testlet decision rule indicate that the assumption of equivalent testlets was war ranted. This procedure is demonstrated with data from the Architect Registration Examination.

This publication has 3 references indexed in Scilit: