Item Location Effects and Their Implications for IRT Equating and Adaptive Testing

Publisher Website
Google Scholar
Abstract
A context effect occurs when examinees' item re sponding behavior is affected by the location of an item within a test. Recent advances in testing practice, most notably adaptive testing and certain innovative equating schemes, require items to be more invariant across intended usages than earlier methods. In this paper, location effects are identified as a form of mul tidimensionality, and examples of testing situations where location effects are important are described. Then, the susceptibility to item location effects of 10 item types from the Graduate Record Examination General Test is investigated by comparing the item difficulty parameters of sets of items across intended usages. Results are replicated using a second form of the test. Two of the 10 item types, analysis of expla nations and logical diagrams, are clearly affected by item location in the population tested. One common item type, reading comprehension, appears to be af fected somewhat by item context in this population. It is strongly advised that these item types not be used in an adaptive testing program without first assessing their susceptibility to location effects within the popu lation (and subpopulations) of interest.