“The Whole is More than the Sum of its Parts”: The Effects of Grouping and Randomizing Items on the Reliability and Validity of Questionnaires

Publisher Website
Google Scholar
Abstract
This investigation was concerned with the widely spread practice to extract subscales from extensive psychodiagnostic inventories and to present their items in questionnaires as homogeneous item-blocks. By way of examining the effects of the mode of item-presentation on the reliability and validity, the frequently used and validated SCL-90-R was analyzed as a prototype of multidimensional symptom self-report inventories. Two studies were conducted in different contexts of application (study I: controlled group testing, study II: non-controlled individual testing) and involved different groups of subjects (study I: 130 nursing school students, study II: 134 university students). In both studies the standard item-arrangement was contrasted with the item-block presentation, which groups together items measuring the same dimension. The results revealed significant effects of item-blocking on the mean values, on the reliability, and the validity of the questionnaire. The findings seriously call into question the admissibility of subscale-extraction for self-report inventories. We conclude that the Gestalt paradigm, “The whole is more than the sum of its parts,” is valid for multidimensional psychodiagnostic inventories.