Pooling Data Across Transparently Different Groups of Key Informants: Measurement Equivalence and Survey Research*

Publisher Website
Google Scholar
Abstract
Survey research is often deployed in the study of situational issues facing organizations and functions within organizations. One particular survey research approach can be described as follows: (1) survey questionnaires involving perceptual questions about a situational issue are administered to key informants, one key informant per unit of analysis; (2) key informants vary in a transparent manner across units of analysis such that groups of these key informants are discernible; and (3) perceptual responses, after data collection, are then pooled to create a single larger data set for subsequent statistical manipulations. In this methodological note, we draw attention to this particular survey research approach and ask the question: When is it appropriate to pool data provided by key informants with transparently different demographics across units of analysis so as to create a single larger data set for statistical manipulations? We use a simple example and data from a published study to motivate the relevance and gravity of this methodological question. Offering the concept and empirical assessment of measurement equivalence as the answer to this methodological question of data pooling, we prescribe and demonstrate, with the total quality management→customer satisfaction relationship, the procedural steps for evaluating the seven subdimensions of measurement equivalence. In conclusion, we highlight methods that should be adopted, before data collection, to minimize the risk of violating measurement equivalence. After data collection and for the instances when the empirical assessment for measurement equivalence advises against pooling of such data, we also offer suggestions for analyzing such data and presenting associated statistical results.