Unobserved Heterogeneity as an Alternative Explanation for “Reversal” Effects in Behavioral Research
- 1 December 2000
- journal article
- Published by Oxford University Press (OUP) in Journal of Consumer Research
- Vol. 27 (3) , 324-344
- https://doi.org/10.1086/317588
Abstract
Behavioral researchers use analysis of variance (ANOVA) tests of differences between treatment means or chi-square tests of differences between proportions to provide support for empirical hypotheses about consumer behavior. These tests are typically conducted on data from “between-subjects” experiments in which participants were randomly assigned to conditions. We show that, despite using internally valid experimental designs such as this, aggregation biases can arise in which the theoretically critical pattern holds in the aggregate even though it holds for no (or few) individuals. First, we show that crossover interactions—often taken as strong evidence of moderating variables—can arise from the aggregation of two or more segments that do not exhibit such interactions when considered separately. Second, we show that certain context effects that have been reported for choice problems can result from the aggregation of two (or more) segments that do not exhibit these effects when considered separately. Given these threats to the conclusions drawn from experimental results, we describe the conditions under which unobserved heterogeneity can be ruled out as an alternative explanation based on one or more of the following: a priori considerations, derived properties, diagnostic statistics, and the results of latent class modeling. When these tests cannot rule out explanations based on unobserved heterogeneity, this is a serious problem for theorists who assume implicitly that the same theoretical principle works equally for everyone, but for random error. The empirical data patterns revealed by our diagnostics can expose the weakness in the theory but not fix it. It remains for the researcher to do further work to understand the underlying constructs that drive heterogeneity effects and to revise theory accordingly.Keywords
This publication has 31 references indexed in Scilit:
- On the Heterogeneity of DemandJournal of Marketing Research, 1998
- A Factor-Analytic Probit Model for Representing the Market Structure in Panel DataJournal of Marketing Research, 1995
- Hypothesized and Confounded Explanations in Theory Tests: A Bayesian AnalysisJournal of Consumer Research, 1992
- Investigating Heterogeneity in Brand Preferences in Logit Models for Panel DataJournal of Marketing Research, 1991
- A maximum likelihood methodology for clusterwise linear regressionJournal of Classification, 1988
- Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior.Journal of Applied Psychology, 1988
- Psychophysical ScalingAnnual Review of Psychology, 1988
- The Concept of External ValidityJournal of Consumer Research, 1982
- On the Estimation of Parameters in Latent Structure AnalysisPsychometrika, 1979
- Within-subjects designs: To use or not to use?Psychological Bulletin, 1976