Assessing the Validity of Randomized Field Experiments

Publisher Website
Google Scholar
Abstract
Randomized field experiments are often logistical failures. They are particularly difficult to implement when the intervention involves a large number of components or service providers. Although surmountable, potential methodological problems with using randomized experiments to evaluate intervention programs under field (i.e., real world) conditions must be anticipated and resolved for the experiment to succeed. This article examines six potential major problems: treatment dilution, treatment contamination or confounding, inaccurate caseflow and power estimates, violations of the random assignment process, changes in the environmental context, and changes in the treatment regimens. It describes the general problem, several methodological developments related to them, and how the authors are attempting to deal with these problems in an experiment at the Research Triangle Institute. Developing designs and implementation strategies to deal with these six problems is crucial to extending the usefulness of randomized field experiments to evaluate social policies and experiments.