The Role of Known Effects in Observational Studies

Publisher Website
Google Scholar
Abstract
When treatments are not randomly assigned, treated and control subjects may be quite different prior to treatment, so that straight forward comparisons of responses in treated and control groups may give a distorted impression of the effect of the treatment. While adjustments for observed pretreatment differences can help, there is often reason for concern that important differences were not measured and not controlled by statistical adjustments. This paper concerns methods for detecting and indicating of such unobserved pretreatment differences. The methods discussed here use known effects of the treatment on certain supplementary responses included in the study to provide information about unobserved pretreatment differences. Previous work has noted that know effects provide the basis for a statistical test of the assumption that adjustments for observed covariates suffice removing bias. Here, the matter is taken several steps further by addressing the following questions. What are the formal properties of such test? Under what circumstances are the testes effective at detecting unobserved pretreatment differences? Or to put it another way, what sorts of supplementary response variables provide the most effevctive checks? If unobserved pretreatment differences are detected, what can be said about the direction of the biases they produce? These statistical tests are, of course, as influenced by the available sample size as they are by the magnitude of the unobserved pretreatment differences they are intended to detect. Given that the pretreatment differences in question are not observed directly, what can be concluded about their magnitude? To motivate the discussion, two examples are discussed throughout the paper.

This publication has 2 references indexed in Scilit: