Abstract
As a research scientist and clinician employed by the Food and Drug Administration, I review many clinical-trial protocols. The results of some of these trials subsequently appear in the medical literature. On occasion, the published description of the study may differ from the prospective protocol in important aspects of study design or statistical analysis — e.g., study size, clinical endpoints, and statistical tests used. The potential for misuse of statistical analysis and misrepresentation of data when key study or analysis parameters are selected or modified retrospectively is tremendous, and generally such practices cannot be detected by reviewers or readers of the study report. Thus, data that are not convincing when the prospective analytical plan is applied may be "improved" by a decision to study a few more patients, report on only a definable subgroup of subjects for which the data are more convincing, use a different index of organ function or quality of life, apply a different statistical test, and so on. In other cases, such elements of the protocol were not specified in advance, and their retrospective selection is biased toward those that present the data in the "best" light. Although many retrospective decisions are not inherently improper, they must be made known to reviewers of the data to allow appropriate evaluation of the statistical inferences.