On the Number of Significant Findings to be Expected by Chance

Publisher Website
Google Scholar
Abstract
When multiple significance tests are computed, a certain number of “significant” findings will emerge simply because of chance fluctuations. In the present paper, some factors affecting the number of nominally significant results are elaborated and a general method is suggested which permits unbiased inference as to the significance of a set of findings, as a set. The method advocated employs a high speed computer to generate empirically a sampling distribution tailormade to a particular data matrix. The method is illustrated in the case of dichotomous response to inventory items, where it is found that the statistical model still often used as a basis for estimation is overly conservative. Some problems in the application of the method are discussed.

This publication has 5 references indexed in Scilit: