The search for significance: different tests on the same data
- 1 January 1984
- journal article
- research article
- Published by Taylor & Francis in Journal of Statistical Computation and Simulation
- Vol. 19 (1) , 1-21
- https://doi.org/10.1080/00949658408810710
Abstract
In many situations a variety of tests are available to test essentially the same null hypothesis. In practice the statistician who fails to reject with the first test used will sometimes try several others, stopping when he obtains the hoped-for significance. This raises the type I error rate, but no broad study has previously been made to address the question by how much. Here the effect of such multiple testing is investigated by simulating two-sample data and studying five common tests: the t, Wilcoxon-Mann-Whitney,t on logs, Yuen-Dixon trimmed t, and Welch's test.Keywords
This publication has 13 references indexed in Scilit:
- Use of a preliminary test in comparing two sample meansCommunications in Statistics - Simulation and Computation, 1981
- The two-sample Winsorized tCommunications in Statistics - Simulation and Computation, 1980
- The robustness of the two—samplet—test over the Pearson systemJournal of Statistical Computation and Simulation, 1978
- Comparison of some two sample means tests by simulationCommunications in Statistics - Simulation and Computation, 1976
- Size and Power of Tests for Equality of Means of Two Normal Populations with Unequal VariancesJournal of the American Statistical Association, 1975
- Relation between the shape of population distribution and the robustness of four simple test statisticsBiometrika, 1975
- A Monte Carlo Study Comparing Various Two-Sample Tests for Differences in MeanTechnometrics, 1968
- The Relation Between Pitman's Asymptotic Relative Efficiency of Two Tests and the Correlation Coefficient Between Their Test StatisticsThe Annals of Mathematical Statistics, 1963
- On comparing different tests of the same hypothesisBiometrika, 1960
- The Use of Range in Place of Standard Deviation in Small SamplesJournal of the Royal Statistical Society Series B: Statistical Methodology, 1935