Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing
- 10 January 2003
- journal article
- research article
- Published by Taylor & Francis in Journal of Biopharmaceutical Statistics
- Vol. 13 (4) , 675-689
- https://doi.org/10.1081/bip-120024202
Abstract
When a large number of statistical tests is performed, the chance of false positive findings could increase considerably. The traditional approach is to control the probability of rejecting at least one true null hypothesis, the familywise error rate (FWE). To improve the power of detecting treatment differences, an alternative approach is to control the expected proportion of errors among the rejected hypotheses, the false discovery rate (FDR). When some of the hypotheses are not true, the error rate from either the FWE- or the FDR-controlling procedure is usually lower than the designed level. This paper compares five methods used to estimate the number of true null hypotheses over a large number of hypotheses. The estimated number of true null hypotheses is then used to improve the power of FWE- or FDR-controlling methods. Monte Carlo simulations are conducted to evaluate the performance of these methods. The lowest slope method, developed by Benjamini and Hochberg (2000) on the adaptive contr...Keywords
This publication has 14 references indexed in Scilit:
- SAM Thresholding and False Discovery Rates for Detecting Differential Gene Expression in DNA MicroarraysPublished by Springer Nature ,2003
- A Direct Approach to False Discovery RatesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2002
- A modified Benjamini–Hochberg multiple comparisons procedure for controlling the false discovery rateJournal of Statistical Planning and Inference, 2002
- Familywise Robustness Criteria for Multiple-Comparison ProceduresJournal of the Royal Statistical Society Series B: Statistical Methodology, 2002
- On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent StatisticsJournal of Educational and Behavioral Statistics, 2000
- A step-down multiple hypotheses testing procedure that controls the false discovery rate under independenceJournal of Statistical Planning and Inference, 1999
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- More powerful procedures for multiple significance testingStatistics in Medicine, 1990
- Multiple Comparison ProceduresWiley Series in Probability and Statistics, 1987
- Plots of P-values to evaluate many tests simultaneouslyBiometrika, 1982