A comment on replication,P‐values and evidence
- 1 January 1992
- journal article
- research article
- Published by Wiley in Statistics in Medicine
- Vol. 11 (7) , 875-879
- https://doi.org/10.1002/sim.4780110705
Abstract
It is conventionally thought that a smallp‐value confers high credibility on the observed alternative hypothesis, and that a repetition of the same experiment will have a high probability of resulting again in statistical significance. It is shown that if the observed difference is the true one, the probability of repeating a statistically significant result, the ‘replication probability’, is substantially lower than expected. The reason for this is a mistake that generates other seeming paradoxes: the interpretation of the post‐trialp‐value in the same way as the pre‐trial α error. The replication probability can be used as a frequentist counterpart of Bayesian and likelihood methods to show thatp‐values overstate the evidence against the null hypothesis.Keywords
This publication has 7 references indexed in Scilit:
- Testing a Point Null Hypothesis: The Irreconcilability of P Values and EvidenceJournal of the American Statistical Association, 1987
- The Effect of Sample Size on the Meaning of Significance TestsThe American Statistician, 1986
- Tests of Significance in Theory and PracticeJournal of the Royal Statistical Society: Series D (The Statistician), 1986
- Clinical Trials and Statistical Verdicts: Probable Grounds for AppealAnnals of Internal Medicine, 1983
- Bayesian Interpretation of Standard Inference StatementsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1965
- Bayesian statistical inference for psychological research.Psychological Review, 1963
- IX. On the problem of the most efficient tests of statistical hypothesesPhilosophical Transactions of the Royal Society A, 1933