The Impact of Data-Analysis Methods on Cumulative Research Knowledge

Publisher Website
Google Scholar
Abstract
The methods of data analysis used in research have a major effect on the development of cumulative knowledge. Traditional methods based on statistical significance testing have systematically retarded the growth of cumulative research knowledge by making it virtually impossible to discern the real meaning of research literatures. Meta-analysis makes it possible to demonstrate graphically the high price the research enterprise has paid for its reliance on significance testing. But in addition to these demonstrations, reform will require that researchers come to understand that the benefits they see asflowingfrom the use of significance tests are illusory. In research practice and in training of researchers, we must use and teach appropriate data analysis methods: point estimates of effect sizes and confidence intervals within individual studies, and meta-analysis in the integration of multiple studies to producefinal conclusions. These reforms are essential to the progress of cumulative research knowledge.