The value of goodness‐of‐fit indices in meta‐analysis: A comment on Hall and Rosenthal

Publisher Website
Google Scholar
Abstract
In a useful article regarding testing for moderator variables in meta‐analysis, Hall and Rosenthal (1991) suggest, among other things, that when the sample of studies is large enough, it is acceptable to use ordinary inferential statistics such as the analysis of variance to test for moderator variables. However, because meta‐analytic data points (effect sizes) differ in important ways from those for which ordinary inferential statistics were designed, we recommend that analysts use conventional meta‐analytic statistics, which are at least potentially more informative about the literature in question. Hall and Rosenthal also suggest that tests of effect size variability should play a minimal role in meta‐analytic model tests. We concur that these tests of homogeneity cannot alone test for moderator variables and that at least tests between mean effect sizes should be performed. However, consistent with Hedges and Olkin's (1985) meta‐analytic framework, we show how homogeneity tests provide additional and valuable information regarding how completely a moderator (or set of moderators) explains the variation of effect sizes. Therefore, these goodness‐of‐fit statistics may prove quite valuable, especially in meta‐analyses of highly variable study outcomes. We further recommend that in meta‐analyses for which study outcomes are already consistent, analysts should continue to perform model tests if they have theoretical expectations about moderators. We discuss these and other meta‐analytic model testing concerns.