A preview of this full-text is provided by Hogrefe Publishing.
Content available from European Psychologist
This content is subject to copyright. Terms and conditions apply.
Gerhard Andersson TheRoleof Meta-Analysis in the Significance TestC ontroversy
The Role of Meta-Analysis in the
Significance Test Controversy
Gerhard Andersson
Department of Psychology, Uppsala University, Sweden
The critique against significance testing has been increasingly acknowl-
edged in recent years. This paper focuses on the relation between meta-
analysis and this controversy. A contradiction in the literature can be
seen in that significance testing has been blamed for the poor accumula-
tion of knowledge in psychology, while at the same time meta-analytic
reviews have claimed the opposite. Although a majority of meta-analytic
experts argue against significance testing, this critique cannot account
for the success of meta-analysis. Rather, it may be that meta-analysis has
facilitated the recognition of the significance test critique. Taking the
significance testing critique seriously has important implications for
meta-analysis in that its research base (e. g., studies) is viewed as unre-
liable. Although the significance test controversy may lead to further
fragmentation of psychology, it is not clear that this will negatively affect
the practice of meta-analysis.
Keywords: Meta-analysis, significance test debate, methodological critique, accumulation of research.
Introduction
Meta-analysis consists of a host of techniques used for
quantitatively summarizing findings from a large body
of empirical research. Since the advent of meta-analy-
sis, many of its proponents have argued that it is a way
to avoid the problems associated with significance test-
ing (Hedges & Olkin, 1985; Hunter & Schmidt, 1990;
Rosenthal, 1991). These problems are commonly de-
scribed under the heading of the “significance test con-
troversy,” and have been covered in books (Morrison &
Henkel, 1970; Chow, 1996), and in several psychology
journals recently (e.g., Chow, 1998; Kirk, 1996;
Schmidt, 1996; Shrout, 1997), and in the past (Bakan,
1966; Carver, 1978; Lykken, 1968; Rozeboom, 1960). Al-
though perhaps not familiar to all psychologists, the
practice of significance testing is due to a hybridization
of the contributions of J. Neyman, E.S. Pearson, and
R.A. Fisher (Chow, 1996; Gigerenzer, 1993; Goodman,
1993). More specifically, according to Chow (1996), sig-
nificance testing (based on the null hypothesis proce-
dure) has two sources: A statistical decision (based on
the pvalue) and an inferential procedure (based on con-
ditional syllogisms). The pvalue stands for the proba-
bility of obtaining data as extreme or more extreme
given that the null hypothesis is true: p(data null hy-
pothesis). Although there are different strands in the
critique against significance testing and the use of the
null hypothesis procedure, there are several issues on
which a majority of opponents agree. The criticism
against significance testing is often directed against the
European
Psychologist
ψψ
ψ
ψ
ψ
ψψ
ψ
ψ
ψ
Gerhard Andersson was educated at Uppsala University, Sweden
(MSc and PhD in Clinical Psychology). He is now Associate Professor
of Psychology at Uppsala University. He also holds a position as clinical
psychologist at the Department of Audiology, Uppsala University
Hospital. His current research interests are health psychology, meth-
odology, and clinical psychology. He has published papers on hearing
impairment, dizziness, tinnitus, and optimism. He is also the Editor-
in-Chief of the Scandinavian Journal of Behaviour Therapy.
Correspondence concerning this article should be addressed to Ger-
hard Andersson, Department of Psychology, Uppsala University, Box
1225, SE-751 42 Uppsala, Sweden (tel. +46 18 471-2116, fax +46
18 471-2123, E-mail Gerhard.Andersson@psyk.uu.se).
European Psychologist, Vol. 4, No. 2, June 1999, pp. 75–82
© 1999 Hogrefe & Huber Publishers 75
