(Cooper, Maxwell, Stone, & Sher, 2008). Even before these stan-
dards are adopted as requirements for publication in APA journals
and elsewhere, they provide invaluable guidance for avoiding
some of the problems we identified with these four meta-analyses.
Yet, it will be decades before these new standards are seen in the
quality of the bulk of the trials available for consideration for
inclusion in meta-analyses. Meanwhile, authors contemplating
conducting a meta-analysis must contend with a literature domi-
nated by methodologically flawed studies and decide whether and
how to proceed. One way to resolve this dilemma is not to conduct
a meta-analysis, as Moher, Jadad, and Klassen (1998) suggest:
There is often a perception that the statistical combination of data
across studies is the most important part of a systematic review. We
take such a view cautiously. We believe that a well-reported, system-
atic qualitative review is much better than an inappropriately con-
ducted and reported quantitative review or meta-analysis (pp. 916).
The risk of publishing meta-analyses that are premature because
of limitations in the number and quality of available studies is that
the accumulation of better, larger-scale studies and the integration
of their results in a future meta analysis will be discouraged
because of a false impression that the research question was
References marked with an asterisk indicate studies included in the
*Alaranta, H., Rytokoski, U., Rissanen, A., Talo, S., Ronnemaa, T.,
Puukka, P., et al. (1994). Intensive physical and psychosocial training-
program for patients with chronic low-back-pain—a controlled clinical-
trial. Spine, 19, 1339 –1349.
Astin, J. A., Beckner, W., Soeken, K., Hochberg, M. C., & Berman, B.
(2002). Psychological interventions for rheumatoid arthritis: A meta-
analysis of randomized controlled trials. Arthritis & Rheumatism-
Arthritis Care & Research, 47, 291–302.
*Barlow, J. H., Turner, A. P., & Wright, C. C. (2000). A randomized
controlled study of the Arthritis Self-Management Programme in the
UK. Health Education Research, 15, 665– 680.
*Bordeleau, L., Szalai, J. P., Ennis, M., Leszcz, M., Speca, M., Sela, R., et
al. (2003). Quality of life in a randomized trial of group psychosocial
support in metastatic breast cancer: Overall effects of the intervention
and an exploration of missing data. Journal of Clinical Oncology, 21,
*Brox, J. I., Sorensen, R., Friis, A., Nygaard, O., Indahl, A., Keller, A., et
al. (2003). Randomized clinical trial of lumbar instrumented fusion and
cognitive intervention and exercises in patients with chronic low back
pain and disc degeneration. Spine, 28, 1913–1921.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.
Conn, V. S., Hafdahl, A. R., Porock, D. C., McDaniel, R., & Nielsen, P. J.
(2006). A meta-analysis of exercise interventions among people treated
for cancer. Supportive Care in Cancer, 14, 699–712.
Cooper, H., Maxwell, S., Stone, A., & Sher, K. J. (2008). Reporting
standards for research in psychology: Why do we need them? What
might they be? American Psychologist, 63, 839– 851.
Coyne, J. C., Lepore, S. J., & Palmer, S. C. (2006). Efficacy of psychos-
ocial interventions in cancer care: Evidence is weaker than it first looks.
Annals of Behavioral Medicine, 32, 104 –110.
Coyne, J. C., Thombs, B., & Hagedoorn, M. (2008). A meta-analysis of
psychosocial interventions for cancer patients gone awry. Annals of
Behavioral Medicine, 37, 94 –96.
Dixon, K. E., Keefe, F. J., Scipio, C. D., Perri, L. M., & Abernethy, A. P.
(2007). Psychological interventions for arthritis pain management in
adults: A meta-analysis. Health Psychology, 26, 241–250.
*Goodwin, P. J., Leszcz, M., Ennis, M., Koopmans, J., Vincent, L., Guther,
H., et al. (2001). The effect of group psychosocial support on survival in
metastatic breast cancer. New England Journal of Medicine, 345, 1719 –
*Hammond, A., & Freeman, K. (2001). One-year outcomes of a randomized
controlled trial of an educational-behavioral joint protection program for
people with rheumatoid arthritis. Rheumatology, 40, 1044 –1051.
*Hernandez-Reif, M., Field, T., Krasnegor, J., & Theakston, H. (2001).
Lower back pain is reduced and range of motion increased after massage
therapy. International Journal of Neuroscience, 106, 131–145.
Higgins, J. P. T., & Altman, D. G. (2008). chap. 8: Assessing risk of bias
in included studies. In J. P. T. Higgins & S. Green (Eds.), Cochrane
handbook for systematic reviews of interventions version 5.0.1 (updated
September 2008). The Cochrane Collaboration. Available from
Hoffman, B. M., Papas, R. K., Chatkoff, D. K., & Kerns, R. D. (2007).
Meta-analysis of psychological interventions for chronic low back pain.
Health Psychology, 26, 1–9.
Howard, G. S., Hill, T. L., Maxwell, S. E., Baptista, T. M., Farias, M. H.,
Coelho, C., et al. What’s wrong with research literatures? And how to
make them right. Review of General Psychology, 13, 146–166.
Ioannidis, J. P. (2008a). Why most discovered true associations are in-
flated. Epidemiology, 19, 640 – 648.
Ioannidis, J. P. A. (2008b). Interpretation of tests of heterogeneity and bias
in meta-analysis. Journal of Evaluation in Clinical Practice, 14, 951–
Irwin, M. R., Cole, J. C., & Nicassio, P. M. (2006). Comparative meta-
analysis of behavioral interventions for insomnia and their efficacy in
middle-aged adults and in older adults 55⫹ years of age. Health Psy-
chology, 25, 3–14.
Jacobsen, P. B., Donovan, K. A., Vadaparampil, S. T., & Small, B. J.
(2007). Systematic review and meta-analysis of psychological and
activity-based interventions for cancer-related fatigue. Health Psychol-
ogy, 26, 660 – 667.
Juni, P., Altman, D. G., & Egger, M. (2001). Systematic reviews in health
care—Assessing the quality of controlled clinical trials. British Medical
Journal, 323, 42– 46.
Juni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The hazards of
scoring the quality of clinical trials for meta-analysis. Journal of the
American Medical Association-Journal of the American Medical Asso-
ciation, 282, 1054 –1060.
Kangas, M., Bovbjerg, D. H., & Montgomery, G. H. (2008). Cancer-
related fatigue: A systematic and meta-analytic review of non-
pharmacological therapies for cancer patients. Psychological Bulle-
tin, 134, 700 –741.
*Kankaanpaa, M., Taimela, S., Airaksinen, O., & Hanninen, O. (1999).
The efficacy of active rehabilitation in chronic low back pain—Effect on
pain intensity, self-experienced disability, and lumbar fatigability. Spine,
24, 1034 –1042.
Keefe, F. J., Caldwell, D. S., Baucom, D., Salley, A., Robinson, E.,
Timmons, K., et al. (1996). Spouse-assisted coping skills training in the
management of osteoarthritic knee pain. Arthritis Care and Research, 9,
*Keefe, F. J., Blumenthal, J., Baucom, D., Affleck, G., Waugh, R., Cald-
well, D. S., et al. (2004). Effects of spouse-assisted coping skills training
and exercise training in patients with osteoarthritic knee pain: A ran-
domized controlled study. Pain, 110, 539 –549.
*Kraaimaat, F. W., Brons, M. R., Geenen, R., & Bijlsma, J. W. J. (1995).
The effect of cognitive-behavior therapy in patients with rheumatoid-
arthritis. Behavior Research and Therapy, 33, 487–495.
Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A. (1998).
Advantages of excluding underpowered studies in meta-analysis: