Ain't Necessarily So: Review and Critique of Recent Meta-Analyses of Behavioral Medicine Interventions in Health Psychology

Department of Psychiatry, University of Pennsylvania School of Medicine, 3535 Market St., Room 676, Philadelphia, PA 19104, USA.
Health Psychology (Impact Factor: 3.95). 03/2010; 29(2):107-16. DOI: 10.1037/a0017633
Source: PubMed

ABSTRACT We examined four meta-analyses of behavioral interventions for adults (Dixon, Keefe, Scipio, Perri, & Abernethy, 2007; Hoffman, Papas, Chatkoff, & Kerns, 2007; Irwin, Cole, & Nicassio, 2006; and Jacobsen, Donovan, Vadaparampil, & Small, 2007) that have appeared in the Evidence Based Treatment Reviews section of Health Psychology.
Narrative review.
We applied the following criteria to each meta-analysis: (1) whether each meta-analysis was described accurately, adequately, and transparently in the article; (2) whether there was an adequate attempt to deal with methodological quality of the original trials; (3) the extent to which the meta-analysis depended on small, underpowered studies; and (4) the extent to which the meta-analysis provided valid and useful evidence-based recommendations.
Across the four meta-analyses, we identified substantial problems with the transparency and completeness with which these meta-analyses were reported, as well as a dependence on small, underpowered trials of generally poor quality.
Results of our exercise raise questions about the clinical validity and utility of the conclusions of these meta-analyses. Results should serve as a wake up call to prospective authors, reviewers, and end-users of meta-analyses now appearing in the literature.


Available from: James C Coyne, May 28, 2015
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Strong opinions for or against the use of systematic reviews to inform policymaking have been published in the medical literature. The purpose of this paper was to examine whether funding sources and author financial conflicts of interest were associated with whether an opinion article was supportive or critical of the use of systematic reviews for policymaking. We examined the nature of the arguments within each article, the types of disclosures present, and whether these articles are being cited in the academic literature. We searched for articles that expressed opinions about the use of systematic reviews for policymaking. We included articles that presented opinions about the use of systematic reviews for policymaking and categorized each article as supportive or critical of such use. We extracted all arguments regarding the use of systematic reviews from each article and inductively coded each as internal or external validity argument, categorized disclosed funding sources, conflicts of interest, and article types, and systematically searched for undisclosed financial ties. We counted the number of times each article has been cited in the "Web of Science." We report descriptive statistics. Articles that were critical of the use of systematic reviews (n = 25) for policymaking had disclosed or undisclosed industry ties 2.3 times more often than articles that were supportive of the use (n = 34). We found that editorials, comments, letters, and perspectives lacked published disclosures nearly twice as often (60% v. 33%) as other types of articles. We also found that editorials, comments, letters, and perspectives were less frequently cited in the academic literature than other article types (median number of citations = 5 v. 19). It is important to consider whether an article has industry ties when evaluating the strength of the argument for or against the use of systematic reviews for policymaking. We found that journal conflict of interest disclosures are often inadequate, particularly for editorials, comments, letters, and perspectives and that these articles are being cited as evidence in the academic literature. Our results further suggest the need for more consistent and complete disclosure for all article types.
    11/2014; 3(1):122. DOI:10.1186/2046-4053-3-122
  • [Show abstract] [Hide abstract]
    ABSTRACT: People struggle to act on the goals that they set themselves, and this gap between intention and action is likely to be exacerbated by mental health problems. Evidence suggests that forming specific if-then plans (or 'implementation intentions') can promote goal attainment and a number of studies have applied such techniques in clinical contexts. However, to date, the extent to which planning can help people with mental health problems has not been systematically examined. The present review used meta-analysis to investigate the effect of if-then planning on goal attainment among people with a DSM-IV/ICD-10 diagnosis (i.e., clinical samples) or scores above a relevant cut-off on clinical measures (i.e., analogue samples). In total, 29 experimental studies, from 18 records, met the inclusion criteria. Excluding one outlying (very large) effect, forming implementation intentions had a large-sized effect on goal attainment (d+ = 0.99, k = 28, N = 1,636). Implementation intentions proved effective across different mental health problems and goals, and in studies with different methodological approaches. Taken together, the findings suggest that forming implementation intentions can be a useful strategy for helping people with mental health problems to achieve various goals and might be usefully integrated into existing treatment approaches. However, further studies are needed addressing a wider range of mental health problems. This meta-analysis suggests that prompting people with mental health problems to form if-then plans (known as 'implementation intentions') specifying when, where, and how they will achieve their goals can be beneficial. The findings proved robust across a range of methodologies, samples, and focal goals, suggesting that forming implementation intentions can help people with a range of mental health problems to achieve a range of different goals. We provide guidance to researchers and practitioners in how to promote the formation of implementation intentions. © 2015 The British Psychological Society.
    British Journal of Clinical Psychology 05/2015; DOI:10.1111/bjc.12086 · 1.90 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Abstract Background: Systematic reviews now routinely assess methodological quality to gauge the validity of the included studies and of the synthesis as a whole. Although trends from higher quality studies should be clearer, it is uncertain how often meta-analyses incorporate methodological quality in models of study results either as predictors, or, more interestingly, in interactions with theoretical moderators. Objective and methods: We survey 200 meta-analyses in three health promotion domains to examine when and how meta-analyses incorporate methodological quality. Results: Although methodological quality assessments commonly appear in contemporary meta-analyses (usually as scales), it is rarely incorporated in analyses, and still more rarely analysed in interaction with theoretical determinants of the success of health promotions. The few meta-analyses (2.5%) that did include such an interaction analysis showed that moderator results remained significant in higher quality studies or were present only among higher quality studies. We describe how to model quality interactively with theoretically derived moderators and discuss strengths and weaknesses of this approach and in relation to current meta-analytic practice. Conclusions: In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon, but also about the role that methodological quality itself plays.
    Psychology and Health 08/2014; DOI:10.1080/08870446.2014.953533 · 2.13 Impact Factor