Article

Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews

Faculty of Health, University of East Anglia, Norwich NR4 7TJ.
BMJ (online) (Impact Factor: 16.38). 02/2009; 338(apr03 1):b1147. DOI: 10.1136/bmj.b1147
Source: PubMed

ABSTRACT To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions.
Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used.
Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned.
The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases.
Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems. APPENDIX 1: PubMed search strategy. APPENDIX 2: Characteristics of identified reports. APPENDIX 3: Identified studies. References of included studies.

0 Followers
 · 
90 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Network meta-analysis (NMA) has emerged as a useful analytical tool allowing comparison of multiple treatments based on direct and indirect evidence. Commonly, a hierarchical Bayesian NMA model is used, which allows rank probabilities (the probability that each treatment is best, second best, and so on) to be calculated for decision making. However, the statistical properties of rank probabilities are not well understood. This study investigates how rank probabilities are affected by various factors such as unequal number of studies per comparison in the network, the sample size of individual studies, the network configuration, and effect sizes between treatments. In order to explore these factors, a simulation study of four treatments (three equally effective treatments and one less effective reference) was conducted. The simulation illustrated that estimates of rank probabilities are highly sensitive to both the number of studies per comparison and the overall network configuration. An unequal number of studies per comparison resulted in biased estimates of treatment rank probabilities for every network considered. The rank probability for the treatment that was included in the fewest number of studies was biased upward. Conversely, the rank of the treatment included in the most number of studies was consistently underestimated. When the simulation was altered to include three equally effective treatments and one superior treatment, the hierarchical Bayesian NMA model correctly identified the most effective treatment, regardless of all factors varied. The results of this study offer important insight into the ability of NMA models to rank treatments accurately under several scenarios. The authors recommend that health researchers use rank probabilities cautiously in making important decisions.
    Clinical Epidemiology 01/2014; 6:451-60. DOI:10.2147/CLEP.S69660
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Network meta-analyses (NMAs) are complex methodological approaches that may be challenging for non-technical end-users, such as policymakers and clinicians, to understand. Consideration should be given to identifying optimal approaches to presenting NMAs that help clarify analyses. It is unclear what guidance researchers currently have on how to present and tailor NMAs to different end-users. A systematic review of NMA guidelines was conducted to identify guidance on how to present NMAs. Electronic databases and supplementary sources were searched for NMA guidelines. Presentation format details related to sample formats, target audiences, data sources, analysis methods and results were extracted and frequencies tabulated. Guideline quality was assessed following criteria developed for clinical practice guidelines. Seven guidelines were included. Current guidelines focus on how to conduct NMAs but provide limited guidance to researchers on how to best present analyses to different end-users. None of the guidelines provided reporting templates. Few guidelines provided advice on tailoring presentations to different end-users, such as policymakers. Available guidance on presentation formats focused on evidence networks, characteristics of individual trials, comparisons between direct and indirect estimates and assumptions of heterogeneity and/or inconsistency. Some guidelines also provided examples of figures and tables that could be used to present information. Limited guidance exists for researchers on how best to present NMAs in an accessible format, especially for non-technical end-users such as policymakers and clinicians. NMA guidelines may require further integration with end-users' needs, when NMAs are used to support healthcare policy and practice decisions. Developing presentation formats that enhance understanding and accessibility of NMAs could also enhance the transparency and legitimacy of decisions informed by NMAs.
    PLoS ONE 12/2014; 9(12):e113277. DOI:10.1371/journal.pone.0113277 · 3.53 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Several phase I/II studies of chemoradiotherapy for gastric cancer have reported promising results, but the significance of preoperative radiotherapy in addition to chemotherapy has not been proven. In this study, a systematic literature search was performed to capture survival and postoperative morbidity and mortality data in randomised clinical studies comparing preoperative (chemo)radiotherapy or chemotherapy versus surgery alone, or preoperative chemoradiotherapy versus chemotherapy for gastric and/or gastro-oesophageal junction (GOJ) cancer. Hazard ratios (HRs) for overall mortality were extracted from the original studies, individual patient data provided from the principal investigators of eligible studies or the earlier published meta-analysis. The incidences of postoperative morbidities and mortalities were also analysed. In total 18 studies were eligible and data were available from 14 of these. The meta-analysis on overall survival yielded HRs of 0.75 (95% CI 0.65-0.86, P < 0.001) for preoperative (chemo)radiotherapy and 0.83 (95% CI 0.67-1.01, P = 0.065) for preoperative chemotherapy when compared to surgery alone. Direct comparison between preoperative chemoradiotherapy and chemotherapy resulted in an HR of 0.71 (95% CI 0.45-1.12, P = 0.146). Combination of direct and adjusted indirect comparisons yielded an HR of 0.86 (95% CI 0.69-1.07, P = 0.171). No statistically significant differences were seen in the risk for postoperative morbidity or mortality between preoperative treatments and surgery alone, or preoperative (chemo)radiotherapy and chemotherapy. Preoperative (chemo)radiotherapy for gastric and GOJ cancer showed significant survival benefit over surgery alone. In comparisons between preoperative chemotherapy and (chemo)radiotherapy, there is a trend towards improved survival when adding radiotherapy, without increased postoperative morbidity or mortality. Copyright © 2014 Elsevier Ltd. All rights reserved.
    European Journal of Surgical Oncology 11/2014; 41(3). DOI:10.1016/j.ejso.2014.11.039 · 2.89 Impact Factor

Full-text (2 Sources)

Download
33 Downloads
Available from
May 27, 2014