Rank Reversal in Indirect Comparisons

Department of Health Management and Policy, University of Michigan, Ann Arbor, MI, USA
Value in Health (Impact Factor: 3.28). 12/2012; 15(8):1137-40. DOI: 10.1016/j.jval.2012.06.001
Source: PubMed


To describe rank reversal as a source of inconsistent interpretation intrinsic to indirect comparison (Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epi 1997;50:683-91) of treatments and to propose best practice.
We prove our main points with intuition, examples, graphs, and mathematical proofs. We also provide software and discuss implications for research and policy.
When comparing treatments by indirect means and sorting them by effect size, three common measures of comparison (risk ratio, risk difference, and odds ratio) may lead to vastly different rankings.
The choice of risk measure matters when making indirect comparisons of treatments. The choice should depend primarily on the study design and the conceptual framework for that study.

Download full-text


Available from: Jason J. Wang, Jan 30, 2014
  • Source
    • "We suggest selecting the outcome metric for analysis on the basis of homogeneity and mathematical properties alone and then using empirical evidence on heterogeneity to compute results on the scale desired for interpretation (vanValkenhoef and Ades, 2013;Dias et al, 2013). Choice of an outcome measure is especially important in network meta-analysis, because it can affect the ranking of treatments (Norton et al, 2012).Caldwell et al (2012)discussed selecting the scale of measurement in network metaanalysis and showed that the larger evidence base in such analyses may enable a data driven approach to selecting the scale. This research set out to compare consistency across studies in meta-analyses using various types of outcome data. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper investigates how inconsistency (as measured by the I(2) statistic) among studies in a meta-analysis may differ, according to the type of outcome data and effect measure. We used hierarchical models to analyse data from 3873 binary, 5132 continuous and 880 mixed outcome meta-analyses within the Cochrane Database of Systematic Reviews. Predictive distributions for inconsistency expected in future meta-analyses were obtained, which can inform priors for between-study variance. Inconsistency estimates were highest on average for binary outcome meta-analyses of risk differences and continuous outcome meta-analyses. For a planned binary outcome meta-analysis in a general research setting, the predictive distribution for inconsistency among log odds ratios had median 22% and 95% CI: 12% to 39%. For a continuous outcome meta-analysis, the predictive distribution for inconsistency among standardized mean differences had median 40% and 95% CI: 15% to 73%. Levels of inconsistency were similar for binary data measured by log odds ratios and log relative risks. Fitted distributions for inconsistency expected in continuous outcome meta-analyses using mean differences were almost identical to those using standardized mean differences. The empirical evidence on inconsistency gives guidance on which outcome measures are most likely to be consistent in particular circumstances and facilitates Bayesian meta-analysis with an informative prior for heterogeneity. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.
    Full-text · Article · Dec 2015 · Research Synthesis Methods
  • Source

    Full-text · Article · Mar 2013 · Value in Health
  • Source

    Full-text · Article · Mar 2013 · Value in Health
Show more