Rank Reversal in Indirect Comparisons

Department of Health Management and Policy, University of Michigan, Ann Arbor, MI, USA
Value in Health (Impact Factor: 3.28). 12/2012; 15(8):1137-40. DOI: 10.1016/j.jval.2012.06.001
Source: PubMed


To describe rank reversal as a source of inconsistent interpretation intrinsic to indirect comparison (Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epi 1997;50:683-91) of treatments and to propose best practice.
We prove our main points with intuition, examples, graphs, and mathematical proofs. We also provide software and discuss implications for research and policy.
When comparing treatments by indirect means and sorting them by effect size, three common measures of comparison (risk ratio, risk difference, and odds ratio) may lead to vastly different rankings.
The choice of risk measure matters when making indirect comparisons of treatments. The choice should depend primarily on the study design and the conceptual framework for that study.

Download full-text


Available from: Jason J. Wang, Jan 30, 2014
  • Source

    Value in Health 03/2013; 16(2):449-51. DOI:10.1016/j.jval.2012.11.012 · 3.28 Impact Factor
  • Source

    Value in Health 03/2013; 16(2):451-2. DOI:10.1016/j.jval.2013.02.003 · 3.28 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background Homogeneity and consistency assumptions underlie network meta-analysis (NMA). Methods exist to assess the assumptions but they are rarely and poorly applied. We review and illustrate methods to assess homogeneity and consistency. Methods Eligible articles focussed on indirect comparison or NMA methodology. Articles were sought by hand-searching and scanning references (March 2013). Assumption assessment methods described in the articles were reviewed, and applied to compare anti-malarial drugs. Results116 articles were included. Methods to assess homogeneity were: comparing characteristics across trials; comparing trial-specific treatment effects; using hypothesis tests or statistical measures; applying fixed-effect and random-effects pair-wise meta-analysis; and investigating treatment effect-modifiers. Methods to assess consistency were: comparing characteristics; investigating treatment effect-modifiers; comparing outcome measurements in the referent group; node-splitting; inconsistency modelling; hypothesis tests; back transformation; multidimensional scaling; a two-stage approach; and a graph-theoretical method.For the malaria example, heterogeneity existed for some comparisons that was unexplained by investigating treatment effect-modifiers. Inconsistency was detected using node-splitting and inconsistency modelling. It was unclear whether the covariates explained the inconsistency. Conclusions Presently, we advocate applying existing assessment methods collectively to gain the best understanding possible regarding whether assumptions are reasonable. In our example, consistency was questionable; therefore the NMA results may be unreliable. Copyright © 2013 John Wiley & Sons, Ltd.
    Research Synthesis Methods 12/2013; 4(4). DOI:10.1002/jrsm.1085
Show more