Impact of allocation concealment on conclusion drawn from meta-analyses of randomized trials

IT University of Copenhagen, København, Capital Region, Denmark
International Journal of Epidemiology (Impact Factor: 9.18). 09/2007; 36(4). DOI: 10.1093/ije/dym087
Source: OAI


Background Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials. Material Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions. Methods We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant. Results Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56–79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect. Conclusion Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.

Download full-text


Available from: Karsten Juhl Jørgensen, Jun 24, 2014
    • "Mit dieser Methode wird demnach der Gefahr einer Verzerrung von Studienergebnissen entgegen gewirkt. Mittlerweile ist auch bekannt, dass ganze Schlussfolgerungen von Meta-Analysen falsch sein können , wenn Studien ohne randomisierten Zuordnungsprozess eingeschlossen werden (Pildal et al. 2007), dies muss jedoch nicht zwangsweise der Fall sein (Torgerson 2007). "

    Zeitschrift für Erziehungswissenschaft 08/2015; DOI:10.1007/s11618-015-0650-6 · 0.99 Impact Factor
  • Source
    • "We extracted data on the effect of self-management interventions on health care utilisation and total costs. To assess study quality, we chose a dichotomous measure based on allocation concealment, as this is consistently associated with treatment effect [20, 21]. Allocation concealment was judged as adequate or inadequate according to the Cochrane risk of bias tool. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: There is increasing interest in the role of 'self-management' interventions to support the management of long-term conditions in health service settings. Self-management may include patient education, support for decision-making, self-monitoring and psychological and social support. Self-management support has potential to improve the efficiency of health services by reducing other forms of utilisation (such as primary care or hospital use), but a shift to self-management may lead to negative outcomes, such as patients who feel more anxious about their health, are less able to cope, or who receive worse quality of care, all of which may impact on their health and quality of life. We sought to determine which models of self-management support are associated with significant reductions in health services utilisation without compromising outcomes among patients with long-term conditions. Methods: We used systematic review with meta-analysis. We included randomised controlled trials in patients with long-term conditions which included self-management support interventions and reported measures of service utilisation or costs, as well as measures of health outcomes (standardized disease specific quality of life, generic quality of life, or depression/anxiety).We searched multiple databases (CENTRAL, CINAHL, Econlit, EMBASE, HEED, MEDLINE, NHS EED and PsycINFO) and the reference lists of published reviews. We calculated effects sizes for both outcomes and costs, and presented the results in permutation plots, as well as conventional meta-analyses. Results: We included 184 studies. Self-management support was associated with small but significant improvements in health outcomes, with the best evidence of effectiveness in patients with diabetic, respiratory, cardiovascular and mental health conditions. Only a minority of self-management support interventions reported reductions in health care utilisation in association with decrements in health. Evidence for reductions in utilisation associated with self-management support was strongest in respiratory and cardiovascular problems. Studies at higher risk of bias were more likely to report benefits. Conclusions: Self-management support interventions can reduce health service utilization without compromising patient health outcomes, although effects were generally small, and the evidence was strongest in respiratory and cardiovascular disorders. Further work is needed to determine which components of self-management support are most effective.
    BMC Health Services Research 08/2014; 14(1):356. DOI:10.1186/1472-6963-14-356 · 1.71 Impact Factor
  • Source
    • "An RCT was assessed to be at low risk of bias based on its performance across two domains – allocation concealment and blinding of outcome assessment. These domains were chosen for bias assessment as the significance of good allocation concealment and outcome assessment blinding in minimising bias and, in particular, overestimation of treatment effect is well supported by empirical evidence [24], [25]. While empirical evidence also exists to support the significance of adequate blinding of participants in reducing exaggeration of estimated treatment effects [25] the inherent difficulty of blinding participants in surgical RCTs necessitated the exclusion of this domain in our assessment. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We investigated the proportion of orthopaedic procedures supported by evidence from randomised controlled trials comparing operative procedures to a non-operative alternative. Orthopaedic procedures conducted in 2009, 2010 and 2011 across three metropolitan teaching hospitals were identified, grouped and ranked according to frequency. Searches of the Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) were performed to identify RCTs evaluating the most commonly performed orthopaedic procedures. Included studies were categorised as "supportive" or "not supportive" of operative treatment. A risk of bias analysis was conducted for included studies using the Cochrane Collaboration's Risk of Bias tool. A total of 9,392 orthopaedic procedures were performed across the index period. 94.6% (8886 procedures) of the total volume, representing the 32 most common operative procedure categories, were used for this analysis. Of the 83 included RCTs, 22.9% (19/83) were classified as supportive of operative intervention. 36.9% (3279/8886) of the total volume of procedures performed were supported by at least one RCT showing surgery to be superior to a non-operative alternative. 19.6% (1743/8886) of the total volume of procedures performed were supported by at least one low risk of bias RCT showing surgery to be superior to a non-operative alternative. The level of RCT support for common orthopaedic procedures compares unfavourably with other fields of medicine.
    PLoS ONE 06/2014; 9(6):e96745. DOI:10.1371/journal.pone.0096745 · 3.23 Impact Factor
Show more