[Show abstract][Hide abstract] ABSTRACT: Background:
Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgement of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in). We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs.
Two reviewers independently screened title, abstract and full text, and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome.
We included 202 SR (98 Cochrane and 104 non-Cochrane), the majority of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio: 2.26, 95%-CI: 1.08 to 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs.
Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimise the interpretation of their findings.
Journal of clinical epidemiology 11/2015; DOI:10.1016/j.jclinepi.2015.11.002 · 3.42 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Background:
Meta-analyses of continuous outcomes typically provide enough information for decision-makers to evaluate the extent to which chance can explain apparent differences between interventions. The interpretation of the magnitude of these differences - from trivial to large - can, however, be challenging. We investigated clinicians' understanding and perceptions of usefulness of 6 statistical formats for presenting continuous outcomes from meta-analyses (standardized mean difference, minimal important difference units, mean difference in natural units, ratio of means, relative risk and risk difference).
We invited 610 staff and trainees in internal medicine and family medicine programs in 8 countries to participate. Paperbased, self-administered questionnaires presented summary estimates of hypothetical interventions versus placebo for chronic pain. The estimates showed either a small or a large effect for each of the 6 statistical formats for presenting continuous outcomes. Questions addressed participants' understanding of the magnitude of treatment effects and their perception of the usefulness of the presentation format. We randomly assigned participants 1 of 4 versions of the questionnaire, each with a different effect size (large or small) and presentation order for the 6 formats (1 to 6, or 6 to 1).
Overall, 531 (87.0%) of the clinicians responded. Respondents best understood risk difference, followed by relative risk and ratio of means. Similarly, they perceived the dichotomous presentation of continuous outcomes (relative risk and risk difference) to be most useful. Presenting results as a standardized mean difference, the longest standing and most widely used approach, was poorly understood and perceived as least useful.
None of the presentation formats were well understood or perceived as extremely useful. Clinicians best understood the dichotomous presentations of continuous outcomes and perceived them to be the most useful. Further initiatives to help clinicians better grasp the magnitude of the treatment effect are needed.
Canadian Medical Association Journal 10/2015; DOI:10.1503/cmaj.150430 · 5.96 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: SUN(^_^)D, the Strategic Use of New generation antidepressants for Depression, is an assessor-blinded, parallel-group, multicenter pragmatic mega-trial to examine the optimum treatment strategy for the first- and second-line treatments for unipolar major depressive episodes. The trial has three steps and two randomizations. Step I randomization compares the minimum and the maximum dosing strategy for the first-line antidepressant. Step II randomization compares the continuation, augmentation or switching strategy for the second-line antidepressant treatment. Step III is a naturalistic continuation phase. The original protocol was published in 2011, and we hereby report its updated protocol including the statistical analysis plan.
We implemented two important changes to the original protocol. One is about the required sample size, reflecting the smaller number of dropouts than had been expected. Another is in the organization of the primary and secondary outcomes in order to make the report of the main trial results as pertinent and interpretable as possible for clinical practices. Due to the complexity of the trial, we plan to report the main results in two separate reports, and this updated protocol and the statistical analysis plan have laid out respective primary and secondary outcomes and their analyses. We will convene the blind interpretation committee before the randomization code is broken.
This paper presents the updated protocol and the detailed statistical analysis plan for the SUN(^_^)D trial in order to avoid reporting bias and data-driven results.
ClinicalTrials.gov: NCT01109693 (registered on 21 April 2010).
[Show abstract][Hide abstract] ABSTRACT: Background: Community-acquired pneumonia (CAP) is common and often severe. Purpose: To examine the effect of adjunctive corticosteroid therapy on mortality, morbidity, and duration of hospitalization in patients with CAP. Data Sources: MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials through 24 May 2015. Study Selection: Randomized trials of systemic corticosteroids in hospitalized adults with CAP. Data Extraction: Two reviewers independently extracted study data and assessed risk of bias. Quality of evidence was assessed with the Grading of Recommendations Assessment, Development, and Evaluation system by consensus among the authors. Data Synthesis: The median age was typically in the 60s, and approximately 60% of patients were male. Adjunctive corticosteroids were associated with possible reductions in all-cause mortality (12 trials; 1974 patients; risk ratio [RR], 0.67 [95% CI, 0.45 to 1.01]; risk difference [RD], 2.8%; moderate certainty), need for mechanical ventilation (5 trials; 1060 patients; RR, 0.45 [CI, 0.26 to 0.79]; RD, 5.0%; moderate certainty), and the acute respiratory distress syndrome (4 trials; 945 patients; RR, 0.24 [CI, 0.10 to 0.56]; RD, 6.2%; moderate certainty). They also decreased time to clinical stability (5 trials; 1180 patients; mean difference,-1.22 days [CI, -2.08 to -0.35 days]; high certainty) and duration of hospitalization (6 trials; 1499 patients; mean difference, -1.00 day [CI, -1.79 to -0.21 days]; high certainty). Adjunctive corticosteroids increased frequency of hyperglycemia requiring treatment (6 trials; 1534 patients; RR, 1.49 [CI, 1.01 to 2.19]; RD, 3.5%; high certainty) but did not increase frequency of gastrointestinal hemorrhage. Limitations: There were few events and trials for many outcomes. Trials often excluded patients at high risk for adverse events. Conclusion: For hospitalized adults with CAP, systemic corticosteroid therapy may reduce mortality by approximately 3%, need for mechanical ventilation by approximately 5%, and hospital stay by approximately 1 day. Primary Funding Source: None.
Annals of internal medicine 10/2015; 163(7):519-528. DOI:10.7326/M150715 · 17.81 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Objectives: To describe how systematic reviewers are reporting missing data for dichotomous outcomes, handling them in the analysis and assessing the risk of associated bias. Methods: We searched MEDLINE and the Cochrane Database of Systematic Reviews for systematic reviews of randomised trials published in 2010, and reporting a meta-analysis of a dichotomous outcome. We randomly selected 98 Cochrane and 104 non-Cochrane systematic reviews. Teams of 2 reviewers selected eligible studies and abstracted data independently and in duplicate using standardised, piloted forms with accompanying instructions. We conducted regression analyses to explore factors associated with using complete case analysis and with judging the risk of bias associated with missing participant data.
BMJ Open 09/2015; 5(9). DOI:10.1136/bmjopen-2015-009368 · 2.27 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Objective:
Many strong recommendations issued by the World Health Organization (WHO) are based on low or very low quality (low certainty) evidence (discordant recommendations). Many such discordant recommendations are inconsistent with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. We sought to understand why WHO makes discordant recommendations inconsistent with GRADE guidance.
Study design and setting:
We interviewed panel members involved in guidelines approved by WHO (2007-2012) that included discordant recommendations. Interviews, recorded and transcribed, focused on use of GRADE including the reasoning underlying, and factors contributing to, discordant recommendations.
Four themes emerged: strengths of GRADE, challenges and barriers to GRADE, strategies to improve GRADE application, and explanations for discordant recommendations. Reasons for discordant recommendations included skepticism about the value of making conditional recommendations; political considerations; high certainty in benefits (sometimes warranted, sometimes not) despite assessing evidence as low certainty; and concerns that conditional recommendations will be ignored.
WHO panelists make discordant recommendations inconsistent with GRADE guidance for reasons that include limitations in their understanding of GRADE. Ensuring optimal application of GRADE at WHO and elsewhere likely requires selecting panelists who have a commitment to GRADE principles, additional training of panellists, and formal processes to maximize adherence to GRADE principles.
Journal of clinical epidemiology 09/2015; DOI:10.1016/j.jclinepi.2015.09.006 · 3.42 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Objective:
We compared the distribution of heterogeneity in meta-analyses of binary and continuous outcomes.
Study design and setting:
We searched citations in Medline and Cochrane databases for meta-analyses of randomized trials published in 2012 that reported a measure of heterogeneity of either binary or continuous outcomes. Two reviewers independently performed eligibility screening and data abstraction. We evaluated the distribution of I(2) in meta-analyses of binary and continuous outcomes and explored hypotheses explaining the difference in distributions.
After full-text screening, we selected 671 meta-analyses evaluating 557 binary and 352 continuous outcomes. Heterogeneity as assessed by I(2) proved higher in continuous than in binary outcomes: the proportion of continuous and binary outcomes reporting an I(2) of 0% was 34% versus 52% respectively and reporting an I(2) of 60-100% was 39% versus 14%. In continuous but not binary outcomes, I(2) increased with larger number of studies included in a meta-analysis. Increased precision and sample size do not explain the larger I(2) found in meta-analyses of continuous outcomes with a larger number of studies.
Meta-analyses evaluating continuous outcomes showed substantially higher I(2) than meta-analyses of binary outcomes. Results suggest differing standards for interpreting I(2) in continuous versus binary outcomes may be appropriate.
Journal of clinical epidemiology 09/2015; DOI:10.1016/j.jclinepi.2015.09.005 · 3.42 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Background:
The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with GRADE guidance.
To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations.
Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published manuscript on GRADE methodology.
Five eligible methodologists participated in detailed semi-structured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes.
Three themes emerged from the interviews of five methodologists: i) The perceived role of methodologists in the process, ii) Contributors to discordant recommendations and iii) Strategies for improvement. Salient findings included i) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; ii) both financial and non-financial conflicts of interest among panel members as an explanation for discordant recommendations; and iii) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels.
These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.
Journal of clinical epidemiology 09/2015; DOI:10.1016/j.jclinepi.2015.09.003 · 3.42 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Background and Purpose—Central poststroke pain is a chronic neuropathic disorder that follows a stroke. Current research on its management is limited, and no review has evaluated all therapies for central poststroke pain.
Methods—We conducted a systematic review of randomized controlled trials to evaluate therapies for central poststroke
pain. We identified eligible trials, in any language, by systematic searches of AMED, CENTRAL, CINAHL, DARE,
EMBASE, HealthSTAR, MEDLINE, and PsychINFO. Eligible trials (1) enrolled ≥10 patients with central poststroke pain; (2) randomly assigned them to an active therapy or a control arm; and (3) collected outcome data ≥14 days after treatment. Pairs of reviewers, independently and in duplicate, screened titles and abstracts of identified citations, reviewed full texts of potentially eligible trials, and extracted information from eligible studies. We used a modified Cochrane tool to evaluate risk of bias of eligible studies, and collected patient-important outcomes according to recommendations by the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials. We conducted, when possible, random effects meta-analyses, and evaluated our certainty in treatment effects using the Grading of Recommendations Assessment, Development, and Evaluation System.
Results—Eight eligible English language randomized controlled trials (459 patients) tested anticonvulsants, an antidepressant, an opioid antagonist, repetitive transcranial magnetic stimulation, and acupuncture. Results suggested that all therapies had little to no effect on pain and other patient-important outcomes. Our certainty in the treatment estimates ranged from very low to low.
Conclusions—Our findings are inconsistent with major clinical practice guidelines; the available evidence suggests no beneficial effects of any therapies that researchers have evaluated in randomized controlled trials.
[Show abstract][Hide abstract] ABSTRACT: Guidelines suggest percutaneous intervention (PCI) of only the culprit artery in patients presenting with ST-segment elevation myocardial infarction (STEMI) and multivessel coronary artery disease. However, recent randomized controlled trials (RCTs) suggest benefit to performing PCI of other stenotic vessels at the same time as culprit vessel PCI.
[Show abstract][Hide abstract] ABSTRACT: Questions:
In Switzerland, evaluation of work capacity in individuals with mental disorders has come under criticism. We surveyed stakeholders about their concerns and expectations of the current claim process.
We conducted a nationwide online survey among five stakeholder groups. We asked 37 questions addressing the claim process and the evaluation of work capacity, the maximum acceptable disagreement in judgments on work capacity, and its documentation.
Response rate among 704 stakeholders (95 plaintiff lawyers, 285 treating psychiatrists, 129 expert psychiatrists evaluating work capacity, 64 social judges, 131 insurers) varied between 71% and 29%. Of the lawyers, 92% were dissatisfied with the current claim process, as were psychiatrists (73%) and experts (64%), whereas the majority of judges (72%) and insurers (81%) were satisfied. Stakeholders agreed in their concerns, such as the lack of a transparent relationship between the experts' findings and their conclusions regarding work capacity, medical evaluations inappropriately addressing legal issues, and the experts' delay in finalising the report. Findings mirror the characteristics that stakeholders consider important for an optimal work capacity evaluation. For a scenario where two experts evaluate the same claimant, stakeholders considered an inter-rater difference of 10%‒20% in work capacity at maximum acceptable.
Plaintiff lawyers, treating psychiatrists and experts perceive major problems in work capacity evaluation of psychiatric claims whereas judges and insurers see the process more positively. Efforts to improve the process should include clarifying the basis on which judgments are made, restricting judgments to areas of expertise, and ensuring prompt submission of evaluations.
[Show abstract][Hide abstract] ABSTRACT: Community-acquired pneumonia (CAP) is common and often severe.
To examine the effect of adjunctive corticosteroid therapy on mortality, morbidity, and duration of hospitalization in patients with CAP.
MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials through 24 May 2015.
Randomized trials of systemic corticosteroids in hospitalized adults with CAP.
Two reviewers independently extracted study data and assessed risk of bias. Quality of evidence was assessed with the Grading of Recommendations Assessment, Development and Evaluation system by consensus among the authors.
The median age was typically in the 60s, and approximately 60% of patients were male. Adjunctive corticosteroids were associated with possible reductions in all-cause mortality (12 trials; 1974 patients; risk ratio [RR], 0.67 [95% CI, 0.45 to 1.01]; risk difference [RD], 2.8%; moderate certainty), need for mechanical ventilation (5 trials; 1060 patients; RR, 0.45 [CI, 0.26 to 0.79]; RD, 5.0%; moderate certainty), and the acute respiratory distress syndrome (4 trials; 945 patients; RR, 0.24 [CI, 0.10 to 0.56]; RD, 6.2%; moderate certainty). They also decreased time to clinical stability (5 trials; 1180 patients; mean difference, -1.22 days [CI, -2.08 to -0.35 days]; high certainty) and duration of hospitalization (6 trials; 1499 patients; mean difference, -1.00 day [CI, -1.79 to -0.21 days]; high certainty). Adjunctive corticosteroids increased frequency of hyperglycemia requiring treatment (6 trials; 1534 patients; RR, 1.49 [CI, 1.01 to 2.19]; RD, 3.5%; high certainty) but did not increase frequency of gastrointestinal hemorrhage.
There were few events and trials for many outcomes. Trials often excluded patients at high risk for adverse events.
For hospitalized adults with CAP, systemic corticosteroid therapy may reduce mortality by approximately 3%, need for mechanical ventilation by approximately 5%, and hospital stay by approximately 1 day.
Annals of internal medicine 08/2015; DOI:10.7326/M15-0715 · 17.81 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Asymptomatic deep-vein thrombosis (DVT) detected by mandatory venography, a surrogate outcome, comprises most of the efficacy outcome events in recent thromboprophylaxis trials. The validity of this surrogate to estimate trade-off between thrombotic and bleeding events in these clinical trials requires a consistent relationship between asymptomatic DVT and symptomatic venous thromboembolism (VTE). In this systematic review of high quality VTE prevention trials, we examined the consistency of the ratios of asymptomatic DVT to symptomatic VTE across a broad range of indications. Studies were identified from citations listed in the chapters on VTE prevention in the antithrombotic guidelines by the American College of Chest Physicians, 2012. A study was eligible if it: 1) was a randomised trial comparing an anticoagulant with standard of care; 2) included at least 500 participants; 3) reported asymptomatic or all DVT rates; and 4) reported symptomatic VTE rates. Of the 26 eligible trials, 19 trials were conducted in orthopaedic patients, five in general surgery patients and two in general medical patients. The overall median rates (ranges) for asymptomatic DVT and symptomatic VTE were 9.11 % (0.75 to 54.87 %) and 0.49 % (0.00 to 3.10 %), respectively. The median ratio was 14.53, with a wide range (2.75 to 103.86). Wide variability in the ratios persisted despite indication- and anticoagulant-specific analyses. In VTE prevention trials of alternative anticoagulants, the wide variability in the ratios of asymptomatic DVT to symptomatic VTE precludes judging the trade-off between thrombotic and bleeding events on the basis of composite outcomes dominated by venographic DVT.
Thrombosis and Haemostasis 07/2015; 114(5). DOI:10.1160/TH14-12-1006 · 4.98 Impact Factor