Content uploaded by Janus Christian Jakobsen
Author content
All content in this area was uploaded by Janus Christian Jakobsen on Dec 01, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods
Abstract and Figures
Thresholds for statistical significance when assessing meta-analysis results are being insufficiently demonstrated by traditional 95% confidence intervals and P-values. Assessment of intervention effects in systematic reviews with meta-analysis deserves greater rigour.
Methodologies for assessing statistical and clinical significance of intervention effects in systematic reviews were considered. Balancing simplicity and comprehensiveness, an operational procedure was developed, based mainly on The Cochrane Collaboration methodology and the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidelines.
We propose an eight-step procedure for better validation of meta-analytic results in systematic reviews (1) Obtain the 95% confidence intervals and the P-values from both fixed-effect and random-effects meta-analyses and report the most conservative results as the main results. (2) Explore the reasons behind substantial statistical heterogeneity using sensitivity analyses. (3) To take account of problems with multiplicity adjust the thresholds for significance according to the number of primary outcomes. (4) Calculate required information sizes ([almost equal to] the a priori required number of participants for a meta-analysis to be conclusive) for all outcomes and analyse each outcome with trial sequential analysis. Report whether the trial sequential monitoring boundaries for benefit, harm, or futility are crossed. (5) Calculate Bayes factors for all primary outcomes. (6) Use subgroup analyses and sensitivity analyses to assess the potential impact of bias on the review results. (7) Assess the risk of publication bias. (8) Assess the clinical significance of the statistically significant review results.
If followed, the proposed eight-step procedure will increase the validity of assessments of intervention effects in systematic reviews of randomised clinical trials.
Figures - uploaded by Janus Christian Jakobsen
Author content
All figure content in this area was uploaded by Janus Christian Jakobsen
Content may be subject to copyright.
... Thus, no former or presently planned review has systematically reviewed the beneficial and harmful effects of duloxetine taking into account both the risk of random errors and the risk of systematic errors in all randomised clinical trials on major depressive disorder [58]. Hence, we planned this systematic review to assess the beneficial and harmful effects of duloxetine versus 'active' placebo, placebo or no intervention in the treatment of major depressive disorder. ...
... College Station, TX: StataCorp LLC). We will undertake meta-analysis according to the recommendations stated in the Cochrane Handbook for Systematic Reviews of Interventions and the eight-step assessment suggested by Jakobsen et al. [58]. When analysing continuous outcomes, we will calculate mean differences (MDs) with 95% confidence intervals (CIs). ...
... The more conservative point estimate is the estimate with the highest P value. We plan to assess a total of five primary and secondary outcome therefore we will consider P ≤ 0.016 as statistically significant [58]. We will investigate possible heterogeneity through subgroup analyses. ...
Background
Major depression significantly impairs quality of life, increases the risk of suicide, and poses tremendous economic burden on individuals and societies. Duloxetine, a serotonin norepinephrine reuptake inhibitor, is a widely prescribed antidepressant. The effects of duloxetine have, however, not been sufficiently assessed in earlier systematic reviews and meta-analyses.
Methods/design
A systematic review will be performed including randomised clinical trials comparing duloxetine with ‘active’ placebo, placebo or no intervention for adults with major depressive disorder. Bias domains will be assessed, an eight-step procedure will be used to assess if the thresholds for clinical significance are crossed. We will conduct meta-analyses. Trial sequential analysis will be conducted to control random errors, and the certainty of the evidence will be assessed using GRADE. To identify relevant trials, we will search Cochrane Central Register of Controlled Trials, Medical Literature Analysis and Retrieval System Online, Excerpta Medica database, PsycINFO, Science Citation Index Expanded, Social Sciences Citation Index, Conference Proceedings Citation Index—Science and Conference Proceedings Citation Index—Social Science & Humanities. We will also search Chinese databases and Google Scholar. We will search all databases from their inception to the present. Two review authors will independently extract data and perform risk of bias assessment. Primary outcomes will be the difference in mean depression scores on Hamilton Depression Rating Scale between the intervention and control groups and serious adverse events. Secondary outcomes will be suicide, suicide-attempts, suicidal ideation, quality of life and non-serious adverse events.
Discussion
No former systematic review has systematically assessed the beneficial and harmful effects of duloxetine taking into account both the risks of random errors and the risks of systematic errors. Our review will help clinicians weigh the benefits of prescribing duloxetine against its adverse effects and make informed decisions.
Systematic review registration
PROSPERO 2016 CRD42016053931
... Meta-analysis We will undertake the meta-analysis according to The Cochrane Handbook for Systematic Reviews of Interventions [36], Keus et al. [61], and our eight-step procedure suggested by Jakobsen et al. [62]. We will use the statistical software Stata version 16 to analyse data [51]. ...
... We will assess our intervention effects with both random-effects model meta-analyses (Hartung-Knapp-Sidik-Jonkman) [63] and fixed-effect model meta-analyses (Mantel-Haenszel for dichotomous outcomes and inverse variance for continuous outcomes) [36,64]. We will use the more conservative point estimate of the two [62]. The more conservative point estimate is the estimate with the highest p-value. ...
... The more conservative point estimate is the estimate with the highest p-value. We assess a total of six primary and secondary outcomes, and we will therefore consider a p-value of 0.014 or less as the threshold for statistical significance [62]. We will investigate possible heterogeneity through subgroup analyses. ...
Background
Major depressive disorder is one of the most common, burdensome, and costly psychiatric disorders worldwide. Antidepressants are frequently used to treat major depressive disorder. It has been shown repeatedly that antidepressants seem to reduce depressive symptoms with a statistically significant effect, but the clinical importance of the effect sizes seems questionable. Both beneficial and harmful effects of antidepressants have not previously been sufficiently assessed. The main objective of this review will be to evaluate the beneficial and harmful effects of antidepressants versus placebo, ‘active placebo’, or no intervention for adults with major depressive disorder.
Methods/design
A systematic review with meta-analysis will be reported as recommended by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), bias will be assessed with the Cochrane Risk of Bias tool-version 2 (ROB2), our eight-step procedure will be used to assess if the thresholds for clinical significance are crossed, Trial Sequential Analysis will be conducted to control for random errors, and the certainty of the evidence will be assessed with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. To identify relevant trials, we will search both for published and unpublished trials in major medical databases from their inception to the present. Clinical study reports will be obtained from regulatory authorities and pharmaceutical companies. Two review authors will independently screen the results of the literature searches, extract data, and perform risk of bias assessment. We will include any published or unpublished randomised clinical trial comparing one or more antidepressants with placebo, ‘active placebo’, or no intervention for adults with major depressive disorder. The following active agents will be included: agomelatine, amineptine, amitriptyline, bupropion, butriptyline, cianopramine, citalopram, clomipramine, dapoxetine, demexiptiline, desipramine, desvenlafaxine, dibenzepin, dosulepin, dothiepin, doxepin, duloxetine, escitalopram, fluoxetine, fluvoxamine, imipramine, iprindole, levomilnacipran, lofepramine, maprotiline, melitracen, metapramine, milnacipran, mirtazapine, nefazodone, nortriptyline, noxiptiline, opipramol, paroxetine, protriptyline, quinupramine, reboxetine, sertraline, trazodone, tianeptine, trimipramine, venlafaxine, vilazodone, and vortioxetine. Primary outcomes will be depressive symptoms, serious adverse events, and quality of life. Secondary outcomes will be suicide or suicide attempt, suicidal ideation, and non-serious adverse events.
Discussion
As antidepressants are commonly used to treat major depressive disorder in adults, a systematic review evaluating their beneficial and harmful effects is urgently needed. This review will inform best practice in treatment and clinical research of this highly prevalent and burdensome disorder.
Systematic review registration
PROSPERO CRD42020220279
... We will perform meta-analyses with 95% using both a xed-effect and random-effects model [64]. In the case of statistical heterogeneity (I² > 70%), data will be reported using the random-effects model and prediction interval [52,59,60]. ...
... We estimate the threshold for clinical relevance using Bayes factors [64]. The Bayes factor is a likelihood ratio that indicates the relative strength of evidence for two theories [76][77][78]. ...
Background
The overall mortality attributable to cardiovascular diseases (coronary heart disease, sudden cardiac death/sudden cardiac arrest, stroke/transient ischemic attack, and peripheral arterial disease) is higher in the Black population when compared to the White population. Essential hypertension (EH) is the most important modifiable risk factor for cardiovascular diseases. The prevalence of hypertension among Black adults is also higher. Diuretics are antihypertensive drugs, and their role in the primary prevention of clinical cardiovascular outcomes in the Black population with essential hypertension remains unknown. To assess the clinical benefits and harms of diuretics, as a primary prevention approach, compared with placebo or any other antihypertensive medications.
Methods
We will search the Cochrane Central Register of Controlled Trials (to update), OVID MEDLINE (1946 to update), Embase (1980 to update), LILACS (1986 to update), and Web of Science (to update). We will manually search the reference lists of the included papers and contact researchers in the field. There will be no language restrictions in the search. We will include parallel-design and crossover randomized clinical trials that has adult Black people with essential hypertension as the population. The primary outcomes are all-cause mortality, myocardial infarction, stroke, and serious adverse events. Pregnant women will be excluded from the study.
We will perform study selection, risk of bias assessment, and data extraction in duplicate. We will estimate risk ratios (RRs) with a 95% confidence interval (95% CI) for dichotomous outcomes. For continuous outcomes, such as health-related quality of life, systolic blood pressure, and diastolic blood pressure, we will calculate the mean difference with 95% CI or the standardized mean difference with 95% CI. We will measure statistical heterogeneity using the I² statistic and use a fixed-effects and random-effects model. We will conduct a sequential trial analysis.
Discussion
Our aim is to provide external validity and try to solve conflicts about the evidence regarding use of diuretics as a primary prevention of cardiovascular outcomes in Black people with essential hypertension to guide appropriate clinical practice.
Systematic review registration
PROSPERO registration number: CRD42021240864
... For all the outcome, fixed-effect and random-effects meta-analyses were both conducted and the most conservative result would be reported. If the results of two analyses were similar, then the result with widest CI would be chosen as our main result (37). The χ 2 test of heterogeneity was simultaneously employed to confirm a fixed effects model. ...
The practice-based evidence suggests that it is possible to use eye movement desensitization and reprocessing (EMDR) to treat major depressive disorder (MDD), but its specific efficacy is unknown. A systematic search was carried out for randomized controlled trials comparing EMDR with a control condition group in MDD patients. Two meta-analyses were conducted, with symptom reduction as primary outcome and remission as exploratory outcome. Eight studies with 320 participants were included in this meta-analysis. The first meta-analysis showed that EMDR outperformed “No Intervention” in decreasing depressive symptoms (standardized mean difference [SMD] = −0.81, 95% CI = −1.22 to −0.39, p < 0.001, low certainty), but statistically significant differences were not observed in improving remission (risk ratio = 1.20, 95% CI = 0.87–1.66, p = 0.25, very low certainty). The second showed the superiority of EMDR over CBT in reducing depressive symptoms (mean difference [MD] = −7.33, 95% CI = −8.26 to −6.39, p < 0.001, low certainty), and improving remission (risk ratio = 1.95, 95% CI = 1.24–3.06, p = 0.004, very low certainty). Besides, anxiety symptoms and level of functioning could not be included as secondary outcome due to the lack of data. The present meta-analysis suggests that EMDR is more effective in treating MDD than “No Intervention” and CBT, particularly in individuals who have traumatic experience. However, this result should be considered with caution due to small sample size and low quality of trails.
... 78 To minimise the risk of multiplicitiy, i.e. falsely rejecting the null hypothesis, we will include only the six to eight most important moderators. We will then adjust the P values and CIs of the primary and secondary outcomes for multiplicity using the method described by Jakobsen et al. 79 Such characteristics could be age at baseline, sex, ethnicity, country of birth, education status, employment status, marital status, severity of BPD, psychosocial impairment, treatment adherence, comorbidity, previous mental illness, medications (psychotropic), mental illness in the family, socioeconomic factors, criminal behaviour, personality traits, previous trauma, IQ, suicide attempts, anger, chronic feelings of emptiness, impulsivity, interpersonal problems, abandonment, psychotic-like symptoms, depression, and self-harm incidents. We will examine the published papers and verify which moderators are investigated. ...
Introduction
The heterogeneity in people with borderline personality disorder (BPD) and the range of specialised psychotherapies means that people with certain BPD characteristics might benefit more or less from different types of psychotherapy. Identifying moderating characteristics of individuals is a key to refine and tailor standard treatments so they match the specificities of the individual participant. The objective of this is to improve the quality of care and the individual outcomes. We will do so by performing three systematic reviews with meta-analyses of individual participant data (IPD). The aim of these reviews is to investigate potential predictors and moderating patient characteristics on treatment outcomes for patients with BPD.
Methods and analysis
We performed comprehensive searches in 22 databases and trial registries up to October 6th 2020. These will be updated with a top-up search up until June 2021. Our primary meta-analytic method will be the one-stage random-effects approach. To identify predictors, we will use the one-stage model that accounts for interaction between covariates and treatment allocation. Heterogeneity in case-mix will be assessed with a membership model based on a multinomial logistic regression where study membership is the outcome. A random-effects meta-analysis is chosen to account for expected levels of heterogeneity.
Ethics and dissemination
The statistical analyses will be conducted on anonymised data that have already been approved by the respective ethical committees that originally assessed the included trials. The three IPD reviews will be published in high-impact factor journals and their results will be presented at international conferences and national seminars.
PROSPERO registration number
CRD42021210688.
More than three decades after the first randomised clinical trials using pharmacological treatment for Alzheimer’s disease, symptomatic treatment with no long-term benefits is still the only treatment available. Currently three acetylcholinesterase inhibitors—donepezil, galantamine and rivastigmine—are prescribed to treat patients with Alzheimer’s disease besides the glutamatergic antagonist memantine.
This is a protocol for a Cochrane Review (intervention). The objectives are as follows:
To assess the benefits and harms of modified dietary fat intake in the treatment of gallstone disease.
Background
Anxiety disorders are the most prevalent psychiatric problem across the lifespan, with typical onset during the pediatric period. Prior literature has examined cognitive mechanisms associated with youth anxiety and identified interpretation bias, the threatening appraisal of ambiguity, as a ubiquitous correlate and likely mechanism. A small set of studies have examined interpretation bias and anxiety in parent-child dyads, although results about this potential relationship are conflicted. This systematic review and meta-analysis investigates the association between parent interpretation bias with child interpretation bias and child anxiety.
Methods
Eight studies met the rigorous inclusion criteria, which required direct assessment of interpretation bias in both youth and parent.
Results
Meta-analysis with a random effects model indicated a small and significant correlation between parent and child interpretation biases (r = 0.14, p < .01), as well as a small and significant correlation between parent interpretation bias and child anxiety (r = 0.20, p = .01).
Limitations
As only eight studies were included in this meta-analysis, reflecting the state of the extant literature, it is possible that, as data accumulate and this work continues in the future, results may or may not be replicated.
Conclusions
Despite variability in findings across the included empirical studies, the current meta-analysis suggests that a correlational relationship between parent interpretation bias and child bias/anxiety exists. This work has implications for conceptualizing parent interpretation bias as a possible explanatory mechanism underlying youth interpretation bias and anxiety.
Background:
Neonatal sepsis is a major cause of morbidity and mortality. It is the third leading cause of neonatal mortality globally constituting 13% of overall neonatal mortality. Despite the high burden of neonatal sepsis, high-quality evidence in diagnosis and treatment is scarce. Possibly due to the diagnostic challenges of sepsis and the relative immunosuppression of the newborn, many neonates receive antibiotics for suspected sepsis. Antibiotics have become the most used therapeutics in neonatal intensive care units. The last Cochrane Review was updated in 2004. Given the clinical importance, an updated systematic review assessing the effects of different antibiotic regimens for early-onset neonatal sepsis is needed.
Objectives:
To assess the beneficial and harmful effects of different antibiotic regimens for early-onset neonatal sepsis.
Search methods:
We searched the following electronic databases: CENTRAL (2020, Issue 8); Ovid MEDLINE; Embase Ovid; CINAHL; LILACS; Science Citation Index EXPANDED and Conference Proceedings Citation Index - Science on 12 March 2021. We searched clinical trials databases and the reference lists of retrieved articles for randomised controlled trials (RCTs) and quasi-RCTs.
Selection criteria:
We included RCTs comparing different antibiotic regimens for early-onset neonatal sepsis. We included participants from birth to 72 hours of life at randomisation.
Data collection and analysis:
Three review authors independently assessed studies for inclusion, extracted data, and assessed risk of bias. We used the GRADE approach to assess the certainty of evidence. Our primary outcome was all-cause mortality, and our secondary outcomes were: serious adverse events, respiratory support, circulatory support, nephrotoxicity, neurological developmental impairment, necrotising enterocolitis, and ototoxicity. Our primary time point of interest was at maximum follow-up.
Main results:
We included five RCTs (865 participants). All trials were at high risk of bias. The certainty of the evidence according to GRADE was very low. The included trials assessed five different comparisons of antibiotics. We did not conduct any meta-analyses due to lack of relevant data. Of the five included trials one trial compared ampicillin plus gentamicin with benzylpenicillin plus gentamicin; one trial compared piperacillin plus tazobactam with amikacin; one trial compared ticarcillin plus clavulanic acid with piperacillin plus gentamicin; one trial compared piperacillin with ampicillin plus amikacin; and one trial compared ceftazidime with benzylpenicillin plus gentamicin. None of the five comparisons found any evidence of a difference when assessing all-cause mortality, serious adverse events, circulatory support, nephrotoxicity, neurological developmental impairment, or necrotising enterocolitis; however, none of the trials were near an information size that could contribute significantly to the evidence of the comparative benefits and risks of any particular antibiotic regimen. None of the trials assessed respiratory support or ototoxicity. The benefits and harms of different antibiotic regimens remain unclear due to the lack of well-powered trials and the high risk of systematic errors.
Authors' conclusions:
Current evidence is insufficient to support any antibiotic regimen being superior to another. Large RCTs assessing different antibiotic regimens in early-onset neonatal sepsis with low risk of bias are warranted.
BACKGROUND:
Our previous systematic review has demonstrated that antioxidant supplements may increase mortality. We have now updated this review.
OBJECTIVES:
To assess the beneficial and harmful effects of antioxidant supplements for prevention of mortality in adults.
METHODS:
Search methods: We searched The Cochrane Library, Medline, Embase, Lilacs, the Science Citation Index Expanded, and Conference Proceedings Citation Index-Science to February 2011. We scanned bibliographies of relevant publications and asked pharmaceutical companies for additional trials. Selection criteria: We included all primary and secondary prevention randomized clinical trials on antioxidant supplements (beta-carotene, vitamin A, vitamin C, vitamin E, and selenium) versus placebo or no intervention. Data collection and analysis: Three authors extracted data. Random-effects and fixed-effect model meta-analyses were conducted. Risk of bias was considered in order to minimize the risk of systematic errors. Trial sequential analyses were conducted to minimize the risk of random errors. Random effects model meta-regression analyses were performed to assess sources of intertrial heterogeneity.
MAIN RESULTS:
Seventy-eight randomized trials with 296,707 participants were included. Fifty-six trials including 244,056 participants had low risk of bias. Twenty-six trials included 215,900 healthy participants. Fifty-two trials included 80,807 participants with various diseases in a stable phase. The mean age was 63 years (range 18 to 103 years). The mean proportion of women was 46%. Of the 78 trials, 46 used the parallel-group design, 30 the factorial design, and 2 the cross-over design. All antioxidants were administered orally, either alone or in combination with vitamins, minerals, or other interventions. The duration of supplementation varied from 28 days to 12 years (mean duration 3 years; median duration 2 years). Overall, the antioxidant supplements had no significant effect on mortality in a random-effects model meta-analysis (21,484 dead/183,749 (11.7%) versus 11,479 dead/112,958 (10.2%); 78 trials, relative risk (RR) 1.02, 95% confidence interval (CI) 0.98 to 1.05) but significantly increased mortality in a fixed-effect model (RR 1.03, 95% CI 1.01 to 1.05). Heterogeneity was low with an I2- of 12%. In meta-regression analysis, the risk of bias and type of antioxidant supplement were the only significant predictors of intertribal heterogeneity. Meta-regression analysis did not find a significant difference in the estimated intervention effect in the primary prevention and the secondary prevention trials. In the 56 trials with a low risk of bias, the antioxidant supplements significantly increased mortality (18,833 dead/146,320 (12.9%) versus 10,320 dead/97,736 (10.6%); RR 1.04, 95% CI 1.01 to 1.07). This effect was confirmed by trial sequential analysis. Excluding factorial trials with potential confounding showed that 38 trials with low risk of bias demonstrated a significant increase in mortality (2822 dead/26,903 (10.5%) versus 2473 dead/26,052 (9.5%); RR 1.10, 95% CI 1.05 to 1.15). In trials with low risk of bias, beta-carotene (13,202 dead/96,003 (13.8%) versus 8556 dead/ 77,003 (11.1%); 26 trials, RR 1.05, 95% CI 1.01 to 1.09) and vitamin E (11,689 dead/97,523 (12.0%) versus 7561 dead/73,721 (10.3%); 46 trials, RR 1.03, 95% CI 1.00 to 1.05) significantly increased mortality, whereas vitamin A (3444 dead/24,596 (14.0%) versus 2249 dead/16,548 (13.6%); 12 trials, RR 1.07, 95% CI 0.97 to 1.18), vitamin C (3637 dead/36,659 (9.9%) versus 2717 dead/ 29,283 (9.3%); 29 trials, RR 1.02, 95% CI 0.98 to 1.07), and selenium (2670 dead/39,779 (6.7%) versus 1468 dead/22,961 (6.4%); 17 trials, RR 0.97, 95% CI 0.91 to 1.03) did not significantly affect mortality. In univariate meta-regression analysis, the dose of vitamin A was significantly associated with increased mortality (RR 1.0006, 95% CI 1.0002 to 1.001, P = 0.002).
AUTHORS' CONCLUSIONS:
We found no evidence to support antioxidant supplements for primary or secondary prevention. Beta-carotene and vitamin E seem to increase mortality, and so may higher doses of vitamin A. Antioxidant supplements need to be considered as medicinal products and should undergo sufficient evaluation before marketing.
Background:
The Quality of Reporting of Meta-analyses (QUOROM) conference was convened to address standards for improving the quality of reporting of meta-analyses of clinical randomised controlled trials (RCTs).
Methods:
The QUOROM group consisted of 30 clinical epidemiologists, clinicians, statisticians, editors, and researchers. In conference, the group was asked to identify items they thought should be included in a checklist of standards. Whenever possible, checklist items were guided by research evidence suggesting that failure to adhere to the item proposed could lead to biased results. A modified Delphi technique was used in assessing candidate items.
Findings:
The conference resulted in the QUOROM statement, a checklist, and a flow diagram. The checklist describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. It is organised into 21 headings and subheadings regarding searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with "trial flow", study characteristics, and quantitative data synthesis; research documentation was identified for eight of the 18 items. The flow diagram provides information about both the numbers of RCTs identified, included, and excluded and the reasons for exclusion of trials.
Interpretation:
We hope this report will generate further thought about ways to improve the quality of reports of meta-analyses of RCTs and that interested readers, reviewers, researchers, and editors will use the QUOROM statement and generate ideas for its improvement.
We have shown in a previous Statistics Note 1 how we can calculate a confidence interval (CI) from a P value. Some published articles report confidence intervals, but do not give corresponding P values. Here we show how a confidence interval can be used to calculate a P value, should this be required. This might also be useful when the P value is given only imprecisely (eg, as P<0.05). Wherever they can be calculated, we are advocates of confidence intervals as much more useful than P values, but we like to be helpful.
The method is outlined in the box below in which we have distinguished two cases.
#### Steps to obtain the P value from the CI for an estimate of effect ( Est )
##### (a) P from CI for a difference
The heterogeneity statistic I(2), interpreted as the percentage of variability due to heterogeneity between studies rather than sampling error, depends on precision, that is, the size of the studies included.
METHODS: Based on a real meta-analysis, we simulate artificially 'inflating' the sample size under the random effects model. For a given inflation factor M = 1, 2, 3,... and for each trial i, we create a M-inflated trial by drawing a treatment effect estimate from the random effects model, using s(i)(2)/M as within-trial sampling variance.
RESULTS: As precision increases, while estimates of the heterogeneity variance tau(2) remain unchanged on average, estimates of I(2) increase rapidly to nearly 100%. A similar phenomenon is apparent in a sample of 157 meta-analyses.
CONCLUSION: When deciding whether or not to pool treatment estimates in a meta-analysis, the yard-stick should be the clinical relevance of any heterogeneity present. tau(2), rather than I(2), is the appropriate measure for this purpose.
A practical guide to analysing partially observed data.
Collecting, analysing and drawing inferences from data is central to research in the medical and social sciences. Unfortunately, it is rarely possible to collect all the intended data. The literature on inference from the resulting incomplete data is now huge, and continues to grow both as methods are developed for large and complex data structures, and as increasing computer power and suitable software enable researchers to apply these methods.
This book focuses on a particular statistical method for analysing and drawing inferences from incomplete data, called Multiple Imputation (MI). MI is attractive because it is both practical and widely applicable. The authors aim is to clarify the issues raised by missing data, describing the rationale for MI, the relationship between the various imputation models and associated algorithms and its application to increasingly complex data structures.
Multiple Imputation and its Application:
Discusses the issues raised by the analysis of partially observed data, and the assumptions on which analyses rest.
Presents a practical guide to the issues to consider when analysing incomplete data from both observational studies and randomized trials.
Provides a detailed discussion of the practical use of MI with real-world examples drawn from medical and social statistics.
Explores handling non-linear relationships and interactions with multiple imputation, survival analysis, multilevel multiple imputation, sensitivity analysis via multiple imputation, using non-response weights with multiple imputation and doubly robust multiple imputation.
Multiple Imputation and its Application is aimed at quantitative researchers and students in the medical and social sciences with the aim of clarifying the issues raised by the analysis of incomplete data data, outlining the rationale for MI and describing how to consider and address the issues that arise in its application.
Subjectivity and ContextBayes Theorem for Two HypothesesComparing Simple Hypotheses: Likelihood Ratios and Bayes FactorsExchangeability and Parametric Modelling*Bayes Theorem for General QuantitiesBayesian Analysis with Binary DataBayesian Analysis with Normal DistributionsPoint Estimation, Interval Estimation and Interval HypothesesThe Prior DistributionHow to Use Bayes Theorem to Interpret Trial ResultsThe "Credibility" of Significant Trial Results*Sequential Use of Bayes Theorem*PredictionsDecision-makingDesignUse of Historical DataMultiplicity, Exchangeability and Hierarchical ModelsDealing with Nuisance Parameters*Computational IssuesSchools of BayesiansA Bayesian ChecklistFurther ReadingKey PointsExercises
Objective. —To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects. Design. —An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects. Data Sources. —Meta-analyses from the Cochrane Pregnancy and Childbirth Database. Main Outcome Measures. —The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding. Results. —Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects ( P P =.01), with odds ratios being exaggerated by 17%. Conclusions. —This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. ( JAMA . 1995;273:408-412)