No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Different meta-analysis methods can change judgements about imprecision of effect estimates: a meta-epidemiological study
Abstract
Objectives
To empirically evaluate five commonly used meta-analysis methods and their impact on imprecision judgements about effect estimates. The two fixed-effect model methods were the inverse variance method based on normal distribution and the Mantel-Haenszel method. The three random-effects model methods were the DerSimonian and Laird, the Hartung-Knapp-Sidik-Jonkman and the profile likelihood approaches.
Design
Meta-epidemiological study.
Setting
Meta-analyses published between 2007 and 2019 in the 10 general medical journals with the highest impact factors that evaluated a medication or device for chronic medical conditions and included at least 5 randomised trials.
Main outcome measures
Discordance in the judgements of imprecision of effect estimates based on two definitions: when either boundary of 95% CI of the OR changed by more than 15% or changed in relation to the null.
Results
We analysed 88 meta-analyses including 1114 trials with an average of 12.60 trials per meta-analysis and average I ² of 26% (range: 0%–96%). The profile likelihood failed to converge in three meta-analyses (3%). Discordance in imprecision judgements based on the two definitions, respectively, occurred between the fixed normal distribution and fixed Mantel-Haenszel method (8% and 2%), between the DerSimonian and Laird and Hartung-Knapp-Sidik-Jonkman methods (19% and 10%), between the DerSimonian and Laird and profile likelihood methods (9% and 5%), and between the Hartung-Knapp-Sidik-Jonkman and profile likelihood methods (5% and 13%). Discordance was greater when fewer studies and greater heterogeneity was present.
Conclusion
Empirical evaluation of studies of chronic medical conditions showed that conclusions about the precision of the estimates of the efficacy of a drug or device frequently changed when different pooling methods were used, particularly when the number of studies within a meta-analysis was small and statistical heterogeneity was substantial. Sensitivity analyses using more than one method may need to be considered in these two scenarios.
ResearchGate has not been able to resolve any citations for this publication.
Objectives
To evaluate the association of study design features and treatment effects in randomised controlled trials (RCTs) evaluating therapies for individuals with chronic medical conditions.
Design
Meta-epidemiological study.
Setting
RCTs from meta-analyses published in the 10 general medical journals with the highest impact factor published between 1 January 2007 and 10 June 2019 and evaluated a drug, procedure or device treatment of chronic medical conditions.
Main outcome measures
The association between trial design features and the effect size, reporting a ratio of ORs (ROR) and 95% confidence interval (CI).
Results
We included 1098 trials from 86 meta-analyses. The most common outcome in the trials was mortality (52%), followed by disease progression (16%) and adverse events (12%). Lack of blinding of patients and study personnel was associated with a larger treatment effect (ROR 1.12; 95% CI 1.00 to 1.25). There was no statistically significant association with random sequence generation, allocation concealment, blinding of outcome assessors, incomplete outcome data, whether trials were stopped early, study funding, type of interventions or with type of outcomes (objective vs subjective).
Conclusion
The meta-epidemiological study did not demonstrate a clear pattern of association between risk of bias indicators and treatment effects in RCTs in chronic medical conditions. The unpredictability of the direction of bias emphasises the need to make every attempt to adhere to blinding, allocation concealment and reduce attrition bias.
Trial registration number
Not applicable.
Globally, approximately one in three of all adults suffer from multiple chronic conditions (MCCs). This review provides a comprehensive overview of the resulting epidemiological, economic and patient burden.
There is no agreed taxonomy for MCCs, with several terms used interchangeably and no agreed definition, resulting in up to three-fold variation in prevalence rates: from 16% to 58% in UK studies, 26% in US studies and 9.4% in Urban South Asians.
Certain conditions cluster together more frequently than expected, with associations of up to three-fold, e.g. depression associated with stroke and with Alzheimer's disease, and communicable conditions such as TB and HIV/AIDS associated with diabetes and CVD, respectively. Clusters are important as they may be highly amenable to large improvements in health and cost outcomes through relatively simple shifts in healthcare delivery.
Healthcare expenditures greatly increase, sometimes exponentially, with each additional chronic condition with greater specialist physician access, emergency department presentations and hospital admissions. The patient burden includes a deterioration of quality of life, out of pocket expenses, medication adherence, inability to work, symptom control and a high toll on carers. This high burden from MCCs is further projected to increase.
Recommendations for interventions include reaching consensus on the taxonomy of MCC, greater emphasis on MCCs research, primary prevention to achieve compression of morbidity, a shift of health systems and policies towards a multiple-condition framework, changes in healthcare payment mechanisms to facilitate this change and shifts in health and epidemiological databases to include MCCs.
Published research should be reported to evidence users with clarity and transparency that facilitate optimal appraisal and use of evidence and allow replication by other researchers. Guidelines for such reporting are available for several types of studies but not for meta-epidemiological methodology studies. Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.
Objective:
To clarify the GRADE (grading of recommendations assessment, development and evaluation) definition of certainty of evidence and suggest possible approaches to rating certainty of the evidence for systematic reviews, health technology assessments and guidelines.
Study design and setting:
This work was carried out by a project group within the GRADE Working Group, through brainstorming and iterative refinement of ideas, using input from workshops, presentations, and discussions at GRADE Working Group meetings to produce this document, which constitutes official GRADE guidance.
Results:
Certainty of evidence is best considered as the certainty that a true effect lies on one side of a specified threshold, or within a chosen range. We define possible approaches for choosing threshold or range. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. Less contextualized approaches, more appropriate for systematic reviews and health technology assessments, include using specified ranges of magnitude of effect, e.g. ranges of what we might consider no effect, trivial, small, moderate, or large effects.
Conclusion:
It is desirable for systematic review authors, guideline panelists, and health technology assessors to specify the threshold or ranges they are using when rating the certainty in evidence.
Background:
Improving survival and extending the longevity of life for all populations requires timely, robust evidence on local mortality levels and trends. The Global Burden of Disease 2015 Study (GBD 2015) provides a comprehensive assessment of all-cause and cause-specific mortality for 249 causes in 195 countries and territories from 1980 to 2015. These results informed an in-depth investigation of observed and expected mortality patterns based on sociodemographic measures.
Methods:
We estimated all-cause mortality by age, sex, geography, and year using an improved analytical approach originally developed for GBD 2013 and GBD 2010. Improvements included refinements to the estimation of child and adult mortality and corresponding uncertainty, parameter selection for under-5 mortality synthesis by spatiotemporal Gaussian process regression, and sibling history data processing. We also expanded the database of vital registration, survey, and census data to 14 294 geography-year datapoints. For GBD 2015, eight causes, including Ebola virus disease, were added to the previous GBD cause list for mortality. We used six modelling approaches to assess cause-specific mortality, with the Cause of Death Ensemble Model (CODEm) generating estimates for most causes. We used a series of novel analyses to systematically quantify the drivers of trends in mortality across geographies. First, we assessed observed and expected levels and trends of cause-specific mortality as they relate to the Socio-demographic Index (SDI), a summary indicator derived from measures of income per capita, educational attainment, and fertility. Second, we examined factors affecting total mortality patterns through a series of counterfactual scenarios, testing the magnitude by which population growth, population age structures, and epidemiological changes contributed to shifts in mortality. Finally, we attributed changes in life expectancy to changes in cause of death. We documented each step of the GBD 2015 estimation processes, as well as data sources, in accordance with Guidelines for Accurate and Transparent Health Estimates Reporting (GATHER).
Findings:
Globally, life expectancy from birth increased from 61·7 years (95% uncertainty interval 61·4-61·9) in 1980 to 71·8 years (71·5-72·2) in 2015. Several countries in sub-Saharan Africa had very large gains in life expectancy from 2005 to 2015, rebounding from an era of exceedingly high loss of life due to HIV/AIDS. At the same time, many geographies saw life expectancy stagnate or decline, particularly for men and in countries with rising mortality from war or interpersonal violence. From 2005 to 2015, male life expectancy in Syria dropped by 11·3 years (3·7-17·4), to 62·6 years (56·5-70·2). Total deaths increased by 4·1% (2·6-5·6) from 2005 to 2015, rising to 55·8 million (54·9 million to 56·6 million) in 2015, but age-standardised death rates fell by 17·0% (15·8-18·1) during this time, underscoring changes in population growth and shifts in global age structures. The result was similar for non-communicable diseases (NCDs), with total deaths from these causes increasing by 14·1% (12·6-16·0) to 39·8 million (39·2 million to 40·5 million) in 2015, whereas age-standardised rates decreased by 13·1% (11·9-14·3). Globally, this mortality pattern emerged for several NCDs, including several types of cancer, ischaemic heart disease, cirrhosis, and Alzheimer's disease and other dementias. By contrast, both total deaths and age-standardised death rates due to communicable, maternal, neonatal, and nutritional conditions significantly declined from 2005 to 2015, gains largely attributable to decreases in mortality rates due to HIV/AIDS (42·1%, 39·1-44·6), malaria (43·1%, 34·7-51·8), neonatal preterm birth complications (29·8%, 24·8-34·9), and maternal disorders (29·1%, 19·3-37·1). Progress was slower for several causes, such as lower respiratory infections and nutritional deficiencies, whereas deaths increased for others, including dengue and drug use disorders. Age-standardised death rates due to injuries significantly declined from 2005 to 2015, yet interpersonal violence and war claimed increasingly more lives in some regions, particularly in the Middle East. In 2015, rotaviral enteritis (rotavirus) was the leading cause of under-5 deaths due to diarrhoea (146 000 deaths, 118 000-183 000) and pneumococcal pneumonia was the leading cause of under-5 deaths due to lower respiratory infections (393 000 deaths, 228 000-532 000), although pathogen-specific mortality varied by region. Globally, the effects of population growth, ageing, and changes in age-standardised death rates substantially differed by cause. Our analyses on the expected associations between cause-specific mortality and SDI show the regular shifts in cause of death composition and population age structure with rising SDI. Country patterns of premature mortality (measured as years of life lost [YLLs]) and how they differ from the level expected on the basis of SDI alone revealed distinct but highly heterogeneous patterns by region and country or territory. Ischaemic heart disease, stroke, and diabetes were among the leading causes of YLLs in most regions, but in many cases, intraregional results sharply diverged for ratios of observed and expected YLLs based on SDI. Communicable, maternal, neonatal, and nutritional diseases caused the most YLLs throughout sub-Saharan Africa, with observed YLLs far exceeding expected YLLs for countries in which malaria or HIV/AIDS remained the leading causes of early death.
Interpretation:
At the global scale, age-specific mortality has steadily improved over the past 35 years; this pattern of general progress continued in the past decade. Progress has been faster in most countries than expected on the basis of development measured by the SDI. Against this background of progress, some countries have seen falls in life expectancy, and age-standardised death rates for some causes are increasing. Despite progress in reducing age-standardised death rates, population growth and ageing mean that the number of deaths from most non-communicable causes are increasing in most countries, putting increased demands on health systems.
Funding:
Bill & Melinda Gates Foundation.
Background:
Random-effects meta-analysis is commonly performed by first deriving an estimate of the between-study variation, the heterogeneity, and subsequently using this as the basis for combining results, i.e., for estimating the effect, the figure of primary interest. The heterogeneity variance estimate however is commonly associated with substantial uncertainty, especially in contexts where there are only few studies available, such as in small populations and rare diseases.
Methods:
Confidence intervals and tests for the effect may be constructed via a simple normal approximation, or via a Student-t distribution, using the Hartung-Knapp-Sidik-Jonkman (HKSJ) approach, which additionally uses a refined estimator of variance of the effect estimator. The modified Knapp-Hartung method (mKH) applies an ad hoc correction and has been proposed to prevent counterintuitive effects and to yield more conservative inference. We performed a simulation study to investigate the behaviour of the standard HKSJ and modified mKH procedures in a range of circumstances, with a focus on the common case of meta-analysis based on only a few studies.
Results:
The standard HKSJ procedure works well when the treatment effect estimates to be combined are of comparable precision, but nominal error levels are exceeded when standard errors vary considerably between studies (e.g. due to variations in study size). Application of the modification on the other hand yields more conservative results with error rates closer to the nominal level. Differences are most pronounced in the common case of few studies of varying size or precision.
Conclusions:
Use of the modified mKH procedure is recommended, especially when only a few studies contribute to the meta-analysis and the involved studies' precisions (standard errors) vary.
Meta-analyses are typically used to estimate the overall/mean of an outcome of interest. However, inference about between-study variability, which is typically modelled using a between-study variance parameter, is usually an additional aim. The DerSimonian and Laird method, currently widely used by default to estimate the between-study variance, has been long challenged. Our aim is to identify known methods for estimation of the between-study variance and its corresponding uncertainty, and to summarise the simulation and empirical evidence that compares them. We identified 16 estimators for the between-study variance, seven methods to calculate confidence intervals, and several comparative studies. Simulation studies suggest that for both dichotomous and continuous data the estimator proposed by Paule and Mandel and for continuous data the restricted maximum likelihood estimator are better alternatives to estimate the between-study variance. Based on the scenarios and results presented in the published studies, we recommend the Q-profile method and the alternative approach based on a 'generalised Cochran between-study variance statistic' to compute corresponding confidence intervals around the resulting estimates. Our recommendations are based on a qualitative evaluation of the existing literature and expert consensus. Evidence-based recommendations require an extensive simulation study where all methods would be compared under the same scenarios. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd Legal Statement.
A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies-the DerSimonian-Laird (DL) estimator-can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence, focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.
The DerSimonian and Laird approach (DL) is widely used for random effects meta-analysis, but this often results in inappropriate type I error rates. The method described by Hartung, Knapp, Sidik and Jonkman (HKSJ) is known to perform better when trials of similar size are combined. However evidence in realistic situations, where one trial might be much larger than the other trials, is lacking. We aimed to evaluate the relative performance of the DL and HKSJ methods when studies of different sizes are combined and to develop a simple method to convert DL results to HKSJ results.
We evaluated the performance of the HKSJ versus DL approach in simulated meta-analyses of 2-20 trials with varying sample sizes and between-study heterogeneity, and allowing trials to have various sizes, e.g. 25% of the trials being 10-times larger than the smaller trials. We also compared the number of "positive" (statistically significant at p < 0.05) findings using empirical data of recent meta-analyses with > =3 studies of interventions from the Cochrane Database of Systematic Reviews.
The simulations showed that the HKSJ method consistently resulted in more adequate error rates than the DL method. When the significance level was 5%, the HKSJ error rates at most doubled, whereas for DL they could be over 30%. DL, and, far less so, HKSJ had more inflated error rates when the combined studies had unequal sizes and between-study heterogeneity. The empirical data from 689 meta-analyses showed that 25.1% of the significant findings for the DL method were non-significant with the HKSJ method. DL results can be easily converted into HKSJ results.
Our simulations showed that the HKSJ method consistently results in more adequate error rates than the DL method, especially when the number of studies is small, and can easily be applied routinely in meta-analyses. Even with the HKSJ method, extra caution is needed when there are = <5 studies of very unequal sizes.
Meta-analysis (MA) is a statistical methodology that combines the results of several independent studies considered by the analyst to be 'combinable'. The simplest approach, the fixed-effects (FE) model, assumes the true effect to be the same in all studies, while the random-effects (RE) family of models allows the true effect to vary across studies. However, all methods are only correct asymptotically, while some RE models assume that the true effects are normally distributed. In practice, MA methods are frequently applied when study numbers are small and the normality of the effect distribution unknown or unlikely. In this article, we discuss the performance of the FE approach and seven frequentist RE MA methods: DerSimonian-Laird, Q-based, maximum likelihood, profile likelihood, Biggerstaff-Tweedie, Sidik-Jonkman and Follmann-Proschan. We covered numerous scenarios by varying the MA sizes (small to moderate), the degree of heterogeneity (zero to very large) and the distribution of the effect sizes (normal, skew-normal and 'extremely' non-normal). Performance was evaluated in terms of coverage (Type I error), power (Type II error) and overall effect estimation (accuracy of point estimates and error intervals).
Objective
To provide updated guidance on when GRADE users should consider rating down more than one level for imprecision using a minimally contextualized approach.
Study design and setting
Based on the first GRADE guidance addressing imprecision rating in 2011, a project group within the GRADE Working Group conducted iterative discussions and presentations at GRADE Working Group meetings to produce this guidance.
Results
GRADE suggests aligning imprecision criterion for systematic reviews and guidelines ——using the approach that relies on thresholds and confidence intervals of absolute effects as a primary criterion for imprecision rating (i.e., CI approach). Based on the CI approach, when a CI appreciably crosses the threshold(s) of interest, one should consider rating down two or three levels. When the CI does not cross the threshold(s) and the relative effect is large, one should implement the optimal information size (OIS) approach. If the sample size of the meta-analysis is far less than the OIS, one should consider rating down more than one level for imprecision.
Conclusion
GRADE provides updated guidance for imprecision rating in a minimally contextualized approach, with a focus on the circumstances in which one should seriously consider rating down two or three levels for imprecision.
The revised edition of the Handbook offers the only guide on how to conduct, report and maintain a Cochrane Review ? The second edition of The Cochrane Handbook for Systematic Reviews of Interventions contains essential guidance for preparing and maintaining Cochrane Reviews of the effects of health interventions. Designed to be an accessible resource, the Handbook will also be of interest to anyone undertaking systematic reviews of interventions outside Cochrane, and many of the principles and methods presented are appropriate for systematic reviews addressing research questions other than effects of interventions. This fully updated edition contains extensive new material on systematic review methods addressing a wide-range of topics including network meta-analysis, equity, complex interventions, narrative synthesis, and automation. Also new to this edition, integrated throughout the Handbook, is the set of standards Cochrane expects its reviews to meet. Written for review authors, editors, trainers and others with an interest in Cochrane Reviews, the second edition of The Cochrane Handbook for Systematic Reviews of Interventions continues to offer an invaluable resource for understanding the role of systematic reviews, critically appraising health research studies and conducting reviews.
Meta-analyses are an important tool within systematic reviews to estimate the overall effect size and its confidence interval for an outcome of interest. If heterogeneity between the results of the relevant studies is anticipated, then a random-effects model is often preferred for analysis. In this model, a prediction interval for the true effect in a new study also provides additional useful information. However, the DerSimonian and Laird method - frequently used as the default method for meta-analyses with random effects - has been long challenged due to its unfavourable statistical properties. Several alternative methods have been proposed that may have better statistical properties in specific scenarios. In this paper, we aim to provide a comprehensive overview of available methods for calculating point estimates, confidence intervals and prediction intervals for the overall effect size under the random-effects model. We indicate whether some methods are preferable than others by considering the results of comparative simulation and real-life data studies.
Studies combined in a meta-analysis often have differences in their design and conduct that can lead to heterogeneous results. A random-effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian-Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta-analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian-Laird estimator is negatively biased in scenarios with small studies, and in scenarios with a rare binary outcome. Results also show the Paule-Mandel method has considerable positive bias in meta-analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta-analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta-analysis. The estimated summary effect of the meta-analysis and its confidence interval derived from the Hartung-Knapp-Sidik-Jonkman method is more robust to changes in the heterogeneity variance estimate and shows minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.
Trustworthy clinical practice guidelines should be based on a systematic review of the literature, provide ratings of the quality of evidence and the strength of recommendations, consider patient values, and be developed by a multidisciplinary panel of experts. The quality of evidence reflects our certainty that the evidence warrants a particular action. Transforming evidence into a decision requires consideration of the quality of evidence, balance of benefits and harms, patients' values, available resources, feasibility of the intervention, acceptability by stakeholders, and effect on health equity. Empirical evidence shows that adherence to guidelines improves patient outcomes; however, adherence to guidelines is variable. Therefore, guidelines require active dissemination and innovative implementation strategies.
Objective:
To evaluate the presence of extreme findings and fluctuation in effect size in endocrinology.
Study design and settings:
We systematically identified all meta-analyses published in 2014 in the field of endocrinology. Within each meta-analysis, the effect size of the primary binary outcome was compared across studies according to their order of publication. We pooled studies using the DerSimonian and Laird random-effects method. Heterogeneity was evaluated using the I(2) and tau(2).
Results:
12% of the included 100 meta-analyses reported the largest effect size in the very first published study. The largest effect size occurred in the first 2 earliest studies in 31% of meta-analyses. When the effect size was the largest in the first published study, it was three times larger than the final pooled effect (ratio of rates: 3.26; 95% CI: 1.80, 5.90). The largest heterogeneity measured by I(2) was observed in 18% of the included meta-analyses when combining the first 2 studies or 17% when combing the first three studies.
Conclusions:
In endocrinology, early studies reported extreme findings with large variability. This behavior of the evidence needs to be taken into account when used to formulate clinical policies.
This paper investigates the impact of the number of studies on meta-analysis and meta-regression within the random-effects model framework. It is frequently neglected that inference in random-effects models requires a substantial number of studies included in meta-analysis to guarantee reliable conclusions. Several authors warn about the risk of inaccurate results of the traditional DerSimonian and Laird approach especially in the common case of meta-analysis involving a limited number of studies. This paper presents a selection of likelihood and non-likelihood methods for inference in meta-analysis proposed to overcome the limitations of the DerSimonian and Laird procedure, with a focus on the effect of the number of studies. The applicability and the performance of the methods are investigated in terms of Type I error rates and empirical power to detect effects, according to scenarios of practical interest. Simulation studies and applications to real meta-analyses highlight that it is not possible to identify an approach uniformly superior to alternatives. The overall recommendation is to avoid the DerSimonian and Laird method when the number of meta-analysis studies is modest and prefer a more comprehensive procedure that compares alternative inferential approaches. R code for meta-analysis according to all of the inferential methods examined in the paper is provided.
© The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav.
Clinical decisions should be based on the totality of the best evidence and not the results of individual studies. When clinicians apply the results of a systematic review or meta-analysis to patient care, they should start by evaluating the credibility of the methods of the systematic review, ie, the extent to which these methods have likely protected against misleading results. Credibility depends on whether the review addressed a sensible clinical question; included an exhaustive literature search; demonstrated reproducibility of the selection and assessment of studies; and presented results in a useful manner. For reviews that are sufficiently credible, clinicians must decide on the degree of confidence in the estimates that the evidence warrants (quality of evidence). Confidence depends on the risk of bias in the body of evidence; the precision and consistency of the results; whether the results directly apply to the patient of interest; and the likelihood of reporting bias. Shared decision making requires understanding of the estimates of magnitude of beneficial and harmful effects, and confidence in those estimates.
• Contains additional discussion and examples on left truncation as well as material on more general censoring and truncation patterns. • Introduces the martingale and counting process formulation swil lbe in a new chapter. • Develops multivariate failure time data in a separate chapter and extends the material on Markov and semi Markov formulations. • Presents new examples and applications of data analysis.
Aggressive glycemic control has been hypothesized to prevent renal disease in patients with type 2 diabetes mellitus. A systematic review was conducted to summarize the benefits of intensive vs conventional glucose control on kidney-related outcomes for adults with type 2 diabetes.
Three databases were systematically searched (January 1, 1950, to December 31, 2010) with no language restrictions to identify randomized trials that compared surrogate renal end points (microalbuminuria and macroalbuminuria) and clinical renal end points (doubling of the serum creatinine level, end-stage renal disease [ESRD], and death from renal disease) in patients with type 2 diabetes receiving intensive glucose control vs those receiving conventional glucose control.
We evaluated 7 trials involving 28 065 adults who were monitored for 2 to 15 years. Compared with conventional control, intensive glucose control reduced the risk for microalbuminuria (risk ratio, 0.86 [95% CI, 0.76-0.96]) and macroalbuminuria (0.74 [0.65-0.85]), but not doubling of the serum creatinine level (1.06 [0.92-1.22]), ESRD (0.69 [0.46-1.05]), or death from renal disease (0.99 [0.55-1.79]). Meta-regression revealed that larger differences in hemoglobin A1c between intensive and conventional therapy at the study level were associated with greater benefit for both microalbuminuria and macroalbuminuria. The pooled cumulative incidence of doubling of the serum creatinine level, ESRD, and death from renal disease was low (<4%, <1.5%, and <0.5%, respectively) compared with the surrogate renal end points of microalbuminuria (23%) and macroalbuminuria (5%).
Intensive glucose control reduces the risk for microalbuminuria and macroalbuminuria, but evidence is lacking that intensive glycemic control reduces the risk for significant clinical renal outcomes, such as doubling of the serum creatinine level, ESRD, or death from renal disease during the years of follow-up of the trials.
The procedure suggested by DerSimonian and Laird is the simplest and most commonly used method for fitting the random effects model for meta-analysis. Here it is shown that, unless all studies are of similar size, this is inefficient when estimating the between-study variance, but is remarkably efficient when estimating the treatment effect. If formal inference is restricted to statements about the treatment effect, and the sample size is large, there is little point in implementing more sophisticated methodology. However, it is further demonstrated, for a simple special case, that use of the profile likelihood results in actual coverage probabilities for 95% confidence intervals that are closer to nominal levels for smaller sample sizes. Alternative methods for making inferences for the treatment effect may therefore be preferable if the sample size is small, but the DerSimonian and Laird procedure retains its usefulness for larger samples.
GRADE suggests that examination of 95% confidence intervals (CIs) provides the optimal primary approach to decisions regarding imprecision. For practice guidelines, rating down the quality of evidence (i.e., confidence in estimates of effect) is required if clinical action would differ if the upper versus the lower boundary of the CI represented the truth. An exception to this rule occurs when an effect is large, and consideration of CIs alone suggests a robust effect, but the total sample size is not large and the number of events is small. Under these circumstances, one should consider rating down for imprecision. To inform this decision, one can calculate the number of patients required for an adequately powered individual trial (termed the "optimal information size" [OIS]). For continuous variables, we suggest a similar process, initially considering the upper and lower limits of the CI, and subsequently calculating an OIS. Systematic reviews require a somewhat different approach. If the 95% CI excludes a relative risk (RR) of 1.0, and the total number of events or patients exceeds the OIS criterion, precision is adequate. If the 95% CI includes appreciable benefit or harm (we suggest an RR of under 0.75 or over 1.25 as a rough guide) rating down for imprecision may be appropriate even if OIS criteria are met.
In a meta-analysis of a set of clinical trials, a crucial but problematic component is providing an estimate and confidence interval for the overall treatment effect theta. Since in the presence of heterogeneity a fixed effect approach yields an artificially narrow confidence interval for theta, the random effects method of DerSimonian and Laird, which incorporates a moment estimator of the between-trial components of variance sigma B2, has been advocated. With the additional distributional assumptions of normality, a confidence interval for theta may be obtained. However, this method does not provide a confidence interval for sigma B2, nor a confidence interval for theta which takes account of the fact that sigma B2 has to be estimated from the data. We show how a likelihood based method can be used to overcome these problems, and use profile likelihoods to construct likelihood based confidence intervals. This approach yields an appropriately widened confidence interval compared with the standard random effects method. Examples of application to a published meta-analysis and a multicentre clinical trial are discussed. It is concluded that likelihood based methods are preferred to the standard method in undertaking random effects meta-analysis when the value of sigma B2 has an important effect on the overall estimated treatment effect.
Meta-analyses aim to provide a full and comprehensive summary of related studies which have addressed a similar question. When the studies involve time to event (survival-type) data the most appropriate statistics to use are the log hazard ratio and its variance. However, these are not always explicitly presented for each study. In this paper a number of methods of extracting estimates of these statistics in a variety of situations are presented. Use of these methods should improve the efficiency and reliability of meta-analyses of the published literature with survival-type endpoints.
Meta-analysis may be used to estimate an overall effect across a number of similar studies. A number of statistical techniques are currently used to combine individual study results. The simplest of these is based on a fixed effects model, which assumes the true effect is the same for all studies. A random effects model, however, allows the true effect to vary across studies, with the mean true effect the parameter of interest. We consider three methods currently used for estimation within the framework of a random effects model, and illustrate them by applying each method to a collection of six studies on the effect of aspirin after myocardial infarction. These methods are compared using estimated coverage probabilities of confidence intervals for the overall effect. The techniques considered all generally have coverages below the nominal level, and in particular it is shown that the commonly used DerSimonian and Laird method does not adequately reflect the error associated with parameter estimation, especially when the number of studies is small.
Users’ guides to the medical literature: A manual for evidence-based clinical practice
- M H Murad
- V M Montori
- Jpa Ioannidis
Stata module to provide comprehensive meta-analysis
- Admetan