Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Department of Epidemiology and Healthcare Research, Kyoto University Graduate School of Medicine, Yoshida-Konoe-cho, Sakyo-ku, Kyoto, Japan.
Journal of Epidemiology (Impact Factor: 3.02). 12/2005; 15(6):235-43. DOI: 10.2188/jea.15.235
Source: PubMed


This study evaluates the statistical and discriminatory powers of three statistical test methods (Begg's, Egger's, and Macaskill's) to detect publication bias in meta-analyses.
The data sources were 130 reviews from the Cochrane Database of Systematic Reviews 2002 issue, which considered a binary endpoint and contained 10 or more individual studies. Funnel plots with observers'agreements were selected as a reference standard. We evaluated a trade-off between sensitivity and specificity by varying cut-off p-values, power of statistical tests given fixed false positive rates, and area under the receiver operating characteristic curve.
In 36 reviews, 733 original studies evaluated 2,874,006 subjects. The number of trials included in each ranged from 10 to 70 (median 14.5). Given that the false positive rate was 0.1, the sensitivity of Egger's method was 0.93, and was larger than that of Begg's method (0.86) and Macaskill's method (0.43). The sensitivities of three statistical tests increased as the cut-off p-values increased without a substantial decrement of specificities. The area under the ROC curve of Egger's method was 0.955 (95% confidence interval, 0.889-1.000) and was not different from that of Begg's method (area=0.913, p=0.2302), but it was larger than that of Macaskill's method (area=0.719, p=0.0116).
Egger's linear regression method and Begg's method had stronger statistical and discriminatory powers than Macaskill's method for detecting publication bias given the same type I error level. The power of these methods could be improved by increasing the cut-off p-value without a substantial increment of false positive rate.

1 Follower
13 Reads
  • Source
    • "Statistical heterogeneity of included preclinical studies will be measured using the I2 test with 95% uncertainty intervals [45]. If there are sufficient number of studies (≥10), an evaluation for the presence of publication bias will be conducted with funnel plot techniques, and Egger’s regression test [46]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Acute respiratory distress syndrome (ARDS) in humans is caused by an unchecked proinflammatory response that results in diffuse and severe lung injury, and it is associated with a mortality rate of 35 to 45%. Mesenchymal stromal cells (MSCs; ‘adult stem cells’) could represent a promising new therapy for this syndrome, since preclinical evidence suggests that MSCs may ameliorate lung injury. Prior to a human clinical trial, our aim is to conduct a systematic review to compare the efficacy and safety of MSC therapy versus controls in preclinical models of acute lung injury that mimic some aspects of the human ARDS. Methods/Design We will include comparative preclinical studies (randomized and non-randomized) of acute lung injury in which MSCs were administered and outcomes compared to animals given a vehicle control. The primary outcome will be death. Secondary outcomes will include the four key features of preclinical acute lung injury as defined by the American Thoracic Society consensus conference (histologic evidence of lung injury, altered alveolar capillary barrier, lung inflammatory response, and physiological dysfunction) and pathogen clearance for acute lung injury models that are caused by infection. Electronic searches of MEDLINE, Embase, BIOSIS Previews, and Web of Science will be constructed and reviewed by the Peer Review of Electronic Search Strategies (PRESS) process. Search results will be screened independently and in duplicate. Data from eligible studies will be extracted, pooled, and analyzed using random effects models. Risk of bias will be assessed using the Cochrane risk of bias tool, and individual study reporting will be assessed according to the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines. Discussion The results of this systematic review will comprehensively summarize the safety and efficacy of MSC therapy in preclinical models of acute lung injury. Our results will help translational scientists and clinical trialists to determine whether sufficient evidence exists to perform a human clinical trial. These results may also guide future acute lung injury preclinical and clinical research.
    Full-text · Article · May 2014 · Systematic Reviews
  • Source
    • "Meta analysis was carried out by using random-effects or fixed effects model based on the pooled effect estimates in the presence (p≤0.1 or I2≥50%) or absence (p>0.1 or I2<50%) of heterogeneity [14]. (3) To assess the presence of publication bias statistically, Begg’s test and Egger’s regression test were preformed where there were three or more studies [15]–[16]. p<0.05 was considered representative of statistically significant publication bias. (4) In this meta-analysis, in order to better investigate possible reasons of between-study heterogeneity, Studies were categorized into subgroups based on ethnicity status.(5)All "
    [Show abstract] [Hide abstract]
    ABSTRACT: To determine the association between HLA-DRB1 haplotypes and risk of cervical cancer in unselected and samples from Chinese ethnicities. A comprehensive search for articles from their inception to April 1st, 2013 was conducted from PubMed, Medline, Elsevier Science, Springer Link, Cochrane Library database, China biology medical literature database (CBM),China National Knowledge Infrastructure (CNKI),VIP,and Chinese literature database(Wang fang). A total of 1596 patients with cervical cancer and 2048 controls from the 12 studies on the relationship between gene polymorphism of HLA-DRB l and cervical cancer were performed and data were analyzed and processed using Review Manager 5.0 and Stata 11.0. Among the 13 family alleles, two (DRB1*03 and DRB1*08) were found to be negatively associated with cervical cancer in all studies or in Uighur subgroups, and two (DRB1*10 and DRB1*15) were positively associated with in all studies or in Uighur subgroups. Among the 25 specific alleles, six (DRB1*0301, *0403,*0404, *0803, *1312 and *1502) were associated with an increased risk cervical cancer in all studies. No significant association was established for other HLA-DRB1 family alleles and specific alleles. Ethnicity partially explained the race influence of DRB1*12, DRB1*14, DRB1*0301, DRB1*0403, DRB1*0404, DRB1*0803, DRB1*1312 and DRB1*1502 phenotypes. Our results support the hypothesis that the HLA-DRB1 family alleles and specific alleles might influence the susceptibility or resistance to cervical cancer, suggesting that immune regulation may play a key role in this disease, although further investigations are still needed.
    Full-text · Article · Feb 2014 · PLoS ONE
  • Source
    • "Publication bias was identified using Egger’s linear regression method and a funnel plot. A P-value < 0.05 in Egger’s linear regression indicated the presence of potential publication bias [22]. All statistical tests for this meta-analysis were performed with STATA software (version 10.0; StataCorp LP, College Station, TX). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Disruption of apoptosis has been implicated in carcinogenesis. Specifically, various single-nucleotide polymorphisms (SNPs) in apoptotic genes, such as FAS-1377 G/A SNP, have been associated with cancer risk. FAS-1377 G/A SNP has been shown to alter FAS gene promoter transcriptional activity. Down-regulation of FAS and cell death resistance is key to many cancers, but an association between FAS-1377 G/A SNP and cancer risk is uncertain. Therefore, we conducted a meta-analysis of the current literature to clarify this relationship. From PubMed and Chinese language (CNKI and WanFang) databases, we located articles published up to March 5, 2013, obtaining 44 case-control studies from 41 different articles containing 17,858 cases and 24,311 controls based on search criteria for cancer susceptibility related to the FAS gene -1377 G/A SNP. Odds ratios (ORs) and 95% confidence intervals (CI) revealed association strengths. Data show that the -1377 G allele was protective against cancer risk. Similar associations were detected in "source of control," ethnicity and cancer type subgroups. Lower cancer risk was found in both smokers with a GG+GA genotype and in non-smokers with the GG+GA genotype, when compared to smokers and nonsmokers with the AA genotype. Males carrying the -1377G allele (GG+GA) had lower cancer incidence than those with the AA genotype. Individuals who carried both FAS-1377(GG+GA)/FASL-844(TT+TC) genotypes appeared to have lower risk of cancer than those who carried both FAS-1377 AA/FASL-844 CC genotypes. The FAS-1377 G/A SNP may decrease cancer risk. Studies with larger samples to study gene-environment interactions are warranted to understand the role of FAS gene polymorphisms, especially -1377 G/A SNP, in cancer risk.
    Preview · Article · Aug 2013 · PLoS ONE
Show more