Systematic evaluation and comparison of statistical tests for publication bias.

Department of Epidemiology and Healthcare Research, Kyoto University Graduate School of Medicine, Yoshida-Konoe-cho, Sakyo-ku, Kyoto, Japan.
Journal of Epidemiology (Impact Factor: 2.86). 12/2005; 15(6):235-43. DOI: 10.2188/jea.15.235
Source: PubMed

ABSTRACT This study evaluates the statistical and discriminatory powers of three statistical test methods (Begg's, Egger's, and Macaskill's) to detect publication bias in meta-analyses.
The data sources were 130 reviews from the Cochrane Database of Systematic Reviews 2002 issue, which considered a binary endpoint and contained 10 or more individual studies. Funnel plots with observers'agreements were selected as a reference standard. We evaluated a trade-off between sensitivity and specificity by varying cut-off p-values, power of statistical tests given fixed false positive rates, and area under the receiver operating characteristic curve.
In 36 reviews, 733 original studies evaluated 2,874,006 subjects. The number of trials included in each ranged from 10 to 70 (median 14.5). Given that the false positive rate was 0.1, the sensitivity of Egger's method was 0.93, and was larger than that of Begg's method (0.86) and Macaskill's method (0.43). The sensitivities of three statistical tests increased as the cut-off p-values increased without a substantial decrement of specificities. The area under the ROC curve of Egger's method was 0.955 (95% confidence interval, 0.889-1.000) and was not different from that of Begg's method (area=0.913, p=0.2302), but it was larger than that of Macaskill's method (area=0.719, p=0.0116).
Egger's linear regression method and Begg's method had stronger statistical and discriminatory powers than Macaskill's method for detecting publication bias given the same type I error level. The power of these methods could be improved by increasing the cut-off p-value without a substantial increment of false positive rate.

1 Follower
  • Source
    • "150) guarantees the stability of meta-analysis results. We did not perform Egger's test because visual inspection of the funnel plots revealed no noticeable asymmetry in square distribution as a sign of publication bias (Hayashino et al., 2005). However, " global " and " first-episode " metaanalyses reported a slight asymmetry due to one study with a large global sample size but with a small sample of firstepisode patients (Spoletini et al., 2009). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Although several structural MRI studies report significant thalamus volume reduction in patients with schizophrenia, many other studies do not. Therefore, the present meta-analyses aimed to clarify whether a reduction in thalamic volume characterizes patients diagnosed with schizophrenia by considering first-episode and chronic phases of the illness and right and left thalamus separately. Using Pubmed databases, we made a detailed literature search for structural MRI studies on patients with schizophrenia that reported physical volumetric measures of the right and left thalamus. Thirteen structural MRI studies were considered eligible for meta-analysis of the entire sample of patients and of the healthy control subjects. Individual meta-analyses were also performed on 6 studies of first-episode patients only and on 7 studies of chronic patients only. These were followed by additional meta-analyses to investigate the role of the factors "illness phase" and "side" on thalamic volume reduction. Overall, the patient group showed a significant bilateral thalamus volume reduction compared to healthy control subjects. This was found in both first-episode and chronic patients. Furthermore, left thalamus was smaller than right in both patients and healthy control subjects. When only studies that used physical volumetric measures were considered, the present meta-analyses confirmed that thalamic volume reduction characterizes patients with schizophrenia, both at the first-episode and chronic phases of the illness.
    Schizophrenia Research 10/2010; 123(1):1-14. DOI:10.1016/j.schres.2010.07.007 · 4.43 Impact Factor
  • Source
    • "We planned to draw funnel plots to investigate any relationships between effect size and study precision, closely related to sample size (Egger 1997). For meaningful funnel plots, a large number of trials with a spread of sample sizes are required ( Glasziou 2001; Hayashino 2005). We planned to draw funnel plots if there were at least seven trials with appropriate data. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Intimate partner abuse is common in all societies and damages the health of survivors and their children in the short and long term. Advocacy may decrease the impact of this abuse on women's health. To assess the effects of advocacy interventions conducted within or outside of health care settings on women who have experienced intimate partner abuse. We searched: CENTRAL and DARE (Cochrane Library Issue 3, 2008), MEDLINE (1966 to 31/7/08), EMBASE (1980 to 2008 week 30), and 11 other databases, to end July 2008. We also searched relevant websites, reference lists and forward citation tracking of included studies, and handsearched six key journals. We contacted principal investigators and experts in the field. Randomised controlled trials comparing advocacy interventions for women with experience of intimate partner abuse against usual care. Two reviewers independently assessed trial quality and undertook data extraction. For binary outcomes we calculated a standardised estimation of the odds ratio (OR) and for continuous data we calculated either a standardised mean difference (SMD) or a weighted mean difference (WMD), both with a 95% confidence interval. We included ten trials involving 1527 participants. The studies were heterogeneous in respect of: intensity of advocacy, outcome measures and duration of follow-up (immediately post-intervention to three years), permitting meta-analysis for only a minority of outcomes. Intensive advocacy (12 hours or more duration) may help terminate physical abuse in women leaving domestic violence shelters or refuges at 12-24 months follow-up (OR 0.43, 95% CI 0.23 to 0.80), but not at up to 12 months follow-up. The evidence indicates that intensive advocacy may improve quality of life at up to 12 months follow-up, but the confidence intervals included zero (WMD 0.23, 95% CI 0.00 to 0.46). Depression did not improve following intensive advocacy at up to 12 months follow-up (WMD -0.05, 95% CI -0.19 to 0.09), nor did psychological distress (SMD -0.16, 95% CI -0.39 to 0.06). Only two meta-analyses of brief advocacy interventions (less than 12 hours duration) were possible; an increased use of safety behaviours was consistent with the receipt of brief advocacy both at up to 12 months (WMD 0.60, 95% CI 0.14 to 1.06) and at 12-24 months (WMD 0.48, 95% CI 0.04 to 0.92) follow up. Based on the evidence reviewed, it is possible that intensive advocacy for women recruited in domestic violence shelters or refuges reduces physical abuse one to two years after the intervention but we do not know if it has a beneficial effect on their quality of life and mental health. Similarly, there is insufficient evidence to show if less intensive interventions in healthcare settings for women who still live with the perpetrators of violence are effective.
    Cochrane database of systematic reviews (Online) 02/2009; 5(3):CD005043. DOI:10.1002/14651858.CD005043.pub2 · 5.94 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We investigated the association of the NRG1 gene and schizophrenia using meta-analytic techniques, combining all published data while restricting our analysis to studies investigating the most commonly reported single marker (SNP8NRG221533). We also investigated whether ancestry (European vs East Asian) and study design (family-based vs case-control) moderated any association. We found no evidence for an association of SNP8NRG221533 with schizophrenia, and significant between-study heterogeneity, which persisted when family-based studies were combined separately. However, when haplotype-based P-values were combined, there was evidence in support of an association of NRG1 with schizophrenia, and no evidence of between-study heterogeneity. Our meta-analysis provides support for the association of NRG1 with schizophrenia, but indicates that firmly establishing the role of NRG1 gene in schizophrenia by genetic association requires much larger sample sizes than have hitherto been reported. Association analyses and replications should take place at the level of the gene, rather than at the level of SNP, haplotype, or functional variant. Meta-analysis would then be carried out on the basis of the combination of P-values.
    Molecular Psychiatry 07/2006; 11(6):539-46. DOI:10.1038/ · 15.15 Impact Factor
Show more