Evidence based medicine: The case of the misleading funnel plot

Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, MA 02111, USA.
BMJ (online) (Impact Factor: 17.45). 10/2006; 333(7568):597-600. DOI: 10.1136/bmj.333.7568.597
Source: PubMed
  • Source
    • "Subgroup analyses were based on clinical judgment , similarity of circumstances in which the studies have been conducted and the publication index. Whilst empirical methods such as Funnel plots [14] [15] were part of preliminary investigations, the final estimates for an average effect on the BD outcomes relied on robust clinical and statistical compatibility, i.e., with evidence consistent with studies homogeneity [16].P e t o 's method for pooled odds ratios was used to compare subjects with carotid artery plaques within BD and control groups because of its good performance when events are very rare [17]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: To evaluate subclinical atherosclerosis in Behcet disease (BD), we performed a systematic review and meta-analysis of studies where atherosclerosis was determined by flow-mediated dilatation (FMD) and endothelial-mediated dilatation (EMD) and by measurement of intima media thickness (IMT) of carotid arteries. Systematic search of EMBASE and PubMed databases from January 2000 to January 2014 according to PRISMA guidelines. Nine studies met the inclusion criteria on FMD/EMD, 11 on IMT and 4 on both. BD had lower FMD than controls (SMD = -0.89, 95% CI: -0.660 to -1.11, p < 0.001), which was confirmed by subgroup analyses on active and inactive patients (SMD = -1.17, 95% CI: -1.45 to -0.89 and SMD = -0.72, 95% CI: -0.97 to -0.46, p = 0.0001 for both). EMD was lower in BD but with a large estimate (SMD = 0.38, 95% CI: -0.79 to -0.03, p = 0.06, I(2) = 82.2%). IMT was greater in BD and the large estimate (SMD = 0.95, 95% CI: 0.63-1.28, p < 0.0001, I(2) = 87.6%) persisted after subgroup analysis on active and inactive patients (I(2) = 88.4% and 86.7%, respectively). Pooling IMT studies by a Newcastle Ottawa Scale of 5 and 6/7 yielded lower estimates (SMD = 0.54, 95% CI: 0.32-0.75, p < 0.0001, I(2) = 58.7% and SMD = 1.72, 95% CI: 1.35-2.09 p < 0.05, I(2) = 48.6%). FMD is impaired in BD even in inactive state and IMT is greater despite a degree of statistical heterogeneity that reflects the clinical heterogeneity of BD. Future prospective studies should account for risk stratification of atherosclerosis in BD. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.
    Full-text · Article · Jul 2015 · Seminars in arthritis and rheumatism
  • Source
    • "Where publication bias was shown to be present, the sensitivity of meta-analytic estimates to that bias was assessed using the trimfill function in the metafor package. We note that funnel plot asymmetry alone can be a misleading test of publication bias, and must be interpreted with caution when heterogeneity is present and sample sizes are small (Lau et al. 2006; Sterne et al. 2011). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Environmental sex reversal (ESR), whereby environmental effects (e.g. exogenous chemicals) override genetic sex determination, is a commonly used technique in aquaculture and physiology research. We performed a systematic review and meta-analyses of the literature that compares the sperm characteristics of masculinized genotypic females to wild-type males. We detected no mean differences between the ejaculate volume, sperm motility, duration or linearity of each type of male. We found some large mean differences in sperm concentration (d = 2.541, CI = −0.004 to 5.086), reproductive success (d = −1.400, CI = −2.943 to 0.142), semen osmolality (d = 1.850, CI = 0.622 to 3.077) and sperm velocity (d = −0.933, CI=−1.426 to −0.441); in the case of the latter two traits, the mean effect was statistically significant. However, any significance did not stand up to a more conservative analysis. Additionally, heterogeneity was high and we found that where large differences between the sperm of sex-reversed and wild-type males are reported, these effects are attributable to sperm sampling methodology. Overall, we found little evidence for large systematic differences between the sperm produced by masculinized and wild-type male fish. Thus, masculinized genotypic females may enjoy reproductive success comparable to genotypic males. This conclusion leads to two potential implications: (i) sex-reversed fish may influence the dynamics of wild populations and (ii) aquaculture practices may use ESR to produce males with sperm quality similar to that of genotypic males. Most studies appear to have been performed in aquaculture species (i.e. Salmonidae); thus future experiments in non-model organisms may provide important insights in to the uniformity of the effects described.
    Full-text · Article · Sep 2014 · Fish and Fisheries
  • Source
    • "An arbitrary minimum number of ten studies is needed to perform funnel plot analysis and Egger tests to objectify publication bias. A lower number of studies results in diminished power and subsequently is prone to misinterpretation [35], [36]. It is not possible to pool our data in such a way that this requirement is met. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background It is suggested that the body posture during urination can influence urodynamic parameters in patients with Lower Urinary Tract Symptoms (LUTS) to an extent approaching pharmacological interventions. In this article, the influence of body position during micturition on maximum urinary flow rate (Qmax), voiding time (TQ) and post-void residual volume (PVR) in healthy males and patients with LUTS is analyzed by means of a systematic review and meta-analysis. Evidence Acquisition A systematic search was conducted in 14 medical databases. Studies comparing urodynamic parameters in standing versus sitting position were eligible for inclusion. Studies were stratified according to health status of included male participants: healthy individuals and patients with LUTS. Standardized mean differences for Qmax, TQ and PVR were pooled in a random effects model. Results Eleven articles were included. In men with LUTS, a significantly lower PVR (−24.96 ml; 95%CI −48.70 to −1.23) was shown in sitting position compared to standing. In accordance, Qmax was increased (1.23 ml/s; 95%CI −1.02 to 3.48), and TQ was decreased (−0.62 s; 95%CI −1.66 to 0.42) in sitting position, although these differences did not reach statistical significance. In healthy men, Qmax (0.18 ml/s; 95% CI −1.67 to 2.02), TQ (0.49 s; 95%CI −3.30 to 4.27) and PVR (0.43 ml; 95%CI −0.79 to 1,65) were similar in sitting and standing position. Conclusion For healthy men, no difference is found in any of the urodynamic parameters. In patients with LUTS, the sitting position is linked with an improved urodynamic profile.
    Full-text · Article · Jul 2014 · PLoS ONE
Show more