[Show abstract][Hide abstract] ABSTRACT: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries.
Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2 = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2 = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6-2.5). The probability of publication within two years after study completion ranged from 7% to 30%.
A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased.
PLoS ONE 12/2014; 9(12):e114023. · 3.53 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Practicing physicians are faced with many medical decisions daily. These are mainly influenced by personal experience but should also consider patient preferences and the scientific evidence reflected by a constantly increasing number of medical publications and guidelines. With the objective of optimal medical treatment, the concept of evidence-based medicine is founded on these three aspects. It should be considered that there is a high risk of misinterpreting evidence, leading to medical errors and adverse effects without knowledge of the methodological background.
[Show abstract][Hide abstract] ABSTRACT: The discontinuation of randomized clinical trials (RCTs) raises ethical concerns and often wastes scarce research resources. The epidemiology of discontinued RCTs, however, remains unclear.
To determine the prevalence, characteristics, and publication history of discontinued RCTs and to investigate factors associated with RCT discontinuation due to poor recruitment and with nonpublication.
Retrospective cohort of RCTs based on archived protocols approved by 6 research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics and planned recruitment from included protocols. Last follow-up of RCTs was April 27, 2013.
Completion status, reported reasons for discontinuation, and publication status of RCTs as determined by correspondence with the research ethics committees, literature searches, and investigator surveys.
After a median follow-up of 11.6 years (range, 8.8-12.6 years), 253 of 1017 included RCTs were discontinued (24.9% [95% CI, 22.3%-27.6%]). Only 96 of 253 discontinuations (37.9% [95% CI, 32.0%-44.3%]) were reported to ethics committees. The most frequent reason for discontinuation was poor recruitment (101/1017; 9.9% [95% CI, 8.2%-12.0%]). In multivariable analysis, industry sponsorship vs investigator sponsorship (8.4% vs 26.5%; odds ratio [OR], 0.25 [95% CI, 0.15-0.43]; P < .001) and a larger planned sample size in increments of 100 (-0.7%; OR, 0.96 [95% CI, 0.92-1.00]; P = .04) were associated with lower rates of discontinuation due to poor recruitment. Discontinued trials were more likely to remain unpublished than completed trials (55.1% vs 33.6%; OR, 3.19 [95% CI, 2.29-4.43]; P < .001).
In this sample of trials based on RCT protocols from 6 research ethics committees, discontinuation was common, with poor recruitment being the most frequently reported reason. Greater efforts are needed to ensure the reporting of trial discontinuation to research ethics committees and the publication of results of discontinued trials.
JAMA The Journal of the American Medical Association 03/2014; 311(10):1045-51. · 29.98 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Many clinical studies are ultimately not fully published in peer-reviewed journals. Underreporting of clinical research is wasteful and can result in biased estimates of treatment effect or harm, leading to recommendations that are inappropriate or even dangerous.
We assembled a cohort of clinical studies approved 2000-2002 by the Research Ethics Committee of the University of Freiburg, Germany. Published full articles were searched in electronic databases and investigators contacted. Data on study characteristics were extracted from protocols and corresponding publications. We characterized the cohort, quantified its publication outcome and compared protocols and publications for selected aspects.
Of 917 approved studies, 807 were started and 110 were not, either locally or as a whole. Of the started studies, 576 (71%) were completed according to protocol, 128 (16%) discontinued and 42 (5%) are still ongoing; for 61 (8%) there was no information about their course. We identified 782 full publications corresponding to 419 of the 807 initiated studies; the publication proportion was 52% (95% CI: 0.48-0.55). Study design was not significantly associated with subsequent publication. Multicentre status, international collaboration, large sample size and commercial or non-commercial funding were positively associated with subsequent publication. Commercial funding was mentioned in 203 (48%) protocols and in 205 (49%) of the publications. In most published studies (339; 81%) this information corresponded between protocol and publication. Most studies were published in English (367; 88%); some in German (25; 6%) or both languages (27; 6%). The local investigators were listed as (co-)authors in the publications corresponding to 259 (62%) studies.
Half of the clinical research conducted at a large German university medical centre remains unpublished; future research is built on an incomplete database. Research resources are likely wasted as neither health care professionals nor patients nor policy makers can use the results when making decisions.
PLoS ONE 02/2014; 9(2):e87184. · 3.53 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Despite the fact that there are more than twenty thousand biomedical journals in the world, research into the work of editors and publication process in biomedical and health care journals is rare. In December 2012, the Esteve Foundation, a non-profit scientific institution that fosters progress in pharmacotherapy by means of scientific communication and discussion organized a discussion group of 7 editors and/or experts in peer review biomedical publishing. They presented findings of past editorial research, discussed the lack of competitive funding schemes and specialized journals for dissemination of editorial research, and reported on the great diversity of misconduct and conflict of interest policies, as well as adherence to reporting guidelines. Furthermore, they reported on the reluctance of editors to investigate allegations of misconduct or increase the level of data sharing in health research. In the end, they concluded that if editors are to remain gatekeepers of scientific knowledge they should reaffirm their focus on the integrity of the scientific record and completeness of the data they publish. Additionally, more research should be undertaken to understand why many journals are not adhering to editorial standards, and what obstacles editors face when engaging in editorial research.
[Show abstract][Hide abstract] ABSTRACT: To identify factors associated with discrepant outcome reporting in randomized drug trials.
Cohort study of protocols submitted to a Swiss ethics committee 1988-1998: 227 protocols and amendments were compared with 333 matching articles published during 1990-2008. Discrepant reporting was defined as addition, omission, or reclassification of outcomes.
Overall, 870 of 2,966 unique outcomes were reported discrepantly (29.3%). Among protocol-defined primary outcomes, 6.9% were not reported (19 of 274), whereas 10.4% of reported outcomes (30 of 288) were not defined in the protocol. Corresponding percentages for secondary outcomes were 19.0% (284 of 1,495) and 14.1% (334 of 2,375). Discrepant reporting was more likely if P values were <0.05 compared with P ≥ 0.05 [adjusted odds ratio (aOR): 1.38; 95% confidence interval (CI): 1.07, 1.78], more likely for efficacy compared with harm outcomes (aOR: 2.99; 95% CI: 2.08, 4.30) and more likely for composite than for single outcomes (aOR: 1.48; 95% CI: 1.00, 2.20). Cardiology (aOR: 2.34; 95% CI: 1.44, 3.79) and infectious diseases (aOR: 1.77; 95% CI: 1.01, 3.13) had more discrepancies compared with all specialties combined.
Discrepant reporting was associated with statistical significance of results, type of outcome, and specialty area. Trial protocols should be made freely available, and the publications should describe and justify any changes made to protocol-defined outcomes.
Journal of clinical epidemiology 09/2013; · 5.48 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Erik von Elm and colleagues discuss plans to increase access and global reach of Cochrane Reviews through translations into other languages. Please see later in the article for the Editors' Summary.
PLoS Medicine 09/2013; 10(9):e1001516. · 14.00 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Sind manuelle Therapien wirksam bei akutem lumbalem Rückenschmerz? Hintergrund Lumbaler Rückenschmerz ist eine häu-fig auftretende Funktionsstörung der Wirbelsäule, die eine grosse Belastung für Patienten und die Gesellschaft allge-mein darstellt. Er führt zu verringerter Lebensqualität, Arbeitsausfällen sowie hohen medizinischen Kosten. Manuelle Therapien der Wirbelsäule werden von verschiedenen Gesundheitsfachleuten (z. B. Physiotherapeuten, Chiroprakti-kern, Osteopathen, Manualtherapeu-ten) praktiziert. In der vorliegenden, aktualisierten Cochrane Review werden Schmerzen als akut betrachtet, wenn sie weniger als sechs Wochen dauern. Resultat Es wurden zwanzig randomisierte Ver-gleichsstudien mit insgesamt 2647 Teil-nehmern ab 18 Jahren erfasst. Dabei wurden Studienpopulationen mit oder ohne ausstrahlenden Schmerz berück-sichtigt; bezüglich klinischem Setting oder Schmerztypus wurde nicht einge-schränkt. Die in den Studien durchge-führten Manipulationen unterschieden sich nach Behandlungskonzept, Intensi-tät (mit oder ohne Impuls) und Bewe-gungsrichtung: osteopathisch (1 Studie), rotational (3 Studien), chiropraktisch (2 Studien), High velocity low amplitu-de (HVLA)-Manipulation (8 Studien), Manipulation nach Cyriax, Kaltenborn, Lewit, Janda (1 Studie), Manipulation nach Stoddart und Maitland (1 Studie), Manipulation nach Flynn (1 Studie), ge-lenkserweiternde Manipulation (1 Stu-die), sowie Manipulation in Seitenlage (2 Studien). Diese manuelle Therapien wurde mit folgenden Kontrollbehandlungen ver-glichen: inaktive Behandlungen (7 Stu-dien), Scheinbehandlungen (1 Studie), andere nicht-manuelle Behandlungen (8 Studien) und manuelle Therapie zu-sätzlich zu anderen Therapien (4 Stu-dien). In 3 Studien wurden verschie-dene manuelle Therapien miteinander verglichen. Die Behandlungsergebnisse wurden nach ihrer Dauer eingeteilt. Primäre Endpunkte waren Schmerz, rü-ckenspezifischer funktionaler Status und Symptomverbesserung bzw. Genesung; sekundäre Endpunkte waren Arbeits-wiederaufnahme und Lebensqualität. Insgesamt war die Qualität der Evidenz im Bezug auf die primären Endpunkte gering bis sehr gering. Nur 6 der 20 Stu-dien waren methodisch gut. Manuelle Therapien verringerten die Schmerz-intensität während eines Monats nicht besser als inaktive Behandlungen (3 Studien: MD 0,14; 95% CI: −0,69 bis 0,96) oder Scheinbehandlungen (1 Stu-die: MD 0,50; 95% CI: −1,39 bis 0,39). Auch bezüglich des rückenspezifischen funktionalen Status nach 3 Monaten bestand kein signifikanter Unterschied (1 Studie: SMD −0,28; 95% CI: −0,59 bis 0,02). Im Vergleich zu anderen Kon-trollinterventionen und bei Nachbeob-achtungen bis zu 12 Monaten erwiesen sich die manuellen Therapien ebenfalls als nicht wirksamer.
[Show abstract][Hide abstract] ABSTRACT: Health professionals and policymakers aspire to make healthcare decisions based on the entire relevant research evidence. This, however, can rarely be achieved because a considerable amount of research findings are not published, especially in case of 'negative' results - a phenomenon widely recognized as publication bias. Different methods of detecting, quantifying and adjusting for publication bias in meta-analyses have been described in the literature, such as graphical approaches and formal statistical tests to detect publication bias, and statistical approaches to modify effect sizes to adjust a pooled estimate when the presence of publication bias is suspected. An up-to-date systematic review of the existing methods is lacking.Methods/design: The objectives of this systematic review are as follows:[bullet] To systematically review methodological articles which focus on non-publication of studies and to describe methods of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses.[bullet] To appraise strengths and weaknesses of methods, the resources they require, and the conditions under which the method could be used, based on findings of included studies.We will systematically search Web of Science, Medline, and the Cochrane Library for methodological articles that describe at least one method of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses. A dedicated data extraction form is developed and pilot-tested. Working in teams of two, we will independently extract relevant information from each eligible article. As this will be a qualitative systematic review, data reporting will involve a descriptive summary.
Results are expected to be publicly available in mid 2013. This systematic review together with the results of other systematic reviews of the OPEN project (To Overcome Failure to Publish Negative Findings) will serve as a basis for the development of future policies and guidelines regarding the assessment and handling of publication bias in meta-analyses.
[Show abstract][Hide abstract] ABSTRACT: Manuelle Therapien sind beim lumbalen Rücken-schmerz nicht wirksamer als andere Behand -lungen – so das Fazit einer aktualisierten Cochrane Review. Unsere AutorInnen erklären, wie das Resultat zustande gekommen ist und was es für die Praxis bedeutet.
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND: Selective publication of studies, which is commonly called publication bias, is widely recognized. Over the years a new nomenclature for other types of bias related to non-publication or distortion related to the dissemination of research findings has been developed. However, several of these different biases are often still summarized by the term 'publication bias'. METHODS: As part of the OPEN Project (To Overcome failure to Publish nEgative fiNdings) we will conduct a systematic review with the following objectives:- To systematically review highly cited articles that focus on non-publication of studies and to present the various definitions of biases related to the dissemination of research findings contained in the articles identified.- To develop and discuss a new framework on nomenclature of various aspects of distortion in the dissemination process that leads to public availability of research findings in an international group of experts in the context of the OPEN Project.We will systematically search Web of Knowledge for highly cited articles that provide a definition of biases related to the dissemination of research findings. A specifically designed data extraction form will be developed and pilot-tested. Working in teams of two, we will independently extract relevant information from each eligible article.For the development of a new framework we will construct an initial table listing different levels and different hazards en route to making research findings public. An international group of experts will iteratively review the table and reflect on its content until no new insights emerge and consensus has been reached. DISCUSSION: Results are expected to be publicly available in mid-2013. This systematic review together with the results of other systematic reviews of the OPEN project will serve as a basis for the development of future policies and guidelines regarding the assessment and prevention of publication bias.
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND: Meta-analyses are particularly vulnerable to the effects of publication bias. Despite methodologists' best efforts to locate all evidence for a given topic the most comprehensive searches are likely to miss unpublished studies and studies that are published in the gray literature only. If the results of the missing studies differ systematically from the published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention's effects.As part of the OPEN project (www.open-project.eu) we will conduct a systematic review with the following objectives:[black small square] To assess the impact of studies that are not published or published in the gray literature on pooled effect estimates in meta-analyses (quantitative measure)[black small square] To assess whether the inclusion of unpublished studies or studies published in the gray literature leads to different conclusions in meta-analyses (qualitative measure) METHODS: Inclusion criteria: Methodological research projects of a cohort of meta-analyses (, which compare the effect of the inclusion or exclusion of unpublished studies or studies published in the gray literature.Literature search: To identify relevant research projects we will conduct electronic searches in Medline, Embase and The Cochrane Library; check reference lists; and contact experts.Outcomes: 1) The extent to which the effect estimate in a meta-analyses changes with the inclusion or exclusion of studies that were not published or published in the gray literature; and 2) the extent to which the inclusion of unpublished studies impacts the meta-analyses' conclusions.Data collection: Information will be collected on the area of health care; the number of meta-analyses included in the methodological research project; the number of studies included in the meta-analyses; the number of study participants; the number and type of unpublished studies; studies published in the gray literature and published studies; the sources used to retrieve studies that are unpublished, published in the gray literature, or commercially published; and the validity of the methodological research project.Data synthesis: Data synthesis will involve descriptive and statistical summaries of the findings of the included methodological research projects. DISCUSSION: Results are expected to be publicly available in the middle of 2013.
Systematic reviews. 05/2013; 2(1):24.
This article is viewable in ResearchGate's enriched format
RG Format enables you to read in context with side-by-side figures, citations, and feedback from experts in your field.
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND: Systematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular. METHODS: The objectives of this systematic review are as follows:To investigate the epidemiology of published systematic reviews of animal studies until present.To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias.To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies.Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior.In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions.Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews. DISCUSSION: Results are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND: Methodological research has found that non-published studies often have different results than those that are published, a phenomenon known as publication bias. When results are not published, or are published selectively based on the direction or the strength of the findings, healthcare professionals and consumers of healthcare cannot base their decision-making on the full body of current evidence. METHODS: As part of the OPEN project (www.open-project.eu) we will conduct a systematic review with the following objectives:1.To determine the proportion and/or rate of non-publication of studies by systematically reviewing methodological research projects that followed up a cohort of studies thata.received research ethics committee (REC) approval,b.were registered in trial registries, orc.were presented as abstracts at conferences.2.To assess the association of study characteristics (for example, direction and/or strength of findings) with likelihood of full publication.To identify reports of relevant methodological research projects we will conduct electronic database searches, check reference lists, and contact experts. Published and unpublished projects will be included. The inclusion criteria are as follows:a.RECs: methodological research projects that examined the subsequent proportion and/or rate of publication of studies that received approval from RECs;b.Trial registries: methodological research projects that examine the subsequent proportion and/or rate of publication of studies registered in trial registries;c.Conference abstracts: methodological research projects that examine the subsequent proportion and/or rate of full publication of studies which were initially presented at conferences as abstracts.Primary outcomes: Proportion/rate of published studies; time to full publication (mean/median; cumulative publication rate by time).Secondary outcomes: Association of study characteristics with full publication.The different questions (a, b, and c) will be investigated separately. Data synthesis will involve a combination of descriptive and statistical summaries of the included methodological research projects. DISCUSSION: Results are expected to be publicly available in mid 2013.
Systematic reviews. 01/2013; 2(1):2.
This article is viewable in ResearchGate's enriched format
RG Format enables you to read in context with side-by-side figures, citations, and feedback from experts in your field.