Poor-Quality Medical Research: What Can Journals Do

Cancer Research UK/NHS Centre for Statistics in Medicine, Institute of Health Sciences, Old Road, Headington, Oxford OX3 7LF, England.
JAMA The Journal of the American Medical Association (Impact Factor: 35.29). 07/2002; 287(21):2765-7. DOI: 10.1001/jama.287.21.2765
Source: PubMed


The aim of medical research is to advance scientific knowledge and hence--directly or indirectly--lead to improvements in the treatment and prevention of disease. Each research project should continue systematically from previous research and feed into future research. Each project should contribute beneficially to a slowly evolving body of research. A study should not mislead; otherwise it could adversely affect clinical practice and future research. In 1994 I observed that research papers commonly contain methodological errors, report results selectively, and draw unjustified conclusions. Here I revisit the topic and suggest how journal editors can help.

Full-text preview

Available from:
  • Source
    • "They are considered a form of post-publication peer review and an essential part of scientific debate [3-7]. LTE provide a platform to comment on, clarify and correct published research by alerting readers about aspects of a paper that may have been overlooked by authors, peer-reviewers and editors [5-8]. Some authors have argued that an article is not fully peer-reviewed until after publication [9,10] and that authors of LTE may have more credibility than pre-publication peer reviewers because their opinions are signed, published and thus transparent [3]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: To identify published letters to the editor (LTE) written in response to randomized controlled trials (RCTs), determine the topics addressed in the letters, and to examine if these topics were affected by the characteristics and results of the RCTs. Comparative cross-sectional study of a representative sample of RCTs from a set of high-impact medical journals (BMJ, Lancet, NEJM, JAMA, and Annals of Internal Medicine). RCTs and their published LTE were searched from these 5 journals in 2007. Data were collected on RCTs and their characteristics (author affiliation, funding source, intervention, and effect on the primary outcome) and the topics addressed in published LTE related to these RCTs. Analysis included chi-square and regression analysis (RCT characteristics) and thematic analysis (LTE topics). Of 334 identified RCTs, 175 trials had at least one LTE. Of these, 381 published LTE were identified. Most RCTs, tested drug interventions (68%), were funded by government (54%) or industry (33%), and described an intervention that had a positive impact on the primary outcome (62%). RCT authors were primarily affiliated with an academic centre (78%). Ninety percent of the 623 LTE topics concerned methodological issues regarding the analysis, intervention, and population in the RCT. There was a significant association between funding source and impact on outcomes (p = 0.002) or type of intervention tested (p = 0.001) in these trials. Clinical and "Other" LTE topics were more likely to be published in response to a government funded RCT (p = 0.005 and p = 0.033, respectively); no other comparisons were significant. This study showed that most LTE are about methodological topics, but found little evidence to support that these topics are affected by the characteristics or results of the RCTs. The lack of association may be explained by editorial censorship as a small proportion of LTE that are submitted are actually published.
    BMC Research Notes 10/2013; 6(1):414. DOI:10.1186/1756-0500-6-414
  • Source
    • "The internet has made it possible to have a fast and effective peer review, which filters articles and thereby improves the quality of any journal (Altman, 2002[1]). Although, it is now a well accepted fact that peer review is crucial to the selection and publication of qualitative scientific work, identifying and motivating high-quality peer reviewers is still an unsolved question. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Science is strengthened not by research alone, but by publication of original research articles in international scientific journals that gets read by a global scientific community. Research publication is the 'heart' of a journal and the 'soul' of science - the outcome of collective efforts of authors, editors and reviewers. The publication process involves author-editor interaction for which both of them get credit once the article gets published - the author directly, the editor indirectly. However, the remote reviewer who also plays a key role in the process remains anonymous and largely unrecognised. Many potential reviewers therefore, stay away from this 'highly honorary' task. Appropriate peer review controls quality of an article and thereby ensures quality and integrity of the journal. Recognising and rewarding the role of the reviewer is therefore vital. In this article, we propose a novel idea of Reviewer Index (RI), Reviewer Index Directory (RID) and Global Reviewer Index Directory (GRID), which will strengthen science by focussing on the reviewer, as well as the author. By adopting this innovative Reviewer Centric Approach, a new breed of well-trained reviewers of high quality and sufficient quantity will be available for eternity. Moreover, RI, RID and GRID would also enable grading and ethical rewarding of reviewers.
    Mens Sana Monographs 03/2013; 11(1):274-84. DOI:10.4103/0973-1229.109347
  • Source
    • "Quite often, research reports and abstracts do not contain proper information for meeting required standards and serving the multiple needs of their end users [1], be they researchers or health care providers. Poor methodology and reporting are widespread [2]. As a result, published studies often cannot be replicated by researchers, translated into clinical practice, or used to inform public health policy. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Manuscripts and abstracts from biomedical journals frequently do not contain proper information for meeting required standards and serving the multiple needs of their end users. Reporting guidelines and checklists help researchers to meet those standards by providing rules or principles for specific research areas. Rheumatology research includes a broad range of heterogeneous research areas, each with its own requirements, producing several distinct categories of articles. Our objectives with this article are to raise awareness of the existence and importance of reporting guidelines, to present a structured overview of reporting guidelines that rheumatology journals could apply, and to encourage their use by journal authors, editors, and reviewers, including those of Arthritis Research & Therapy. Internationally recognized reporting guidelines exist for a diversity of research areas. We encourage colleagues to consult the 'Enhancing the QUAlity and Transparency Of health Research' (EQUATOR) network when writing scientific papers. EQUATOR is an international initiative that seeks to improve the reliability and value of biomedical research literature by promoting transparent and accurate reporting of studies. We propose specific reporting guidelines for a number of study designs: animal research, randomized trials, reliability and agreement studies, systematic reviews with and without meta-analyses, diagnostic test accuracy studies, and also observational research including cross-sectional, cohort, and case-control studies. We encourage authors, editors, and reviewers to adhere to and enforce the use of the appropriate guidelines when writing, reading, and reviewing scientific papers.
    Arthritis research & therapy 02/2013; 15(1):109. DOI:10.1186/ar4145 · 3.75 Impact Factor
Show more