Assessing the Quality of Randomized Controlled Trials: An Annotated Bibliography of Scales and Checklists

Clinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa Civic Hospital, Ontario, Canada.
Controlled Clinical Trials 03/1995; 16(1):62-73. DOI: 10.1016/0197-2456(94)00031-W
Source: PubMed


Assessing the quality of randomized controlled trials (RCTs) is important and relatively new. Quality gives us an estimate of the likelihood that the results are a valid estimate of the truth. We present an annotated bibliography of scales and checklists developed to assess quality. Twenty-five scales and nine checklists have been developed to assess quality. The checklists are most useful in providing investigators with guidelines as to what information should be included in reporting RCTs. The scales give readers a quantitative index of the likelihood that the reported methodology and results are free of bias. There are several shortcomings with these scales. Future scale development is likely to be most beneficial if questions common to all trials are assessed, if the scale is easy to use, and if it is developed with sufficient rigor.

48 Reads
  • Source
    • "In order to compare the quality of the included studies with the previous systematic review of Vancampfort et al. (2012) and to assess for potential bias in the trials, each study was evaluated with the previously validated 5-point Jadad scale (Jadad et al., 1996; Clark et al., 1999; Moher et al., 1995). A score of 0 to 5 is assigned, with higher scores indicating higher quality in the conduct or reporting of a trial. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This systematic review summarizes the most recent evidence from randomized controlled trials (RCTs) considering the effectiveness of physical therapy interventions (aerobic exercises, strength exercises, relaxation training, basic body awareness exercises, or a combination of these) within the multidisciplinary management of schizophrenia. Two authors searched PubMed, PsycINFO, EMBASE, Web of Science, Physiotherapy Evidence Database (PEDro), and the Cochrane Library considering RCTs published from July 1, 2011-October 1, 2014. Thirteen RCTs representing 549 participants met the inclusion criteria. Overall, the results demonstrate that aerobic exercise significantly reduces psychiatric symptoms, potentially improves mental and physical quality of life and reduces metabolic risk and weight. Specifically, yoga reduces psychiatric symptoms, whilst Tai-chi and progressive muscle relaxation may also have benefits to patients. Two RCTs reported on adverse events. No adverse event was observed supporting the notion that physical therapy is safe in people with schizophrenia. There was considerable heterogeneity in the design, implementation and outcomes in the included studies precluding a meaningful meta-analysis. In general, the quality of physical therapy RCTS is improving and current research demonstrates that physical therapy approaches are valuable interventions and can help improve the psychiatric, physical and quality of life of people with schizophrenia. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.
    Psychiatry Research 08/2015; 229(3). DOI:10.1016/j.psychres.2015.07.083 · 2.47 Impact Factor
    • "It is widely accepted that the quality of a study may affect its results (Chambless & Hollon, 1998; Lincoln & Rief, 2004; Moher et al., 1995; Schulz, Chalmers, Hayes, & Altman, 1995). Aspects of study quality may influence the calculated effect size directly or indirectly, for example, the calculated effect size may increase when study dropouts are not included in the analyses or may decrease when several outcome domains are considered (see Liebherz, Schmidt, & Rabung, submitted for review). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Although it is widely accepted that the quality of a study may affect its results, there is no agreed specific standard for assessing the quality of psychotherapy outcome studies. This study aims to review the existing variety of relevant quality assessment criteria. We identified relevant quality measures based on a systematic literature search. To determine the specific relevance of the available quality criteria, we consulted experts in the field of psychotherapy outcome research. Nineteen different measures providing 185 different quality criteria were included. Four measures specifically focused psychotherapy studies, none of these were designed for studies without a control group. Experts did judge 20% of the items as "absolutely indispensable." Quality criteria that are exclusively related to the context of psychotherapy research are rare. Further research is required to examine the empirical relation between specific quality criteria and study results.
    Psychotherapy Research 07/2015; DOI:10.1080/10503307.2015.1044763 · 1.75 Impact Factor
  • Source
    • "More recently, the Cochrane Collaboration has espoused a new tool designed to appraise the risk of bias inherent in nonrandomized studies [7], although it has yet to find wider application. Similarly, in an assessment of interventional SRs, Moher et al. [8] identified 25 unique tools used in the evaluation of methodological quality of clinical trials. Many of these were found to lack accepted theoretical basis and varied in respect of their size, complexity, and regarding the weight attributed to individual methodological aspects such as randomization procedures or blinding. "
    [Show abstract] [Hide abstract]
    ABSTRACT: To assess the use of quality assessment tools among a cross-section of systematic reviews (SRs) and to further evaluate whether quality was used as a parameter in the decision to include primary studies within subsequent meta-analysis. We searched PubMed for systematic reviews (interventional, observational and diagnostic) published in Core Clinical Journals between January 1st and March 31(st), 2014. 310 systematic reviews were identified. Quality assessment was undertaken in 223 (71.9%) with isolated use of the Cochrane risk of bias tool (26%, n= 58) and the Newcastle Ottawa Scale (15.3%, n= 34) most common. A threshold level of primary study quality for subsequent meta-analysis was used in 13.2% (41/310) of reviews. Overall, fifty-four combinations of quality assessment tools were identified with a similar preponderance of tools used among observational and interventional reviews. Multiple tools were used in 11.6% (n= 36) of SRs overall. We found that quality assessment tools were used in a majority of SRs; however, a threshold level of quality for meta-analysis was stipulated in just 13.2% (n= 41). This cross-sectional analysis provides further evidence of the need for more active or intuitive editorial processes to enhance the reporting of systematic reviews. Copyright © 2015 Elsevier Inc. All rights reserved.
    Journal of clinical epidemiology 07/2015; DOI:10.1016/j.jclinepi.2015.06.023 · 3.42 Impact Factor
Show more