Rating the methodological quality of single-subject designs and n-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale

Northern Clinical School, Faculty of Medicine, University of Sydney, Australia.
Neuropsychological Rehabilitation (Impact Factor: 1.96). 09/2008; 18(4):385-401. DOI: 10.1080/09602010802009201
Source: PubMed


Rating scales that assess methodological quality of clinical trials provide a means to critically appraise the literature. Scales are currently available to rate randomised and non-randomised controlled trials, but there are none that assess single-subject designs. The Single-Case Experimental Design (SCED) Scale was developed for this purpose and evaluated for reliability. Six clinical researchers who were trained and experienced in rating methodological quality of clinical trials developed the scale and participated in reliability studies. The SCED Scale is an 11-item rating scale for single-subject designs, of which 10 items are used to assess methodological quality and use of statistical analysis. The scale was developed and refined over a 3-year period. Content validity was addressed by identifying items to reduce the main sources of bias in single-case methodology as stipulated by authorities in the field, which were empirically tested against 85 published reports. Inter-rater reliability was assessed using a random sample of 20/312 single-subject reports archived in the Psychological Database of Brain Impairment Treatment Efficacy (PsycBITE). Inter-rater reliability for the total score was excellent, both for individual raters (overall ICC = 0.84; 95% confidence interval 0.73-0.92) and for consensus ratings between pairs of raters (overall ICC = 0.88; 95% confidence interval 0.78-0.95). Item reliability was fair to excellent for consensus ratings between pairs of raters (range k = 0.48 to 1.00). The results were replicated with two independent novice raters who were trained in the use of the scale (ICC = 0.88, 95% confidence interval 0.73-0.95). The SCED Scale thus provides a brief and valid evaluation of methodological quality of single-subject designs, with the total score demonstrating excellent inter-rater reliability using both individual and consensus ratings. Items from the scale can also be used as a checklist in the design, reporting and critical appraisal of single-subject designs, thereby assisting to improve standards of single-case methodology.

  • Source
    • "These build on proposals by Kratchowill et al (2010, 2013) but are not used widely in practice and their validity has yet to be tested. Interestingly, while Tate et al. (2008) required statistical analysis, Tate et al. (2013) do not (ROBiNT item 13: Data Analysis), stating, " Controversy remains about whether the appropriate method of analysis in single-case reports is visual or statistical. Nonetheless, 2 points are awarded if systematic visual analysis is used according to steps specified by Kratochwill et al. (2010; 2013), or visual analysis is aided by quasi-statistical techniques, or statistical methods are used where a rationale is provided for their suitability. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: In Howard, Best, and Nickels (2015, Optimising the design of intervention studies: Critiques and ways forward. Aphasiology, 2015.), we presented a set of ideas relevant to the design of single-case studies for evaluation of the effects of intervention. These were based on our experience with intervention research and methodology, and a set of simulations. Our discussion and conclusions were not intended as guidelines (of which there are several in the field) but rather had the aim of stimulating debate and optimising designs in the future. Our paper achieved the first aim—it received a set of varied commentaries, not all of which felt we were optimising designs, and which raised further points for debate.
    Full-text · Article · Dec 2015 · Aphasiology
    • "In this paper we focused on a specific rehabilitation period within a longer follow up and rehabilitation history, and on the method chosen by the multidisciplinary team to verify the effectiveness of an intensive rehabilitative treatment. We studied the effects of the focused rehabilitative training by the adapting a single case design (Tate et al., 2008). Tate et al.'s methodology to develop items for the Single Case Experimental Design (SCED) suggests building either a reversal/ withdrawal or a multiple baseline across behaviors design. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We report the clinical and rehabilitative follow up of M, a female child carrying a compound heterozygous pathogenic mutations in the TCTN1 gene and affected by Joubert Syndrome (JS). JS is a congenital cerebellar ataxia characterized by "the molar tooth sign" on axial MRI, a pathognomonic neuroradiological malformation involving the cerebellum and brainstem. JS presents with high phenotypic/cognitive variability, and little is known about cognitive rehabilitation programs. We describe the therapeutic settings, intensive rehabilitation targets and outcome indexes in M's cognitive development. Using a single case evidence-based approach, we attempt to distinguish the effectiveness of the intervention from the overall developmental trend. We assume that an adequate amount of focused, goal directed treatment in a relative short period of time can be at least as effective as one provided in longer time, and much less interfering with the child's everyday life. We conclude by discussing specific issues in cognitive development and rehabilitation in JS and, more broadly, in cerebellar malformations.
    No preview · Article · Oct 2015 · Research in developmental disabilities
  • Source
    • "A replicated single case design was employed with five participants . This design is recommended when new treatments are developed and evaluated [5] [6]. Single case designs provide an intensive study of the individual, which includes systematic observation, manipulation of variables, repeated measurement before and during the intervention, and mainly visual data analysis . "
    [Show abstract] [Hide abstract]
    ABSTRACT: Dealing with chronic pain is difficult and affects physiological as well as psychological well-being. Patients with chronic pain are often reporting concurrent emotional problems such as low mood and depressive symptoms. Considering this, treatments need to involve strategies for improving mood and promoting well-being in this group of patients. With the rise of the positive psychology movement, relatively simple intervention strategies to increase positive feelings, cognitions, and behaviours have become available. So far, the evidence for positive psychology techniques mainly comes from studies with healthy participants, and from studies with patients expressing emotional problems such as depression or anxiety as their main complaint. This study describes an initial attempt to explore the potential effects of a positive psychology intervention in a small sample of patients suffering from chronic pain.
    Full-text · Article · Apr 2015 · Scandinavian Journal of Pain
Show more