Web-based training and interrater reliability testing for scoring the Hamilton Depression Rating Scale

Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, United States.
Psychiatry Research (Impact Factor: 2.47). 09/2008; 161(1):126-30. DOI: 10.1016/j.psychres.2008.03.001
Source: PubMed


Despite the importance of establishing shared scoring conventions and assessing interrater reliability in clinical trials in psychiatry, these elements are often overlooked. Obstacles to rater training and reliability testing include logistic difficulties in providing live training sessions, or mailing videotapes of patients to multiple sites and collecting the data for analysis. To address some of these obstacles, a web-based interactive video system was developed. It uses actors of diverse ages, gender and race to train raters how to score the Hamilton Depression Rating Scale and to assess interrater reliability. This system was tested with a group of experienced and novice raters within a single site. It was subsequently used to train raters of a federally funded multi-center clinical trial on scoring conventions and to test their interrater reliability. The advantages and limitations of using interactive video technology to improve the quality of clinical trials are discussed.

Download full-text


Available from: Christopher John Groening
  • Source
    • "Despite the clear importance of training, descriptions of training provided in clinical trials are usually not adequately reported (Mulsant et al., 2002) and are typically limited to one to three paragraphs. Articles devoted to training focus solely on inter-rater reliability aspects of single measures (e.g., Jeglic et al., 2007; Kobak, Engelhardt, & Lipsitz, 2006; Kobak, Lipsitz, Williams, Engelhardt, & Bellew, 2005; Rosen et al., 2008), to the exclusion of the training provided on study procedures not specific to a given measure (e.g., using data capture systems, recruitment methods). Furthermore, effectively training research staff for multisite randomized clinical trials poses additional challenges due to geographic distribution of staff, difficulty scheduling training session thus leading to extended training timelines, hiring decisions resulting in staff with varying skills, and limited opportunities for trainers to work face-to-face with staff when needed. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: Descriptions of and recommendations for meeting the challenges of training research staff for multisite studies are limited despite the recognized importance of training on trial outcomes. The STRIDE (Stimulant Reduction Intervention using Dosed Exercise) study is a multisite randomized clinical trial that was conducted at nine addiction treatment programs across the United States within the National Drug Abuse Treatment Clinical Trials Network (CTN) and evaluated the addition of exercise to addiction treatment as usual (TAU), compared to health education added to TAU, for individuals with stimulant abuse or dependence. Research staff administered a variety of measures that required a range of interviewing, technical, and clinical skills. Purpose: In order to address the absence of information on how research staff are trained for multisite clinical studies, the current manuscript describes the conceptual process of training and certifying research assistants (RAs) for STRIDE. Methods: Training was conducted using a three-stage process to allow staff sufficient time for distributive learning, practice, and calibration leading up to implementation of this complex study. Results: Training was successfully implemented with staff across nine sites. Staff demonstrated evidence of study and procedural knowledge via quizzes and skill demonstration on six measures requiring certification. Overall, while the majority of staff had little to no experience in the six measures, all RAs demonstrated ability to correctly and reliably administer the measures throughout the study. Conclusions: Practical recommendations are provided for training research staff and are particularly applicable to the challenges encountered with large, multisite trials.
    Full-text · Article · Dec 2013 · Addiction Research and Theory
  • [Show abstract] [Hide abstract]
    ABSTRACT: The aim of this study was to compare two methods to conduct CGAS rater training. A total of 648 raters were randomized to training (CD or seminar), and rated five cases before and 12 months after training. The ICC at baseline/end of study was 0.71/0.78 (seminar), 0.76/0.78 (CD), and 0.67/0.79 (comparison). There were no differences in training effect in terms of agreement with expert ratings, which speaks in favor of using the less resource-demanding CD. However, the effect was modest in both groups, and untrained comparison group improved of the same order of magnitude, which proposes more extensive training.
    No preview · Article · Jul 2011 · Administration and Policy in Mental Health and Mental Health Services Research
  • [Show abstract] [Hide abstract]
    ABSTRACT: Using the database of the National Institute of Mental Health-sponsored acute treatment of late life mania study (GERI-BD), we assessed the role of social support in the presentation of late life bipolar mania. In the first 100 subjects randomized in geriatric BD, we explored the demographic, clinical, and social support characteristics (assessed using the Duke Social Support Index) and aspects of manic presentation. We selected two dependent variables: symptom severity, as determined by the Young Mania Rating Scale (YMRS) at baseline, and duration of episode. We selected nine potential independent variables on the basis of Pearson correlation coefficients. We derived two final models using multiple regression analysis employing an iterative process. In our severity model, being married was associated with a higher YMRS score (p = 0.05), whereas higher social interaction scores with non-family members were associated with a lower YMRS score (p = 0.011). In the episode duration model, longer duration was associated with a higher Hamilton Depression Rating Scale score (p = 0.03) and higher social interaction scores with non-family members (p = 0.0003), younger age (p = 0.04), higher number of persons in one's family social network (p = 0.017), and higher instrumental support scores (p = 0.0062). In late life mania, more social interaction with one's community appears to be associated with less severe symptoms at presentation for treatment, however, it can also be associated with slightly longer the duration of episode. Two aspects of the Duke Social Support Index are associated with a shorter episode duration prior to seeking treatment: being part of a larger family network and a having a higher level of instrumental support prior to treatment. The Instrumental Support Subscale measures the degree of assistance that is available for the respondent in performing daily tasks. These findings suggest that in older adults with BD, close social interactions and support are important in limiting the length of the illness episode prior to treatment. Social interactions involving non-family members may be less important in moderating the intensity of the symptoms at presentation. Copyright © 2014 John Wiley & Sons, Ltd.
    No preview · Article · Oct 2014 · International Journal of Geriatric Psychiatry