Treatment adherence and competency ratings among therapists, supervisors, study-related raters and external raters in a clinical trial of 12-Step facilitation for stimulant users

Journal of Substance Abuse Treatment (Impact Factor: 3.14). 09/2014; 47(3). DOI: 10.1016/j.jsat.2014.05.008

ABSTRACT This study investigated the correspondence among four groups of raters on adherence to STAGE-12, a manualized 12-step facilitation (TSF) group and individual treatment targeting stimulant abuse. The four rater groups included the study therapists, supervisors, study-related ("TSF expert") raters, and non-project related ("external") raters. Results indicated that external raters rated most critically mean adherence - the mean of all the adherence items - and global performance. External raters also demonstrated the highest degree of reliability with the designated expert. Therapists rated their own adherence lower, on average, than did supervisors and TSF expert raters, but therapist ratings also had the poorest reliability. Findings highlight the challenges in developing practical, but effective methods of fidelity monitoring for evidence based practice in clinical settings. Recommendations based on study findings are provided.

1 Follower
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: AIMS: The study evaluated the effectiveness of an 8-week combined group plus individual 12-step facilitative intervention on stimulant drug use and 12-step meeting attendance and service. DESIGN: Multisite randomized controlled trial, with assessments at baseline, mid-treatment, end of treatment, and 3- and 6-month post-randomization follow-ups (FUs). SETTING: Intensive outpatient substance treatment programs. PARTICIPANTS: Individuals with stimulant use disorders (n=471) randomly assigned to treatment as usual (TAU) or TAU into which the Stimulant Abuser Groups to Engage in 12-Step (STAGE-12) intervention was integrated. MEASUREMENTS: Urinalysis and self-reports of substance use and 12-step attendance and activities. INTERVENTION: Group sessions focused on increasing acceptance of 12-step principles; individual sessions incorporated an intensive referral procedure connecting participants to 12-step volunteers. FINDINGS: Compared with TAU, STAGE-12 participants had significantly greater odds of self-reported stimulant abstinence during the active 8-week treatment phase; however, among those who had not achieved abstinence during this period, STAGE-12 participants had more days of use. STAGE-12 participants had lower Addiction Severity Index Drug Composite scores at and a significant reduction from baseline to the 3-month FU, attended 12-step meetings on a greater number of days during the early phase of active treatment, engaged in more other types of 12-step activities throughout the active treatment phase and the entire FU period, and had more days of self-reported service at meetings from mid-treatment through the 6-month FU. CONCLUSIONS: The present findings are mixed with respect to the impact of integrating the STAGE-12 intervention into intensive outpatient drug treatment compared with TAU on stimulant drug use. However, the results more clearly indicate that individuals in STAGE-12 had higher rates of 12-step meeting attendance and were engaged in more related activities throughout both the active treatment phase and the entire 6-month FU period than did those in TAU.
    Journal of substance abuse treatment 05/2012; 44(1). DOI:10.1016/j.jsat.2012.04.004 · 2.90 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This study examined the correspondence of treatment integrity ratings (adherence and competence) among community program therapists, supervisors, and observers for therapists who used motivational enhancement therapy (MET) within a National Institute on Drug Abuse Clinical Trials Network protocol. The results suggested there was reasonable agreement between the three groups of raters about the presence or absence of several fundamental MET strategies. Moreover, relative to observers, therapists and supervisors were more positive in their evaluations of the therapists' MET adherence and competence. These findings underscore the need for objective monitoring of therapists' performance when using empirically supported treatments and for adequately training therapists and supervisors to evaluate their treatment implementation in community programs, and are consistent with observations that different perspectives on the therapeutic process are not interchangeable.
    Psychotherapy Research 04/2009; 19(2):181-93. DOI:10.1080/10503300802688460 · 1.75 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Objective: This study evaluated the accuracy of youth, caregiver, therapist, and trained raters relative to treatment experts on ratings of therapist adherence to a substance abuse treatment protocol for adolescents. Method: Adherence ratings were provided by youth and caregivers in an ongoing trial evaluating a Contingency Management (CM) intervention for youth in juvenile drug court. These ratings were compared to those provided by therapists and trained raters, and each rater type was compared to ratings provided by CM treatment experts. Data were analyzed using item-response-theory-based Many-Facet Rasch Models. Results: Relative to treatment experts, youth and caregivers were significantly more likely to endorse the occurrence of CM components. In contrast, therapists and trained raters were much more consistent with treatment experts. In terms of practical significance, youth and caregivers each had a 97% estimated probability of indicating that a typical treatment component had occurred. By comparison, the probability was 31%, 19%, and 26% for therapists, trained raters, and treatment experts, respectively. Conclusions: Youth and caregivers were highly inaccurate relative to treatment experts, whereas, therapists and trained raters were generally consistent with treatment experts. The implications of these findings for therapist adherence measurement are considered. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
    Journal of Consulting and Clinical Psychology 05/2013; 81(4). DOI:10.1037/a0033021 · 4.85 Impact Factor
Show more