Therapist Self-Report of Evidence-Based Practices in Usual Care for Adolescent Behavior Problems: Factor and Construct Validity

ArticleinAdministration and Policy in Mental Health and Mental Health Services Research 41(1) · November 2012with11 Reads
DOI: 10.1007/s10488-012-0442-8 · Source: PubMed
Abstract
This study introduces a therapist-report measure of evidence-based practices for adolescent conduct and substance use problems. The Inventory of Therapy Techniques-Adolescent Behavior Problems (ITT-ABP) is a post-session measure of 27 techniques representing four approaches: cognitive-behavioral therapy (CBT), family therapy (FT), motivational interviewing (MI), and drug counseling (DC). A total of 822 protocols were collected from 32 therapists treating 71 adolescents in six usual care sites. Factor analyses identified three clinically coherent scales with strong internal consistency across the full sample: FT (8 items; α = .79), MI/CBT (8 items; α = .87), and DC (9 items, α = .90). The scales discriminated between therapists working in a family-oriented site versus other sites and showed moderate convergent validity with therapist reports of allegiance and skill in each approach. The ITT-ABP holds promise as a cost-efficient quality assurance tool for supporting high-fidelity delivery of evidence-based practices in usual care.
    • "The intervention has strong outcomes for adolescent substance abuse and delinquency in a series of randomized controlled trials. Comparison conditions in these studies included active treatments or usual care in non-research community clinics (Hogue et al. 2014; Von Sydow et al. 2013; Williams and Chang 2000). MDFT treatment process and implementation studies (Hogue and Liddle 2009) support the model's putative mechanisms of change (Henderson et al. 2009Henderson et al. , 2010 Hogue et al. 2008; Diamond et al. 2006; Schmidt et al. 1996), and economic analyses indicate MDFT is less expensive than standard care (Zavala et al. 2005). "
    [Show abstract] [Hide abstract] ABSTRACT: To address a growing public health problem with youth cannabis use, five Western European countries – Belgium, France, Germany, the Netherlands and Switzerland – collaborated on a cannabis treatment research effort. After deliberation, the research priority chosen was to implement and rigorously evaluate a treatment program for adolescents with cannabis use disorders – virtually unavailable in Western Europe at the time. Adolescent cannabis use disorders were even denied by some policy makers as bona fide public health problems. The most promising candidate for the treatment program to be studied, based on cross-national expert analyses and an exhaustive review of research findings to date, was Multidimensional Family Therapy (MDFT), developed in the USA. When pilot training with candidate clinicians began, some claimed it was “too American.” Some did not understand its innovation at first glance, stating that aspects of MDFT interventions were already part of daily clinical work. Others worried whether the senior role of psychiatrists would be jeopardized, and if the approach engaged in too much outreach, and would be a threat to in-office work. Still others said the model might be too practical, and ignore the need for depth-oriented, psychodynamic treatment – still dominant in parts of Europe. While at the outset MDFT presented as a cultural shock, concerns disappeared when the approach was taught, attempted and integrated into the regular practice settings. The multi-country randomized controlled trial was designed with considerable discussion and collaboration. Referred to as INCANT (International Cannabis Need of Treatment), this study, the first independent replication of MDFT, showed that most adolescents with cannabis use disorders in these five countries have multiple behavioral problems, including criminality, truancy and mental co-morbidity. MDFT proved to be more effective than a high level treatment as usual in reducing cannabis dependence and on other problem behavior measures as well. Positive outcomes were seen in all the five countries. And given the clinical outcomes, the therapist competence and fidelity outcomes, and the capacity of the sites to absorb this new clinical approach, MDFT was found to be feasible and adaptable to representative regular clinical care Western Europe settings, adding expanded treatment alternative to standard care. The challenges of conducting a multi-national randomized controlled trial in real world, non-research settings foreshadowed subsequent efforts to sustain implementation of this evidence-based treatment program. While retaining the core principles, structure and interventions of the approach, the MDFT implementation strategy has been adapted in each of these European countries, as they vary in accreditation requirements, reimbursement rules, public and private position of treatment centers for youth with multiple problem behavior, regard for certain professional groups (e.g., social workers), and referral processes. Facilitating MDFT implementation in Europe has been like executing an EU financial crisis policy, but we are getting there.
    Full-text · Chapter · Jan 2015 · Psychological Assessment
  • [Show abstract] [Hide abstract] ABSTRACT: Developing therapist-report fidelity tools to support quality delivery of evidence-based practices in usual care is a top priority for implementation science. This study tested the reliability and accuracy of two groups of community therapists who reported on their use of family therapy (FT) and motivational interviewing/cognitive-behavioral therapy (MI/CBT) interventions during routine treatment of inner-city adolescents with conduct and substance use problems. Study cases (n = 45) were randomized into two conditions: (a) Routine Family Therapy (RFT), consisting of a single site that featured family therapy as its standard of care for behavioral treatment; or (b) Treatment As Usual (TAU), consisting of five sites that featured non-family approaches. Therapists and trained observational raters provided FT and MI/CBT adherence ratings on 157 sessions (104 RFT, 53 TAU). Overall therapist reliability was adequate for averaged FT ratings (ICC = .66) but almost non-existent for MI/CBT (ICC = .06); moreover, both RFT and TAU therapists were more reliable in reporting on FT than on MI/CBT. Both groups of therapists overestimated the extent to which they implemented FT and MI/CBT interventions. Results offer support for the feasibility of using existing therapist-report methods to anchor quality assurance procedures for FT interventions in real-world settings, though not for MI/CBT.
    Article · Apr 2014
  • [Show abstract] [Hide abstract] ABSTRACT: Observational measures to assess implementation integrity (the extent to which components of an evidence-based treatment are delivered as intended) are needed. The authors evaluated the reliability of the scores and the validity of the score interpretations for the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale (TPOCS-RS; McLeod, 2010) and assessed the potential of the TPOCS-RS to assess treatment differentiation, a component of implementation integrity. The TPOCS-RS includes 5 theory-based subscales (Cognitive, Behavioral, Psychodynamic, Client-Centered, and Family). Using the TPOCS-RS, coders independently rated 954 sessions conducted with 89 children (M age = 10.56, SD = 2.00; age 7-15 years; 65.20% White) diagnosed with a primary anxiety disorder who received different treatments (manual-based vs. nonmanualized) across settings (research vs. practice). Coders produced reliable ratings at the item level (M intraclass correlation coefficient = .76, SD = .18). Analyses support the construct validity of the Cognitive and Behavioral subscale scores and, to a lesser extent, the Psychodynamic, Family, and Client-Centered subscale scores. Correlations among the TPOCS-RS subscale scores and between the TPOCS-RS subscale scores and observational ratings of the alliance and client involvement were moderate suggesting independence of the subscale scores. Moreover, the TPOCS-RS showed promise for assessing implementation integrity as the TPOCS-RS subscale scores, as hypothesized, discriminated between manual-guided treatment delivered across research and practice settings and nonmanualized usual care. The findings support the potential of the TPOCS-RS Cognitive and Behavioral subscales to assess treatment differentiation in implementation research. Results for the remaining subscales are promising, although further research is needed. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
    Full-text · Article · Oct 2014
Show more

Recommended publications

Discover more