Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors

ArticleinJournal of Consulting and Clinical Psychology 75(6):829-41 · January 2008with27 Reads
DOI: 10.1037/0022-006X.75.6.829 · Source: PubMed
Treatment integrity refers to the degree to which an intervention is delivered as intended. Two studies evaluated the adequacy of treatment integrity procedures (including establishing, assessing, evaluating, and reporting integrity; therapist treatment adherence; and therapist competence) implemented in psychotherapy research, as well as predictors of their implementation. Randomized controlled trials of psychosocial interventions published in 6 influential psychological and psychiatric journals were reviewed and coded for treatment integrity implementation. Results indicate that investigations that systematically addressed treatment integrity procedures are virtually absent in the literature. Treatment integrity was adequately addressed for only 3.50% of the evaluated psychosocial interventions. Journal of publication and treatment approach predicted integrity implementation. Skill-building treatments (e.g., cognitive-behavioral) as compared with non-skill-building interventions (e.g., psychodynamic, nondirective counseling) were implemented with higher attention to integrity procedures. Guidelines for implementation of treatment integrity procedures need to be reevaluated.
    • "An RCT design as such cannot even guarantee that the experimental conditions (i.e., the treatment packages that were implemented) conform to the treatment models that are to be contrasted. The latter requires adequate measures of treatment integrity, including adherence, competence and differentiation [50]. Training the therapists in TM-specific manuals before they are allowed to take part as therapists in the actual trial does not guarantee that the treatments they carry out as part of the trial show adequate adherence, competence or differentiation. "
    [Show abstract] [Hide abstract] ABSTRACT: Background Randomized controlled trials (RCTs) are considered the best methodology for studying the efficacy of psychotherapy. Optimally an RCT design makes it possible to conclude that if one treatment has a better outcome than another, this is due to the treatment package (TP) as it was implemented in this particular context, rather than other factors beyond the treatment (= high internal validity). Strong internal validity does not, however, provide evidence for the treatment model (TM) that provides the theoretical basis of the TP, because the TP that is tested may differ from the comparison condition in a number of other ways that suggest alternative explanations for the effects. These alternative treatment contrasts represent threats to construct validity of the conclusions. Maximal construct validity requires (1) that the treatments are clearly contrasted on the experimental factors (treatment integrity), and (2) that alternative treatment contrasts can be eliminated. The analysis of alternative explanations is a neglected topic in psychotherapy research. To approach this problem, a methodology for the analysis of treatment contrasts is suggested and tested. Methods Two indexes were defined: (1) a Treatment Integrity Index (TII) and (2) an Alternative Treatment Contrast Index (ATCI). This methodological approach was applied to eight comparative RCTs of treatments for Borderline Personality Disorder (BPD), which were coded for a set of treatment contrasts independently by three coders. ResultsThe analysis of the RCTs of treatments for BPD showed that construct validity differed widely between the different studies but was generally low (low TII and ATCI), and that it is therefore difficult to draw causal conclusions from this research. The publication policies of scientific journals in this area seldom require the systematic data relevant to an analysis of alternative explanations of the effects, which is needed to provide evidence for a particular TM. Conclusions Research on psychotherapy needs to be refocused from treatment packages (TP) to treatment models (TM). This requires an improved conceptualization of the methodological principles and skills involved, and the development of valid measures of these, but also improved reporting standards concerning treatment-construct validity in scientific journals.
    Full-text · Article · Dec 2016
    • "Research to improve client outcomes in community mental health has been hindered by an inability to accurately and inexpensively measure fidelity [1]. Fidelity includes adherence, or how closely the components of a protocol are followed, and competence, or how skillfully the components are implemented and how responsive the therapist is to the client and situation [2]. Fidelity has been identified in implementation science frameworks as the mechanism by which desired outcomes are achieved [3][4][5], and as an indicator of quality of care [6]. "
    [Show abstract] [Hide abstract] ABSTRACT: Background This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods. Methods/designThis trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews. DiscussionResults will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice. Trial registrationNCT02820623, June 3rd, 2016.
    Full-text · Article · Dec 2016
    • "A detailed treatment protocol has been published previously (Ford et al., 2011). Multiple strategies were implemented to ensure treatment fidelity (Borrelli et al., 2005; Perepletchikova et al., 2007) including trial physiotherapists being provided with a detailed manual, 2- days of training and standardised electronic clinical notes (Ford et al., 2011; Hahne et al., 2011). The clinical notes were reviewed by the primary researcher twice during treatment providing individual feedback as well as in monthly meetings. "
    [Show abstract] [Hide abstract] ABSTRACT: Low back disorders are prevalent and directional preference management is a common treatment with mixed evidence for effectiveness. To determine the effectiveness of individualised directional preference management plus guideline-based advice versus advice alone in participants with reducible discogenic pain of 6-week to 6-month duration. Pre-planned secondary analysis of a multicentre, parallel group randomised controlled trial. Participants were randomly allocated to receive a 10-week physiotherapy program of 10-sessions of individualised directional preference management plus guideline-based advice (n=40) or 2-sessions of advice alone (n=38). Primary outcomes were back pain, leg pain and activity limitation. Outcomes were taken at baseline and 5, 10, 26, and 52-weeks. Between-group differences significantly favoured directional preference management compared with advice for back pain at 5-weeks (1.28; 95% CI 0.34 to 2.23) and 10-weeks (1.45; 95% CI 0.51 to 2.40), and leg pain at 10-weeks (1.21; 95% CI 0.04 to 2.39). These short-term differences were not maintained. There were no significant differences between-groups for activity limitation. Secondary outcomes and responder analyses favoured directional preference management suggesting between-group differences were clinically important. In people with reducible discogenic pain, individualised directional preference management plus guideline-based advice resulted in significant and rapid improvement in short-term back and leg pain compared with advice alone. These effects were not maintained at long-term and there were no differences in activity limitation. Individualised directional preference management could be considered for patients with reducible discogenic pain seeking rapid pain relief however further research is indicated.
    Article · Jun 2016
Show more

Recommended publications

Discover more