Treatment Integrity in Psychotherapy Research: Analysis of the Studies and
Examination of the Associated Factors
Francheska Perepletchikova, Teresa A. Treat, and Alan E. Kazdin
Treatment integrity refers to the degree to which an intervention is delivered as intended. Two studies
evaluated the adequacy of treatment integrity procedures (including establishing, assessing, evaluating,
and reporting integrity; therapist treatment adherence; and therapist competence) implemented in psy-
chotherapy research, as well as predictors of their implementation. Randomized controlled trials of
psychosocial interventions published in 6 influential psychological and psychiatric journals were re-
viewed and coded for treatment integrity implementation. Results indicate that investigations that
systematically addressed treatment integrity procedures are virtually absent in the literature. Treatment
integrity was adequately addressed for only 3.50% of the evaluated psychosocial interventions. Journal
of publication and treatment approach predicted integrity implementation. Skill-building treatments (e.g.,
cognitive–behavioral) as compared with non-skill-building interventions (e.g., psychodynamic, nondi-
rective counseling) were implemented with higher attention to integrity procedures. Guidelines for
implementation of treatment integrity procedures need to be reevaluated.
Keywords: treatment integrity, treatment fidelity, adherence, competence, treatment outcome
The main goals in treatment outcome research are specification
of a treatment and evaluation of its feasibility and efficacy. Inter-
pretations of treatment effects or the lack of treatment effects
require some assurance that the treatment was carried out as it was
designed (Kazdin, 2003). Treatment integrity (also known as treat-
ment fidelity) refers to the extent to which the intervention was
implemented as intended. Treatment integrity encompasses three
aspects: (a) therapist treatment adherence, the degree to which the
therapist utilizes prescribed procedures and avoids proscribed pro-
cedures; (b) therapist competence, the level of the therapist’s skill
and judgment; (c) and treatment differentiation, whether treat-
ments differ from each other along critical dimensions (e.g., Waltz,
Addis, Koerner, & Jacobson, 1993).
Therapist treatment adherence and treatment differentiation are
closely related. A measure of therapist treatment adherence is
sufficient for determination of whether treatments are in fact
distinct (Waltz et al., 1993). When therapists adhere closely to the
manual for each treatment (e.g., by implementing procedures pre-
scribed for Treatment A and by avoiding procedures prescribed for
Treatment B as well as other proscribed procedures), intervention
purity is preserved. Manipulation checks on treatment delivery
(i.e., assessment of treatment adherence) ensure that tasks pertain-
ing to each treatment do not overlap.
The relationship between therapist treatment adherence and
competence is less straightforward. Research examining the rela-
tionship between these two aspects has produced conflicting re-
sults, which range from no significant association (e.g., Paivio,
Holowaty, & Hall, 2004) to high correlation (e.g., Barber et al.,
2006; Shaw et al., 1999). Empirical examination of the association
between therapist treatment adherence and competence may be
challenging, due to the inherent conditionality between these two
aspects: competence presupposes adherence, but adherence does
not presuppose competence (McGlinchey & Dobson, 2003). Con-
ceptual distinction, on the other hand, is more evident. Whereas
adherence represents a quantitative aspect of treatment integrity
(how frequently the therapist implements procedures prescribed by
the manual and avoids those proscribed), competence is its qual-
itative aspect (how well prescribed procedures are implemented).
Even if adherent, therapists can deliver treatment in an incompe-
tent manner that threatens the validity of the interpretations about
the obtained outcome. Failure to evaluate competence may result
in an inability to establish which factors, treatment, or treatment
provider resulted in the treatment effect or lack of effect. As noted
by Nezu and Nezu (2005), the intervention may not equal the
The breakdown in treatment integrity may pose threats to the
experimental validity of a study and can have serious implications
for inferences drawn about the relationship between treatment and
Frankenberger, & Bocian, 2000; Kazdin, 2003; Moncher & Prinz,
1991). If a treatment was not executed as planned, it is not possible
to establish which manipulation (intervention or alternative fac-
tors) resulted in a change on dependent measures, which would
threaten the internal validity. Lack of treatment integrity can
hinder attempts to replicate the study and to evaluate its external
validity. Generality of the findings cannot be established without
an exact description of what has actually been done to the depen-
Francheska Perepletchikova, Teresa A. Treat, and Alan E. Kazdin,
Department of Psychology, Yale University.
This work was presented in part at the 18th Annual Convention of the
Association for Psychological Science, New York, New York, May 28,
2006, and was supported in part by the Robert M. Leylan Dissertation
Fellowship. We are very grateful to Susan Nolen-Hoeksema, Peter
Salovey, and Douglas Mennin for their intellectual contributions and to
Daniel J. Bauer for his help with statistical analyses.
Correspondence concerning this manuscript should be addressed to
Francheska Perepletchikova at email@example.com
Journal of Consulting and Clinical Psychology
2007, Vol. 75, No. 6, 829–841
Copyright 2007 by the American Psychological Association
dent variable. When an intervention is not provided as planned, the
construct validity of the experiment is also compromised. Impre-
cision in intervention delivery can cause ambiguity in evaluating
what the intervention was and why it produced the effect. Further,
when treatment is not implemented as intended, unsystematic error
may be introduced into the data, which compromises statistical
conclusion validity. By increasing the within-group variability,
such “noise” reduces the obtained effect size and statistical power
and thus decreases the likelihood of detecting the effect.
This report consists of two studies. Study 1 evaluated the ade-
quacy of treatment integrity procedures in the context of random-
ized controlled trials (RCTs) of psychotherapy published in influ-
ential psychiatric and psychological journals. Study 2 examined
factors that were potential correlates of the implementation of
integrity procedures, including treatment approach, corresponding
author’s educational background, the number of treatment com-
parisons, treatment characteristics, article type, and journal of
Study 1: Analysis of Treatment Outcome Studies
Multiple recommendations have been provided in the literature
on implementation of treatment integrity procedures (e.g., Carroll
& Nuro, 2002; Gresham, 1997; Gresham et al., 2000; Schlosser,
2002; Waltz et al., 1993). These recommendations can be divided
into four domains: establishing, assessing, evaluating, and report-
ing integrity. Establishing treatment integrity encompasses the
operational definition of an intervention and the training and
supervision of therapists. Treatment integrity depends on the com-
pleteness and clarity of the criteria that define the intervention
(Kazdin, 2003). Detailing treatments in a manual reduces the
variability in treatment implementation and enhances treatment
integrity (e.g., Drozd & Goldfried, 1996). However, clear and
unambiguous specification of the independent variable does not
ensure that the manipulation will be implemented as planned
without careful training of therapists. Training procedures can be
roughly divided into indirect and direct categories (e.g., Sterling-
Turner, Watson, Wildmon, Watkins, & Little, 2001). The indirect
category includes didactic instructions about the intervention and
written materials describing the rationale, scripts, and activities.
The direct category includes opportunities for practice and in-
volves procedures such as role-playing, rehearsal, feedback, and
periodic booster sessions. A faithful rendition of the treatment is
more likely with direct training procedures (e.g., Kratochwill,
Elliott, & Busse, 1995; Sterling-Turner, Watson, & Moore, 2002).
Therapists have to be supervised continuously to ensure accuracy
of treatment implementation and to reduce therapeutic drift, which
refers to gradual deviation from the treatment protocol (Kazdin,
Treatment integrity can be assessed via direct, indirect, and
hybrid strategies. Direct observations can be conducted by trained
staff present in the treatment setting, who view sessions through a
one-way mirror, via monitors, or by videotaping. Indirect methods
include therapist self-reports, debriefing clients on what was done
during the treatment sessions, written homework assignments, and
data collection sheets. Although these methods are less costly and
laborious than are direct strategies, they are subject to distortion in
self-representation, altered perception of the past, and poor recol-
lection. Research that relies primarily on indirect evaluations of
treatment integrity is likely to be weak in its ability to measure
integrity accurately. As indirect measures of integrity offer imme-
diate access to therapist adherence and to competence levels (Ber-
gan & Kratochwill, 1990; Gresham, 1989), they can be used to
supplement observational data and to adjust implementation (e.g.,
by directing therapist attention to omitted material and by encour-
aging the practice of inadequately executed procedures). Perfor-
mance feedback may increase integrity when low levels are de-
tected during treatment sessions (Codding, Feinberg, Dunn, &
Assessment of treatment integrity should encompass all three
aspects involved in its specification: therapist treatment adherence,
therapist competence, and treatment differentiation (Waltz et al.,
1993). Therapist treatment adherence measures are sufficient for
evaluation of treatment differentiation but only if they include
proscribed procedures (i.e., procedures to avoid, as they may dilute
intervention purity) as well as prescribed tasks. Therapist compe-
tence should not be assumed on the basis of experience and
training but rather should be verified independently by measure-
ment of how sensitively the treatment protocol is applied to indi-
vidual clients. Data on the validity and reliability of integrity
measures should be presented (see Perepletchikova & Kazdin,
2005 for discussion of validation methods).
Evaluation of treatment integrity encompasses procedures such
as ensuring the accuracy of the representation of the obtained
integrity data, training of raters, assessing interrater reliability, and
controlling for measure reactivity. Accuracy of the representation
of integrity levels depends upon the collection of data across
treatment phases, therapists, situations, sessions, and cases. For
example, some treatment phases (e.g., assessment of the pathol-
ogy) may be simpler than are others (e.g., training of skills).
Higher integrity ratings may be achieved when data are collected
primarily during the administration of more straightforward tasks.
Rater competence requires rigorous training in all of the major and
minor treatment components, including subtle aspects of the treat-
ment and the treatment manual. Thus, raters who are skilled in the
delivery of the treatment being evaluated seem to be the most
suitable for integrity rating. Regardless of who performs the rat-
ings, interrater reliability checks are important for ensuring ade-
quate assessment of integrity. Additionally, as any assessment may
be reactive, reactivity should be assessed and controlled. Thera-
pists’ self-reports can be biased and distorted by self-interest.
Observations can alter performance of the therapist and may result
in higher adherence to specified procedures during the observed
sessions. To ameliorate reactivity, staff can perform “spot checks”
of treatment implementation on a variable time schedule, where
therapists are interviewed by staff members at random times and
without notification. Also, reactivity may be lower when all of the
sessions are videotaped or observed, which may preclude the
fluctuation in integrity due to the presence or absence of an
Treatment integrity should be reported in terms of overall in-
tegrity, component integrity, and session integrity (Gresham, 1997;
Schlosser, 2002). Overall integrity reflects the integrity of treat-
ment components across sessions. Component integrity refers to
the integrity of implementing each treatment component across
sessions. Session integrity refers to the integrity of all treatment
components within each session. Although overall integrity may
be high, treatment may be implemented with low adherence and
PEREPLETCHIKOVA, TREAT, AND KAZDIN
scribed procedures along with prescribed tasks. Such examination
may increase awareness of the importance of treatment integrity
and may provide guidelines for ensuring adequacy of the imple-
Addressing treatment integrity is expensive and laborious. Fu-
ture research may evaluate ways to attain a satisfactory balance of
the costs and benefits of attending to integrity. For example, how
can one best achieve accurate representation of integrity data?
What are the optimal number and length of observations? Valida-
tion of integrity measures represents a particular challenge, as such
measures may encompass two separate constructs—therapist treat-
ment adherence and therapist competence. Additionally, treat-
ments may differ in their operational definitions, components, and
requirements for competent implementation. Does this mean that
integrity measures may have to be developed and validated spe-
cifically for each treatment, or can ways be devised to create more
general measures of integrity? Further empirical examination of
the relation between the different aspects of treatment integrity
might provide a much-needed insight into the question of the
incremental utility of evaluating various aspects of integrity.
The empirical evaluation of process-oriented psychotherapies
warrants greater consideration. With the increased use of empiri-
cally supported interventions in clinical practice and training, these
potentially efficacious treatments may become obsolete just be-
cause they resist empirical testing with the current methods. In-
deed, the predominant majority of validated treatments—85%—
are skill-building interventions (O’Donohue et al., 2000).
Future research might examine additional predictors of integrity
implementation. Such predictors might include treatment, therapist
and client characteristics, various barriers to implementation of
treatment integrity procedures (e.g., lack of editorial requirement),
and characteristics of the journal of publication (e.g., specific vs.
general scope, high vs. low impact factor, psychiatric vs. psycho-
logical). Future research might also evaluate whether studies that
more adequately address integrity procedures and achieve higher
integrity levels are more likely to find significant differences
between therapies. A demonstration that attending to treatment
integrity advances science, rather than merely meets a reporting
requirement, may serve as a powerful incentive for researchers to
adhere to treatment integrity regulations.
Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemination
of evidence-based practices: Addressing practitioners’ concern about
manual-based psychotherapies. Clinical Psychology: Science and Prac-
tice, 6, 430–441.
Barber, J. P., Gallop, R., Crits-Christoph, P., Frank, A., Thase, M. E.,
Weiss, R. D., & Connolly Gibbons, M. B. (2006). The role of therapist
adherence, therapist competence, and alliance in predicting outcome of
individual drug counseling: Results from the National Institute Drug
Abuse Collaborative Cocaine Treatment Study. Psychotherapy Re-
search, 16, 229–240.
Bergan, J., & Kratochwill, T. (1990). Behavioral consultation and therapy.
New York: Plenum Press.
Binder, J. (1993). Observations on the training of therapists in time-limited
dynamic psychotherapy. Psychotherapy, 30, 592–598.
Bohart, A. C., O’Hara, M., & Leitner, L. M. (1998). Empirically violated
treatments: Disenfranchisement of humanistic and other psychothera-
pies. Psychotherapy Research, 8, 141–157.
Borrelli, B., Sepinwall, D., Ernst, D., Bellg, A. J., Czajkowski, S., Breger,
R., DeFrancesco, C., Levesque, et al. (2005). A new tool to assess
treatment fidelity and evaluation of treatment fidelity across 10 years of
health behavior research. Journal of Consulting and Clinical Psychol-
ogy, 73, 852–860.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models:
Applications and data analysis methods. Newbury Park, CA: Sage.
Carroll, K. M., & Nuro, K. F. (2002). One size cannot fit all: A stage model
for psychotherapy manual development. Clinical Psychology: Science
and Practice, 9, 396–406.
Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun,
K. S., et al. (1998). Update on empirically validated therapies, II.
Clinical Psychologist, 51, 3–16.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported
therapies. Journal of Consulting and Clinical Psychology, 66, 1–18.
Codding, R. S., Feinberg, A. B., Dunn, E. K., & Pace, G. M. (2005).
Effects of immediate performance feedback on implementation of be-
havior support plans. Journal of Applied Behavior Analysis, 38, 205–
Crits-Christoph, P. (1997). Limitations of the Dodo bird verdict and the
role of clinical trials in psychotherapy research: Comment on Wampold
et al. (1997). Psychological Bulletin, 122, 216–220.
Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G., Knat-
terud, G. L., et al. (2003). Evidence-based behavioral medicine: What is
it and how do we achieve it? Annals of Behavior Medicine, 26, 161–171.
Drozd, J. F., & Goldfried, M. R. (1996). A critical evaluation on the
state-of-the-art in psychotherapy outcome research. Psychotherapy, 33,
Gresham, F. M. (1989). Assessment or treatment integrity in school con-
sultation and prereferral interventions. School Psychology Review, 18,
Gresham, F. M. (1997). Treatment integrity in single-subject research. In
R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and
analysis of single-case research (pp. 93–117). Mahwah, NJ: Erlbaum.
Gresham, F. M., Donald, L., MacMillan, D. L., Beebe-Frankenberger,
M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabil-
ities intervention research: Do we really know how treatments are
implemented? Learning Disabilities Research & Practice, 15, 198–205.
Gresham, F. M., Gansle, K. A., & Noell, G. H. (1993). Treatment integrity
in applied behavior analysis with children. Journal of Applied Behavior
Analysis, 26, 257–263.
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S.
(1993). Treatment integrity in school-based intervention studies: 1980–
1990. School Psychology Review, 22, 254–272.
Heck, R. H., & Thomas, S. L. (2000). An introduction to multilevel
modeling techniques. Mahwah, NJ: Erlbaum.
Henry, W. P. (1998). Science, politics, and the politics of science: The use
and misuse of empirically validated treatments. Psychotherapy Re-
search, 8, 126–140.
Johnston, J., & Pennypacker, H. S. (1980). Strategies and tactics of human
behavioral research. Hillsdale, NJ: Erlbaum.
Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.).
Boston: Allyn & Bacon.
Kendall, P. C., & Chu, B. C. (2000). Retrospective self-reports of therapist
flexibility in a manual-based treatment for youths with anxiety disorders.
Journal of Clinical Child Psychology, 29, 209–220.
Kratochwill, T. R., Elliott, S. N., & Busse, R. T. (1995). Behavioral
consultation: A five-year evaluation of consultant and client outcomes.
School Psychology Quarterly, 10, 87–117.
McGlinchey, J. B., & Dobson, K. S. (2003). Treatment integrity concerns
in cognitive therapy for depression, Journal of Cognitive Psychotherapy:
An International Quarterly, 17, 299–318.
Moher, D., Schulz, K. F., & Altman, D. (2001). The CONSORT statement:
Revised recommendations for improving the quality of reports of
PEREPLETCHIKOVA, TREAT, AND KAZDIN
parallel-group randomized trials. Journal of the American Medical As-
sociation, 285, 1987–1991.
Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome
studies. Clinical Psychology Review, 11, 247–266.
Nezu, A. M., & Nezu, C. M. (2005). Comments on “Evidence-based
behavior medicine: What it is and how do we achieve it?”: The inter-
ventionist does not always equal the intervention—the role of therapist
competence. Annals of Behavior Medicine, 29, 80.
O’Donohue, W., Buchanan, J. A., & Fisher, J. E. (2000). Characteristics of
empirically supported treatments. Journal of Psychotherapy Practice
and Research, 9, 69–74.
Paivio, S. C., Holowaty, K. A. M., & Hall, I. (2004). The influence of
therapist adherence and competence on client reprocessing of child
abuse memories. Psychotherapy: Theory, Research, Practice, Training,
Patterson, G. R., & Chamberlain, P. (1994). A functional analysis of
resistance during parent training therapy. Clinical Psychology: Science
and Practice, 1, 53–70.
Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and
therapeutic change: Issues and research recommendations. Clinical Psy-
chology: Science and Practice, 12, 365–383.
Peterson, L., Homer, A., & Wonderlich, S. (1982). The integrity of inde-
pendent variables in behavior analysis. Journal of Applied Behavior
Analysis, 15, 477–492.
Roberts, J. K., & Monaco, J. P. (2006, April). Effect size measures for the
two-level linear multilevel model. Paper presented at the annual meeting
of the American Education Research Association.
Rogers Wiese, M. R. (1992). A critical review of parent training research.
Psychology in the School, 29, 229–236.
Schlosser, R. W. (2002). On the importance of being earnest about treat-
ment integrity. Augmentative and Alternative Communication, 18, 36–
Shadish, W. R., & Ragsdale, K. (1996). Random versus nonrandom as-
signment in controlled experiments. Do you get the same answer?
Journal of Consulting and Clinical Psychology, 64, 1290–1305.
Shaw, B. F., Elkin, I. E., Yamaguchi, J., Olmsted, M., Vallis, T. M.,
Dobsson, K. S., et al. (1999). Therapist competence ratings in relation to
clinical outcome in cognitive therapy of depression. Journal of Consult-
ing and Clinical Psychology, 67, 837–846.
Snijders, T., & Bosker, N. (1999). Multilevel analysis: An introduction to
basic and advanced multilevel modeling. London: Sage Publications.
Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects
of direct training and treatment integrity on treatment outcomes in
school consultation. School Psychology Quarterly, 17, 47–77.
Sterling-Turner, H. E., Watson, T. S., Wildmon, M., Watkins, C., & Little,
E. (2001). Investigating relationship between training type and treatment
integrity. School Psychology Quarterly, 16, 56–67.
Tabachnick, B. G., & Fidell, S. L. (2001). Using multivariate statistics (4th
ed.). Needham Heights, MA: Allyn & Bacon.
Thomson ISI. (2002). Journal Citation Reports. Retrieved January 12,
2005, from www.isinet.com/products/evaltools/jcr/
Tinsley, H. E. A., & Weiss, D. J. (2000). Interrater reliability and agree-
ment. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied
multivariate statistics and mathematical modeling (pp. 95–124). San
Diego, CA: Academic Press.
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the
integrity of a psychotherapy protocol: Assessment of adherence and
competence. Journal of Consulting and Clinical Psychology, 61, 620–
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., &
Ahn, H. (1997). A meta-analysis of outcome studies comparing bona
fide psychotherapies: Empirically, “all must have prizes.” Psychological
Bulletin, 122, 203–215.
Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The em-
pirical status of empirically supported psychotherapies: Assumptions,
findings, and reporting in controlled clinical trials. Psychological Bul-
letin, 130, 631–663.
Wilson, G. T. (1998). Manual-based treatment and clinical practice. Clin-
ical Psychology: Science and Practice, 5, 363–375.
Received June 21, 2006
Revision received May 31, 2007
Accepted June 4, 2007 ?
Instructions to Authors
For Instructions to Authors, please consult the February 2007 issue of the volume or visit
www.apa.org/journals/ccp and click on the “Instructions to authors” link in the Journal Info box on
TREATMENT INTEGRITY IN PSYCHOTHERAPY RESEARCH