Content uploaded by Derek Milne
Author content
All content in this area was uploaded by Derek Milne
Content may be subject to copyright.
ORIGINAL PAPER
Supervising Cognitive-Behavioral Psychotherapy:
Pressing Needs, Impressing Possibilities
Robert P. Reiser
•
Derek Milne
Published online: 9 November 2011
Springer Science+Business Media, LLC 2011
Abstract While accounts of the practice of cognitive
behavioral therapy (CBT) supervision have been available
over the past 15 years and have demonstrated consistency
in terms of an overall model specifying the structure,
process and content of supervision sessions, a number of
gaps can be identified in the literature on the clinical
supervision of CBT. Comprehensive consensus statements
of the competencies underlying clinical supervision have
been developed both in the United States (Falender et al.,
J Clin Psychol, 60, 771–785, 2004) and in the United
Kingdom (Roth and Pilling, IAPT supervision competen-
cies framework, University College London, 1992), but
there has been a lack of attention to relevant theory and to
procedural detail. As a consequence, the literature on CBT
supervision has only recently begun to assimilate concepts
from the wider supervision field (e.g. developmental
models), and there is as yet no manual that would allow
supervisors to make explicit determinations of competency.
Furthermore, supervision in general continues to suffer
both from the absence of compelling empirical support
demonstrating its effectiveness in improving clinical out-
comes, and there is a lack of explicitly-defined supervisory
procedures that can be reliably observed and measured.
Unfortunately, this combination of factors hampers the
development of CBT supervision (e.g. it complicates
efforts to provide a standardized framework for training in
supervision). As an important starting point in ‘treating’ the
underdeveloped state of CBT supervision, we outline a
formulation of its current condition, then suggest some
promising interventions.
Keywords Competency Supervision
Cognitive behavioral therapy Training
What is the Current State of Supervising in Cognitive
Behavioral Psychotherapy?
In this review of the state of cognitive behavioral therapy
(CBT) supervision, we will address the current state of
knowledge about supervising CBT, identify pressing needs
that are stymieing further development of the CBT super-
vision model and practice, and finally point to future pos-
sibilities that might further unblock the potential for
improving training and practice related to CBT supervision.
Cognitive therapy or CBT was initially introduced in a
seminal text in 1979 (Beck et al. 1979) and the practice of
CBT supervision has been defined over the past 15 years in
key articles by Padesky (1996), Liese and Beck (1997),
Beck et al. (2008) and more recently by Newman (2010).
In many ways, the practice of CBT finds itself in a highly
advantageous position, as regards increasing acceptance by
governmental bodies (McHugh and Barlow 2010), insur-
ance, governmental and managed care mandates for
empirically-supported treatments with demonstrated effec-
tiveness, and broad empirical support, in terms of extensive
clinical trials covering a variety of diagnostic groups and
problems (Butler et al. 2006). To crown this progress, in
the UK, CBT is in the unique position of being the subject
of a governmental mandate, called ‘Improving Access to
Psychological Therapies’ (IAPT), which is designed to
enhance the availability of cognitive behavioral therapists,
R. P. Reiser (&)
Palo Alto University, Palo Alto, CA, USA
e-mail: robert.reiser3@gmail.com
D. Milne
Clinical Psychology, Newcastle University,
Newcastle upon Tyne, UK
e-mail: Derek.milne@newcastle.ac.uk
123
J Contemp Psychother (2012) 42:161–171
DOI 10.1007/s10879-011-9200-6
guided by trained supervisors (Improving Access to Psy-
chological Therapies [IAPT]; Clark et al. 2009).
Given this advantageous state of affairs for CBT involv-
ing a strong, dynamic and evolving evidence-base and
increasing acceptance by governmental, managed care and
insurance companies, we would expect that clinical super-
vision of CBT would be equally enriched and well-devel-
oped with a strong evidence-base as to its effectiveness in
improving training and client outcomes. Indeed, given the
accepted complexity of supervision and its growing
engagement with a competency-based model (Falender et al.
2004; Fleming 2004; Newman 2010; Roth and Pilling 1992),
we would expect that research into the effectiveness of
supervision of CBT, including a fine-grained analysis and
determination of what essential components are associated
with improved training and client outcomes, would be well
underway. In fact, as Milne and others have noted in a series
of reviews of supervision ranging from 1997 through 2008,
this is not at all the case and, based on the current state of
supervision research, this remains an elusive goal (Ellis and
Ladany 1997; Freitas 2002; Milne and James 2000; Milne
2008; Milne et al. 2010).
The current basis of CBT supervision practice largely
rests on descriptions of CBT supervision provided by Liese
and Beck (1997) and Padesky (1996) almost 15 years ago.
Surveys of practitioners have indicated that these two texts
continue to be seminal and authoritative as regards current
CBT supervision practices: ‘‘The most frequently cited
influences on… supervision practice were Padesky (1996)
and Liese and Beck (1997).’’ (Townend et al. 2002, p. 490).
These fundamental reference texts spelled out the overall
structure of supervision and created a template for best
practices that seems to have survived the decades largely
intact. A more recent update by Beck et al. (2008) closely
parallels these original texts. In summary, the structure of
supervision closely parallels the structure of a CBT session,
with the following elements: a check in, a bridge back to
the last session, setting the agenda, working through
agenda items, summarizing, assigning homework and get-
ting feedback. Further considerations are given to the
relationship aspects of supervision, the use of Socratic
questioning and guided discovery, the importance of case
formulation, and planning and adhering to a collaborative
model. One of the unique principles of CBT supervision
continues to be direct observation through audio or vid-
eotapes, and the use of the Cognitive Therapy Rating Scale
(CTRS: Young and Beck 1988)) to rate the therapists’ level
of competence, reflecting the empirical roots of CBT.
The literature delineating the elements of CBT supervi-
sion has tended to be largely descriptive, emphasizing
principles—relationship factors, collaboration, guided dis-
covery, structure (Padesky 1996)—rather than explicit
procedures and a corresponding rigorously manualized
approach. A more recent account of CBT supervision
(Newman 2010), while linking supervision to a competency
framework and Rodalfa’s cube model (Rodolfa et al.
2005)—specifying foundational and functional competen-
cies—does not provide sufficient detail to allow any real
specification of procedures. Even the most recent and
impressive effort by Roth and Pilling (1991), although
identifying broad areas of competencies for CBT supervi-
sion, notes the lack of reliable tools for objective mea-
surement, and the need for refinement of competencies into
anchored rating scales, in order to assess competence. In
many ways this echoes Milne and James’ earlier conclu-
sions in their 2000 review of effective CBT supervision:
they noted a gap in the use of instructional or training
manuals (‘‘clearly more manual development is required’’,
p. 122). It is quite clear that research on supervision inter-
ventions in general continues to lag badly behind psycho-
therapy research—these narrative accounts of supervision
would clearly be considered completely unacceptable
(unable to be reliably replicated) as compared to the type of
explicit procedurally detailed knowledge available in
treatment manuals. In turn, research on CBT supervision
lags behind the general field, an embarrassing situation,
given the empirical heritage of CBT (Milne 2008).
While the practice of CBT has been supported by a rich
tradition of empirical science, the supervision of CBT
continues to lack a compelling underlying empirical basis
(Ellis and Ladany 1997; Milne and James 2000; Lambert
and Ogles 2004; Wheeler and Richards 2007). This is
particularly at odds with CBT, which has distinguished
itself through a rigorous scientific tradition, including
adherence to techniques with an underlying method that
emphasizes empiricism, rapid development of manualized
versions of treatment with explicit attention to procedural
details, and insistence on consistent demonstrations of its
effectiveness in multiple, randomized controlled trials
(Butler et al. 2006). One could argue that each of the above
elements (commitment to scientific rigor, development of
explicit procedural details and demonstrations of effec-
tiveness) is absent as regards the current status of CBT
supervision. This appraisal is supported by a review pub-
lished in 2006, in which Armstrong and Freeston indicated
that ‘‘the base for efficacy [of CBT supervision] is very
limited’’ (p. 349) and could be contrasted sharply with the
strong evidence base for CBT. Roth and Pilling (1992)
have reached a similar conclusion.
This lack of development is unfortunate, in that our
current knowledge of what makes for effective practice in
CBT supervision is limited, and consequently our under-
standing of what constitutes effective training that
improves the outcomes of therapy is equally limited. Milne
(2008) has argued that the further development of CBT
supervision has suffered from a reflexive approach in
162 J Contemp Psychother (2012) 42:161–171
123
which supervision is constructed on a psychotherapy-based
model. He notes that this narrow approach underestimates
the complexity of supervision and needs to be supple-
mented with an enriched point of view incorporating
educational, developmental, and learning principles. Fur-
thermore, he argues that there is evidence of very limited
fidelity to the CBT model in our current practice. This
sorry state of affairs has actually persisted for some time,
so we should reiterate Watkins’ (1997) earlier conclusion,
one which he intended to apply to all psychotherapy-based
models of supervision:
Psychotherapy-based models of clinical supervision
have generally shown an amazing amount of stability
over the past 25 to 30 years, with their being no truly
new therapy-based theories of supervision emerging
and with even existing therapy-based models show-
ing limited changes or revisions within themselves.
(p. 570)
In summary, despite its acknowledged role in supporting
and guiding therapists, CBT supervision has developed
little over the last few decades, in marked contrast to CBT.
This is not to say that CBT supervision has been in any
sense ineffective or inappropriate to date, as it is noted that
there is much to commend the ‘reflexive’ basis of current
practice (Milne 2008). However, we do believe that CBT
supervision can be significantly enhanced, conceptually,
empirically and practically. In order to contribute to this
developmental task, we next turn our attention to the
challenges that we face in taking CBT supervision forward,
in keeping with the same rigorous scientific tradition that
prevails in CBT more generally (Butler et al. 2006).
What are the Most Pressing Developmental Tasks
Facing CBT Supervision?
Defining the Competencies of CBT Supervision
Significant recent advances have occurred in defining the
competencies of CBT supervision. The IAPT initiative has
included an initial effort to define the competencies asso-
ciated with delivering the major psychological treatments
(Roth and Pilling 2008) with a detailing of the competen-
cies associated with supervising these treatments (Roth and
Pilling 1992). By using expert consensus panels, combined
with a review of training procedures that have established
effectiveness in controlled clinical trials, the IAPT initia-
tive has addressed some of the weaknesses in the super-
vision competencies model proposed by Falender et al.
(2004), as the latter relied solely on expert consensus.
Furthermore, the IAPT consensus statement ‘‘A Compe-
tence Framework for the Supervision of Psychological
Therapies’’ clearly and transparently delineates its sources
for competency statements, in addition to identifying
expert reference group members and citing references.
Finally, the approach was more broadly interdisciplinary,
by contrast with the sole reliance on professional psy-
chologists in the Falender et al. (2004) consensus
statement.
Some notable additional features in the IAPT compe-
tency framework that add value to prior supervision com-
petency frameworks (Falender et al. 2004) and CBT
specific supervision frameworks (Beck et al. 2008; New-
man 2010) include the emphasis on gauging supervisee
competence though both direct observation and the use of
session-by-session, standardized outcome monitoring tools.
These include objective measures of client progress and
client self-report; a stronger consideration of the value of
direct observation, including the use of audio and video
recordings in supervision; and a fuller consideration of the
use of educational principles in enhancing learning.
Possible limitations to the IAPT approach to defining
supervision competencies include the fact that supervisory
competencies may be constrained unnecessarily to the
types of trials chosen (e.g. would trials involving clients
with psychosis call for completely similar training pack-
ages, or might there be unique features to supervising CBT
with psychosis not present in CBT with anxiety or
depressive disorders?). A second potential limitation is the
fact that supervision competencies have been selected from
supervision packages that may not be fully representative
of the full developmental learning spectrum necessary for
supervisors who work with very inexperienced trainees, or
trainees in community settings. It is possible that the
framework could become unbalanced and unwieldy from a
developmental training perspective, when confined to
randomized controlled trials with highly selected patient
groups in academic settings. Indeed, unlike the comparable
model within the United States, that develops competency
expectations for psychologists (Fouad et al. 2009), there is
no clear specification of supervision competencies expec-
ted at different training levels. Milne (2008, 2009) has also
addressed this issue of the need for a developmentally-
informed learning model in detail.
In their concluding remarks on a recent review of psy-
chotherapy-based supervision, Falender and Shafranske
(2010) make a similar observation as to the direction that
will need to be taken in terms of operationalizing
competencies:
This work will be advanced by employing a compe-
tency-based model of supervision (Falender and
Shafranske 2004; Kaslow et al. 2004) in which the
component aspects of competencies (i.e., knowledge,
skills, attitudes/values) are operationally identified,
J Contemp Psychother (2012) 42:161–171 163
123
approaches to self-assessment and evidence-based
assessment formative assessment are developed (see
e.g., Milne 2009; Reiser and Milne 2009) and a range
of learning strategies are employed. Consistent with
the call for competency-based standards, the field
of clinical supervision is challenged to respond.
(Falender and Shafranske 2010, p. 49).
Efforts to Standardize CBT Supervision Training
There have been initial attempts at defining standards for
training in order to achieve competency in supervision,
including CBT supervision (e.g. American Psychological
Association-APA 2007; Falender et al. 2004). However,
these efforts are hampered by the fact that there is a lack of
international consensus in the field as to what constitutes
minimal requirements for training in supervision. Despite
the availability of a consensus statement on a competency
framework for the provision of supervision in psychology
in the United States since 2004 (Falender et al. 2004), there
is still considerable divergence of opinion as to what
comprises competent training in supervision. Falender
et al. (2004) identified six core competencies, comprising
requisite knowledge, skills, values, ‘‘social context over-
arching issues’’ (p. 778), training, and ongoing assessment
of supervision competencies. In this consensus statement,
several very generic components for effective training and
assessment of supervision were identified: coursework,
supervision of supervision including some form of live
supervision with feedback (Falender et al. 2004, p. 778).
Whilst this is similar to two UK consensus statements
(British Psychological Society 2009; Roth and Pilling
2008), important differences exist concerning the required
competencies. There appears to be greater consensus
regarding the best methods for training supervisors to attain
these competencies (Falender et al. 2004; Kaslow et al.
2004; Milne et al. 2009). However, underscoring our
concern over the variability in consensus statements, sur-
vey data suggest wide variability in what is actually cov-
ered within workshops, at least in the UK (Fleming 2004).
This also appears to be a concern in the United States, as
there seems to be a continuing gap between consensus
statements on achieving competency in supervision and
actual practices in doctoral training programs. In particular,
consensus recommendations as to requiring some form of
observation (audio, video, live) and feedback do not appear
to us to have gathered steam within the practice and
training spheres. In a recent survey by Rings et al. (2009),
training directors in the US differed substantially on
endorsements of what constitutes adequate training, with
training directors only modestly endorsing a model con-
sistent with Falender et al.’s competency statement (2004).
Specifically, when training directors were asked to endorse
supervision training competencies, there was only weak
agreement with the items: ‘‘Supervisor has completed
coursework in supervision’’ and ‘‘Supervisor has received
supervision of his or her supervision, including some form
of observation (audio or video) with critical feedback about
supervision of supervision’’. In a post-hoc analysis, Rings
et al. determined that training directors’ preferences were
associated with their own level and type of training in
supervision, suggesting that the field has a long way to go
in terms of overcoming prejudices and biases inherent in
our own training backgrounds.
To indicate that the lack of consensus is not restricted to
the US and the UK, a second recent survey of directors of
clinical training in Canadian psychology programs (Hadji-
stavropoulos et al. 2010) also highlighted a lack of stan-
dardization in training for supervisors. While about 50% of
the responding programs required at least some specified
coursework, the number of hours required was highly var-
iable, ranging from 3 to 39. While the use of videotapes for
reviewing supervision was highly rated as potentially the
most helpful resource, it was apparent that few programs—
only 20% of respondents—used either audio or videotapes
of supervision in training The authors concluded: ‘‘The
picture that emerged is that training in clinical psychology
supervision is in its infancy in Canada, with approximately
40% of the respondents indicating that they do not require
structured coursework or a required practicum in clinical
supervision.’’ (Hadjistavropoulos et al. 2010, p. 209).
In summary, this state of affairs closely parallels the
zeitgeist of ‘‘remarkable neglect of the development pro-
cess and evaluation of supervisor competence’’ (p. 72)
cited by Falendar and Shafranske (2004). These authors
also noted wide variations in the availability, consistency
and quality of supervision training and evaluation methods
in graduate counseling and clinical psychology programs.
In short, it appears that little has changed in the status of
supervision training efforts in graduate training programs
in the past 10 years. If anything, we can conclude that there
is a continuing bias in the field along the lines noted by
Falender and Shafranske (2004): ‘‘It has been assumed that
any adequate clinician can be an adequate supervisor, that
one learns supervision skills from one’s own supervisors,
that one can simply learn by doing’’ (p. 72)
In the treatment fidelity model espoused by Bellg et al.
(2004), improving the training of providers has distinct
fidelity improvement strategies aimed at standardizing the
training received, measuring skill acquisition and pre-
venting drift (p. 446). Standardization of training com-
prises a number of elements, including ‘‘using standardized
training materials, conducting role-playing, and observing
actual intervention and evaluating adherence to protocol.’’
(p. 447). If we apply to this model to supervision training,
164 J Contemp Psychother (2012) 42:161–171
123
we would need to have a well-standardized curriculum
involving standard training materials (e.g., a supervision
training manual specifying procedures), make use of
structured practice including role-play and live observa-
tion, and then measure participant skill acquisition through
standardized checklists (Bellg et al. 2004, p. 447). Even in
the IAPT’s most recent effort at dissemination of compe-
tencies (IAPT, Supervision Guidance 2011), it is clear that
a number of these features, especially a procedural level
manual and a reliable competency checklist have yet to be
developed:
Currently, there are no reliable tools for assessing the
competences of supervisors. Claimed competence
within therapy may not be related to objective com-
petence as measured independently from therapy
tapes (e.g. Brosan, Reynolds, & Moore 2006). The
Supervision Competence Framework may help in the
development of rating scales that could be applied to
taped supervision sessions in order to assess super-
visors’ competence. (p. 10)
In conclusion, it appears that we have a long way to go
in terms of developing a parallel process for training
supervisors that would be sufficiently robust to withstand
standards routinely applied to clinical trials. However,
some preliminary work, following the fidelity framework,
suggests that the above requirements can be addressed:
Milne (2010) reported a successful dissemination of a
supervisor training manual in the UK, when evaluated
simply in terms of the acceptability of the approach to the
25 trainers and 256 supervisors involved. Additional
developments are summarized below, in our discussion of
some of the ways that we might enhance CBT supervision.
Next we continue our review of fidelity issues, turning our
attention to how supervision is delivered.
Can We Develop a Consensus on What Constitutes
Effective Supervision?
Despite the above consensus statements on a competency
framework for the provision of supervision and for training
in supervision, surveys that tap into actual supervision
practices in the field suggest further challenges in achiev-
ing high fidelity. For instance, Townend et al. (2002) sur-
veyed cognitive behavioral therapists accredited by the
British Association of Behavioural and Cognitive Psycho-
therapists (BABCP) in the UK (later replicated in Townend
et al. 2007), in order to get an accurate picture of their
supervision practices in the field. It should be noted that
cognitive behavioral therapists accredited by BABCP
undergo a rigorous accreditation both for qualification to
practice and for qualification to supervise. There is no
comparable accreditation process in the United States for
supervisors, with the exception of the Academy of Cog-
nitive Therapy, which credentials cognitive behavioral
therapists based on a review of experience, and submission
of an audiotaped treatment session, along with a cognitive
case formulation. Hence, it would be expected that a survey
of this BABCP group would give a picture of best practices
in the field.
Respondents to Townend et al.’s survey indicated that
they generally followed the overall format and model of
CBT supervision as specified in Padesky (1996) and Liese
and Beck (1997), including structured sessions (i.e. goals
and an agenda) which addressed case formulation, prob-
lem-solving, therapeutic techniques, as well as the thera-
peutic relationship. While about 50% of respondents
reported using agenda-setting, personal goal-setting, dis-
cussion of cognitive processes and homework tasks
‘‘sometimes or ‘‘often’’, only a minority (less than 20%)
used more active techniques such as role-play, and only 5%
of respondents reported using direct observation review of
audio or video tapes ‘‘often’’. It is interesting to note that
only 64% of these respondents reported receiving any form
of supervision training themselves, as only a minority
(17%) had completed a formal post-graduate training
course in supervision (p. 492). Overall, the picture is one of
relatively poor adherence and poor fidelity to the CBT
supervision model espoused by Padesky (1996) and Liese
and Beck (1997), with especially weak adherence in the
area of active techniques such as role-play, and the use of
audio or video tapes with a formal rating system (e.g. the
CTRS) of competence with feedback to the therapist. This
is illustrated by the conclusion drawn by Townend et al.
(2002):
The most surprising result was the limited use made
by experienced CBPs of audio/video tapes or real-life
observation and review of actual practice using
instruments such as the CTS. What this means is that
supervision is mainly about what supervisees say they
are doing in practice and not what may actually be
taking place. The supervisee could therefore drift
from acceptable practice without the therapist or their
supervisor being cognizant that practice standards
have degraded, if indeed they had been established in
the first place. (p. 497)
This same observation of poor fidelity to the model was
also echoed and reaffirmed in Milne and James’ (2000)
review, which suggested that managing and agenda-setting
were observed very infrequently; and again in a review by
Milne (2008
), which underscored the rarity of active,
experiential learning within the supervision session. In
conclusion, it appears that, despite the development of
consensus statements on supervision starting almost
10 years ago, and despite widely-accepted accounts of
J Contemp Psychother (2012) 42:161–171 165
123
CBT supervision, there remains a gap between the theory,
as represented in the literature (including competency
statements and consensus recommendations), and the cur-
rent state of supervisory practice.
To summarize the ‘pressing needs’ that we have iden-
tified, the good news is that there are widely-accepted
statements regarding the CBT supervision model (Padesky
1996; Liese and Beck 1997) and several expert consensus-
statements from the UK and the US which are compatible
with that model. Furthermore, naturalistic studies of CBT
supervision suggest that it can be effective in developing
the supervisee’s competence and in promoting clinical
effectiveness (Milne 2008, 2009). The bad news is that the
CBT supervision model has yet to be operationalized (e.g.
in the form of a supervisor training manual), and the actual
practice of CBT supervision does not accurately reflect our
current knowledge-base about what might constitute
effective practice. There is also no generally accepted
method for training supervisors, and consensus statements
lack the specificity and procedural detail to serve as useful
templates for competent supervisory practice. Furthermore,
there is no generally-accepted method for credentialing
supervisors in CBT, with the exception of the BABCP
credentialing process. In particular, there is no published,
psychometrically adequate observational measure available
to operationalize or capture elements of competent super-
vision at a behavioral level. To make matters worse,
supervisors in practice appear reluctant to make use of
supervision formats (audio, video, live) in which formal
measures might be applied to establish and enhance com-
petency in CBT. By contrast with the model, much of the
content and process of CBT supervision as it is currently
practiced appears to be discursive discussions of case for-
mulation and case conceptualization, and it therefore lacks
enactive and observational features. This means that
experiential learning is likely to be severely limited (e.g. no
specific, observationally-based or contiguous feedback).
One might conclude that this disappointing state of affairs
reflects a systemic ‘failure to thrive’, with a paradoxical
lack of commitment to research and development within
the key CBT organizations (e.g. BABCP; ACT), under-
achievement within the local supervisor training arrange-
ments, and developmental delay within individual CBT
supervision (Milne 2008). We now turn from this bleak
developmental diagnosis to some options for curing or
‘hot-housing’ CBT supervision, in terms of these same
three levels of the system.
What are the Future Possibilities for Supervising CBT?
Despite two consensus-based efforts to define the compe-
tencies of supervision (Falender et al. 2004; Roth and
Pilling 2008), these statements have not been sufficiently
detailed to allow us to create an instrument for the evalu-
ation of CBT supervision. In fact, Falender et al. (2004)
anticipated this potential outcome and noted that:
Because there has been only minimal effort devoted to
assessment of outcomes or of supervision process or
results, the task is monumental to translate competencies
into measurable criteria. Included within such a task would
be additional identification and clarification of the con-
structs comprising supervision competencies. (p. 781)
As an illustration of how research and development
could be brought to bear on the deficiency in national
leadership, we outline developments in defining the
supervision competencies through nationally-prepared,
CBT supervision guidelines; we note the national piloting
of a manual for supervisor training; and we describe a new
instrument that has been specifically designed to evaluate
CBT supervision.
Guidelines were discussed in a sister paper within this
issue (Milne and Reiser, in press), where it was noted that
guidelines are ‘a set of statements that recommend specific
professional conduct’ (APA 2002, p. 1048), a tool for
assisting supervisors to function optimally. To summarize
some of the underlying work referenced in that paper, we
previously (Milne and Dunkerley 2010) reviewed the evi-
dence for clinical supervision, developed a model of clin-
ical supervision, seeking multi-disciplinary professional
and service-user input at every stage of the guideline
development process. Based on this process, four guide-
lines were designed and evaluated (i.e. the supervisory
relationship; assessing learning needs and collaborative
agenda-setting; facilitating learning; and evaluation: Milne
and Dunkerley 2010). An acceptability evaluation sug-
gested that the overall rating for all guidelines was in the
‘acceptable’ range, mid-way to the best available rating of
‘good’. Also, the criterion of accuracy was met by all four
guidelines. These guidelines were used within the follow-
ing account of a national supervisor training initiative,
where they received further positive reactions.
In order to address this need for the manualization of
training, Milne (2010) has piloted a supervision training
manual for workshop leaders. The training manual included
didactic materials (e.g. a PowerPoint slideshow, summa-
rizing the relevant literature), DVD clips of appropriate
supervision (i.e. consistent with the guidelines), and a set of
program evaluation tools. One group of trainers was ran-
domly assigned to receive telephone-based consultation
prior to the workshop, whilst the second group worked from
the manual alone. The 25 trainers and 256 supervisors rated
the manual favorably overall, with the supervisors in the
manual plus consultation group rating their workshop lead-
ers more favorably in terms of ‘competence’, ‘interpersonal
effectiveness’ and ‘motivation’ than the manual-only group.
166 J Contemp Psychother (2012) 42:161–171
123
Finally, Milne and Reiser (2008) have developed an
instrument designed to reliably detect supervisor and trai-
nee behaviors within the supervision session. This direct
observational tool, called ‘‘Supervision, Adherence and
Guidance Evaluation’’ (SAGE) is an empirically-derived
instrument that assesses the competence of supervisors on
23 behaviors pertinent to effective supervision. SAGE has
evolved from two closely-related instruments: Teacher’s
PETS (Milne et al. 2002) and CBT STARS (Blackburn et al.
2001). SAGE allows an observer to systematically code
supervision sessions, so that judgments about competence,
feedback and fidelity evaluations can be conducted.
In summary, we have noted three examples of how we
might develop CBT supervision at the national level of the
supervisory system. All three examples are decidedly
preliminary pieces of research and development, so are
only intended to illustrate the ‘impressing possibilities’ for
CBT supervision at this level. We next consider the
intermediate level, that of the local organization (e.g. a
clinical psychology Doctoral training program), in relation
to developing the way that we train CBT supervisors. The
final section considers the most specific level of the system,
the individual supervisor.
Strategies to Improve Training in Supervision
What are some of the future possibilities for CBT super-
visor training, ones that have the potential to enhance
training effects and fidelity to an empirically-supported
training model? Just as we suggest that the CBT supervi-
sion model can be enhanced by drawing on the wider lit-
erature (e.g. developmental models), so there is a critical
need to revamp our training systems to bring them in line
with empirically-supported methods within the staff train-
ing literature, as these methods are likely to enhance
training outcomes.
In a recent comprehensive review of training in evi-
dence-based practices (EBP), Beidas and Kendall (2010)
concluded that training clearly influences therapist knowl-
edge, attitudes and self-reported behavior, but that actual
behavioral changes were harder to achieve:
Importantly, training engenders self-reported behav-
ior change that does not always match actual
behavior change—pointing to the importance of
including independently-rated behavior in all future
assessments of EBP training. The mismatch in per-
ception and actual behavior is highly problematic and
has important implications because clinicians may
believe they are implementing a treatment with
fidelity and skill (Miller and Mount 2001). (p. 25)
It appeared from their findings that actual behavior
change was associated with both the quality and method of
training: active learning strategies, including coaching and
feedback, were noted to be particularly effective (p. 25). It
is worth looking in detail at one of the reviewed studies that
demonstrated behavioral changes (Miller et al. 2004). In
this randomized trial, trainees assigned to five training
conditions were compared using performance on audio-
taped work samples at baseline, 4, 8 and 12 months post-
intervention. Trainees who had been randomized to a 2-day
clinical workshop-only condition showed immediate
improvement post-training, but then deterioration in skills
at 4 months. The most significant improvements occurred
for the group that was assigned to the workshop plus
feedback and ongoing coaching, which involved an aver-
age of 5 telephone contacts over the period of the study. It
is clear from the results of this study that acquisition of
procedural knowledge is an active, experiential process,
one which requires ongoing effort and so probably cannot
be achieved within didactic formats.
Similarly, psychotherapy training studies (presumably
analogous to supervisory training) have consistently shown
that coursework and other didactic approaches do not result
in enduring behavioral changes (Miller and Mount 2001;
Miller et al. 2004; Sholomskas et al. 2005; Fixsen et al. 2005;
Rakovshik and McManus 2010; Beidas and Kendall 2010).
In their recent comprehensive review of evidence-based
training in CBT, Rakovshik and McManus (2010) concluded
‘‘Results from didactic instruction only would seem to sup-
port the conclusion often voiced in training reviews: tradi-
tional and primarily didactic forms of dissemination are
necessary, but insufficient in producing sustained change in
therapist behavior and patient outcome.’’ (p. 511).
In keeping with this view, a parallel survey of training
and supervision methodologies within clinical trials, con-
ducted by Roth et al. (2010), noted that effective training
procedures typically involved a rich mix of didactic and
experiential methods combined with direct observation,
including fidelity checklists and standardized competence
ratings. In short, stable behavioral changes are more likely
to occur with active learning involving enriched opportu-
nities for observation, modeling along with coaching, and
individually-tailored feedback that is guided by standard-
ized ratings of competence.
Clearly, against these evidence-based training criteria,
CBT supervisor training is still under development, suffer-
ing from an outdated and unproven training methodology.
These findings have important implications for the
training of CBT supervisors and suggest that our typical
strategies involving workshops alone (typically emphasiz-
ing didactic methods to promote declarative knowledge)
will have little effect on changing supervisor behavior. To
illustrate, in a recent study of CBT therapists who had
received didactic training plus 1 year of weekly consulta-
tion (using the Cognitive Therapy Rating Scale to rate their
J Contemp Psychother (2012) 42:161–171 167
123
audiotapes) demonstrated significant reductions in depres-
sion for clients versus therapists in the TAU condition
(Simons et al. 2010). Useful new internet-based technolo-
gies which allow for rapid and secure transmission of digital
audio-taped material can be combined with periodic tele-
phone—based consultation and feedback, in order to
enhance learning and secure high fidelity to the supervision
model over time. This model has the potential to reduce
costs related to staff travel time and high resource costs of
on-site workshops, which entail loss of a full day of work
to attend.
In addition to improvements in the method of training,
the content of training could also be standardized, in order
to present the competencies of supervision systematically,
so disseminating evidence-based guidelines for supervision
in a consistent fashion. An additional developmental pro-
gression would be to add a suitable evaluation of super-
visor training. Even the most recent and promising efforts
in this arena (Guidance for Commissioning IAPT Super-
visor Training, Revised 2011) acknowledged this deficit,
alongside the obstacles to remediation:
It would also be desirable for the competencies of
supervisors attending these courses to be assessed and
evaluated…. Such assessments, whilst highly desir-
able, are demanding of course staff time and would
need to be adequately resourced. Moreover, reliable
scales of supervision competence have yet to be
developed and agreed.’’ (p. 3)
According to the treatment fidelity model (Bellg et al.
2004), in addition to the goal of standardizing training, we
should also consistently measure skill acquisition over
time, to prevent competence drift (p. 446). Bellg et al. also
suggested observing the actual intervention (i.e. the train-
ing), measuring participant skill acquisition, and evaluating
adherence to an intervention protocol. Through direct
observation and the use of standardized checklists (Bellg
et al. 2004). A small-scale but explicit application of this
fidelity framework has suggested that this kind of sys-
tematic evaluation of supervisor training is feasible, albeit
with weak instrumentation in this ‘case study’ (Culloty
et al. 2010).
Strategies to Enhance Learning in Supervision
The third and final ‘future possibility’ that we consider is at
the level of the individual supervisor. In some striking
ways, a set of recommendations can be applied to the
individual practice of supervision that parallel those noted
above in relation to training: in both cases there is a need to
foster experiential learning. Once again, there is a strong
consensus that certain types of supervisory interventions
are more likely to be effective. These include educational
role play, enactive methods, review of audio and video
tapes in every session using a CTRS-type checklist to
rating competency, and giving constructive, detailed
feedback. We will now substantiate this statement. Milne
(2008) has discussed several specific enhancements to the
traditional practice of CBT supervision, including aug-
menting traditional CBT agenda-setting into a mini-learn-
ing needs assessment in each supervision session, adding
an explicit problem-solving cycle, enhancing the experi-
ential aspects of supervision (including the deliberate
processing of emotional relations in supervision), and
applying broader educational principles based on Kolb’s
(1984) experiential learning model (Milne 2008). As
already noted, these principles were incorporated into an
evidence-based training manual, together with other ele-
ments required to develop CBT supervision (Milne 2009).
In some ways, this method elaborates Shanfield et al.’s
(1992) earlier findings, based on independent ratings of
videotapes of supervision, using the Psychotherapy Super-
vision Inventory. ‘Excellent’ supervisors were rated higher
in levels of empathy, tended to be focused on the concerns
of the resident/trainee, and had a more experiential focus.
Bennett-Levy (2006) has suggested a comparable and
useful framework for the acquisition of therapist skills which
involves a tripartite model, involving declarative, proce-
dural, and reflective knowledge. According to this frame-
work, declarative knowledge is viewed as insufficient to
develop procedural and reflective knowledge. The declara-
tive system concerns factual knowledge (e.g. ‘knowing that
something is true’), whereas procedural knowledge concerns
‘how to’ and ‘when to’ rules. Experienced therapists use
procedural knowledge to make determinations about
appropriate interventions. Bennett-Levy’s description of
how therapists acquire skills suggests an approach to
supervision quite consistent with the suggestions we can
glean from the empirical literature on supervisor training:
Taking the example of cognitive therapists learning
to create successful behavioural experiments, novice
therapists may learn these skills through a series of
teaching strategies: a brief lecture, and classroom
demonstration, followed by a role-play setting up a
behavioral experiment with another trainee and get-
ting feedback. Next, they transfer these newfound
skills to clinical situations. With repeated use, eval-
uation and feedback, they refine these basic skills
until they become relatively automatic and fluent.
Didactic learning, modelling, practice and feedback
therefore form the key learning mechanisms for the
relative newcomer. (Bennett-Levy 2006, p. 59)
In a second paper, Bennett-Levy et al. (2009) identified
six common training and supervision methods: modeling/
demonstration, reading, role-play, self-experiential work,
168 J Contemp Psychother (2012) 42:161–171
123
lectures, and reflective practice. He argued that the Declar-
ative, Procedural Reflexive (DPR) knowledge model can be
applied in order to optimize training and supervision.
Summary and Conclusions
While there is a pressing need and even well-funded gov-
ernment initiatives to enhance CBT supervision, there are
systemic barriers to its development. We identified illus-
trations at the national, local and individual levels of the
supervisory system. Specifically, national efforts to define
and operationalize the specific competencies of CBT
supervision appear to be at an early, ‘arrested’ stage of
development, both in the UK and the US. Locally, our
training strategies for supervisors have yet to coalesce
around a convincing empirical base that offers the promise of
effective training. Lastly, the individual practice of CBT
supervision does not appear to be performed with fidelity to
the model, as defined by Padesky (1996) and Liese and Beck
(1997). Harshly, one might conclude that CBT supervision is
still in an infantile state of development. This is particularly
unsatisfactory, viewed in the context of the strong empirical
heritage of CBT. This state of affairs echoes a general
refrain, first voiced by Watkins in his seminal text ‘‘Hand-
book of Psychotherapy Supervision’’ (Watkins 1997): ‘‘If
psychotherapy supervision is really all that important, then
why is training on how to supervise and become a supervisor
so limited?…Something does not compute.’’ (p. 604). To
encourage healthy if retarded development, we have high-
lighted some promising possibilities, in relation to each of
these three levels of activity. At the national level there are
some promising signs of progress, such as the dissemination
of carefully-prepared CBT supervision guidelines, and a tool
for evaluating CBT supervision competencies. Locally,
there is now a supervisor training manual with good
acceptability to trainers and supervisors. And at the level of
the individual supervisor we have noted the encouraging
growth potential of experiential methods, allied to classic
aspects of CBT, such as objective measurement and cor-
rective feedback. We conclude that these systemic impedi-
ments to the development of CBT supervision can be
overcome, enabling CBT supervision to achieve its poten-
tial. As ever in such large-scale innovations, effective
leadership is vital, and we should therefore give especial
attention to the IAPT approach to supervisor development.
References
American Psychological Association (APA). (2002). Guidelines on
multicultural education, training, research, practice, and orga-
nizational change for psychologists. Washington, DC: Author.
American Psychological Association. (2007). Assessment of compe-
tency benchmarks work group: A developmental model for the
defining and measuring competence in professional psychology.
Washington DC: Author.
Armstrong, P. V., & Freeston, M. H. (2006). Conceptualising and
formulating cognitive therapy supervision. In N. Tarrier (Ed.),
Case formulation in cognitive behaviour therapy: The treatment
of challenging and complex clinical cases. London: Routledge.
Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive
therapy of depression. New York: Guilford.
Beck, J., Sarnat, J. E., & Barenstein, V. (2008). Psychotherapy- based
approaches to supervision. In C. Falender & E. Shafranske
(Eds.), Casebook for clinical supervision—a competency based
approach. Washington, DC: American Psychological
Association.
Beidas, R. S., & Kendall, P. C. (2010). Training therapists in
evidence-based practice: A critical review of studies from a
systems-contextual perspective. Clinical Psychology: Science
and Practice, 17, 1–30.
Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory,
M., et al. (2004). Enhancing treatment fidelity in health behavior
change studies: Best practices and recommendations from the
NIH Behavior Change Consortium. Health Psychology, 23,
443–451.
Bennett-Levy, J. (2006). Therapist skills: A cognitive model of their
acquisition and refinement. Behavioural and Cognitive Psycho-
therapy, 34, 57–78.
Bennett-Levy, J., McManus, F., Westling, B. E., & Fennell, M.
(2009). Acquiring and refining CBT skills and competencies:
Which training methods are perceived to be most effective?
Behavioural and Cognitive Psychotherapy, 37, 571–583.
Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S.,
Garland, A., et al. (2001). The Revised Cognitive Therapy Scale
(CTS-R): Psychometric properties. Behavioural and Cognitive
Psychotherapy, 29(4), 431–447.
British Psychological Society. (2009). Directory for applied psychol-
ogy practice supervisors. Leicester: BPS.
Brosan, L., Reynolds, S., & Moore, R. G. (2006). Factors associated
with competence in cognitive therapists. Behavioural and
Cognitive Psychotherapy, 35, 179–190.
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006).
The empirical status of cognitive-behavioral therapy: A review
of meta-analyses. Clinical Psychology Review, 26, 17–31.
Clark, D. M., Layard, R., Smithies, R., Richards, D. A., Suckling, R.,
& Wright, B. (2009). Improving access to psychological therapy:
Initial evaluation of two UK demonstration sites. Behaviour
Research and Therapy, 47, 910–920.
Culloty, T., Milne, D. L., & Sheikh, A. I. (2010). Evaluating the
training of clinical supervisors: A pilot study using the fidelity
framework. The Cognitive Behaviour Therapist, 3, 132–144.
Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees
and clients in clinical supervision: An integrative review. In C.
E. Watkins (Ed.), Handbook of psychotherapy supervision (pp.
447–507). Chichester: Wiley.
Falender, C., Cornish, J. A. E., Goodyear, R., Hatcher, R., Kaslow, N.
J., Leventhal, G., et al. (2004). Defining competencies in
psychology supervision: A consensus statement. Journal of
Clinical Psychology, 60, 771–785.
Falender, C. A., & Shafranske, E. P. (2004). Clinical supervision: A
competency-based approach. Washington, DC: American Psy-
chological Association.
Falender, C., & Shafranske, E. (2010). Psychotherapy-based super-
vision models in an emerging competency-based era: A
commentary. Psychotherapy Theory, Research, Practice, Train-
ing, 47, 45–50.
J Contemp Psychother (2012) 42:161–171 169
123
Fixsen, D.L., Naoom, S.F., Blase
´
, K.A., Friedman, R.M., & Wallace,
F. (2005). Implementation research: A synthesis of the literature.
Tampa, FL: University of South Florida, Louis de la Parte
Florida Mental Health Institute, The National Implementation
Research Network (FMHI Publication #231).
Fleming, I. (2004). Training clinical psychologists as supervisors. In
I. Fleming & L. Steen (Eds.), Supervision and clinical psychol-
ogy: Theory, practice and perspectives. New York: Brunner-
Routledge.
Fouad, N. A., Grus, C. L., Hatcher, R. L., Kaslow, N. J., Hutchings, P.
S., Madson, M., et al. (2009). Competency benchmarks: A model
for the understanding and measuring of competence in profes-
sional psychology across training levels. Training and Education
in Professional Psychology, 3, S–S26.
Freitas, G. J. (2002). The impact of psychotherapy supervision on
client outcome: A critical examination of two decades of
research. Psychotherapy: Theory/Research/Practice/Training,
39, 354–367.
Hadjistavropoulos, H., Kehler, M., & Hadjistavropoulos, T. (2010).
Training graduate students to be clinical supervisors: A survey of
Canadian professional psychology programs. Canadian Psychol-
ogy, 51(3), 206–212.
IAPT Education and Training Group (2011). Guidance for Commis-
sioning IAPT Supervisor Training (Revised 2011). Retrieved
from IAPT website: http://www.iapt.nhs.uk/silo/files/guidance-
for-commissioning-iapt-supervisor-training-revised-2011.pdf.
Kaslow, N. J., Borden, K. A., Collins, F. L., Jr., Forrest, L., Illfelder-
Kaye, J., & Nelson, P. D. (2004). Competencies conference:
Future directions in education and credentialing in professional
psychology. Journal of Clinical Psychology, Special Issue, 60,
699–712.
Kolb, D. A. (1984). Experiential learning. Englewood-Cliffs, NJ:
Prentice-Hall.
Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness
of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s
handbook of psychotherapy and behavior change. New York:
Wiley.
Liese, B. S., & Beck, J. S. (1997). Cognitive therapy supervision. In
C. E. Watkins (Ed.), Handbook of psychotherapy supervision
(pp. 114–133). Chichester: Wiley.
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and
implementation of evidence-based psychological treatments: A
review of current efforts. American Psychologist, 65, 73–84.
Miller, W. R., & Mount, K. A. (2001). A small study of training in
motivational interviewing: does one workshop change clinician
and client behavior? Behavioural and Cognitive Psychotherapy,
29, 457–471.
Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano,
M. (2004). A randomized trial of methods to help clinicians learn
motivational interviewing. Journal of Consulting and Clinical
Psychology, 72(6), 1050–1062.
Milne, D. L. (2008). CBT supervision: From reflexivity to special-
ization. Behavioural and Cognitive Psychotherapy, 36, 779–786.
Milne, D. L. (2009). Evidence-based clinical supervision. Chichester:
BPS Blackwell.
Milne, D. L. (2010). Can we enhance the training of clinical
supervisors? A national pilot study of an evidence-based
approach. Clinical Psychology and Psychotherapy, 17, 321–328.
Milne, D. L., & Dunkerley, C. (2010). Towards evidence-based
clinical supervision: The development and evaluation of four
CBT guidelines. The Cognitive Behaviour Therapist, 3, 43–57.
Milne, D. L., & James, I. A. (2000). A systematic review of effective
cognitive-behavioural supervision. British Journal of Clinical
Psychology, 39, 111–127.
Milne, D. L., James, I. A., Keegan, D., & Dudley, M. (2002).
Teacher’s PETS: A new observational measure of experiential
training interactions. Clinical Psychology and Psychotherapy, 9,
187–199.
Milne, D. L. & Reiser, R. (2008). Supervision: Adherence and
guidance evaluation (SAGE). Unpublished instrument, available
from the first author.
Milne, D. L., & Reiser, R. (in press). A rationale for evidence-based
clinical supervision. Journal of Contemporary Psychotherapy.
Milne, D. L., Reiser, R., Aylott, H., Dunkerley, C., Fitzpatrick, H., &
Wharton, S. (2010). The systematic review as an empirical
approach to improving CBT supervision. International Journal
of Cognitive Therapy, 3, 278–294.
Milne, D.L., Scaife, J., & Cliffe,T. (2009). How should we train
effective supervisors? A British consensus on facilitating
experiential learning. Clinical Psychology Forum, 203.
Newman, C. F. (2010). Competency in conducting cognitive–behav-
ioral therapy: Foundational, functional, and supervisory aspects.
Professional Psychology: Research and Practice, 47(1), 12–19.
Padesky, C. A. (1996). Developing cognitive therapist competency:
Teaching and supervision models. In P. M. Salkovskis (Ed.),
Frontiers of cognitive therapy. London: Guilford Press.
Rakovshik, S. G., & McManus, F. (2010). Establishing evidence-
based training in cognitive behavioral therapy: A review of
current empirical findings and theoretical guidance. Clinical
Psychology Review, 30, 496–516.
Reiser, R.., & Milne, D. L. (2009, November). Assessing competence
in CBT supervision. Clinical roundtable presented at the annual
meeting of Association of Behavioural and Cognitive Therapies,
New York, NY.
Rings, J. A., Genuchi, M. C., Hall, M. D., Angelo, M., & Erickson
Cornich, J. A. (2009). Is there consensus among predoctoral
internship training directors regarding clinical supervision com-
petencies? A descriptive analysis. Training and Education in
Professional Psychology, 3, 140–147.
Rodolfa, E., Bent, R., Eisman, E., Nelson, P., Rehm, L., & Ritchie, P.
(2005). A cube model for competency development: Implica-
tions for psychology educators and regulators. Professional
Psychology: Research and Practice, 36(4), 347–354.
Roth, A., & Pilling, S. (1992). IAPT supervision competencies
framework. Retrieved from University College London.
http://www.ucl.ac.uk/clinical-psychology/CORE/supervision_
framework.htm.
Roth, A., & Pilling, S. (2008). A competence framework for the
supervision of psychological therapies. Retrieved March 30,
2009, from http://www.ucl.ac.uk/clinical-psychology/CORE/
supervision_framework.htm.
Roth, A., Pilling, S., & Turner, J. (2010). Therapist training and
supervision in CBT in major trials for depression and anxiety.
Behavioral and Cognitive Psychotherapy, 38(3), 291–302.
Simons, A. D. et al. (2010). Training and dissemination of cognitive
behavior therapy for depression in adults: A preliminary
examination of therapist competence and client outcomes.
Journal of Consulting and Clinical Psychology, 78(5), 751–756.
Shanfield, S. B., Mohl, P. C., Mathews, K. L., & Hetherly, V. (1992).
Quantitative assessment of the behavior of supervisors. Amer-
ican Journal of Psychiatry, 149(3), 352–357.
Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S.
A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain:
A dissemination trial of three strategies of training clinicians in
cognitive-behavioral therapy. Journal of Consulting and Clinical
Psychology, 73, 106–115.
Townend, M., Iannetta, L., & Freeston, M. H. (2002). Clinical
supervision in practice: A survey of UK cognitive-behavioural
psychotherapists accredited by the BABCP. Behavioural and
Cognitive Psychotherapy, 30, 485–500.
Townend, M., Iannetta, L., Freeston, M. & Hayes, J. (2007). Supervi-
sion practices of UK cognitive-behavioural psychotherapists,
170 J Contemp Psychother (2012) 42:161–171
123
2001 and 2006. Paper presented at the 5th World Congress of
Behavioural and Cognitive Therapies, Barcelona, 11–14 July.
Watkins, C. E. (Ed.). (1997). The handbook of psychotherapy
supervision. Chichester: Wiley.
Wheeler, S., & Richards, K. (2007). The impact of clinical
supervision on counselors and therapists, their practice and their
clients. A Systematic review of the literature. Counselling and
Psychotherapy Research, 7(1), 54–65.
Young, J. & Beck, A.T. (1988) The cognitive therapy rating scale
manual (revised). Philadelphia, PA: Center for Cognitive
Therapy, University of Pennsylvania.
J Contemp Psychother (2012) 42:161–171 171
123
