Introduction: Using Deliberate Practice to Improve Supervision and Training

Full text

3
The Cycle of Excellence: Using Deliberate Practice to Improve Supervision and Training,
First Edition. Edited by Tony Rousmaniere, Rodney K. Goodyear, Scott D. Miller, and Bruce E. Wampold.
© 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.
1
An ounce of practice is worth more than tons of preaching.
—Mahatma Gandhi
Over the past century, dramatic improvements in performance have been experienced in
sports, medicine, science, and the arts. This is true, for example, in every Olympic sport
(e.g., Lippi, Banfi, Favaloro, Rittweger, & Maffulli,2008). College athletes in running,
swimming, and diving perform better than gold medal winners from the early Olympic
Games (Ericsson,2006). In medicine, the number of diseases that can be treated effec-
tively has steadily increased, while mortality from medical complications has decreased
(Centers for Disease Control,2012; Friedman & Forst,2007). In mathematics, calculus
that previously required decades to learn is now taught in a year of high school
(Ericsson,2006). In the arts, modern professional musicians routinely achieve or exceed
technical skill that previously was attainable only by unique masters like Mozart (Lehmann
& Ericsson,1998).
Unfortunately, the same cannot be said of mental health treatment. Although the number
and variety of psychotherapy models have grown rapidly, the actual effectiveness of psycho-
therapy has not experienced the dramatic improvements seen in the fields described (Miller,
Hubble, Chow, & Seidel,2013). For example, in modern clinical trials, cognitive behavioral
therapy appears to be less effective than was demonstrated in the original trials from the
1970s (Johnsen & Friborg,2015). That we have remained on this performance plateau is
clearly not due to a lack of desire for improvement—virtually all mental health clinicians
want to be more effective. So what have we been missing? How can we get better at helping
our clients? In this book, we outline procedures that lead to increasing the effectiveness of
psychotherapy.
Introduction
Tony Rousmaniere, Rodney K. Goodyear, Scott D. Miller, and Bruce E. Wampold

1 Introduction
4
The Overall Effectiveness of Psychotherapy
First, let’s step back to examine the big picture concerning the effectiveness of psychothera-
pists. Good news: The consistent finding across decades of research is that, as a field, we
successfully help our clients. Studies examining the effectiveness of clinicians working
across the field, from community mental health centers, to university counseling centers, to
independent practice, show that, on average, mental health clinicians produce significant
positive change for their clients (Lambert,2013; Wampold & Imel,2015). The average psy-
chologically distressed person who receives psychotherapy will be better off than 80% of the
distressed people who do not (Hubble, Duncan, & Miller,1999; Wampold & Imel,2015).
Dozens of studies show that the effects of psychotherapy and counseling are at least as large
as the effects of psychotropic medications and that psychotherapy and counseling are less
expensive, have fewer troubling side effects, and last longer (Forand, DeRubeis, &
Amsterdam,2013; Gotzsche, Young, & Crace,2015).
Opportunity for Improvement
Although the big picture is positive, there is room for improvement. For example, in clinical
trials, only 60% of clients achieve clinical “recovery,” and between 5% and 10% actually dete-
riorate during treatment (Lambert, 2013). The percentage of clients who terminate care
prematurely falls between 20% and 60%, depending on how “prematurely” is defined (Swift,
Greenberg, Whipple, & Kominiak,2012), and these rates have remained largely unchanged
for the past five decades.
Furthermore, there is considerable between‐clinician variability in effectiveness. Whereas
the most effective therapists average 50% better client outcomes and 50% fewer dropouts
than therapists in general (Miller et al., 2013), these “super shrinks” (Miller, Hubble, &
Duncan,2007) are counterbalanced by those therapists who produce, on average, no change
or may even cause most of their clients to deteriorate (Baldwin & Imel,2013; Kraus, Caston-
guay, Boswell, Nordberg, & Hayes,2011; Wampold & Brown,2005). So there is clear room
for many therapists to demonstrably increase their effectiveness.
How, then, can clinicians become more effective? Some may assume that the best way to
get better at something is simply to do it a lot. A significant body of research documents
thatmusicians, chess players, and athletes, in the correct circumstances, improve with time
and experience (at least up to the point of competency; Ericsson & Pool,2016). However,
psychotherapy is a field in which practitioners’ proficiency does not automatically increase
with experience (Tracey, Wampold, Goodyear, & Lichtenberg, 2015; Tracey, Wampold,
Lichtenberg, & Goodyear,2014). Two large studies have shown that “time in the saddle” itself
does not automatically improve therapist effectiveness (Goldberg, Rousmaniere etal., 2016;
Owen, Wampold, Rousmaniere, Kopta, & Miller,2016). One of these studies, based on the
outcomes of 173 therapists over a period of time up to 18 years, found considerable variance
in the outcomes achieved by the therapists over time. Although some of the therapists were
able to continually improve, client outcomes on average tended to decrease slightly as the

Current Strategies for Improving Effectiveness 5
therapists gained more experience (Goldberg, Rousmaniere et al., 2016). Another study
examined the change in outcomes of 114 trainees over an average of 45 months. As in the
Goldberg, Rousmaniere et al. (2016) study, in the Owen et al. (2016) study, there was consid-
erable variance in the outcomes achieved by trainees over time. Although trainees, on
average, demonstrated small‐size growth in outcomes over time, this growth was moderated
by client severity, and some trainees demonstrated worse outcomes over time, leading
theauthors to observe that “trainees appear to have various trajectories in their ability to
fosterpositive client outcomes over time, and at times not a positive trajectory” (p. 21).
Current Strategies for Improving Effectiveness
What accounts for the failure to improve? Answering that question requires first looking at
the four most widely used methods for improving therapist effectiveness: supervision, con-
tinuing education (CE), the dissemination of evidence‐based treatments, and outcome
feedback systems.
Supervision provides trainees with important professional preparation. For example,
supervision has been shown to provide basic helping skills, improve trainees’ feelings about
themselves as therapists and understanding about being a therapist, and enhance trainees’
ability to create and maintain stronger therapeutic alliances, the component of therapy
most associated with positive outcomes (e.g., Hill et al., 2015; Hilsenroth, Kivlighan, &
Slavin‐Mulford,2015; Wampold & Imel,2015). However, evidence concerning the impact
of supervision—as it has been practiced—on improving client outcomes is mixed at best
(Bernard & Goodyear, 2014; Rousmaniere, Swift, Babins‐Wagner, Whipple, & Ber-
zins, 2016). Indeed, prominent supervision scholars (e.g., Beutler & Howard, 2003;
Ladany,2007) have questioned the extent to which supervision improves clinical outcomes.
Summarizing the research in this area, Watkins (2011) reported, “[W]e do not seem to be
any more able now, as opposed to 30 years ago, to say that supervision leads to better out-
comes for clients” (p. 252).
Continuing education (CE) (“further education” in the United Kingdom) is a second
method for improving, or at least maintaining, therapist effectiveness. Many jurisdictions
require CE to maintain licensure, certification, or registration necessary for practice. CE is
commonly delivered via a passive‐learning format, such as lecture or video (perhaps with
some discussion). This format may be effective at imparting knowledge about particular
topics (laws, ethics, new treatments, etc.), but typically it includes little interactive practice
or corrective feedback for participants and thus has questionable impact on actual skill
development. Research from CE in medicine has demonstrated that passive‐learning for-
mats have “little or no beneficial effect in changing physician practice” (Bloom, 2005,
p. 380). Summarizing concerns about the limits of CE, Neimeyer and Taylor (2010)
reported, “Acentral concern follows from the field’s failure to produce reliable evidence
that CE translates into discernibly superior psychotherapy or outcomes, which serves
as the cornerstone of the warrant underlying CE and its related commitment to the welfare
of the consumer” (p. 668).

1 Introduction
6
A third prominent method for improving therapist effectiveness that has gained consider-
able momentum over the past half century is the dissemination of evidence‐based treat-
ments (EBTs, also called empirically supported treatments or psychological treatments with
research support). Using EBTs to improve the quality of mental health care is based on a
two‐step process: (a) clinical trials are used to determine which specific therapy models are
effective for treating specific psychiatric disorders, and (b) these models are disseminated by
training therapists to be competent in the EBTs. Over the years, hundreds of EBTs have been
tested in clinical trials for an ever‐increasing range of disorders, and the results of these trials
commonly show EBTs to be more effective than no treatment. However, there is a paucity of
evidence that becoming competent in EBTs improves the effectiveness of individual thera-
pists in actual practice (Laska, Gurman, & Wampold,2014). For example, Branson, Shafran,
and Myles (2015) found no relationship between cognitive behavioral therapy competence
and patient outcome. In fact, large studies frequently show that clinicians in general practice
achieve the same outcomes as those deemed competent in clinical trials (Wampold &
Imel,2015). In a meta‐analysis of clinical trials comparing an EBT to a treatment‐as‐usual
condition, Wampold et al. (2011) showed that when treatment as usual involved legitimate
psychotherapy, the outcomes of treatment as usual and EBT were not statistically different.
Notably, clinical trials often show more variability in outcomes among clinicians than
between treatments, suggesting that more attention is needed to skill acquisition by indi-
vidual clinicians (based on their personal clients’ outcome data) across all treatment models
(Baldwin & Imel,2013; Miller et al.,2007; Wampold & Imel,2015). In summary, competence
in evidence‐based treatment models does not appear to be itself sufficient for improving the
effectiveness of psychotherapy by individual clinicians in actual practice.
A fourth method for improving therapist effectiveness that has been increasingly
adopted over the past two decades is feedback systems, also called practice‐based evi-
dence, in which clinicians monitor their clients’ progress by examining outcome data
session to session. Feedback systems have been shown to improve the quality of psycho-
therapy, in part by identifying and preventing failing cases (Lambert & Shimokawa,2011).
In fact, two feedback systems—the Partners for Change Outcome Management System
(PCMOS,2013) and OQ‐Analyst—have such a powerful impact on client outcome that
they are now considered an “evidence‐based practice” by the Substance Abuse and Mental
Health Services Administration. However, feedback systems have not been shown to lead
to the development of clinical expertise for individual therapists (Miller et al., 2013;
Tracey et al.,2014). That is, although therapists who receive feedback about particular
clients can alter the treatment for those particular clients, receiving the feedback does not
appear to reliably generalize to other cases or improve therapists’ overall clinical skills.
Each of these methods for professional improvement has clear value. However, despite
the attention that has been given to strengthening supervision and training (American Psy-
chological Association, 2015), CE (Wise et al.,2010), the dissemination of empirically based
treatments (McHugh & Barlow,2010), and routine clinical feedback (Lambert,2010), over-
all psychotherapy outcomes have not improved over the past 40 years (Miller et al.,2013).
Simply put, our field has lacked a successful model for therapist skill advancement. So, we
return to our question: How can clinicians become more effective? To help answer this
question, let’s look beyond our field and see what we can learn from others.

The Cycle of Excellence 7
The Science of Expertise
During the past two decades, a growing body of research has examined the methods pro-
fessionals use to attain expertise (e.g., Ericsson,1996,2009). The science of expertise has
been concerned with identifying how professionals across a wide range of fields—from
musicians, to chess players, to athletes, to surgeons—move from average to superior per-
formance. The findings confirm results cited earlier regarding the development of expertise
in psychotherapy: Simply accumulating work experience does not itself lead to expert per-
formance ( Ericsson,2006). Rather, researchers have identified a universal set of processes
that accounts for the development of expertise as well as a step‐by‐step process that can
be followed to improve performance within a particular discipline ( Ericsson, Charness,
Feltovich, & Hoffman,2006).
The Cycle of Excellence
Informed by findings reported by researchers (Ericsson,1996, 2009; Ericsson, Charness,
Feltovich, & Hoffman, 2006; Ericsson, Krampe, & Tesch‐Romer, 1993) and writers
(Colvin,2008; Coyle,2009; Shenk,2010; Syed,2010) on the subject of expertise, Miller et al.
(2007) identified three components critical for superior performance. Working in tandem
to create a “cycle of excellence,” these components include:
1. Determining a baseline level of effectiveness, including strengths and skills that need
improvement;
2. Obtaining systematic, ongoing, formal feedback; and
3. Engaging in deliberate practice. (See Figure1.1.)
A brief description of each step follows.
In order to improve, it is essential to know how well one fares in a given practice domain,
including strengths and skills that need improvement. Top performers, research shows, are
constantly comparing what they do to their own “personal best,” the performance of others,
and existing standards or baselines (Ericsson, 2006). As reviewed, in the realm of
Determine a baseline level
of effectiveness, including strengths
and skills that need improvement
Engage in
deliberate practice
Obtain systematic,
ongoing, formal feedbac
k
Figure 1.1 Cycle of Excellence.

1 Introduction
8
psychotherapy, numerous well‐established outcome measurement systems are available to
clinicians for assessing their baseline (Miller et al.,2013). Each of these systems provides
therapists with real‐time comparisons of their results with national and international norms
(Lambert, 2010; Miller, Duncan, Sorrell, & Brown,2005). Specific clinical strengths and
skills that need improvement can be identified by supervisors, trainers, or peers, depending
on the developmental level of the therapist.
The second element in the Cycle of Excellence is obtaining formal, ongoing feedback. Feed-
back comes from two sources: (a) empirical outcome measures and (b) coaches and teach-
ers—in psychotherapy, these often are referred to as supervisors—whose job it is to identify
the skills that need to be developed and provide specific suggestions and training experiences
specifically designed to enhance the individual’s performance. High‐level performers, it turns
out, both seek out and have more access to such mentoring from recognized experts
(Hunt,2006). As discussed earlier, research has shown that ongoing feedback from supervi-
sors can improve trainees’ clinical skills, such as the ability to build a strong therapeutic work-
ing alliance (e.g., Hill et al.,2015; Hilsenroth, Ackerman, Clemence, Strassle, & Handler, 2015).
Although feedback is necessary for improvement, it is not itself sufficient. Creating a
Cycle of Excellence requires an additional essential step: engaging in deliberate practice
(Ericsson,2006). Briefly, this type of practice is focused, systematic, and carried out over
extended periods of time. Generally, it involves identifying where one’s performance falls
short, seeking guidance from recognized experts, setting aside time for reflecting on feed-
back received, and then developing, rehearsing, executing, and evaluating a plan for
improvement (Ericsson,1996, 2006; Ericsson et al.,1993). Deliberate practice involves a
tight focus on repetitively practicing specific skills until they become routine. Because it
requires sustained concentration and continuous corrective feedback outside the trainee’s
comfort zone, deliberate practice typically is not enjoyable or immediately rewarding
(Coughlan, Williams, McRobert, & Ford, 2013; Ericsson & Pool,2016). Deliberate practice
intentionally causes a manageable level of strain to stimulate growth and adaptation: “[E]lite
performers search continuously for optimal training activities, with the most effective dura-
tion and intensity, that will appropriately strain the targeted physiological system to induce
further adaptation without causing overuse and injury” (Ericsson, 2006, p. 12). For these
reasons, deliberate practice is distinctly different from the two activities most common for
therapists: routine performance and passive learning, as illustrated in Table1.1.
How Much Practice Is Enough?
Elite performers across many different domains, including professional musicians, athletes,
and chess players, devote hours to deliberate practice every day, often including weekends
(Ericsson,1996,2006; Ericsson et al.,1993). Researchers have found that achieving expert
performance does not just take a few years of training but rather requires much more
effort—thousands of hours of deliberate practice, often requiring 10 to 30 years of sustained
effort and focus (Ericsson,2006). Furthermore, research indicates that continued deliberate
practice throughout the career span is required for maintenance of expert performance
(Ericsson,2006).

How Much Practice Is Enough 9
The concept of the “10,000‐hour” or “10‐year rule” was brought to popular awareness by the
book Outliers (Gladwell,2008), referring to the amount of time necessary to become an expert
in a field. (Research actually has found that the number of hours required for mastery varies
by field; Ericsson & Pool, 2016.) However, a common misconception is that thousands of
hours of routine work experience lead to expert performance. In contrast, researchers have
found something much more challenging: Thousands of hours of deliberate practice, on top of
hours spent in routine work performance, usually are required for expert performance.
Could the same process apply to mental health professionals? Chow, Miller, Seidel, Kane,
and Andrews (2015) recently examined this question by surveying a group of therapists
about the amount of time and effort they dedicated to deliberate practice. Their findings are
strikingly similar to what expertise researchers discovered about other fields: Highly effec-
tive therapists devoted 4.5 times more hours to activities specifically designed to improve
their effectiveness than less effective therapists (Chow et al.,2015). Figure1.2 compares the
findings about therapists from Chow et al. (2015) with the findings from a similar study
about violinists (Ericsson et al.,1993).
Unfortunately, to date, professional training programs have encouraged deliberate prac-
tice to a very limited extent, despite the recognition that training should be “sequential,
cumulative and graded in complexity” (Commission on Accreditation,2013, p. 7). Oppor-
tunities to engage in deliberate practice become even fewer once clinicians complete their
training. For most therapists, a serious focus on skill acquisition ends at the beginning of
their career, right after graduate school. As seen in Figure1.3, performance of the typical
therapist does not improve through the professional career (i.e., after professional training),
a result supported by longitudinal study of therapist outcomes (Goldberg, Rousmaniere
Table 1.1 Comparison of routine performance, passive learning, and deliberate practice.
Activity Definition Examples Goal Characteristics
Routine
performance
Simply
performing work
as usual
Providing therapy To earn an income
by providing
a service
Often feels
enjoyable and
immediately
rewarding
Passive
learning
Learning
without a practice
and feedback
component
Attending lectures
Reading about
psychotherapy
models
To build general
knowledge
about models, theories,
and skills
May be enjoyable
and feel
immediately
rewarding
Deliberate
practice
Repetitively
practicing specific
skills with
continuous
corrective
feedback
Reviewing videos
of therapy sessions
with expert
providing feedback
Repeatedly
role‐playing
solutions to mistakes
made in videotaped
sessions
To address knowledge
deficits specific to
therapist; works exactly
at therapist’s
performance threshold;
makes specific skills
routine and automatic
by moving performance
into procedural memory
Feels
challenging and
hard; not
inherently
enjoyable or
immediately
rewarding

1 Introduction
10
et al.,2016). It appears that students in graduate training acquire skills (e.g., Hill et al.,2015)
and improve their outcomes over the course of training, although the improvement in out-
comes may be quite gradual and not consistent (Owen et al.,2016). It is worth noting that
even for domains where expertise is clearly visible (e.g., musicians, athletes, chess players),
–200 12345678
300
800
1300
1800
2300
Accumulated Practice (hrs)
2800
3300
3800
Estimated Accumulation of Solitary Practice in the
First 8 Years of Professional Practice
Therapists
Year of Experience
0
46810 12
Age of the Musicians (Years)
14 16 18 20
1000
2000
3000
4000
5000
6000
7000
8000
Estimated Accumulated
Practice (Hours)
9000
10000
11000
Violinists
Professionals
Best Experts
Good Experts
Least Accomplished Experts
Amateurs
Pianists
Musicians
1st Quartile (N=7)
2nd, 3rd, &4th
Quartile Combined
(N=8)
Figure 1.2 Comparing the relationship between the hours of deliberate practice and improved
performance for therapists and violinists.
Sources: Chow et al. (2015, p. 342) and Ericsson et al. (1993, p. 379).

Bringing the Science of Expertise to Psychotherapy 11
few achieve a level recognized as expert. Many of us are passably good musicians (we might
sing or play guitar at gatherings or religious services), but we are clearly not in an elite
group. Those who are elite, regardless of natural talent, have engaged in deliberate practice.
Bringing the Science of Expertise to Psychotherapy
Our goal for this book is to bring the science of expertise to the field of mental health. We
do this by proposing a model for using the Cycle of Excellence throughout therapists’
careers, from supervised training to independent practice.
Stage 1: Deliberate Practice in Supervised Training
The first major stage of clinicians’ careers is intensive formal training, with the goal of achiev-
ing professional competency. Trainees in this stage work under supervision. Supervision, one
of the four methods of development discussed earlier, is a relationship in which a more senior
clinician monitors and guides a trainee’s work in order both to facilitate trainee development
and to ensure quality of client care (American Psychological Association, 2015; Bernard &
Goodyear,2014). Supervision provides a strong yet flexible relationship in which a seasoned
expert can identify errors and the skills necessary for improvement, on a case‐by‐case basis.
Supervisors can provide the essential ingredients for deliberate practice (McMahan,2014) by:
1. Explaining and demonstrating models for effective practice (e.g., cognitive behavioral
therapy or psychodynamic psychotherapy);
2. Determining each therapist’s zone of proximal development (i.e., their exact threshold of
understanding and opportunity for improvement);
3. Providing corrective feedback and guidance in style that is congruent and accessible to
the learner;
Expert
Performance
Arrested
Development
Deliberate
Practice
Typical Therapists
Graduate
School
Early
Career
Supervised
Practice
Independent Practice
Middle
Career
Late
Career
Hours of Deliberate Practice
Figure 1.3 Improved performance via deliberate practice.

1 Introduction
12
4. Offering emotional encouragement to boost the learner’s morale and buffer against the
emotional challenges inherent in deliberate practice (Duckworth, Kirby, Tsukayama,
Berstein, & Ericsson,2011); and
5. Teaching trainees how to work appropriately within various professional domains (clini-
cal, legal, administrative, etc.).
During their first few years of graduate school, trainees are not only learning their craft
but also being socialized into the culture of their field. Supervision is the perfect opportu-
nity to instill the habits and attitudes necessary for a “culture of expertise” that will help
clinicians use deliberate practice throughout their careers.
Stage 2: Deliberate Practice in Independent Practice
After clinicians complete their formal training and become licensed, they move into the sec-
ond (and final) major stage of their career: independent practice. At this point, they become
responsible for their own learning, which generally can be of several types (Lichtenberg &
Goodyear,2012): incidental learning (i.e., spontaneous, unplanned learning that might occur
through, e.g., reviewing a manuscript or hearing a radio interview with an expert); CE expe-
riences; and intentional, self‐directed learning. Deliberate practice concerns that third type
of learning and has the goals of maintaining competency and gradually developing mastery
of the craft. The mechanisms to support deliberate practice are varied in this st age, and include:
● advanced training with experts,
● skill assessment and case consultation with experts or peers, and
● solo study (e.g., watching videotapes of one’s own work).
Table1.2 describes the different goals, settings, areas of focus, and methods of deliberate
practice for each career stage.
Table 1.2 Deliberate practice goals, settings, areas, and methods across the career span.
Deliberate Practice
Goals Settings Areas of Focus Methods
Career
Stage 1:
Supervised
Training
Achieve
professional
competency
Under supervision Attain competency
in all basic skills
Videotape review,
clinical role‐plays,
assigned
homework, etc.
Career
Stage 2:
Independent
Practice
Assess skills,
maintain
competency,
develop expertise,
leading to mastery
of craft
In consultation with
experts, peers, and
solo study
Develop advanced
skills in areas of
specialty
Address specific
deficiencies
Videotape review
with experts,
peers, and by
oneself
Advanced training
with experts,
self‐study, etc.

About This Book 13
Sources of Motivation to Engage in Deliberate Practice
Students enter training programs in the mental health professions with excitement. They
are highly motivated to seek and capitalize on learning opportunities. But as Stoltenberg
and McNeill (2010) have discussed, students’ motivation fluctuates across time. It is our
impression that most clinicians remain intellectually curious throughout their professional
lives but, once they attain basic competence, the curiosity is manifest more in diffuse ways
than in focused ways. As discussed, deliberate practice is hard work, and learners typically
find it both challenging and inherently unpleasant (Duckworth et al.,2011; Ericsson,2006).
Researchers have identified a subset of very high‐achieving therapists who do engage in
deliberate practice (Miller et al.,2007,2013). They demonstrate grit, which is “perseverance
and passion for long‐term goals” (Duckworth, Peterson, Matthews, & Kelly,2007, p. 1087)
and have “the capacity to stay committed to a challenging, far‐off, but ‘sweet’ goal”
( Duckworth et al.,2011, p. 174). Indeed, Duckworth et al. (2011) found that level of grit
predicted the extent to which spelling bee competitors engaged in deliberate practice and,
in turn, how they performed. Given the challenges of sustaining internal motivation to
engage in the deliberate practice necessary to develop expertise, both institutional support
and effective mechanisms of accountability are essential to encouraging it (Goodyear,2015).
This is especially true for licensed clinicians who can be tempted to “coast” instead of engag-
ing in ongoing deliberate practice. Recent research at a community mental health center in
Canada has shown that agency‐wide support for deliberate practice, led by senior manage-
ment, can improve client outcomes (Goldberg, Babins‐Wagner et al.,2016). In this book, we
describe evidence‐based methods that treatment centers can use to support clinicians’
engagement in the Cycle of Excellence.
About This Book
The goals of this book are to provide clinicians and clinical supervisors with (a) the theory
of deliberate practice and the Cycle of Excellence, (b) a new model to integrate deliberate
practice into clinical training and independent practice, and (c) case examples of how delib-
erate practice is being used across a range of psychotherapy settings. This book is organized
into four parts.
Part I: The Cycle of Excellence reviews the science of clinical outcomes, expertise, and
supervision and proposes a new model for integrating deliberate practice into clinical prac-
tice at every stage of a career, from supervised training to independent practice.
Part II: Tracking Performance focuses on an essential ingredient of deliberate practice:
empirically tracking therapist effectiveness. In the field of mental health, this means meas-
uring client outcome, which has the full richness and complexity of the human experience.
The chapters in this part describe accessible methods supervisors and clinicians can use to
track client outcomes at the case, therapist, and agency levels, using both quantitative and
qualitative methods.
Part III: Applications for Integrating Deliberate Practice into Supervision explores inno-
vative programs for using deliberate practice to enhance psychotherapy training across a

1 Introduction
14
broad spectrum of areas, including psychodynamic psychotherapy, cognitive behavioral
therapy, agency‐level improvement, and CE. This part also includes a chapter that describes
how deliberate practice has been integrated into medical education, presented as a model
and learning opportunity for the field of mental health.
Part IV: Recommendations concludes the volume by pulling together the previous chap-
ters and proposing steps that can be taken to contribute to the mission of improving psy-
chotherapeutic expertise.
Questions from John Norcross, PhD
For each chapter in this volume, we editors have posed several questions to the authors that
a critical reader might ask. Answers by the chapter authors appear at the end of each chap-
ter. For those chapters in which one or two of us were authors, others of our team took the
role of asking challenging questions.
Because the four of us all were authors of this chapter, we reached outside the team and
asked John Norcross, a prominent psychotherapy researcher and trainer, to pose the ques-
tions to us. In his characteristic way, he asked questions that were both insightful and
rigorous.
Question #1. There is yet but a single research study attesting to the effectiveness of deliberate
practice among psychotherapists in routine care. You review the research literature on the
value of deliberate practice among other professionals, but those professions are notable for
working by themselves and with inanimate objects (e.g., chess pieces, musical instruments),
without the reciprocal influence of a client/patient. How do you respond to those who argue
that you are recommending a practice (and writing an entire book) well beyond the support-
ive research evidence with psychotherapists?
Answer from Editors: We wholeheartedly agree with this question’s underlying implication
that clinical supervision and training methods should be subject to rigorous empirical test-
ing. Indeed, we are arguing for a stance of empirical skepticism toward the effectiveness of
all methods of clinical training, old and new. Too many of the field’s current supervision
practices are in wide use because they have been handed down via tradition rather than
having been intentionally adopted on the basis of the research evidence (e.g., Ellis &
Ladany,1997).
In this volume, we are proposing that clinical supervision, training, and CE be reformed
along the principles of deliberate practice. This marks a significant departure from the cur-
rent approaches to clinical supervision and training. For example, we propose (a) to evalu-
ate clinical supervision and training by the impact on client outcomes (rather than adherence
and competence in a treatment model); (b) to emphasize active learning methods, such as
repetitive behavioral rehearsal of clinical skills via role‐plays with corrective feedback
(rather than discussions about psychotherapy theory); and (c) that clinicians receive per-
sonal performance feedback continuously throughout their career (rather than stopping
when they are licensed).

Questions from John Norcross, PhD 15
The question of whether these principles that have been shown to improve performance
across a range of fields apply as well to the practice of psychotherapy is valid. Psychotherapy
is a unique pursuit by virtue of its interpersonal context and demands. When we cite evi-
dence from other fields such as music, athletics, or medicine, our goal is to focus on the
learning processes rather than any implied similarities between psychotherapy and the
functions of those other fields, to make the case that the principles of deliberate practice
improve skill acquisition apply across a wide variety of fields and tasks. (For example, Zen
Buddhism and other spiritual traditions have relied on deliberate practice for millennia.)
Each of these fields is unique, and each has developed its own specific methods of deliberate
practice to specifically address its particular pedagogic challenges.
All these fields rely on a human being having learned a particular skill or set of skills. The
large body of research that forms the science of expertise identifies principles that improve
the effectiveness of human skill acquisition, and we argue this research applies to psycho-
therapy, including the development of necessary interpersonal skills (e.g., Anderson,
McClintock, Himawan, Song, & Patterson,2015; see Chapter3).
In short, our primary concern is with new principles of supervision and training. The
methods we suggest for implementing these principles are largely drawn from the research
evidence (directly or as extrapolations). The next task for our field, though, is for research-
ers and clinicians to develop new methods of supervision and training, based on these prin-
ciples, and then subject them to rigorous empirical testing and evaluation in both clinical
labs and actual practice.
Question #2. Your “cycle of excellence” bears strong resemblance to other, well‐established
models of active learning, such as that by David Kolb. What distinguishes your cycle from
those of others, and what specific research support does your model enjoy?
Answer from Editors: This is an excellent question. And because others likely will wonder
about it as well, we welcome the chance to address it. At the heart of the Cycle of Excellence
model is the assumption that people learn from observing and critiquing their work. This
same assumption has informed training since at least the time of Dewey (1938). In fact,
other prominent models, such as those of Kolb (e.g., Kolb & Fry,1975) and Schön (1988),
owe a huge intellectual debt to Dewey’s observations on the role of experience.
But the Cycle of Excellence differs from these models in at least two fundamental ways.
The first of these concerns the essential role that a coach or supervisor has in providing
feedback and direct instruction. This is in contrast with discovery learning, which so often
is assumed to be common to models such as that of Kolb or Schön. Whereas discovery
learning has an intuitive appeal, Kirschner, Sweller, and Clark (2006) offered a scathing cri-
tique of its effectiveness.
The second fundamental difference is in the role that intentional practice is assumed to
play in skill development. Those models stress the cognitive processes that lead to new
understandings about therapists’ work and how they then might modify what they do. The
models do not, though, focus on the hard work of really practicing and consolidating skills
that lead to effective psychotherapy practice.

1 Introduction
16
What we are proposing is that the Cycle of Excellence and deliberate practice emphasize
maximizing opportunities for behavioral rehearsal and continuous corrective feedback.
The goal is to give trainees ample opportunity to experiment with specific skills, so they can
fail and get correction many times before trying the skills with real clients. This can be
accomplished through the use of role‐plays and other behavioral training drills in supervi-
sion. To illustrate, our model aims to be more like how one learns to drive (behavioral
rehearsal with continuous corrective feedback), while traditional supervision is more like
how one learns philosophy (discussing theory). These two models are not mutually exclu-
sive; trainees, of course, need to learn psychotherapy theory. Rather, we are suggesting that
behavioral rehearsal with continuous corrective feedback be given a stronger emphasis
within the supervision hour than it has been given previously.
Question #3. Many researchers have indeed questioned the value‐added benefit of clinical
supervision on client outcomes. At the same time, the supervision research and supervision
guidelines have converged on a series of best practices, which certainly contain the recom-
mendations for supervision advanced in your chapter. Can you explain how your recommen-
dations (e.g., demonstrate effective practice, provide corrective feedback, offer emotional
encouragement) differ from those generic best practices?
Answer from Editors: The supervision “best practices” that various professions have devel-
oped (e.g., American Psychological Association,2014; Association for Counselor Education
and Supervision,2011) represent the wisdom of supervision experts and stand as expected
supervision competencies. It is inevitable, then, that we would incorporate them in our
model as minimal expectations for effective supervisory practice. To illustrate: All of the
available supervision guidelines stress the importance of supervisors directly observing
their supervisees’ work with clients in contrast to the current dominant practice of relying
on supervisees’ self‐reports of their work. Directly observing supervisees’ work is impera-
tive in our model, which assumes that expertise development is only as effective as the
feedback available to guide it. But our model is not limited to that type of feedback and
requires, for example, the use of information from routine outcome monitoring. In this way,
our model goes beyond expected best practices.
This question also implicitly raises the important distinction between competence and
expertise. Competence, as either a supervisor or a psychotherapist, is about performing
work in an expected way. When professionals are held accountable for competence, they are
responsible for what Lerner and Tetlock (2003) describe as process accountability (see also
Goodyear,2015): To what extent is this person executing a skill set as expected, regardless
of the obtained outcomes?
But in our model, expertise development is assessed in terms of client outcomes. Thera-
pists and supervisors are held to outcome accountability (Lerner & Tetlock,2003): To what
extent is this person achieving intended outcomes, regardless of how she or he performed?
Both forms of accountability are important, although we give outcome accountability the
greater weight: Process should not overshadow outcome. This is especially true as one does
not predict the other (see, e.g., Webb, DeRubeis, & Barber,2010).

References 17
This book is concerned primarily with ways that psychotherapists can move from compe-
tence to expertise. Although we give it less emphasis in the book, we assume as well that
supervisors would commit to a similar process of development and that the measure of
their evolving expertise would be client outcomes, just as it is with therapists.
Question #4: Figure 2.3 in Chapter2 shows huge growth in performance among “experts”
using deliberate practice. Is this figure based on an actual study, in which the experts achieved
twice the performance/client outcomes of “typical therapists”? Or perhaps the figure just rep-
resents a conceptual promise?
Answer from Editors: As the growing research on implementation science makes clear, all
research offers a “conceptual promise.” There is no guarantee of results. Differences in con-
texts, clients, management, and providers make any simple transfer and application of
research findings to real‐world clinical settings challenging in the best of circumstances.
More, as evidenced by the study reported in Figure 2.3, deliberate practice is hard work, the
gains are slow in coming and not immediately rewarding. A further threat is that practition-
ers with average results already consider themselves as effective as the best while devoting
significantly less time to efforts aimed at improving their outcomes. At the same time, this
same graph, together with findings from other studies cited, shows that the most effective
therapists engage in significantly more deliberate practice than their more average counter-
parts. Indeed, with clients of therapists in the top quartile achieving outcomes more than
twice the size of those in the bottom (Wampold & Brown,2005), deliberate practice serves
to benefit both clinicians and recipients of mental health services.
In sum, the promise of deliberate practice depicted in Figure 2.3 is indeed real. However,
its realization depends on two critical factors: (a) continuous effort and (b) long‐term com-
mitment. Although a number of studies are in the works, knowledge regarding the applica-
tion of deliberate practice to improving performance as a psychotherapist is still very much
in its infancy. Evidence regarding what is required for helping practitioners sustain the com-
mitment necessary to realize the gains is, for all points and purposes, unavailable. Until a
time in the future, practitioners and the field will be reliant on studies from other domains
(e.g., sport, music, medicine, computer programming, teaching).
References
Anderson, T., McClintock, A. S., Himawan, L., Song, X., & Patterson, C. L. (2015). A
prospective study of therapist facilitative interpersonal skills as a predictor of treatment
outcome. Journal of Consulting and Clinical Psychology, 84(1), 57–66. doi:10.1037/
ccp0000060
American Psychological Association. (2014). Guidelines for clinical supervision in health
service psychology. Retrieved from http://apa.org/about/policy/guidelines‐supervision.pdf
Association for Counselor Education and Supervision. (2011). Best practices in clinical
supervision. Retrieved from https://goo.gl/3n5qp2

1 Introduction
18
Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. J. Lambert
(Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp.
258–297). Hoboken, NJ: Wiley.
Bernard, J. M., & Goodyear, R. K. (2014). Fundamentals of clinical supervision (5th ed.).
Boston, MA: Merrill.
Beutler, L. E., & Howard, M. (2003). Training in psychotherapy: Why supervision does not
work. Clinical Psychologist, 56(4), 12–16.
Bloom, B. S. (2005). Effects of continuing medical education on improving physician clinical
care and patient health: A review of systematic reviews. International Journal of Technology
Assessment in Health Care, 21(3), 380–385. doi:10.1017/S026646230505049X
Branson, A., Shafran, R., & Myles, P. (2015). Investigating the relationship between
competence and patient outcome with CBT highlights. Behaviour Research and Therapy,
68, 19–26. doi:10.1016/j.brat.2015.03.002
Centers for Disease Control and Prevention. (2012). Death rate from complications of medical
and surgical care among adults aged ≥ 45 years, by age group—United States, 1999–2009.
Retrieved from http://goo.gl/fpElbC
Chow, D., Miller, S.D., Seidel, J., Kane, R., & Andrews, B. (2015). The role of deliberate
practicein the development of highly effective psychotherapists. Psychotherapy, 52(3),
337–345.
Colvin, G. (2008). Talent is overrated: What really separates world‐class performers from
everybody else. New York, NY: Penguin.
Commission on Accreditation. (2013). Guidelines and principles for accreditation of programs
in professional psychology. Washington, DC: American Psychological Association.
Coughlan, E. K., Williams, A. M., McRobert, A. P., & Ford, P. R. (2014). How experts practice:
A novel test of deliberate practice theory. Journal of Experimental Psychology. Learning,
Memory, and Cognition, 40(2), 449–58. doi:10.1037/a0034302
Coyle, D. (2009). The talent code: Greatness isn’t born. It’s grown. Here’s how. New York, NY:
Bantam Dell.
Dewey, J. (1938). Experience and education. New York, NY: Collier Books
Duckworth, A. L., Kirby, T. A., Tsukayama, E., Berstein, H., & Ericsson, K. A. (2011).
Deliberate practice spells success: Why grittier competitors triumph at the National Spelling
Bee. Social Psychological and Personality Science, 2(2), 174–181. doi:10.1177/194855061038
5872
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and
passion for long‐term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101.
doi:10.1037/0022-3514.92.6.1087
Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and clients in clinical
supervision: An integrative review. In C. E. Watkins (Ed.), Handbook of psychotherapy
supervision (pp. 447–507). New York, NY: Wiley.
Ericsson, K. A. (1996). The acquisition of expert performance: An introduction to some of the
issues. In K. A. Ericsson (Ed.), The road to excellence: The acquisition of expert performance
in the arts and sciences, sports, and games (pp. 1–50). Mahwah, NJ: Erlbaum.
Ericsson, K. A. (2006). The influence of experience and deliberate practice on the
development of superior expert performance. In K. A. Ericsson, N. Charness, P. J. Feltovich,

References 19
& R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp.
683–703). Cambridge, UK: Cambridge University Press.
Ericsson, K. A. (2009). Enhancing the development of professional performance: Implications
from the study of deliberate practice. In Development of professional expertise: Toward
measurement of expert performance and design of optimal learning environments (pp.
405–431). New York, NY: Cambridge University Press.
Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (2006). The Cambridge
handbook of expertise and expert performance. Cambridge, UK: Cambridge University Press.
Ericsson, K. A., Krampe, R. T., & Tesch‐Romer, C. (1993). The role of deliberate practice in the
acquisition of expert performance. Psychological Review, 100, 363–406.
Ericsson, K. A., & Pool, R. (2016). Peak: Secrets from the new science of expertise. New York,
NY: Houghton Mifflin Harcourt.
Forand, N. R., DeRubeis, R. J., & Amsterdam, J. D. (2013). Combining medication and
psychotherapy in the treatment of major mental disorders. In M. J. Lambert (Ed.), Bergin
and Garfield’s handbook of psychotherapy and behavior change (pp. 735 – 774). Hoboken,
NJ: Wiley.
Friedman, L. S., & Forst, L. (2007). The impact of OSHA recordkeeping regulation changes on
occupational injury and illness trends in the US: A time‐series analysis.Occupational and
Environmental Medicine, 64(7), 454–460.
Gladwell, M. (2008). Outliers. New York, NY: Little, Brown.
Goldberg, S., Babins‐Wagner, R., Rousmaniere, T. G., Berzins, S., Hoyt, W. T., Whipple,
J.L.,...Wampold, B. E. (2016). Creating a climate for therapist improvement: A case study of
an agency focused on outcomes and deliberate practice. Psychotherapy, 53, 367–375.
Goldberg, S., Rousmaniere, T. G., Miller, S. D., Whipple, J., Nielsen, S. L.., Hoyt, W. T. &
Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A
longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology,
63, 1–11.
Goodyear, R. (2015, November). Using accountability mechanisms more intentionally:
A framework and its implications for training professional psychologists. American
Psychologist, 70(8), 736–743. doi:10.1037/a0039828
Gøtzsche, P. C., Young, A. H., & Crace, J. (2015). Does long term use of psychiatric drugs cause
more harm than good? British Medical Journal, 350, h2435. doi:10.1136/bmj.h2435
Hill, C. E., Baumann, E., Shafran, N., Gupta, S., Morrison, A., Rojas, A.E.P.,...Gelso, C. J.
(2015). Is training effective? A study of counseling psychology doctoral trainees in a
psychodynamic/interpersonal training clinic. Journal of Counseling Psychology, 62, 184–201.
Hilsenroth, M. J., Ackerman, S. J., Clemence, A. J., Strassle, C. G., & Handler, L. (2002). Effects
of structured clinician training on patient and therapist perspectives of alliance early in
psychotherapy. Psychotherapy: Theory, Research, Practice, Training, 39(4), 309–323. doi:10
.1037/0033-3204.39.4.309
Hilsenroth, M. J., Kivlighan, D. M., & Slavin‐Mulford, J. (2015). Structured supervision of
graduate clinicians in psychodynamic psychotherapy: Alliance and technique. Journal of
Counseling Psychology. Advance online publication. doi:10.1037/cou0000058
Hubble, M. A., Duncan, B. L., & Miller, S. D. (1999). The heart and soul of change: What works
in therapy. Washington, DC: American Psychological Association.

1 Introduction
20
Hunt, E. (2006). Expertise, talent, and social encouragement. In K. A. Ericsson, N. Charness,
P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert
performance (pp. 31–38). Cambridge, UK: Cambridge University Press.
Johnsen, T. J., & Friborg, O. (2015). The effects of cognitive behavioral therapy as an anti‐
depressive treatment is falling: A meta‐analysis. Psychological Bulletin. Advance online
publication. 10.1037/bul0000015
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction
does not work: An analysis of the failure of constructivist, discovery, problem‐based,
experiential, and inquiry‐based teaching. Educational Psychologist, 41(2), 75–86.
Kolb. D. A., & Fry, R. (1975). Toward an applied theory of experiential learning. In C. Cooper
(Ed.), Theories of group process (pp. 125–149). London, UK: Wiley.
Kraus, D. R., Castonguay, L., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist
effectiveness: Implications for accountability and patient care. Psychotherapy Research,
21(3), 267–276.
Ladany, N. (2007). Does psychotherapy training matter? Maybe not. Psychotherapy: Theory,
Research, Practice, Training, 44(4), 392–396. doi:10.1037/0033-3204.44.4.392
Lambert, M. J. (2010). Yes, it is time for clinicians to routinely monitor treatment outcome. In
B. L. Duncan, S. D. Miller, B. E. Wampold, & M. A. Hubble (Eds.), The heart and soul of
change: Delivering what works in therapy (pp. 239–266). Washington, DC: American
Psychological Association.
Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.),
Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 169–218).
Hoboken, NJ: Wiley.
Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy: Theory,
Research, Practice, Training, 48, 72–79.
Laska, K. M., Gurman, A. S., & Wampold, B. E. (2014). Expanding the lens of evidence‐based
practice in psychotherapy: A common factors perspective. Psychotherapy, 51, 467–481. doi:
10.1037/a0034332
Lehmann, A. C., & Ericsson, K. A. (1998). The historical development of domains of expertise:
Performance standards and innovations in music. In A. Steptoe (Ed.), Genius and the mind
(pp. 67–94). Oxford, UK: Oxford University Press.
Lerner, J., & Tetlock, P. E. (2003). The impact of accountability on cognitive bias: Bridging
individual, interpersonal, and institutional approaches to judgment and choice. In S.
Schneider & J. Shanteau (Eds.), Emerging perspectives in judgment and decision‐making (pp.
431–457). New York, NY: Cambridge University Press.
Lichtenberg, J. W., & Goodyear, R. K. (2012). Informal learning, incidental learning, and
deliberate continuing education: Preparing psychologists to be effective lifelong learners. In
G. J. Neimeyer & J. M. Taylor (Eds.), Continuing education: Types, roles, and societal impacts
(pp. 71–80). Hauppauge, NY: Nova Science.
Lippi, G., Banfi, G., Favaloro, E. J., Rittweger, J., & Maffulli, N. (2008). Updates on improvement
of human athletic performance: Focus on world records in athletics. British Medical
Bulletin, 87(1), 7–15. Retrieved from http://bmb.oxfordjournals.org/content/87/1/7.long

References 21
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence‐
based psychological treatments: A review of current efforts. American Psychologist, 65(2),
73–84. doi:10.1037/a0018121
McMahan, E. H. (2014). Supervision: A nonelusive component of deliberate practice toward
expertise. American Psychologist, 69, 711–712. doi:/10.1037/a0037832
Miller, S. D., Duncan, B. L., & Hubble, M. A. (2004). Beyond integration: The triumph of
outcome over process in clinical practice. Psychotherapy in Australia, 10, 2–19.
Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, J. (2005). The Partners for Change Outcome
System. Journal of Clinical Psychology: In Session, 61, 199–208.
Miller, S. D., Hubble, M. A, Chow, D. L., & Seidel, J. A. (2013). The outcome of psychotherapy:
Yesterday, today, and tomorrow. Psychotherapy, 50(1), 88–97. doi:10.1037/a0031097
Miller, S. D., Hubble, M. A., and Duncan, B. L. (2007). Supershrinks: Learning from the field’s
most effective practitioners. Psychotherapy Networker, 31, 26–35, 56.
Neimeyer, G. J., & Taylor, J. M. (2010). Continuing education in psychology. In J. C. Norcross,
G. R. VandenBos, & D. K. Freedheim (Eds.), History of psychotherapy: Continuity and
change (pp. 663–671). Washington, DC: American Psychological Association.
Neimeyer, G. J., Taylor, J. M., & Wear, D. M. (2009). Continuing education in psychology:
Outcomes, evaluations, and mandates. Professional Psychology: Research and Practice, 40(6),
617–624.
Owen, J., Wampold, B. E., Rousmaniere, T. G., Kopta, M., & Miller., S. (2016). As good as it
gets? Therapy outcomes of trainees over time. Journal of Counseling Psychology,
63, 12–19.
PCOMS. (2013). Partners for Change Outcome Management System: International Center for
Clinical Excellence. Retrieved from http://legacy.nreppadmin.net/ViewIntervention.
aspx?id=249
Rousmaniere, T. G., Swift, J. K., Babins‐Wagner, R., Whipple, J. L., & Berzins, S. (2016).
Supervisor variance in psychotherapy outcome in routine practice. Psychotherapy Research,
26(2), 196–205.
Schön, D. (1988). Educating the reflective practitioner. San Francisco, CA: Jossey‐Bass.
Shenk, D. (2010). The genius in all of us: Why everything you’ve been told about genetics, talent,
and IQ is wrong. New York, NY: Random House.
Stoltenberg, C. D., & McNeill, B. W. (2010). IDM supervision: An integrative developmental
model for supervising counselors and therapists (3rd ed.). New York, NY: Routledge.
Swift, J. K., Greenberg, R. P., Whipple, J. L., & Kominiak, N. (2012). Practice recommendations
for reducing premature termination in therapy. Professional Psychology: Research and
Practice, 43(4), 379–387. doi:10.1037/a0028291
Syed, M. (2010). Bounce: Mozart, Federer, Picasso, Beckham, and the Science of Success. New
York, NY: HarperCollins.
Tracey, T.J.G., Wampold, B. E., Goodyear, R. K., & Lichtenberg, J. W. (2015). Improving
expertise in psychotherapy. Psychotherapy Bulletin, 50(1), 7–13.
Tracey, T.J.G., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in
psychotherapy: An elusive goal? American Psychologist, 69, 218–229.

1 Introduction
22
Wampold, B. E., & Brown, J. (2005). Estimating variability in outcomes attributable to
therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and
Clinical Psychology, 73(5), 920.
Wampold, B. E., Budge, S. L., Laska, K. M., del Re, A. C., Baardseth, T. P., Fluckiger,
C.,...Gunn, W. (2011). Evidence‐based treatments for depression and anxiety versus
treatment‐as‐usual: A meta‐analysis of direct comparisons. Clinical Psychology Review,
31(8), 1304–1312. Retrieved from http://doi.org/10.1016/j.cpr.2011.07.012
Wampold, B. E., & Imel, Z. (2015). The great psychotherapy debate: The evidence for what
makes psychotherapy work. New York, NY: Routledge.
Watkins, D. (2011). Does psychotherapy supervision contribute to patient outcomes?
Considering thirty years of research. Clinical Supervisor, 30, 235–256. doi:10.1080/07325223
.2011.619417
Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and
treatment outcome: A meta‐analytic review. Journal of Consulting and Clinical Psychology,
78(2), 200–211.
Wise, E. H., Sturm, C. A., Nutt, R. L., Rodolfa, E., Schaffer, J. B., & Webb, C. (2010). Life‐long
learning for psychologists: Current status and a vision for the future. Professional
Psychology: Research and Practice, 41(4), 288–297.