Content uploaded by Scott D. Miller
Author content
All content in this area was uploaded by Scott D. Miller on Jul 11, 2016
Content may be subject to copyright.
Content uploaded by Scott D. Miller
Author content
All content in this area was uploaded by Scott D. Miller on Jun 20, 2016
Content may be subject to copyright.
Featured Expert Review
Feedback Informed Treatment (FIT): Achieving Clinical
Excellence One Person at a Time
— Scott D. Miller, Mark A. Hubble, Jason A. Seidel, Daryl Chow, &
Susanne Bargmann
“It is the big choices we make
that set our direction.
It is the smallest choices we make
that get us to the destination.”
—Shad Helmstetter
Clinical psychology outcomes research and stud-
ies of high performance in other fields indicate
that the critical factors separating high-per-
forming psychotherapists from average therapists have
little to do with experience or the use of empirically-
supported treatments. Instead, there appear to be
systematic differences in how practitioners implement
the tools of their trade (regardless of their therapeu-
tic orientation). As therapists shift their focus from
traditional methods of accumulating knowledge and
experience toward a more empirically-supported meth-
odology for improving performance (including the
formal collection of feedback, a stance of non-defensive
openness, and individually tuned programs of delib-
erate practice), evidence suggests that the individual
practitioner will be able to achieve superior outcomes,
measure these outcomes, and compete more effectively
in the behavioral healthcare marketplace.
A “great debate” is raging in the field of psychotherapy
(Wampold, 2001). On one side are those who hold that
behavioral health interventions are similar to medi-
cal treatments (Barlow, 2004). Therapies work, they
believe, because like penicillin they contain specific
ingredients remedial to the disorder being treated.
Consistent with this perspective, emphasis is placed on
diagnosis, treatment plans, and adherence to so-called
“validated” treatments (Siev, Huppert, & Chambless,
2009; Huppert, Fabbro, & Barlow, 2006; Chambless &
Ollendick, 2001). The “medical model,” as it is termed,
is arguably the dominant view of how psychotherapy
works. It is also the view held by most people who seek
behavioral health treatment.
On the other side of the debate are those who argue
that improvements in effectiveness, and ultimately,
clinical excellence, will not be achieved by mimick-
ing the practices of medicine. In fact, they hold that
psychotherapy is fundamentally incompatible with
the medical view (Wampold, 2001; Duncan, Miller,
Wampold, & Hubble, 2010; Hubble, Duncan, & Miller,
1999). Proponents of what has been termed the “con-
textual” perspective highlight the evidence for the lack
of differential effectiveness among the 250 compet-
ing psychological treatments, suggesting instead that
the efficacy of psychotherapy is more parsimoniously
accounted for by a handful of curative factors shared
by all (Lambert, 1992; Miller, Duncan, & Hubble, 1997).
While each therapist offers their own particular frame-
work for treatment, of particular importance from this
contextual point of view are extratherapeutic factors
and the therapeutic relationship. The former refer to
strengths, resources, life-circumstances—variables that
clients bring to treatment. The therapeutic relationship
includes the emotional bond between the participants
and agreements on goals and tasks.
The challenge for practitioners striving to achieve
excellence—given the sharply diverging points of view
and dizzying array of treatments available—is know-
ing what to do, when to do it, and with whom. For the
independent practitioner, these questions are especially
pressing as therapists continue to lose their share of
a market that increasingly looks for faster, cheaper,
more effective solutions to psychological and rela-
tional problems. Thankfully, recent developments are
on track to providing an empirically robust and clini-
cally feasible answer to the question of “What works
for whom?” Based on the pioneering work of Howard,
Moras, Brill, Martinovich, and Lutz (1996) and others
(c.f., Lambert, 2010; Brown, Dries, & Nace, 1999; Miller,
Duncan, & Hubble, 2005; Duncan et al., 2010), this
approach transcends the “medical versus contextual”
debate by focusing on routine, ongoing monitoring of
engagement in and progress of therapy (Lambert, 2010).
Such data, in turn, are utilized to inform decisions
about the kind of treatment offered as well as whether
to continue, modify, or even end services. Indeed,
multiple, independent randomized clinical trials now
show that formally and routinely assessing and discuss-
ing clients’ experience of the process and outcome of
care effectively doubles the rate of reliable and clini-
cally significant change, decreases drop-out rates by as
much as 50%, and cuts deterioration rates by one-third
(Miller, 2010).
Excellence is within the reach of all clinicians, whether
aligned primarily with the medical or contextual views
of psychotherapy. In short, they can benefit by using
feedback to improve the outcome of the services they
offer one person at a time.
What Kind of Feedback Matters?
“If we don’t change direction,
we’ll end up where we’re going.”
Professor Irwin Corey
Feedback-informed treatment or FIT is based on
several well-established findings from the outcome
literature. The first is: psychotherapy works. Studies
dating back over 35 years document that the average
treated person is better off than 80% of the untreated
sample in most studies (Duncan et al., 2010; Smith
& Glass, 1977; Wampold, 2001). Second, the general
trajectory of change in successful treatment is pre-
dictable, with the majority of measured progress
occurring earlier rather than later (Brown, Dreis, and
Nace, 1999; Hansen, Lambert & Forman 2002). Third,
despite the proven efficacy of psychotherapy, there is
considerable variation in both the engagement in and
outcome of individual episodes of care. With regard to
the former, for example, available evidence indicates
that as many as 50% of those who initiate treatment
drop out before achieving a reliable improvement in
functioning (Bohanske & Franczak, 2010; Kazdin,
1996; Garcia & Weisz, 2002; Swift & Greenberg, 2012;
Wierzbicki & Pekarik, 1993). With regard to the latter,
significant differences in outcome exist between prac-
titioners. Indeed, a large body of evidence shows that
“who” provides a treatment contributes 8 to 9 times
more to outcome than “what” particular treatment is
offered (Wampold, 2005; Miller, Hubble, & Duncan,
2007). Such findings indicate that people seeking treat-
ment would do well to choose their provider carefully
as it is the therapist – not the treatment approach—that
matters most in terms of results. Fourth, and finally,
a sizable portion of the variability in outcome among
clinicians is attributable to the therapeutic alliance.
For example, in a study involving 80 clinicians and 331
clients, Baldwin, Wampold, and Imel (2007) reported
that it was therapist variability in the alliance, rather
than client variability, that predicted outcome. In other
words, therapists who on average, formed stronger alli-
ances, performed better than therapists who did not
form as strong a therapeutic engagement with their
clients. Taken together, the foregoing findings indicate
that real-time monitoring and utilization of outcome
and alliance data can maximize the “fit” between client,
therapist, and treatment. With so many factors at play
influencing outcome at the time of service delivery, it
is practically impossible to know a priori what treat-
ment or treatments delivered by a particular therapist
will reliably work with a specific client. Regardless of
discipline or theoretical orientation, clinicians must
determine if the services being offered are working and
adjust accordingly.
Two simple scales that have proven useful for monitor-
ing the status of the relationship and progress in care
are the Session Rating Scale (SRS [Miller, Duncan, &
Johnson, 2000]), and the Outcome Rating Scale (ORS,
[Miller & Duncan, 2000]). The SRS and ORS measure
alliance and outcome, respectively. Both scales are
short, 4-item, self-report instruments that have been
tested in numerous studies and shown to have solid
reliability and validity (Miller, 2010). Most importantly
perhaps, the brevity of the two measures insures they
are also feasible for use in everyday clinical practice.
After having experimented with other tools, the devel-
opers, along with others (i.e., Brown et al., 1999), found
that “any measure or combination of measures that
[take] more than five minutes to complete, score, and
interpret [are] not considered feasible by the majority
of clinicians” (Duncan & Miller, 2000, p. 96). Indeed,
available evidence indicates that routine use of the ORS
and SRS is high compared to other, longer measures
(e.g., 99% utilization rates of the ORS & SRS, versus
25% utilization rate of the Outcome Questionnaire-45
[Miller, Duncan, Brown, Sparks, & Claud, 2003]).
Administering and scoring the measures is simple
and straightforward. The ORS is administered at the
beginning of the session. The scale asks consumers
of therapeutic services to think back over the prior
week (or since the last visit) and place a hash mark (or
“x”) on four different lines, each representing a differ-
ent area of functioning (e.g., individual, interpersonal,
social, and overall well being). The SRS, by contrast,
is completed at the end of each visit. Here again, the
consumer places a hash mark on four different lines,
each corresponding to a different and important qual-
ity of the therapeutic alliance (e.g., relationship, goals
and tasks, approach and method, and overall). On
both measures, the lines are ten centimeters in length.
Scoring is a simple matter of determining the distance
in centimeters (to the nearest millimeter) between
the left pole and the client’s hash mark on each indi-
vidual item and then adding the four numbers together
to obtain the total score (the scales are available in
numerous languages at www.scottdmiller.com/perfor-
mance-metrics).
In addition to hand scoring, a growing number of
computer-based applications are available which can
simplify the process of administering, scoring, inter-
preting, and aggregating data from the ORS and SRS.
Such programs are especially useful in large and busy
group practices and agencies. Detailed descriptions
of the other applications can be found online at www.
scottdmiller.com.
Creating a “Culture of Feedback”
“My priority is to encourage openness and a
culture that is willing to acknowledge when things
have gone wrong.”
John F. Kennedy
Of course, soliciting clinically meaningful feedback
from consumers of therapeutic services requires more
than administering two scales. Clinicians must work
at creating an atmosphere where clients feel free to
rate their experience of the process and outcome of
services: (1) without fear of retaliation; and (2) with a
hope of having an impact on the nature and quality of
services delivered.
Interestingly, empirical evidence from both business
and healthcare demonstrates that consumers who are
happy with the way failures in service delivery are
handled are generally more satisfied at the end of the
process than those who experience no problems along
the way (Fleming & Asplund, 2007). The most effec-
tive clinicians, it turns out, consistently achieve lower
scores on standardized alliance measures at the outset
of therapy thereby providing an opportunity to discuss
and address problems in the working relationship—a
finding that has now been confirmed in numerous,
independent, real-world clinical samples (Miller,
Hubble, & Duncan, 2007).
Beyond displaying an attitude of openness and receptiv-
ity, creating a “culture of feedback” involves taking time
to introduce the measures in a thoughtful and thorough
manner. Providing the client with a rationale for using
the tools is critical, as is including a description of how
the feedback will be used to guide service delivery
(e.g., enabling the therapist to catch and repair alliance
breaches, prevent dropout, correct deviations from
optimal treatment experiences, etc). Additionally, it is
important that the client understands that the therapist
will not be offended or become defensive in response to
feedback given. Instead, therapists must take clients’
concerns regarding the treatment process seriously
and avoid the temptation to interpret feedback solely in
clinical terms. When introducing the measures at the
beginning of a therapy, the therapist might say:
“(I/We) work a little differently in this (agency/prac-
tice). (My/Our) first priority is making sure that you
get the results you want. For this reason, it is very
important that you are involved in monitoring our prog-
ress throughout therapy. (I/We) like to do this formally
by using a short paper and pencil measure called the
Outcome Rating Scale. It takes about a minute. Basi-
cally, you fill it out at the beginning of each session
and then we talk about the results. A fair amount of
research shows that if we are going to be successful in
our work together, we should see signs of improvement
earlier rather than later. If what we’re doing works,
then we’ll continue. If not, however, then I’ll try to
change or modify the treatment. If things still don’t
improve, then I’ll work with you to find someone or
someplace else for you to get the help you want. Does
this make sense to you?” (Miller & Duncan, 2004;
Miller & Bargmann, 2011).
At the end of each session, the therapist administers
the SRS, emphasizing the importance of the relation-
ship in successful treatment and encouraging negative
feedback:
“I’d like to ask you to fill out one additional form. This
is called the Session Rating Scale. Basically, this is a
tool that you and I will use at the end of each session
to adjust and improve the way we work together. A
great deal of research shows that your experience of our
work together—did you feel understood, did we focus
on what was important to you, did the approach I’m
taking make sense and feel right—is a good predictor of
whether we’ll be successful. I want to emphasize that
I’m not aiming for a perfect score—a 10 out of 10. Life
isn’t perfect and neither am I. What I’m aiming for is
your feedback about even the smallest things—even if
it seems unimportant—so we can adjust our work and
make sure we don’t steer off course. Whatever it might
be, I promise I won’t take it personally. I’m always
learning, and am curious about what I can learn from
getting this feedback from you that will in time help me
improve my skills. Does this make sense?” (Miller &
Bargmann, 2011).
Free copies of the ORS and SRS measures are available
at: www.scottdmiller.com/performance-metrics.com.
The ORS and SRS are collectively called the Partners for
Change Outcome Management System (PCOMS) which
has been certified as an evidence-based practice by the
Substance Abuse and Mental Health Services Adminis-
tration (SAMHSA). A copy of the SAMHSA report can
be found at: http://www.nrepp.samhsa.gov/ViewIn-
tervention.aspx?id=249. Instructional manuals for the
implementation of FIT and the PCOMS are available at
www.scottdmiller.com and further training materials,
articles, networking and educational opportunities, and
instructional videos are available at the International
Center for Clinical Excellence website: http://www.
centerforclinicalexcellence.com.
In one example of how FIT can alter practitioners’
outcomes, Anker, Duncan, & Sparks (2009) conducted
the largest randomized clinical trial in the history of
couples therapy research. The design of the study was
simple. Using the ORS and SRS, the outcomes and alli-
ance ratings of 205 couples in therapy were gathered
during each treatment session. In half of the cases,
clinicians received feedback about the couples’ experi-
ence of the therapeutic relationship and progress in
treatment; in the other half, none. At the conclusion
of the study, couples whose therapist received feedback
experienced twice the rate of reliable and clinically sig-
nificant change as those in the non-feedback condition.
At 6-month follow-up, couples treated by therapists not
receiving feedback had nearly twice the rate of separa-
tion and divorce.
The research evidence is clear: psychotherapy is
effective for a wide range of presenting concerns and
problems. At the same time, too many clients dete-
riorate while in care, an even larger number drop out
before experiencing a reliable improvement in function-
ing, and outcomes vary widely and consistently among
clinicians.
FIT enables practitioners to achieve excellence by
routinely soliciting feedback regarding the client’s per-
ception of the therapeutic alliance and progress and
using the information to guide and improve service
delivery. A significant and growing body of research
documents that, regardless of theoretical orientation or
preferred treatment approach, employing FIT improves
outcome and retention rates and reduces deterioration.
In short, FIT can systematically improve the effective-
ness of independent practitioners of psychotherapy, one
person and one therapy session at a time.
From Feedback to Excellence
“...[E]xperts are always made not born.”
K. Anders Ericsson (2007)
As crucial as the use of feedback measures may be in
delivering better outcomes, their use is not enough to
develop expertise. Our attitudinal perspective plays a
significant influence in our adoption of feedback. For
instance, De Jong, van Sluis, Nugter, Heiser, and Spin-
hoven (2012) found that not every therapist benefits
from the use of formal feedback measures. Only thera-
pists who were committed and held an open attitude
towards the use of feedback benefited from the utiliza-
tion of feedback mechanisms. In other words, feedback
functions like tuning equipment for a musical instru-
ment. It indicates when a note is out of tune, but it does
not necessarily improve the musician’s sense of pitch.
Needless to say, it does not inform the user about how
to compose a classic.
Another issue that hinders the adoption of feedback
measures is attributed to self-assessment bias, also
coined as the “Lake Wobegon” effect (Kruger, 1999). The
phenomenon of self-assessment bias is not uncommon.
Kahneman (2011) termed this “the illusion of valid-
ity,” describing the fallacy of judgments about one’s
own abilities, especially without any feedback from
external sources to confirm or disconfirm one’s intui-
tive responses. For example, Kahneman (2011) found
that experts making political judgments, stock trad-
ers, and financial advisors were not only inaccurate
in their predictions, but also over-confident in their
judgments. Similar self-assessment biases have also
been found with physicians (Davis et al., 2006). Simi-
lar to studies of physicians, self-assessment reports by
psychotherapists have revealed that the least effective
therapists rate themselves as highly as the most effec-
tive therapists (Brown et al., 2006; Hiatt & Hargrave,
1995). Therapists are also more likely to overestimate
their rates of client improvement and underestimate
their rates of client deterioration (Walfish, McAlister,
O’Donnell, & Lambert, 2012). In our recent investiga-
tion with a sub-sample of therapists who have been
routinely measuring their own outcomes over a 5-year
period (Andrews, Wislocki, Short, Chow, & Minami,
2013), their self-assessment of their effectiveness did
not predict actual client outcomes (Chow, 2013; Chow,
Miller, Kane, Thornton, Andrews, n.d.). In fact, the
majority of therapists optimistically viewed that they
have improved over the years. As such, it remains
questionable if self-reported effectiveness actually does
represent actual levels of competency.
Feedback can be helpful when an additional step is in
place: engaging in deliberate practice (Ericsson, 1996;
Ericsson, 2006; Ericsson, 2009; Ericsson, Krampe, &
Tesch-Romer, 1993). Deliberate practice is defined as:
…Individualized training activities especially designed
by a coach or teacher to improve specific aspects of an
individual’s performance through repetition and suc-
cessive refinement. To receive maximal benefit from
feedback, individuals have to monitor their training
with full concentration, which is effortful and limits the
duration of daily training. (Ericsson & Lehmann, 1996,
pp. 278-279)
This type of practice is often focused, systematic,
carried out over extended periods of time, guided by
conscious monitoring of outcomes, and evaluated by
analyses of levels of expertise acquired, identification of
errors, and procedures implemented at reducing errors
(Ericsson, 1996; Ericsson, 2006; Ericsson et al., 1993).
In a study of violinists, for example, “best” and “good”
violinists spent almost three times longer than music
teachers in solitary practice with their instrument,
averaging 3.5 hours per day for each day of the week
including weekends, compared with 1.3 hours per day
for the music teachers (Ericsson et al., 1993).
Based on research in the field of expertise and expert
performance, Ericsson and colleagues noted that
superior performance is not a function of any innate
talent (Ericsson, Nandagopal, & Roring, 2005; Erics-
son, Roring, & Nandagopal, 2007), nor is it reflected
by degrees earned, professional title, or experience.
Rather, it comes from the incremental development of
extended deliberate practice. Deliberate practice was
found to mediate performance in multiple areas of
expertise, such as music (Ericsson et al., 1993; Krampe
& Ericsson, 1996), chess (Gobet & Charness, 2006),
sports (Cote, Ericsson, & Law, 2005), business (Son-
nentag & Kleine, 2000), and medicine and surgery
(Ericsson, 2007b; Mamede et al., 2007; Norman, Eva,
Brooks, & Hamstra, 2006; Schmidt & Rikers, 2007).
Ericsson and colleagues (1993) argue, “The search for
stable heritable characteristics that could predict or
at least account for superior performance of eminent
individuals has been surprisingly unsuccessful” (p.
365), with the exception of certain sporting activities
(e.g., ballet, basketball) that require a specific physical
endowment.
In psychotherapy, neither training clinicians to improve
the alliance nor greater experience conducting ther-
apy have predicted clinical outcomes (Horvath, 2001;
Anderson, Ogles, Patterson, Lambert, and Vermeersch,
2009). As described above, some therapists are consis-
tently better at establishing and maintaining helpful
relationships than others. Evidence that the difference
is attributable to their possession of deeper domain-spe-
cific knowledge (the kind of therapeutic resource that
is attained by deliberate practice) was demonstrated
by Anderson et al. (2009). In that study, differences
in client outcomes between therapists were found to
be unrelated to therapist gender, theoretical orienta-
tion, professional experience, and overall social skills.
Rather, the therapists who exhibited deeper, broader,
and interpersonally nuanced knowledge obtained the
best results. Regardless of presenting problem or client’s
relational style, top-performing therapists were able to
respond collaboratively and empathically, and far less
likely to make remarks or comments that distanced or
offended a client.
Acquiring this kind of understanding, perception,
and sensitivity is a common goal for clinicians from
the full range of theoretical orientations; yet the data
from Anderson et al. (2009) and the broader evidence
from Ericsson and colleagues suggest that some end up
having such knowledge and using it effectively, while
others (of equal experience and social ability), do not.
A recent research study investigated the contribution
of therapist variables, their professional work prac-
tices, professional development activities, and beliefs
regarding learning and personal appraisals of thera-
peutic effectiveness (Chow, 2013; Chow et al., n.d.).
Although preliminary, results from this study are in
line with earlier research on the factors that account for
expertise. Similar to Anderson et al. (2009) and others
(Wampold & Brown, 2005), therapist gender, qualifi-
cations, professional discipline, years of experience,
and time spent conducting therapy were unrelated to
outcome. Similar to findings reported by Walfish et al.
(2012), therapist self-appraisal was not a reliable mea-
sure of effectiveness. Consistent with results obtained
in other professional domains (e.g., Charness, Tuffiash,
Krampe, Reingold, & Vasyukova, 2005; Duckworth,
Kirby, Tsukayama, Berstein, & Ericsson, 2011; Ericsson
et al., 1993; Keith & Ericsson, 2007; Krampe & Ericsson,
1996; Starkes, Deakin, Allard, Hodges, & Hayes, 1996),
the findings by Chow and colleagues (n.d.) provide
preliminary support for the significant role of deliber-
ate practice in the development of expertise among
highly effective therapists. In sum, the amount of time
therapists reported being engaged in solitary activities
intended to improve their skills was related to outcome.
Seventeen therapists were asked, “How many hours
per week (on average) do you spend alone seriously
engaging in activities related to improving your therapy
skills in the current year?” The top quartile (in terms
of clinical outcomes) group of therapists invested about
1.8 times more time on “deliberate practice alone” com-
pared with the second quartile group of therapists. The
top quartile group spent about 3.7 times more time on
“deliberate practice alone” than the third quartile group.
Chow and colleagues (n.d.) also found that compared
to other therapists in their cohort, highly effective
therapists were more likely to report being surprised
by their clients’ feedback. This surprise may signify
qualities about the therapist’s openness, receptivity, and
willingness to receive negative and positive feedback
consistent with the concept of therapists taking a “not-
knowing” stance to the dialogical process of therapy
(Anderson, 1990, 2005; Anderson & Goolishian, 1988).
That is, while the therapist uses his or her expertise in
creating a facilitative environment for the client, the
therapist adopts a responsive and tentative posture,
while conveying a sense of openness and newness
towards the client’s unfolding narrative.
Providing further converging evidence for the delib-
erate and error-centric practice of more effective
practitioners of both psychotherapy and other profes-
sional fields, Najavits and Strupp (1994) found that
effective therapists were more self-critical and reported
making more mistakes then less effective therapists. In
a more recent study, among other predictors, therapist-
reported professional self-doubt (PSD) had a positive
effect on client ratings of working alliance, with higher
levels of PSD suggesting an open attitude towards
admitting their own shortcomings (Nissen-Lie, Monsen,
& Ronnestad, 2010). Taken together, these studies sug-
gest that highly effective therapists’ willingness to
evaluate their contribution to the psychotherapeutic
process, and emphasis on self-correction were associ-
ated with their better performance.
One Therapist at a Time
“A man walking is never in balance, but always
correcting for imbalance.”
Gregory Bateson
Taken together, the findings above point to a viable and
hopeful journey ahead for the field of psychotherapy.
The three key features of knowing one’s performance
baseline, obtaining feedback, and engaging in deliber-
ate practice provide a practical framework for clinicians
who seek to improve their craft of therapy (Miller,
Hubble, Chow, & Seidel, 2013; Tracey, Wampold, Lich-
tenber, & Goodyear, 2014). A craft is defined as “a
collection of learned skills accompanied by experienced
judgment” (Moore, 1994; p. 1). Psychologists who want
to improve at their craft of therapy must continously
reach for objectives just beyond their level of current
ability (Miller, Hubble, & Duncan, 2007).
For independent practitioners to thr ive in a market-
place increasingly driven by demands for quality and
accountability, they must evolve beyond the study of
psychotherapies in general (i.e., premises, models, pro-
cedures, and techniques), and beyond the accumulation
of credentials and years of experience. Instead, evi-
dence points to the likely necessity (and certainly to the
necessity of further research) of working to improve the
outcome of each and every therapist, one client at a time.
References
Anderson, H. (2005). Myths about “not-knowing”. Family Pro-
cess, 44(4), 497-504. doi:10.1111/j.1545-5300.2005.00074.x
Anderson, H., & Goolishian, H. A. (1988). Human systems as
linguistic systems: Preliminary and evolving ideas about
the implications for clinical theory. Family Process, 27(4),
371-393. doi:10.1111/j.1545-5300.1988.00371.x
Anderson, T., Ogles, B. M., Patterson, C. L., Lambert, M. J., &
Vermeersch, D. A. (2009). Therapist effects: Facilitative
interpersonal skills as a predictor of therapist success.
Journal of Clinical Psychology, 65(7), 755-768. doi:10.1002/
jclp.20583
Andrews, W., Wislocki, A. P., Short, F., Chow, D. L., & Minami,
T. (2013). A 5-year evaluation of the human givens
therapy using a practice research network. Mental Health
Review Journal, 18(3), 165-176. doi:10.1108/MHRJ-04-2013-
0011
Anker, M., Duncan, B., & Sparks, J. (2009). Using client feed-
back to improve couple therapy outcomes: A randomized
clinical trial in a naturalistic setting. Journal of Consulting
and Clinical Psychology, 77, 693-704.
Baldwin, S., Wampold, B., & Imel, Z. (2007). Untangling the
alliance-outcome correlation: Exploring the relative
importance of therapist and patient variability in the
alliance. Journal of Consulting and Clinical Psychology, 75,
842-852.
Barlow, D. (2004). Psychological treatments. American Psy-
chologist, 59, 869-878.
Bohanske, B., & Franczak, M. (2010). Transforming public
behavioral health care: A case example of consumer-
directed services, recovery, and the common factors
(pp. 299-322). In B. Duncan, S. Miller, B. Wampold, & M.
Hubble (Eds.). The Heart and Soul of Change. Washington,
D. C.: APA Press.
Brown, G., Dreis, S., & Nace, D. (1999). What really makes a
difference in psychotherapy outcome? Why does man-
aged care want to know? In M. Hubble, D. Duncan, & S.
Miller (eds). The Heart and Soul of Change: What Really
Works in Therapy (389-406). Washington, D.C.: APA Press.
Chambless, D. & Ollendick, T. (2001). Empirically-supported
psychological interventions: Controversies and evidence.
Annual Review of Psychology, 52, 685-716.
Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyu-
kova, E. (2005). The role of deliberate practice in chess
expertise. Applied Cognitive Psychology, 19(2), 151-165.
doi:http://dx.doi.org/10.1002/acp.1106
Chow, D. L. (2013). The study of supershrinks: Development and
deliberate practices of highly effective psychotherapists. Paper
presented at the Achieving Clinical Excellence (ACE)
Conference, May 16-18, 2013, Amsterdam, Holland.
Chow, D., Miller, S. D., Kane, R., Thornton, J., & Andrews,
W. (n.d.). The study of supershrinks: Development and
deliberate practices of highly effective psychotherapists.
Manuscript in preparation.
Cote, J., Ericsson, K., & Law, M. P. (2005). Tracing the Devel-
opment of Athletes Using Retrospective Interview
Methods: A Proposed Interview and Validation Procedure
for Reported Information. Journal of Applied Sport Psychol-
ogy, 17(1), 1-19. doi:10.1080/10413200590907531
Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R.,
Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician
self-assessment compared with obser ved measures of
competence: A systematic review. Journal of the Ameri-
can Medical Association, 296(9), 1094-1102. doi:10.1001/
jama.296.9.1094
de Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spin-
hoven, P. (2012). Understanding the differential impact of
outcome monitoring: Therapist variables that moderate
feedback effects in a randomized clinical trial. Psycho-
therapy Research, 22(4), 464-474. doi:10.1080/10503307.201
2.673023
Duckworth, A. L., Kirby, T. A., Tsukayama, E., Berstein, H., &
Ericsson, K. A. (2011). Deliberate Practice Spells Success.
Social Psychological and Personality Science, 2(2), 174-181.
doi:10.1177/1948550610385872
Duncan, B.L., & Miller, S.D. (2000). The Heroic Client: Doing
Client-Directed, Outcome-Informed Therapy. San Francisco:
Jossey-Bass.
Duncan, B., Miller, S., Wampold, B., & Hubble, M. (eds.)
(2010). The heart and soul of change (2nd Ed.):
Delivering“what works.” Washington, D.C.: APA Press.
Ericsson, A. K. (1996). The acquisition of expert performance:
An introduction to some of the issues. In K. A. Ericsson
(Ed.), The road to excellence: The acquisition of expert perfor-
mance in the arts and sciences, sports, and games (pp. 1-50).
Mahwah, N.J.: Lawrence Erlbaum Associates.
Ericsson, K. A. (2006). The Influence of experience and delib-
erate practice on the development of superior expert
performance. In K. A. Ericsson, N. Charness, P. J. Fel-
tovich & R. R. Hoffman (Eds.), The Cambridge handbook of
expertise and expert performance. (pp. 683-703). Cambridge:
Cambridge University Press.
Ericsson, K. A. (2007). An expert-performance perspective
of research on medical expertise: the study of clini-
cal performance. Medical Education, 41(12), 1124-1130.
doi:10.1111/j.1365-2923.2007.02946.x
Ericsson, K. A. (2009). Enhancing the development of pro-
fessional performance: Implications from the study of
deliberate practice. In K. A. Ericsson (Ed.), Development
of professional expertise: Toward measurement of expert per-
formance and design of optimal learning environments. (pp.
405-431). New York, NY: Cambridge University Press; US.
Ericsson, K. A., Krampe, R. T., & Tesch-Romer, C. (1993). The
role of deliberate practice in the acquisition of expert
performance. Psychological Review, 100(3), 363-406.
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and excep-
tional performance: Evidence of Maximal Adaptation to
Task Constraints. Annual Review of Psychology, 47(1), 273-
305. doi:10.1146/annurev.psych.47.1.273
Ericsson, K. A., Nandagopal, K., & Roring, R. W. (2005). Gift-
edness viewed from the expert-performance perspective.
Journal for the Education of the Gifted, 28(3-4), 287-311.
doi:10.4219/jeg- 2005-335
Ericsson, K., Roring, R. W., & Nandagopal, K. (2007). Misun-
derstandings, agreements, and disagreements: Toward
a cumulative science of reproducibly superior aspects of
giftedness. High Ability Studies, 18(1), 97-115. doi:http://
dx.doi.org/10.1080/13598130701351088
Fleming, J. & Asplund, J. (2007). Human Sigma. New York:
Gallup Press.
Garcia, J. A., & Weisz, J. R. (2002). When youth mental health
care stops: Therapeutic relationships problems and other
reasons for ending youth outpatient treatment. Journal of
Consulting and Clinical Psychology, 70(2), 439-443.
Gobet, F., & Charness, N. (2006). Expertise in chess. In K. A.
Ericsson, N. Charness, P. J. Feltovich & R. R. Hoffman
(Eds.), The Cambridge handbook of expertise and expert
performance. (pp. 523-538). Cambridge: Cambridge Uni-
versity Press.
Hansen, N., Lambert, M. J., & Forman, E. M. (2002). The psy-
chotherapy dose-response effect and its implication for
treatment delivery services. Clinical Psychology: Science
and Practice, 9(3), 329–343.
Hiatt, D., & Hargrave, G. (1995). The characteristics of highly
effective therapists in managed behavioral provid-
ers networks. Behavioral Healthcare Tomorrow, 4, 19-22.
doi:10.1037//0278-7393.26.3.566
Horvath, A. O. (2001). The alliance. Psychotherapy: Theory,
Research, Practice, Training, 38(4), 365-372. doi:http://
dx.doi.org/10.1037/0033-3204.38.4.365
Horvath, A. O., Del Re, A., Fluckiger, C., & Symonds, D.
(2011). Alliance in Individual Psychotherapy. Psycho-
therapy: Theory, Research, Practice, Training, 48(1), 9-16.
doi:10.1037/a0022186
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz,
W. (1996). The evaluation of psychotherapy: Efficacy,
effectiveness, and patient progress. American Psychologist,
51, 1059–1064.
Hubble, M., Duncan, B., & Miller, S. (1999). The heart and soul
of change. Washington, DC: APA press.
Huppert, J., Fabbro, A., & Barlow, D. (2006). Evidence-based
practice and psychological treatments (pp. 131-152). In
C. Goodheart, A. Kazdin, & R. Sternberg (eds.). Evidence-
based psychotherapy: Where practice and research meet.
Washington, DC: APA press.
Kahneman, D. (2011). Thinking, fast and slow. New York, NY:
Farrar, Straus and Giroux.
Kazdin, A. E. (1996). Dropping out of child psychotherapy:
Issues for research and implications for practice. Child
Clinical Psychology, 1, 133-156.
Keith, N., & Ericsson, K. (2007). A deliberate practice account
of typing proficiency in everyday typists. Journal of Exper-
imental Psychology: Applied, 13(3), 135-145. doi:http://
dx.doi.org/10.1037/1076-898X.13.3.135
Krampe, R. T., & Ericsson, K. (1996). Maintaining excellence:
Deliberate practice and elite performance in young and
older pianists. Journal of Experimental Psychology: General,
125(4), 331-359.
Kruger, J. (1999). Lake Wobegon be gone! The “below-average
effect” and the egocentric nature of comparative ability
judgments. Journal of Personality and Social Psychology,
77(2), 221-232. doi:http://dx.doi.org/10.1037/0022-
3514.77.2.221
Lambert, M. (1992). Psychotherapy outcome research: Impli-
cations for Integrative and Eclectic Therapists. In J.
Norcross & M. Goldfried (eds.) Handbook of psychotherapy
integration (pp. 94-129).New York: Basic Books
Lambert, M. (2010). Yes, it is time to routine track patient
outcome. In B. Duncan, S. Miller, B. Wampold, & M.
Hubble (eds.). The heart and soul of change (pp. 239-266).
Washington, D.C.: APA Press.
Mamede, S., Schmidt, H. G., Rikers, R. M. J. P., Penaforte,
J. C., & Coelho-Filho, J. M. (2007). Breaking down
automaticity: case ambiguity and the shift to reflective
approaches in clinical reasoning. Medical Education,
41(12), 1185-1192. doi:10.1111/j.1365-2923.2007.02921.x
Miller, S.D. (2010). Psychometrics of the ORS and SRS: Results
from RCTs and Meta-Analyses of Routine Outcome Monitoring
& Feedback. Chicago, IL: ICCE.
Miller, S. D., & Bargmann, S. (2011). Feedback-informed treat-
ment (FIT): Improving the treatment of male clients one
man at a time (pp. 194-208). In Ashfield, J. (Ed.). Doing
psychotherapy with men: Practicing ethical psychotherapy
and counselling with men. Australian Institute of Male
Health and Studies.
Miller, S.D., & Duncan, B.L. (2000). The outcome rating scale.
Chicago, IL: International Center for Clinical Excellence.
Miller, S.D., & Duncan, B.L. (2004). The Outcome and Session
Rating Scales: Administration and Scoring Manual. Chicago,
IL: ISTC.
Miller, S.D., Duncan, B.L., Brown, J., Sparks, J., & Claud, D.
(2003). The Outcome Rating Scale: A preliminar y study
of the reliability, validity, and feasibility of a brief visual
analog measure. Journal of Brief Therapy, 2, 91-100.
Miller, S.D., Duncan, B.L., & Johnson, L. (2000). The Session
Rating Scale. Chicago, IL: International Center for Clinical
Excellence.
Miller, S.D., Hubble, M.A., and Duncan, B.L. (2007). Super-
shrinks: Learning from the field’s most effective
practitioners. The Psychotherapy Networker, 31, 26-35, 56.
Moore, D. S. (1994). The craft of teaching. Address at the award
ceremony for distinguished college or university teaching of
mathematics. San Francisco, CA. Retrieved from http://
www.stat.purdue.edu/~dsmoore/ articles/Craft.pdf
Najavits, L. M., & Strupp, H. H. (1994). Differences in the
effectiveness of psychodynamic therapists: A process-
outcome study. Psychotherapy: Theory, Research, Practice,
Training, 31(1), 114-123. doi:10.1037/0033-3204.31.1.114
Nissen-Lie, H. A., Monsen, J. T., & Ronnestad, M. H. (2010).
Therapist predictors of early patient-rated working alli-
ance: A multilevel approach. Psychotherapy Research,
20(6), 627-646. doi:10.1080/10503307.2010.497633
Norman, G., Eva, K., Brooks, L., & Hamstra, S. (2006).
Expertise in medicine and surgery. In K. A. Ericsson,
N. Charness, P. J. Feltovich & R. R. Hoffman (Eds.), The
Cambridge handbook of expertise and expert performance.
(pp. 339-353). Cambridge: Cambridge University Press.
Orlinsky, D. E., & Ronnestad, M. H. (2005). How psychothera-
pists develop: A study of therapeutic work and professional
growth. Washington: American Psychological Association.
Schmidt, H. G., & Rikers, R. M. (2007). How expertise devel-
ops in medicine: knowledge encapsulation and illness
script formation. Medical Education, 41(12), 1133-1139.
doi:10.1111/j.1365-2923.2007.02915.x
Siev, J., Huppert, J., & Chambless, D. (2009). The dodobird,
treatment technique, and disseminating empirically sup-
ported treatments. The Behavior Therapist, 32, 69-76.
Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psycho-
therapy outcome studies. American Psychologist, 32(9),
752–760.
Sonnentag, S., & Kleine, B. M. (2000). Deliberate practice at
work: A study with insurance agents. Journal of Occu-
pational and Organizational Psychology, 73(1), 87-102.
doi:http://dx.doi.org/10.1348/096317900166895
Starkes, J. L., Deakin, J. M., Allard, F., Hodges, N., & Hayes,
A. (1996). Deliberate practice in sports: What is it
anyway? In K. A. Ericsson (Ed.), The road to excellence:
The acquisition of expert performance in the arts and sciences,
sports, and games. (pp. 81–106). Mahwah, NJ: Erlbaum.
Swift, J.K., & Greenberg, R.P. (2012). Premature discontinu-
ation in adult psychotherapy: A meta-analysis. Journal of
Consulting and Clinical Psychology, 80, 547-559. doi:10.1037/
a0028226
Tracey, T. J. G., Wampold, B. E., Lichtenberg, J. W., & Good-
year, R. K. (2014). Expertise in psychotherapy: An elusive
goal? American Psychologist, Advanced online publication.
doi:http://dx.doi.org/10.1037/a0035099
Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J.
(2012). An investigation of self-assessment bias in mental
health providers. Psychological Reports, 110(2), 639-644.
doi:http://dx.doi.org/10.2466/02.07.17.PR0.110.2.639-644
Wampold, B. E. (2001). The great psychotherapy debate: Models,
methods, and findings. Mahwah, NJ: Lawrence Erlbaum.
NJ: Lawrence Erlbaum.
Wampold, B. E. (2005). What should be validated? The psycho-
therapist. In J. C. Norcross, L. E. Beutler, & R. F. Levant
(Eds.), Evidence-based Practices in Mental Health (pp. 200–
208, 236–238). Washington, DC: American Psychological
Association.
Wampold, B. E., & Brown, G. S. (2005). Estimating variabil-
ity in outcomes attributable to therapists: A naturalistic
study of outcomes in managed care. Journal of Consulting
& Clinical Psychology, 73(5), 914-923. doi:10.1002/jclp.20110
Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psy-
chotherapy dropout. Professional Psychology: Research and
Practice, 24(2), 190-195.
and .
Providing Divorce and Custody Mediation Services: The Basics
— Lori C. Thomas
Increasingly parties who are seeking to
avoid the financial and emotional, as well
as collateral damage to their children,
have steered away from traditional court-based
divorces process and have instead opted to
purse divorce mediation. In divorce media-
tion, the mediator serves as a neutral third
party who facilitates a discussion between
divorcing parties, as they negotiate a mutu-
ally acceptable agreement in the dissolution of
their marriage. As a neutral third party, the
mediator is not empowered to make decisions
for the parties.
In the process of mediation, parties may negotiate any
aspect of the termination of their marriage including
child custody, child support, and property distribution.
Parties can enter mediation either privately or through
a court-ordered process.
In some jurisdictions, courts will mandate that par-
ties who file a petition for child custody or visitation
attend mediation, with the goal of resolv-
ing their dispute before the court makes a
ruling on their custody matter. Court-ordered
mediations are one way in which courts have
attempted to relieve the court of the endless
backlog of custody cases that flow through the
family court system. In jurisdictions that
contain court-ordered mediations, courts will
typically maintain a list of mediators. There
are varying requirements for getting placed
on a court list. For example, as a mediator in
Chester County Pennsylvania, I was required
to have both basic and advanced mediation training.
Additionally, the court required a specified number of
supervised mediation cases prior to being placed on
the court list. Once placed on the court list, the court
then sets the fee for those court ordered mediations.
Additionally, I was required to conduct a court-specified
number of pro-bono mediations each year.
Divorce and custody mediation are typically governed
by state statute, which provides the minimum qualifi-
