A preview of this full-text is provided by American Psychological Association.
Content available from Canadian Psychology/Psychologie canadienne
This content is subject to copyright. Terms and conditions apply.
Will Publicly Funded Psychotherapy in Canada Be Evidence Based?
A Review of What Makes Psychotherapy Work and a Proposal
Giorgio A. Tasca
University of Ottawa
Joel M. Town and Allan Abbass
Dalhousie University
Jeremy Clarke
London School of Economics
Jurisdictions in Canada, notably in Ontario and Quebec, are proposing a laudable goal of increasing
publicly funded access to psychotherapy. Ontario and Quebec will likely follow the lead of the Increasing
Access to Psychotherapy (IAPT) program in the United Kingdom and train psychotherapists to provide
cognitive– behavioural therapy. Results from IAPT provide some important lessons about taking an
approach that prefers one brand of psychotherapy. We argue that such policy decisions are based on a
medical model approach to psychotherapy that makes erroneous assumptions about what psychotherapy
is, what is the nature of the evidence, and how training and services should be delivered. In this article,
we review critically these assumptions and the state of the art of the research about what makes
psychotherapy work. Psychotherapy is effective and preferable to antidepressant medication. Differences
between psychotherapies account for a small proportion of variance in patient outcomes. The largest
known predictors of patient outcomes are patient factors (coping style, resistance/reactance, interpersonal
problems, culture) and therapeutic relationship factors (therapeutic alliance). Also notable are therapist
factors (facilitative interpersonal skills, empathy, and managing countertransference), and practices like
progress monitoring. Canadian jurisdictions should note that training therapists to adapt treatment and
interpersonal stances to patient characteristics, to develop and maintain the therapeutic relationship, to
enhance therapist facilitative interpersonal skills, and to engage in progress monitoring has a greater
chance of achieving a reduction in the burden caused by depression and anxiety among their citizens than
focusing on delivering a particular brand of psychotherapy.
Keywords: psychotherapy, improving access to psychotherapy, outcomes, patient factors, therapeutic
relationship
In 2017, the provincial government of Ontario, Canada, an-
nounced new funding ($25 million per year for 3 years) to improve
access to psychotherapy among its citizens with mild to moderate
depression and anxiety disorders (Ontario Ministry of Health &
Long Term Care, 2017). This was welcome news for a population
that to date receives insufficient government assistance to access
psychotherapy. For the most part, unless one is fortunate enough to
work for an employer that provides sufficient extended health
benefits for psychological treatment, many Canadians have little
access to timely psychotherapy. This plan by the government of
Ontario is meant to address this shortcoming of the health system
regarding mental health care. Later in the year, the Quebec gov-
ernment announced a similar measure to increase access to psy-
chotherapy (Anderssen, 2017). The challenge at this stage, in
Ontario and Quebec, is how the money will be allocated, how
treatment will be provided, what kind of treatment, and to whom.
To help to make this decision in Ontario, the Ministry of Health
and Long Term Care (MOHLTC) received a consultation from
Health Quality Ontario (HQO), a government funded organisation
that studies and evaluates the provision of health interventions. In
its report, HQO emphasised the need to deliver evidence-based
psychotherapy to those with mild to moderate depression and
anxiety, and made a number of recommendations in that regard
(HQO, n.d.). Among the recommendations was the need to provide
“structured psychotherapy” by trained practitioners. Currently, the
MOHLTC and its partners have interpreted this to mean providing
time-limited cognitive– behavioural therapy (CBT; The Royal
Mental Health Care, 2017). Five main psychiatric hospitals in
Ontario will organise and provide training to community-based
therapists. On the face of it, the MOHLTC appears to be relying on
aspects of the Improving Access to Psychotherapy (IAPT) pro-
gram in the United Kingdom as a successful model for it to follow
Giorgio A. Tasca, School of Psychology, University of Ottawa; Joel M.
Town, Departments of Psychiatry and Psychology and Neuroscience, Dal-
housie University; Allan Abbass, Centre for Emotions and Health, Dal-
housie University; Jeremy Clarke, Centre for Philosophy of Natural and
Social Science, London School of Economics.
Jeremy Clarke was former National Advisor for the Improving Access to
Psychotherapy program in the United Kingdom from 2008 to 2013.
Correspondence concerning this article should be addressed to Giorgio
A. Tasca, School of Psychology, University of Ottawa, 136 Jean-Jacques
Lussier, Ottawa, Ontario, Canada, K1N6N5. E-mail: gtasca@uottawa.ca
Canadian Psychology / Psychologie canadienne
0708-5591/18/$12.00 2018, Vol. 59, No. 4, 293–300
© 2018 Canadian Psychological Association http://dx.doi.org/10.1037/cap0000151
293
(National Health Service [NHS], n.d.). Quebec announced a sim-
ilar plan to follow aspects of the IAPT model (Delorme, 2018).
The HQO report and its interpretation make a number of as-
sumptions about the definition of psychotherapy and the state and
quality of the evidence. These assumptions have implications for
the way in which psychotherapy and the evidence is understood
and how psychotherapy can and should be delivered. In this article,
we argue that if the aim is simply to increase and achieve new
access targets, then the United Kingdom. IAPT model offers a
good template (Clark et al., 2018). But if the aim is to reduce the
burden of depression, then the IAPT example offers some impor-
tant lessons. After a decade of government investing some £1.3
billion, IAPT has certainly increased access to CBT (NHS Digital,
n.d.a;n.d.b). Around 1 million people annually now access IAPT
services, almost all of whom wait less than 6 weeks from referral
to treatment (Baker, 2017). But IAPT has yet to make any real
impact on reducing the burden of depression and its social and
economic consequences (Jorm, Patten, Brugha, & Mojtabai, 2017;
McManus, Bebbington, Jenkins, & Brugha, 2016;Organisation for
Economic Co-Operation and Development, 2014). This is partly
because (a) between 2007 and 2017, substantially less than half of
patients referred, in some cases less than a third, chose to take up
and complete the IAPT treatment on offer (NHS Digital, n.d.c;
Johns, 2017); (b) less than half of IAPT patients who complete
treatment achieve recovery (NHS Digital, n.d.c); and (c) less than
half of treatment completers who achieve recovery from low-
intensity CBT (the most common intervention) remain recovered
at 12 months posttreatment (Ali et al., 2017). Hence, contrary to
IAPT claims that almost 50% of patients recover, in reality only
10% to 20% of patients achieve real lasting recovery in IAPT.
There is also a clear disparity between reported recovery rates in
the most deprived areas of the United Kingdom (35%) and the
least deprived areas (55%; NHS Digital, n.d.b), with most deprived
areas overrepresented by people on social security, which is im-
portant as these were the target group IAPT promised to help, and
the basis for the original claim that the programme ‘would pay for
itself’. It appears that flaws in IAPT’s original design and imple-
mentation in the United Kingdom may have had unintended con-
sequences for those who need help most. Thus, it is not yet clear
that IAPT’s model, based on CBT, is either cost-effective or
sustainable (McCrone, 2013).
We suggest an alternative model of training and delivering
psychotherapy in Canada that is more closely aligned with the
research evidence for what makes psychotherapy work. Such a
model may achieve all of the important aims that IAPT in the
United Kingdom originally set out to deliver (Layard & Clark,
2014). Adopting this in Ontario and Quebec, as we demonstrate
below, would make increased access to psychotherapy a cost-
effective, sustainable policy.
What Is Psychotherapy and What Is
Evidence-Based Practice?
In 2012, the American Psychological Association (APA,
2012) released a report on the Recognition of Psychotherapy
Effectiveness. In it, the APA defined psychotherapy as the “. . .
informed and intentional application of clinical methods and
interpersonal stances derived from established psychological
principles for the purpose of assisting people to modify their
behaviours, cognitions, emotions, and/or other personal char-
acteristics in directions that the participants deem desirable” (p.
1). The phrase “interpersonal stances” is critical to this defini-
tion in that it recognizes the important role of the quality of the
therapeutic relationship in predicting patient outcomes (Nor-
cross & Lambert, 2011). We will return to this issue below
when we review the evidence for the impact of the therapeutic
relationship relative to differences between therapeutic ap-
proaches.
A second key aspect of that APA (2012) report was the defini-
tion of evidence-based practice (EBP), originally put forward in
2006 by APA, as the “. . . integration of the best available research
with clinical expertise in the context of patient characteristics,
culture and preferences” (APA Task Force on Evidence Based
Practice, 2006, p. 273). This represents a broader view of EBP that
goes beyond exclusively relying on establishing empirically sup-
ported treatments (EST; Chambless & Hollon, 1998). The EST
approach developed lists of psychotherapies that were deemed
efficacious if the treatment demonstrated superior effects to a
control condition or was equally effective to another EST in
randomized controlled trials. This approach tends to pit psycho-
therapies against each other in an effort to identify which treatment
is most efficacious. However, the broader EBP approach adopted
by APA in 2006 relies on three key and equal aspects: the research
evidence, clinical expertise, and patient characteristics. As our
review below indicates, it turns out that patient characteristics are
the largest predictor of patient outcomes, which is an important
consideration when designing a public mental health programme
covering a heterogeneous population, and in which health inequal-
ities may limit access for some (Dixon-Woods et al., 2005).
Despite the APA definitions of psychotherapy and of EBP,
which recognise the importance of the therapeutic relationship and
patient factors in determining outcomes, some researchers and
writers conceptualise psychotherapy as if it were a pharmacolog-
ical intervention (Wampold & Imel, 2015). This is indicative of a
medical model of psychotherapy in which its effectiveness is due
to specific interventions (cognitive restructuring, transference in-
terpretations, two-chair techniques) targeting specific causes of
disorders (cognitive distortions, inner conflict, emotional block-
ages). This is much the same as the way that a pharmacologist
conceptualizes a pharmacological intervention (selective serotonin
reuptake inhibitors) that target dysfunction in a specific neu-
rotransmitter system (the serotonin system) as the cause of depres-
sion. A medical model of psychotherapy suggests that some brands
of psychotherapy will be more effective than others because of
their specific interventions. Such a model also suggests that psy-
chotherapy research and practice must control for all so-called
nonspecific aspects of psychotherapy (differences between thera-
pists, the therapeutic relationship) to isolate the effects of the active
ingredient on the mechanism causing the disorder (Wampold & Imel,
2015). However in CBT, for example, there is little research support-
ing the theory that challenging cognitions leads to symptomatic im-
provement (Longmore & Worrell, 2007).
This may appear to be an esoteric discussion at first, but bear
with us. Conceptualising psychotherapy as a medical intervention
has a profound impact on how one views and practices psycho-
therapy, and for how governments fund psychotherapy. Within the
medical model approach, the therapist, the therapeutic relationship,
and patient characteristics (other than specific symptoms) are only
294 TASCA, TOWN, ABBASS, AND CLARKE
peripherally relevant to the workings of psychotherapy or at least
need to be controlled as background noise. Therefore, within a
medical model, psychotherapy practice is about applying a specific
intervention while trying to control for or simply ignoring differ-
ence between therapists, the therapeutic relationship, and patient
characteristics and preferences. This is akin to viewing psycho-
therapy as a medical technology device. Further, a medical model
predicts that some psychotherapy will demonstrate superior effi-
cacy to others because their specific ingredients or actions will
target the purported specific causes of a disorder. Practically, for
funders like government departments, this means that one should
fund one brand of psychotherapy and not another. However, as we
demonstrate below, there is diminishing evidence that any one
form of bona fide psychotherapy is on average more effective than
another bona fide psychotherapy for many disorders, and this is
especially true for depression. Moreover, IAPT’s own outcome
data shows broad equivalence for its different therapies: counsel-
ling, interpersonal psychotherapy, couple therapy, brief dynamic
therapy, and CBT, with the latter being marginally less effective in
recent reports (NHS Digital, n.d.a,n.d.b;Pybis, Saxon, Hill, &
Barkham, 2017;Royal College of Psychiatrists, 2011,2013).
Bona Fide Psychotherapies Are Effective
By bona fide psychotherapies, we mean psychotherapies that are
based on a psychological theory, meant to be effective for a
specific disorder, delivered by a trained therapist, and have dem-
onstrated evidence for their efficacy or effectiveness (Wampold &
Imel, 2015). The field has known for some time that psychother-
apies are more effective than no treatment (Lambert & Bergin,
1994;Grissom, 1996), such that the effect sizes for psychotherapy
are as large as or larger than most common medical interventions
(Rosnow & Rosenthal, 2003). The average effect size for psycho-
therapy versus no treatment is large (d⫽.80 or r⫽.37) indicating
that the average patient is better off than 79% of untreated controls
(Wampold & Imel, 2015).
Psychotherapies are as effective as antidepressant medications
for reducing depressive symptoms (Spielmans, Berman, & Usitalo,
2011). However, there is also evidence that psychotherapy has better
long-term outcomes (de Maat, Dekker, Schoevers, & De Jonghe,
2006;Imel, Malterer, McKay, & Wampold, 2008;Spielmans et al.,
2011), lower drop-out rates (Cuijpers, van Straten, Andersson, &
van Oppen, 2008), and psychotherapy is less susceptible to treat-
ment resistance after an unsuccessful course of therapy (Wampold
& Imel, 2015). Turner, Matthews, Linardatos, Tell, and Rosenthal
(2008) unequivocally demonstrated that the effects of antidepres-
sant medications are inflated by publication bias and active sup-
pression of negative data by the pharmaceutical industry. While
publication bias is also evident in trials of psychotherapy and CBT
in particular, the resulting inflation seems to be smaller (Cuijpers
et al., 2008). Further, a meta-analysis indicated that 75% of pa-
tients would prefer psychotherapy over medications if they had the
choice (McHugh, Whitton, Peckham, Welge, & Otto, 2013), and
that patients who receive their preferred treatment tend to have
better outcomes (Swift, Callahan, & Vollmer, 2011). In the United
Kingdom, a large cross-sectional survey of National Health Ser-
vice patients treated for depression also showed that a choice of
their preferred type of therapy was important to them, and would
impact their outcome (Williams et al., 2016). Yet, by far, most
patients in North America are offered medications for mental
disorders (Olfson & Marcus, 2010) likely because of limited
accessibility to specialized psychotherapy, even if psychother-
apy is cost-effective (Vasiliadis, Dezetter, Latimer, Drapeau, &
Lesage, 2017). Hence, the Ontario and Quebec government plan
for publicly funded psychotherapy is an important step to fill
the gap in access to a range of efficacious psychotherapies for
depression and anxiety disorders.
Bona Fide Psychotherapies Do Not Differ for
Depression Outcomes
A number of bona fide psychotherapies for depression exist, in-
cluding CBT (Beck, 2011), interpersonal psychotherapy (Klerman,
Weissman, Rounsaville, & Chevron, 1984), emotion-focused
therapy (Greenberg, 2010), short-term psychodynamic psycho-
therapy (Luborsky et al., 1995), behavioural activation (Martel,
Dimidjian, & Herman-Dunn, 2010), and others. The best way to
summarise the outcome literature on the relative efficacy of psycho-
logical treatments is to conduct meta-analyses of studies that directly
compare treatments to each other. Meta-analysis can produce the most
reliable estimates of the effect of treatments, and meta-analyses of
studies that directly compare treatments reduce between-study error
and bias. The vast majority of meta-analyses of direct comparisons
show that there is no difference between psychological therapies
when treating patients with depression (see e.g., Cuijpers et al., 2008;
Cuijpers et al., 2013). There is also evidence that the effects of CBT
are declining over time (Johnsen & Friborg, 2015), likely because
more recent studies are of higher quality leading to lower effects, and
the initial enthusiasm of developers of CBT may have inflated early
findings. Also, the particular brand of psychotherapy is not a signif-
icant moderator of drop-out rates from treatment (another indicator of
outcomes; Swift & Greenberg, 2012). To summarize, psychotherapy
is effective, it is just as effective as antidepressant medications,
psychotherapy is preferred by 75% of patients, and differences be-
tween treatments account for a small amount of the variability in
patient outcomes.
Finally, a meta-analysis demonstrated that therapist adherence
to (r⫽.02) and competence in (r⫽.07) a treatment manual of a
specific intervention bear almost no relationship to patient out-
comes (Webb, Derubeis, & Barber, 2010). In other words, the
degree to which a therapist learns to follow and apply a psycho-
logical treatment manual, like CBT (Beck, 2011), interpersonal
psychotherapy (Klerman et al., 1984), or short-term psychody-
namic psychotherapy (Luborsky et al., 1995), has almost no impact
whether their patients get better. If there are no differences be-
tween treatments, and the degree to which one follows a treatment
manual does not improve outcomes, then what is it that accounts
for the positive effects of psychotherapy? And what can these
predictors tell us about how to train therapists to achieve the best
possible patient outcomes for all?
Factors That Reliably Predict Patient Outcomes
In 2011, Norcross and Lambert did a sweeping review of the
psychotherapy outcome and process literature, and also surveyed
the leading psychotherapy researchers in the world about predic-
tors of patient outcomes. They produced a striking pie chart
illustrating the amount of patient outcome variance accounted for
295
WHAT MAKES PSYCHOTHERAPY WORK
by those factors studied over the decades. By far, the largest
known predictor of outcomes was patient factors, which accounted
for 30% of the variance, followed by the therapeutic relationship at
12%, specific treatment effects at 8%
1
, and differences between
therapists at 7%. A remarkable aspect of these findings is that
while most of the funding, training, and research in psychotherapy
focus on specific treatment models (accounting for at most 8% of
the variance in patient outcomes), more important predictors of
outcomes lie in other areas including patient characteristics (30%
of the variance) and the therapeutic relationship (12% of the
variance). This is not to say that specific treatments are not
relevant, but that an overemphasis on their effect represents an
oversimplification of how change occurs in psychotherapy.
Patient Factors
Patient factors include a number of aspects that affect outcomes and
that can be adapted to, managed, or enhanced by trained and knowl-
edgeable therapists. For example, in a meta-analysis, Constantino,
Arnkoff, Glass, Ametrano, and Smith (2011) found a significant
association between patient expectations of receiving benefit and
outcomes, r⫽.24, p⬍.001. In other words the degree to which
a therapist helps to instill a sense of hope in patients may improve
their outcomes. Patient coping style and its impact on therapy has
long been of interest to therapists and researchers. A meta-anslysis
by Beutler, Harwood, Kimpara, Verdirame, and Blau (2011) found
a significant effect of coping style on patient outcomes, r⫽.26,
p⬍.001. Specifically, patients with externalizing coping tend to
do better in symptom-focused treatment, whereas patients with
internalizing coping tend to have better outcomes in insight-
oriented treatments. Another meta-analysis found a significant
association, r⫽.38, p⬍.001, for matching patient resistance/
reactance to therapist directiveness. Patients higher in resistance or
reactance had better outcomes if therapists were less directive,
whereas patients with lower resistance or reactance did better with
therapists who were more directive (Beutler et al., 2011). Simi-
larly, there is evidence from systematic reviews and meta analyses
that higher pretreatment interpersonal problems and interpersonal
distress (McFarquhar, Luyten, & Fonagy, 2018), greater attach-
ment anxiety (Levy, Ellison, Scott, & Bernecker, 2011), higher
functional impairment (Beutler et al., 2002), and greater severity,
complexity, and comorbidity (Bohart & Wade, 2013) are associ-
ated with poorer outcomes. Finally, a meta-analysis found that
adapting psychotherapy to patient culture results in better out-
comes than culturally unadapted psychotherapy (Benish, Quintana,
& Wampold, 2011). This latter issue is well illustrated by data
from IAPT in the United Kingdom. As noted above, patients living
in the most economically deprived areas were less likely to finish
a course of treatment and less likely to recover (NHS Digital,
n.d.b).
Therapeutic Relationship
The most studied therapeutic relationship factor is the therapeu-
tic alliance. Therapeutic alliance is commonly defined as the
emotional bond between therapist and patient plus a collaborative
agreement on the tasks and goals of therapy. In a large meta-
analysis of over 200 studies, Horvath, Del Re, Flückiger, and
Symonds, (2011) found a highly robust relationship between
higher alliance and better patient outcomes, r⫽.28, p⬍.001.
Who measured the alliance (patient, therapist, rater), when it was
measured (early, middle, or late in therapy), which scale was used,
and what type of therapy made little difference to this significant
association. There is now a growing consensus that the alliance is
a necessary precursor to symptom change in all types of therapy
(Zilcha-Mano, 2017). Research is also indicating that the quality of
the alliance fluctuates throughout treatment, and this fluctuation
has implications for treatment. For example, alliance ruptures
(disagreements or tensions between therapists and patients) likely
occur in 50% of cases within the first six sessions of therapy (Muran
et al., 2009). Unresolved alliance ruptures are significantly associated
with poorer patient outcomes (Muran et al., 2009). In a meta-analysis,
Safran, Muran, and Eubanks-Carter (2011) reported that therapists
ability to repair alliance ruptures was associated with better patient
outcomes, r⫽.24, p⫽.002, and that training therapists to identify
and repair alliance ruptures is associated with better pre- to posttreat-
ment patient outcomes, r⫽.56, p⬍.001.
Therapist Effects
Finally, therapist effects (i.e., differences between therapists)
appear to have a small but significant effect on patient outcomes
(Wampold & Brown, 2005). That is, some therapists are reliably
more (or less) effective than others. Training therapists with a
treatment manual is, in part, designed to diminish therapist effects,
but as noted previously, adherence and competence to a manual
appear to have little bearing on patient outcomes (Webb et al.,
2010). Data from IAPT and NHS-funded services in the United
Kingdom also illustrate this point by revealing that considerable
variability in patient dropping out (12.1%) and deterioration
(10.1%) was explained by differences between therapists (Saxon,
Barkham, Foster, & Parry, 2017). Using data collected from
33,243 patients treated for depression within IAPT services, re-
search found that variability between service providers had a
significant effect on patient outcomes while the type of therapy
they received did not (Saxon, Firth, & Barkham, 2017). To illus-
trate the practical importance of even small therapist differences,
one large study in the United States of over 700 therapists and
7000 patients found that 16% of the therapists (n⫽112) were
reliably harmful (Kraus, Castonguay, Boswell, Nordberg, & Hayes,
2011). Those patients (n⫽1,120) treated by reliably harmful thera-
pists on average got worse due to therapy, and this result was main-
tained even after within-therapist case mix variables (size of therapist
caseload, patient diagnosis, patient severity) were controlled. The
implications of this research are that therapists need to be trained in
those factors that lead to better patient outcomes, and that relying on
therapist adherence to a treatment protocol may not improve matters.
What are the therapist factors that are associated with better
outcomes? One potential candidate is therapist facilitative inter-
1
Meta-analyses indicate that differences between treatments account for
approximately 8% of the variance and that this may be the upper bound of
this estimate (see Wampold & Imel, 2015;Wampold & Serlin, 2014).
Wampold and Imel (2015) argued that this percentage is inflated by
researcher allegiance (i.e., the effects of a researcher’s preference for a
particular treatment modality for which they have the greatest affinity).
When researcher allegiance is controlled, the amount of patient outcome
variance accounted for by differences between treatments reduces from 8%
to 1% (see also Lambert & Ogles, 2004).
296 TASCA, TOWN, ABBASS, AND CLARKE
personal skills—which includes verbal fluency, warmth, and emo-
tional expression. Research has shown that therapist facilitative
interpersonal skills predict patient outcomes with large effects
(Anderson, Ogles, Patterson, Lambert, & Vermeersch, 2009). In
particular, therapist empathy is highly associated with positive
patient outcomes (Elliott, Bohart, Watson, & Greenberg, 2011).
Another candidate is therapists’ ability to identity and manage
countertransference (Hayes, Gelso, & Hummel, 2011), which re-
fers to therapists’ emotional reactions triggered by patients but
which implicates a therapist’s unresolved conflicts. In a meta-
analysis, Hayes and colleagues (2011) found that training thera-
pists to manage countertransference was associated to better pa-
tient outcomes with moderate effects. This is consistent with a
meta-analysis indicating that therapists (not patients) are largely
responsible for the alliance-outcome relationship (Del Re, Flückiger,
Horvath, Symonds, & Wampold, 2012).
A therapist practice that robustly improves patient outcomes is
progress monitoring, or the regular assessment of patient outcome
during therapy (e.g., weekly or biweekly). The assessment patient
progress is done with psychometrically sound instruments fol-
lowed by feedback to the therapist on the status of the patient
relative to standardized norms. The feedback process alerts ther-
apists to changes in patient status on a week-to-week basis, which
allows therapists to alter their interventions or to attend to a
previously unknown or negative process in the therapy. In a
meta-analysis, Lambert and Shimokawa (2011) found that provid-
ing therapists with feedback about patient progress was moderately
and significantly associated with positive outcomes, r⫽.24, p⬍
.001. In particular, using a progress monitoring system cut the
percentage of patients who deteriorated by half. This is important
because approximately 8% of patients get worse after psychother-
apy (Lambert, 2010).
An Alternative Model for Creating Capable,
Multidisciplinary Therapy Teams
The Ontario and Quebec plans to fund CBT for mild to mod-
erate depression and anxiety appear to be largely based on a
medical model, following the influential example of IAPT in the
United Kingdom. There is an obvious appeal to follow a model
that has shown it can improve access (Clark et al., 2018). However,
an overemphasis on apparently cost-effective ways to scale-up access
rapidly, such as use of computerized CBT, other low-intensity CBT
interventions, and stepped care is unlikely to yield the kinds of
longer-term benefits that offering a genuine choice of efficacious
psychotherapies could deliver (Ali et al., 2017). We suggest an alter-
native model that heeds these lessons and that follows from the
research on what makes psychotherapy work.
As Wampold and Imel (2015) argued, the medical model does
not apply well to psychotherapy because it de-contextualizes psy-
chotherapy by treating it as a set of technical procedures that are
not impacted by the therapeutic relationship, patient, and therapist.
However, as our review of the evidence indicates, it is precisely
these contextual factors (Norcross & Lambert, 2011) and not
adherence to a particular treatment protocol (Webb et al., 2010)
that predict better patient outcomes. A contextual model of psy-
chotherapy (Wampold & Imel, 2015) holds that (a) therapists need
to work within a theoretical orientation based on established psy-
chological principles and sets of procedures to address symptoms;
(b) patients and therapists need to agree on the tasks (based on
these procedures) and goals of therapy, which in part requires the
development, maintenance, and repair of the therapeutic alliance
(Horvath et al., 2011;Safran et al., 2011); (c) patients and thera-
pists need to like working with each other (another aspect of the
alliance), which in part requires therapists to have a high level of
facilitative interpersonal skills including empathy (Anderson et al.,
2009); and, crucially, (d) therapists must take into account patient
characteristics and preferences when designing and delivering
interventions, including patient coping style, reactance, interper-
sonal style, and culture.
This alternative contextual approach would not give primacy or
preference to a single therapeutic orientation like CBT, as has
happened in the United Kingdom. Instead, it would recognise that
therapists trained in many bona fide psychotherapies are likely to
be effective in treating the wide range of patients with heteroge-
neous presentations of depressive and other disorders (see Cuijpers
et al., 2008,2013;Wampold & Imel, 2015). We argue that the
training of capable multidisciplinary teams of therapists should be
consistent with the existing research to provide truly evidence-
based care to patients.
A Proposal
Rather than exclusively focusing the training of therapists on
only one particular treatment manual or protocol, therapist training
across a number of effective approaches should target those skills
that the evidence indicates directly improves patient outcomes.
Meta analyses indicate that methods like continuous progress
monitoring and feedback (Lambert & Shimokawa, 2011) and
video review of therapist skills (Town et al., 2012) can improve
therapist reflective practice and patient outcomes. Thus, it is im-
portant for evidence-based curriculums to train therapists compe-
tently to (a) develop and maintain a therapeutic alliance (Horvath
et al., 2011), (b) identify and repair alliance ruptures (Safran et al.,
2011), (c) properly assess patient characteristics that affect out-
comes (expectations, coping style, attachment, reactance, func-
tional impairment, interpersonal problems, level of social support,
motivation) and modify interpersonal stances and interventions
according to those patient characteristics (Beutler et al., 2002,
2011;Constantino et al., 2011;Levy et al., 2011;McFarquhar et
al., 2018), (d) adapt their interpersonal stance and intervention
according to a patient’s culture and context (Benish et al., 2011),
(e) enhance their facilitative interpersonal skills (Anderson, Crow-
ley, Himawan, Holmberg, & Uhlin, 2016) such as empathy (Elliott
et al., 2011) and identify and manage countertransference (Hayes
et al., 2011), and (f) use regular progress monitoring to identify
patients who might be deteriorating and then use the feedback to
alter therapist interpersonal stances and interventions.
We recognise that these evidence-based skills that therapists
must learn are complex and multifaceted. However, these skills
reflect what psychotherapy is: the “. . . informed and intentional
application of clinical methods and interpersonal stances derived
from established psychological principles for the purpose of as-
sisting people . . .” (APA, 2012, p. 1). Hence, although the training
of competence in these skills and stances may be more involved
than training someone to adhere to a treatment manual, such
training is more closely aligned with the research evidence and so
will result in better mental health outcomes for the population
297
WHAT MAKES PSYCHOTHERAPY WORK
being treated. Moreover, it offers a better career framework for
practitioners who can be recruited and retained from a wider range
of backgrounds.
We strongly encourage the Ontario MOHLTC, Quebec, and
other Canadian jurisdictions charged with increasing access to
psychotherapy to follow the evidence and not focus solely or
primarily on one brand of psychotherapy, CBT, as happened in the
United Kingdom. They should train therapists across a range of
effective approaches on those interpersonal stances and skills that
the research indicates will improve patient outcomes. Such an
approach would result in capable multidisciplinary teams that
tackle the serious challenge of reducing the burden of depression
and anxiety and their associated health and social inequalities.
Résumé
Certaines régions du Canada, notamment l’Ontario et le Québec,
proposent comme objectif louable d’accroître l’accès aux traite-
ments de psychothérapie financés par les deniers publics. Les deux
provinces suivront probablement l’exemple du programme Increasing
Access to Psychotherapy (IAPT) du Royaume-Uni, et formeront des
psychothérapeutes offrant la thérapie cognitivo-comportementale.
Les résultats de l’IAPT fournissent d’importantes leçons sur
l’adoption d’une approche privilégiant un type de psychothérapie
en particulier. Nous avançons que de telles décisions politiques
sont basées sur un modèle médical a
`l’égard de la psychothérapie
qui fait des suppositions erronées au sujet de la nature de cette
discipline, de la nature des preuves et de la façon dont la formation
et les services doivent être dispensés. Dans cet article, nous ex-
aminons d’un œil critique ces suppositions et faisons le point des
connaissances sur ce qui explique l’efficacité de la psychothérapie.
La psychothérapie est efficace et préférable au traitement au
moyen d’antidépresseurs. Les différences entre les psycho-
thérapies expliquent la petite proportion de la variance dans les
résultats chez les patients. Les principales variables prédictives des
résultats sont reliées au patient (mode d’adaptation, résistance et
réactance, problèmes interpersonnels, culture), et a
`la relation
thérapeutique (alliance thérapeutique). Sont également importants
les facteurs reliés au thérapeute (compétences de facilitation inter-
personnelles, empathie, gestion du contre-transfert) ainsi que les
pratiques, comme le suivi des progrès. Dans leurs décisions, les
régions du Canada doivent aussi tenir compte du fait qu’apprendre
aux thérapeutes a
`adapter le traitement et la posture thérapeutique
selon les caractéristiques du patient, a
`développer et a
`maintenir la
relation thérapeutique, a
`améliorer leurs compétences de facilita-
tion interpersonnelles et a
`surveiller les progrès sont autant
d’éléments qui augmenteront les chances de réduire le fardeau que
créent la dépression et l’anxiété parmi la population, mieux que ne
le fera l’offre d’un type de psychothérapie en particulier.
Mots-clés : psychothérapie, améliorer l’accès a
`la psychothérapie,
résultats, facteurs liés au patient, relation thérapeutique.
References
Ali, S., Rhodes, L., Moreea, O., McMillan, D., Gilbody, S., Leach, C.,...
Delgadillo, J. (2017). How durable is the effect of low intensity CBT for
depression and anxiety? Remission and relapse in a longitudinal cohort
study. Behaviour Research and Therapy, 94, 1– 8. http://dx.doi.org/10
.1016/j.brat.2017.04.006
American Psychological Association (APA). (2012). Recognition of psy-
chotherapy effectiveness. Retrieved from http://www.apa.org/about/
policy/resolution-psychotherapy.aspx
Anderson, T., Crowley, M. E. J., Himawan, L., Holmberg, J. K., & Uhlin,
B. D. (2016). Therapist facilitative interpersonal skills and training
status: A randomized clinical trial on alliance and outcome. Psychother-
apy Research, 26, 511–529. http://dx.doi.org/10.1080/10503307.2015
.1049671
Anderson, T., Ogles, B. M., Patterson, C. L., Lambert, M. J., & Ver-
meersch, D. A. (2009). Therapist effects: Facilitative interpersonal skills
as a predictor of therapist success. Journal of Clinical Psychology, 65,
755–768. http://dx.doi.org/10.1002/jclp.20583
Anderssen, E. (2017, December 4). Quebec to spend $35-million a year on
psychotherapy program. The Globe and Mail. Retrieved from https://
www.theglobeandmail.com/news/national/quebec-to-invest-35-million-
in-psychotherapy-program/article37192490/
American Psychological Association (APA) Presidential Task Force on
Evidence-Based Practice. (2006). Evidence-based practice in psychol-
ogy. American Psychologist, 61, 271–285. http://dx.doi.org/10.1037/
0003-066X.61.4.271
Baker, C. (2017, January 18). Mental health problems: Statistics on prev-
alence and services (Briefing Paper Number 6988). Retrieved from
http://researchbriefings.parliament.uk/ResearchBriefing/Summary/
SN06988
Beck, J. S. (2011). Cognitive behaviour therapy: Basics and beyond (2nd
ed.). New York, NY: Guilford Press.
Benish, S. G., Quintana, S., & Wampold, B. E. (2011). Culturally adapted
psychotherapy and the legitimacy of myth: A direct-comparison meta-
analysis. Journal of Counseling Psychology, 58, 279 –289. http://dx.doi
.org/10.1037/a0023626
Beutler, L. E., Harwood, T. M., Alimohamed, S., & Malik, M. (2002).
Functional impairment and coping style. In J. C. Norcross (Ed.), Psy-
chotherapy relationships that work: Therapist contributions and respon-
siveness to patients (pp. 145–170). New York, NY: Oxford University
Press.
Beutler, L. E., Harwood, T. M., Kimpara, S., Verdirame, D., & Blau, K.
(2011). Coping style. In J. Norcross (Ed.), Psychotherapy relationships
that work (2nd ed., pp. 336 –353). New York, NY: Oxford. http://dx.doi
.org/10.1093/acprof:oso/9780199737208.003.0017
Bohart, A. C., & Wade, A. G. (2013). The client in psychotherapy. In M.
Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and
behavior change (6th ed., pp. 219 –257). Hoboken, NJ: Wiley.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported
therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.
http://dx.doi.org/10.1037/0022-006X.66.1.7
Clark, D. M., Canvin, L., Green, J., Layard, R., Pilling, S., & Janecka, M.
(2018). Transparency about the outcomes of mental health services
(IAPT approach): An analysis of public data. Lancet, 391, 679 – 686.
http://dx.doi.org/10.1016/S0140-6736(17)32133-5
Constantino, M. J., Arnkoff, D. B., Glass, C. R., Ametrano, R. M., &
Smith, J. Z. (2011). Expectations. Journal of Clinical Psychology, 67,
184 –192. http://dx.doi.org/10.1002/jclp.20754
Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., &
Dobson, K. S. (2013). A meta-analysis of cognitive-behavioural therapy
for adult depression, alone and in comparison with other treatments.
Canadian Journal of Psychiatry, 58, 376 –385. http://dx.doi.org/10
.1177/070674371305800702
Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008).
Psychotherapy for depression in adults: A meta-analysis of comparative
outcome studies. Journal of Consulting and Clinical Psychology, 76,
909 –922. http://dx.doi.org/10.1037/a0013075
Delorme, A. (2018, March). Provincial perspectives. Paper presented at the
Mental Health Commission of Canada Round Table Discussion on
298 TASCA, TOWN, ABBASS, AND CLARKE
Expanding Access to Psychotherapies and Psychological Services in
Canada, Ottawa, Canada.
Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D., & Wampold,
B. E. (2012). Therapist effects in the therapeutic alliance-outcome rela-
tionship: A restricted-maximum likelihood meta-analysis. Clinical Psy-
chology Review, 32, 642– 649. http://dx.doi.org/10.1016/j.cpr.2012.07
.002
De Maat, S., Dekker, J., Schoevers, R., & De Jonghe, F. (2006). Relative
efficacy of psychotherapy and pharmacotherapy in the treatment of
depression: A meta-analysis. Psychotherapy Research, 16, 562–572.
http://dx.doi.org/10.1080/10503300600756402
Dixon-Woods, M., Kirk, D., Agarwal, S., Annandale, E., Arthur, T., &
Sutton, A. (2005). Vulnerable groups and access to health care: A
critical interpretive review. Report for the National Co-ordinating Cen-
tre for NHS Service Deliver and OrganisationR&D(NCCSDO).
Retrieved from http://www.netscc.ac.uk/hsdr/files/project/SDO_
FR_08-1210-025_V01.pdf
Elliott, R., Bohart, A. C., Watson, J. C., & Greenberg, L. S. (2011).
Empathy. Psychotherapy: Theory, Research, & Practice, 48, 43– 49.
http://dx.doi.org/10.1037/a0022187
Greenberg, L. S. (2010). Emotion-focused therapy. Washington, DC:
American Psychological Association Press. http://dx.doi.org/10.1037/
e602962010-001
Grissom, R. J. (1996). The magical number .7 ⫾.2: Meta-meta-analysis of
the probability of superior outcome in comparisons involving therapy,
placebo, and control. Journal of Consulting and Clinical Psychology, 64,
973–982. http://dx.doi.org/10.1037/0022-006X.64.5.973
Hayes, J. A., Gelso, C. J., & Hummel, A. M. (2011). Managing counter-
transference. Psychotherapy: Theory, Research, & Practice, 48, 88 –97.
http://dx.doi.org/10.1037/a0022182
Health Quality Ontario (HQO). (n.d.). Major depression: Care for adults
and adolescents. Quality standards: Retrieved from: http://www
.hqontario.ca/portals/0/documents/evidence/quality-standards/qs-
depression-clinical-guide-1609-en.pdf
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011).
Alliance in individual psychotherapy. Psychotherapy, 48, 9 –16. http://
dx.doi.org/10.1037/a0022186
Imel, Z. E., Malterer, M. B., McKay, K. M., & Wampold, B. E. (2008). A
meta-analysis of psychotherapy and medication in unipolar depression
and dysthymia. Journal of Affective Disorders, 110, 197–206. http://dx
.doi.org/10.1016/j.jad.2008.03.018
Johns, D. (2017). Health equity audit: Improving access to psychological
therapies. Nottingham, United Kingdom: Nottingham City Council.
Johnsen, T. J., & Friborg, O. (2015). The effects of cognitive behavioral
therapy as an anti-depressive treatment is falling: A meta-analysis.
Psychological Bulletin, 141, 747–768. http://dx.doi.org/10.1037/bul00
00015
Jorm, A. F., Patten, S. B., Brugha, T. S., & Mojtabai, R. (2017). Has
increased provision of treatment reduced the prevalence of common
mental disorders? Review of the evidence from four countries. World
Psychiatry, 16, 90 –99. http://dx.doi.org/10.1002/wps.20388
Klerman, G. L., Weissman, M. M., Rounsaville, B. J., & Chevron, E. S.
(1984). Interpersonal psychotherapy of depression. New York, NY:
Basic Books.
Kraus, D. R., Castonguay, L., Boswell, J. F., Nordberg, S. S., & Hayes,
J. A. (2011). Therapist effectiveness: Implications for accountability and
patient care. Psychotherapy Research, 21, 267–276. http://dx.doi.org/10
.1080/10503307.2011.563249
Lambert, M. J. (2010). Prevention of treatment failure: The use of mea-
suring, monitoring, and feedback in clinical practice. Washington, DC:
American Psychological Association Press. http://dx.doi.org/10.1037/
12141-000
Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychother-
apy. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy
and behavior change (pp. 143–189). Oxford, England: John Wiley &
Sons.
Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness of
psychotherapy. In M. J. Lambert (Ed.), Handbook of psychotherapy and
behavior change (5th ed.). New York, NY: Wiley.
Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback.
Psychotherapy: Theory, Research, & Practice, 48, 72–79. http://dx.doi
.org/10.1037/a0022238
Layard, R., & Clark, D. M. (2014). Thrive: The power of evidence-based
psychological therapies. London, United Kingdom: Penguin.
Levy, K. N., Ellison, W. D., Scott, L. N., & Bernecker, S. L. (2011).
Attachment style. Journal of Clinical Psychology, 67, 193–203. http://
dx.doi.org/10.1002/jclp.20756
Longmore, R. J., & Worrell, M. (2007). Do we need to challenge thoughts
in cognitive behavior therapy? Clinical Psychology Review, 27, 173–
187. http://dx.doi.org/10.1016/j.cpr.2006.08.001
Luborsky, L., Mark, D., Hole, A., Popp, C., Goldsmith, B., & Cacciola, J.
(1995). Supportive-expressive dynamic psychotherapy of depression: A
time-limited version. In J. Barber & P. Crits-Christoph (Eds.), Dynamic
therapies for the psychiatric disorders: Axis I. New York, NY: Basic
Books.
Martel, C. R., Dimidjian, S., & Herman-Dunn, R. (2010). Behavioral
activation for depression: A clinician’s guide. New York, NY: Guilford
Press.
McCrone, P. (2013). IAPT is probably not cost-effective. The British
Journal of Psychiatry, 202, 383. http://dx.doi.org/10.1192/bjp.202.5.383
McFarquhar, T., Luyten, P., & Fonagy, P. (2018). Changes in interpersonal
problems in the psychotherapeutic treatment of depression as measured
by the Inventory of Interpersonal Problems: A systematic review and
meta-analysis. Journal of Affective Disorders, 226, 108 –123. http://dx
.doi.org/10.1016/j.jad.2017.09.036
McHugh, R. K., Whitton, S. W., Peckham, A. D., Welge, J. A., & Otto, M. W.
(2013). Patient preference for psychological vs. pharmacological treatment
of psychiatric disorders: A meta-analytic review. Journal of Clinical Psy-
chiatry, 74, 595– 602. http://dx.doi.org/10.4088/JCP.12r07757
McManus, S., Bebbington, P., Jenkins, R., & Brugha, T. (2016). Mental
health and wellbeing in England: Adult Psychiatric Morbidity Survey
2014. Leeds, United Kingdom: NHS Digital.
Muran, J. C., Safran, J. D., Gorman, B. S., Samstag, L. W., Eubanks-
Carter, C., & Winston, A. (2009). The relationship of early alliance
ruptures and their resolution to process and outcome in three time-
limited psychotherapies for personality disorders. Psychotherapy: The-
ory, Research, & Practice, 46, 233–248. http://dx.doi.org/10.1037/
a0016085
National Health Service (NHS). (n.d.). Adult improving access to psycho-
logical therapies programme. Retrieved from https://www.england.nhs
.uk/mental-health/adults/iapt/
NHS Digital. (n.d.a). Psychological therapies: Annual report on the use of
IAPT services in England, 2014 –15. Retrieved from http://digital.nhs
.uk/catalogue/PUB19098
NHS Digital. (n.d.b). Psychological therapies: Annual report on the use of
IAPT services in England, 2015–16. Retrieved from https://digital.nhs
.uk/catalogue/PUB22110
NHS Digital. (n.d.c). Psychological therapies: Annual report on the use of
IAPT services in England, 2016 –17. Retrieved from https://digital.nhs
.uk/catalogue/PUB30157.
Norcross, J. C., & Lambert, M. J. (2011). Psychotherapy relationships that
work II. Psychotherapy, 48, 4–8. http://dx.doi.org/10.1037/a0022180
Organisation for Economic Co-operation and Development. (2014). Men-
tal Health and Work: United Kingdom. Paris, France: Author.
Olfson, M., & Marcus, S. C. (2010). National trends in outpatient psycho-
therapy. The American Journal of Psychiatry, 167, 1456 –1463. http://
dx.doi.org/10.1176/appi.ajp.2010.10040570
299
WHAT MAKES PSYCHOTHERAPY WORK
Ontario Ministry of Health and Long-Term Care. (2017, October). Ontario
improving mental health services for people across the province. Re-
trieved from https://news.ontario.ca/mohltc/en/2017/10/ontario-
improving-mental-health-services-for-people-across-the-province.html
Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017). The comparative
effectiveness and efficiency of cognitive behaviour therapy and generic
counselling in the treatment of depression: Evidence from the 2nd U. K.
National Audit of psychological therapies. BMC Psychiatry, 17, 215–
228. http://dx.doi.org/10.1186/s12888-017-1370-7
Rosnow, R. L., & Rosenthal, R. (2003). Effect sizes for experimenting
psychologists. Canadian Journal of Experimental Psychology/Revue
canadienne de psychologie expérimentale, 57, 221–237. http://dx.doi
.org/10.1037/h0087427
Royal College of Psychiatrists. (2011). National Audit of Psychological
Therapies for Anxiety and Depression, National Report 2011. Retrieved
from https://www.rcpsych.ac.uk/pdf/NAPT%202011%20Report%20
.pdf
Royal College of Psychiatrists. (2013). Report of the second round of the
National Audit of Psychological Therapies (NAPT) 2013. London,
United Kingdom: Healthcare Quality Improvement Partnership.
Safran, J. D., Muran, J. C., & Eubanks-Carter, C. (2011). Repairing
alliance ruptures. Psychotherapy: Theory, Research, & Practice, 48,
80 – 87. http://dx.doi.org/10.1037/a0022140
Saxon, D., Barkham, M., Foster, A., & Parry, G. (2017). The contribution
of therapist effects to patient dropout and deterioration in the psycho-
logical therapies. Clinical Psychology & Psychotherapy, 24, 575–588.
http://dx.doi.org/10.1002/cpp.2028
Saxon, D., Firth, N., & Barkham, M. (2017). The relationship between
therapist effects and therapy delivery factors: Therapy modality, dosage,
and non-completion. Administration and Policy in Mental Health, 44,
705–715. http://dx.doi.org/10.1007/s10488-016-0750-5
Spielmans, G. I., Berman, M. I., & Usitalo, A. N. (2011). Psychotherapy
versus second-generation antidepressants in the treatment of depression:
A meta-analysis. Journal of Nervous and Mental Disease, 199, 142–149.
http://dx.doi.org/10.1097/NMD.0b013e31820caefb
Swift, J. K., Callahan, J. L., & Vollmer, B. M. (2011). Preferences. Journal
of Clinical Psychology, 67, 155–165. http://dx.doi.org/10.1002/jclp
.20759
Swift, J. K., & Greenberg, R. P. (2012). Premature discontinuation in adult
psychotherapy: A meta-analysis. Journal of Consulting and Clinical
Psychology, 80, 547–559. http://dx.doi.org/10.1037/a0028226
The Royal Mental Health Care. (2017, October). Ontario improving mental
health services for people across the provinces: The Royal’s role.
Retreived from http://www.theroyal.ca/mental-health-centre/news-and-
events/newsroom/16909/ontario-improving-mental-health-services-for-
people-across-the-province-the-royals-role/
Town, J. M., Diener, M. J., Abbass, A., Leichsenring, F., Driessen, E., &
Rabung, S. (2012). A meta-analysis of psychodynamic psychotherapy
outcomes: Evaluating the effects of research-specific procedures. Psy-
chotherapy: Theory, Research, & Practice, 49, 276 –290. http://dx.doi
.org/10.1037/a0029564
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal,
R. (2008). Selective publication of antidepressant trials and its influence
on apparent efficacy. The New England Journal of Medicine, 358,
252–260. http://dx.doi.org/10.1056/NEJMsa065779
Vasiliadis, H. M., Dezetter, A., Latimer, E., Drapeau, M., & Lesage, A.
(2017). Assessing the costs and benefits of insuring psychological ser-
vices as part of Medicare for depression in Canada. Psychiatric Services,
68, 899 –906. http://dx.doi.org/10.1176/appi.ps.201600395
Wampold, B. E., & Brown, G. S. (2005). Estimating variability in out-
comes attributable to therapists: A naturalistic study of outcomes in
managed care. Journal of Consulting and Clinical Psychology, 73,
914 –923. http://dx.doi.org/10.1037/0022-006X.73.5.914
Wampold, B. E., & Imel, Z. (2015). The great psychotherapy debate (2nd
ed.). New York, NY: Routledge.
Wampold, B. E., & Serlin, R. C. (2014). Meta-analytic methods to test
relative efficacy. Quality & Quantity: International Journal of Method-
ology, 48, 755–765. http://dx.doi.org/10.1007/s11135-012-9800-6
Webb, C. A., Derubeis, R. J., & Barber, J. P. (2010). Therapist adherence/
competence and treatment outcome: A meta-analytic review. Journal of
Consulting and Clinical Psychology, 78, 200 –211. http://dx.doi.org/10
.1037/a0018912
Williams, R., Farquharson, L., Palmer, L., Bassett, P., Clarke, J., Clark,
D. M., & Crawford, M. J. (2016). Patient preference in psychological
treatment and associations with self-reported outcome: National cross-
sectional survey in England and Wales. BMC Psychiatry, 16, 4. http://
dx.doi.org/10.1186/s12888-015-0702-8
Zilcha-Mano, S. (2017). Is the alliance really therapeutic? Revisiting this
question in light of recent methodological advances. American Psychol-
ogist, 72, 311–325. http://dx.doi.org/10.1037/a0040435
Received February 21, 2018
Revision received March 28, 2018
Accepted March 28, 2018 䡲
300 TASCA, TOWN, ABBASS, AND CLARKE