Content uploaded by Jeroen Ruwaard
Author content
All content in this area was uploaded by Jeroen Ruwaard
Content may be subject to copyright.
Efficacy and effectiveness of online CBT:
A decade of Interapy research
Jeroen RUWAARDa,b,1, Alfred LANGEa, Bart SCHRIEKENb, and
Paul EMMELKAMP
a
a University of Amsterdam, Department of Psychology, Amsterdam, the Netherlands
b Interapy PLC, Amsterdam, the Netherlands
Abstract. Since 1996, researchers of the Interapy research group of the University
of Amsterdam have been examining the effects of online cognitive behavioral
treatment (online CBT). Over the years, the group conducted nine controlled trials
of online CBT of a variety of mental health disorders, among a total of 840
participants. These studies suggest that online CBT is a viable and effective
alternative to face to face treatment. Treatment adherence was 82%, and reductions
in psychopathology represented a large between-group effect size of SMD = 0.9
(95% CI: .7 to 1.1), which were maintained over long periods. The research
culminated in the foundation of the Interapy clinic, which received Dutch health
regulatory body approval in 2005. Since then, costs of online CBT are reimbursed
through public health insurance. A large study of treatment outcome of 1500
patients of the Interapy clinic showed that effects in clinical practice are similar to
those observed in the controlled trials, and comparable to selected benchmarks of
naturalistic studies of face to face CBT. The accumulated evidence provides
compelling support for the efficacy and effectiveness of online CBT.
Keywords. Cognitive Behavior Therapy; Computer assisted protocol directed
therapy; Effectiveness Studies; Follow-Up Studies; Internet; Randomized
controlled trial; Treatment Outcome;
Introduction
In 1996, researchers of the University of Amsterdam conducted a small feasibility
study of internet-based psychotherapy. Although the World Wide Web was still in its
infancy at that time, they created a website through which they treated 20 students with
posttraumatic stress. To the surprise of the research team, the results of what they
called “Interapy” were very encouraging. Despite the lack of face to face contact, stress
symptoms of 19 of the 20 students had reduced to normal levels after treatment [1].
The 1996 study was seminal. It was followed by over a decade of research which
resulted in full integration of internet-based treatment in the Dutch public health
system. In this article, we briefly summarize this research from a meta-analytical
perspective. We discuss the Interapy method, the efficacy of Interapy as established in
a series of controlled clinical trials, and the effectiveness of Interapy as observed in
routine clinical practice.
1 Corresponding author: Jeroen Ruwaard. Interapy PLC, P.O. box 3884, 1001 AR Amsterdam, the
Netherlands. E-mail: research@interapy.nl
1. What is Interapy?
Interapy is web-based, manualized, therapist-assisted cognitive behavior therapy
(CBT). Screening, treatment, and outcome measurement are conducted without any
face to face contact. With exception of a diagnostic telephone interview, all interaction
between patients and mental health personnel is online, in a secure website. They
interact through an asynchronous exchange of text-messages, i.e., the dialogue
resembles an e-mail conversation rather than a video-conference or an online chat-
session. This dialogue is governed by a computer system that executes the treatment
manual. The manual defines a fixed sequence of homework assignments that
implement common CBT interventions, which are translated into a format suitable for
delivery over the Internet. Therapist support consists of standardized, default feedback
and instructions that are tailored by the therapists to the specific situation of their
patients. In the feedback, motivational techniques are used to enhance the impact of the
interventions, i.e., to ensure patients understand the purpose of the interventions, that
they set realistic goals, that they do the exercises as prescribed, and that they continue
treatment. These techniques target patients’ motivation for change, the therapeutic
alliance (e.g., by expressing empathy and understanding), and self-esteem and self-
efficacy (e.g., by complimenting the patients with their progress and accomplishments).
Treatments are brief, but intensive. The duration of treatment varies from 5 to 16
weeks, in which therapists provide feedback about two times per week.
2. The Efficacy of Interapy in Controlled Clinical Trials
Over the years, Interapy has been evaluated in the online treatment of depression,
posttraumatic stress, bereavement, work-related stress, panic disorder and bulimia
nervosa, in nine controlled trials [2-10]. Characteristics of these trials are listed in
Table 1.
2.1. Method
Trials included a total of 840 participants. Most studies were randomized controlled
trials (RCT), in which participants were randomly assigned to Interapy treatment
condition, or to a waiting list control condition. One study was a within-subject
baseline-control study, and one study included bibliotherapy as an additional active
experimental control condition. Outcome was assessed through well-validated self-
report measures of primary and secondary symptom severity, such as the Beck
Depression Inventory, the Impact of Event Scale, and the Depression Anxiety Stress
Scales. These measures were administered at baseline, immediately after treatment
(posttest), and one to three years after treatment (long-term follow-up). Outcome was
analyzed in terms of mean change in symptom severity over time, and in terms of
clinical significant change (i.e., reliable recovery [11]). Most trials involved
conservative intention-to-treat analyses: participants, who did not complete
posttreatment measurements, were assumed to have gained nothing.
Table 1. Characteristics of the Interapy efficacy trials.
Study Symptoms Target
population Experimental
condition N Primary
Outcome
Lange 2001 posttraumatic stress student, Dutch Interapy
waiting list
13
12
IES
Lange 2003 posttraumatic stress community, adult,
Dutch
Interapy
waiting list
122
62
IES
Wagner 2006 bereavement community, adult,
German
Interapy
waiting list
26
29
IES
Knaevelsrud 2007 posttraumatic stress community, adult,
German
Interapy
waiting list
49
47
IES
Ruwaard 2007 work-related stress community, adult,
Dutch
Interapy
waiting list
177
62
DASS
Ruwaard 2009 depression community, adult,
Dutch
Interapy
waiting list
36
18
BDI
Ruwaard 2010 panic symptoms community, adult,
Dutch
Interapy
waiting list
27
31
PDSS-SR /
panic diary
Lange 2010 posttraumatic stress Dutch, adolescent,
victims of sexual
abuse
baseline-
control
24 IES
Ruwaard (submitted) bulimic symptoms community, adult,
Dutch
Interapy
waiting list
bibliotherapy
35
35
35
EDE-Q
NOTE: IES: Impact of Event Scale; PDSS-SR: Panic Disorder Severity Scale, Self-rate; BDI: Beck
Depression Inventory; EDE-Q: Eating Disorder Examination-Questionnaire; DASS: Depression Anxiety
Stress Scales.
Figure 1. Forest plot of between-group effect sizes and 95% confidence intervals for the Interapy trials.
2.2. Results
Dropout rates were encouragingly low: an (unweighted) average of 82% of the patients
completed every step of treatment. As illustrated by the left forest plot in Figure 1 (A),
reductions in primary symptom severity were significantly larger with Interapy, in
comparison to the experimental controls. The standardized mean difference (SMD) in
improvement between Interapy and the experimental controls ranged from SMD = .5 to
SMD = 1.3, with a pooled SMD of .9 (95% CI: .7 to 1.1). These are large effects,
roughly equivalent to those of face to face CBT. Studies also revealed higher recovery
rates with Interapy compared with experimental controls. Across the trials, the
unweighted average recovery rate with Interapy was 60% (range: 36% to 85%) and
23% in the experimental comparison groups (range: 9% to 42%). As illustrated by
Figure 1 (B), this equated to a significant, moderate-to-large odds ratio (OR) of 6.1
(95% CI: 4.2 to 9.0). One to three years after treatment, these treatment gains were
found to be maintained.
3. The Effectiveness of Interapy in Routine Clinical Practice
In 2001, the Interapy research team founded the Interapy clinic, with the aim of
implementing online CBT in the public health system. This clinic has been in operation
since. In 2008, the team examined treatment outcome of 1500 patients of the clinic, to
assess the effectiveness of online CBT during routine clinical practice.
3.1. Method
This was an uncontrolled, retrospective pretest-posttest study, with two follow-ups.
Data were collected from unselected, consecutive electronic patient records of the
clinic, which contained scores of self-report questionnaires that were administered at
pretest, posttest, six weeks after treatment and 1 year after treatment. Patients were
Dutch adults (female: 67%; age: Mean = 40; SD = 10) with a GP-referral for
psychotherapy, who started treatment of depression (n = 413; 28%), posttraumatic
stress (n = 478; 32%), panic disorder (n = 139; 9%) or work-related stress (n = 470;
31%). The majority of patients scored above clinical cut-off on the primary outcome
measures (n = 1420, 95%; range 74%- 99% across treatments). Scores of about a
quarter of the patients (24%; n = 364) indicated severe symptomatology. Most (n =
1052; 71%) did not use psychiatric medication. Primary outcome variables were
treatment adherence, primary and secondary symptom severity and recovery rates.
3.2. Results
Due to routine outcome measurement, posttreatment data were available for 79% of the
patients. Treatment dropout was 29%, which is comparable to known dropout rates in
Dutch mental healthcare (30%). Symptom reductions met selected benchmarks of
naturalistic studies of face to face CBT. On the short-term (at post-test and six weeks
follow-up), patients reported significant (P < .001) reductions in symptom severity,
which represented a large pooled (uncontrolled) effect size of SMD = 1.4 (range: 0.7 ≤
SMD ≤ 1.9). Among treatment completers, 71% reliably improved and 52%
experienced a clinically significant change (i.e., recovery). Follow-up measurements
were difficult to interpret given an attrition rate of 67%. Nonetheless, available data
suggested that improvements sustain up to one year after treatment.
4. Conclusion
A decade of research has provided compelling support for the efficacy and
effectiveness of online CBT. Treatment adherence rate is high (82%), effect sizes are
comparable to those of face to face CBT, and method and outcome generalize well to
routine clinical practice. Despite limitations of the present evidence (most comparison
groups were waiting lists, most long-term follow-up results were uncontrolled,
outcome was determined through self-report measures, and the applicability of Interapy
to other disorders is unclear), the results identify online CBT as a valuable addition to
existing treatment options. Online treatment provides relatively easy access, and may
facilitate the timely implementation of new evidence-based treatment methods in
routine healthcare. Dutch regulatory health bodies have recognized this potential. In
2005, these bodies endorsed the services of the Interapy clinic as a valid alternative to
regular, face to face treatment. Since then, costs of online treatment are reimbursed
through public health insurance to all Dutch citizens with a GP-referral for
psychotherapy. The implementation and dissemination of online treatment has started.
References
A. Lange, J.P. van de Ven, B.A. Schrieken, B. Bredeweg, and P.M. Emmelkamp, Internet-mediated,
protocol-driven treatment of psychological dysfunction. Journal of Telemedicine and Telecare 6 (2000):
15-21.
A. Lange, J.P. van de Ven, B. Schrieken, and P.M. Emmelkamp, Interapy, treatment of posttraumatic stress
through the Internet: a controlled trial. Journal of Behavior Therapy and Experimental Psychiatry 32
(2001): 73-90.
A. Lange, D. Rietdijk, M. Hudcovicova, J.P. van de Ven, B. Schrieken, and P.M. Emmelkamp, Interapy: a
controlled randomized trial of the standardized treatment of posttraumatic stress through the internet.
Journal of Consulting and Clinical Psychology 71 (2003): 901-9.
B. Wagner, C. Knaevelsrud, and A. Maercker, Internet-based cognitive-behavioral therapy for complicated
grief: a randomized controlled trial. Death Studies 30 (2006): 429-53.
C. Knaevelsrud and A. Maercker, Internet-based treatment for PTSD reduces distress and facilitates the
development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry 7
(2007): 13.
J. Ruwaard, A. Lange, M. Bouwman, J. Broeksteeg, and B. Schrieken, E-Mailed standardized cognitive
behavioural treatment of work-related stress: a randomized controlled trial. Cognitive Behaviour
Therapy 36 (2007): 179-192.
J. Ruwaard, B. Schrieken, M. Schrijver, J. Broeksteeg, J. Dekker, H. Vermeulen, and A. Lange, Standardized
web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial
with a long-term follow-up. Cognitive Behaviour Therapy 38 (2009): 206-221.
A. Lange and J. Ruwaard, Ethical dilemmas in online research and treatment of sexually abused adolescents.
Journal of Medical Internet Research 12 (2010): e58.
J. Ruwaard, J. Broeksteeg, B. Schrieken, P. Emmelkamp, and A. Lange, Web-based therapist-assisted
cognitive behavioral treatment of panic symptoms: A randomized controlled trial with a three-year
follow-up. Journal of Anxiety Disorders 24 (2010): 387-396.
J. Ruwaard, A. Lange, J. Broeksteeg, A. Renteria-Agirre, B. Schrieken, C. Dolan, and P. Emmelkamp,
Online cognitive behavioral treatment of bulimic symptoms: A randomized controlled trial. Manuscript
submitted for publication (copy on file with first author).
N.S. Jacobson and P. Truax, Clinical significance: a statistical approach to defining meaningful change in
psychotherapy research. Journal of Consulting and Clinical Psychology 59 (1991): 12-9.
