Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Contents lists available at ScienceDirect
Internet Interventions
journal homepage: www.elsevier.com/locate/invent
Internet-based CBT for patients with depressive disorders in primary and
psychiatric care: Is it effective and does comorbidity affect outcome?
Anna-Lena Flygare
a
, Ingemar Engström
b
, Mikael Hasselgren
c
, Markus Jansson-Fröjmark
d
,
Rikard Frejgrim
a
, Gerhard Andersson
d,e
, Fredrik Holländare
c,⁎
a
Centre for Clinical Research, Region Värmland, Älvgatan 49, Karlstad, Sweden
b
University Health Care Research Centre, Faculty of Medicine and Health, Örebro University, 70116 Örebro, Sweden
c
School of Medical Sciences, Örebro University, Örebro, Sweden
d
Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institute, Liljeholmstorget 7b, 117 63 Stockholm, Sweden
e
Department of Behavioural Sciences and Learning, Linköping University, 581 83 Linköping, Sweden
ARTICLE INFO
Keywords:
Depression
Internet
Comorbidity
Personality disorders
Primary healthcare
Psychiatric care
ABSTRACT
Internet-based cognitive behavior therapy (ICBT) has proved effective in reducing mild to moderate depressive
symptoms. However, only a few studies have been conducted in a regular healthcare setting which limits the
generalizability of the results. The influence of psychiatric comorbidity on outcome is not well understood. In the
current study, patients with mild to moderate depressive symptoms in primary and psychiatric care were in-
terviewed using the SCID-I and SCID-II to assess psychiatric diagnoses. Those included were randomly allocated
to ICBT (n= 48) or to an active control condition (n= 47). Both groups received therapist support. At post-
treatment, ICBT had reduced depressive symptoms on the BDI-II more than the active control intervention
(p= .021). However, the difference between groups was no longer significant at the 6-, 12- or 24-month follow-
ups. The within-group effect size after ICBT (BDI-II) was large (d= 1.4). A comorbid anxiety disorder didn't
moderate the outcome, while the presence of a personality disorder predicted significantly less improvement in
depressive symptoms. ICBT had a large effect on depressive symptoms in a sample from regular healthcare. It is
possible to obtain a large effect from ICBT despite comorbid anxiety, however, including patients with a co-
morbid personality disorder in the current form of ICBT seems questionable.
1. Introduction
It has been found that cognitive behavioral therapy (CBT) is trans-
ferable to Internet format, especially when guided by a therapist
(Andersson, 2016) who provides support, encouragement, and occa-
sionally direct therapeutic activities (Johansson and Andersson, 2012).
Meta-analyses show that guided Internet-based CBT (ICBT) is an ef-
fective treatment for depression (Karyotaki et al., 2018;Richards and
Richardson, 2012). ICBT with guidance seems to be equally effective as
face-to-face therapy (Andersson et al., 2016), and offering ICBT as a
complement to standard care expands the availability of effective psy-
chological treatment, as it enables therapists to increase their case load.
There are clear indications that guided internet-based psychological
treatments are more effective than unguided treatments (Richards and
Richardson, 2012), although there are exceptions (Titov et al., 2014).
The guidance is typically supportive in nature, including encourage-
ment and reinforcement (Holländare et al., 2016;Paxling et al., 2013;
Sanchez-Ortiz et al., 2011) but how different aspects of guidance in-
fluence outcome is still not fully understood.
Most studies on internet-based psychological treatment have in-
vestigated CBT-based interventions, (Ruwaard et al., 2009;Andersson
et al., 2005;Hedman et al., 2014;Ruwaard et al., 2012;Williams and
Andrews, 2013;Hedman et al., 2012;Dear et al., 2018;Titov et al.,
2016;Johansson et al., 2019;Mathiasen et al., 2018) although one
study compared internet-based psychodynamic treatment for depres-
sion with an active control condition with positive results (Johansson
et al., 2013). One study compared guided ICBT with individualized e-
mail therapy (Vernmark et al., 2010), and moderate to large effect sizes
were found in both groups. Internet-based psychological treatment
programs can thus vary in content and presentation, yet still be
https://doi.org/10.1016/j.invent.2019.100303
Received 12 September 2019; Received in revised form 12 December 2019; Accepted 14 December 2019
⁎
Corresponding author at: School of Medical Sciences, Örebro University, Södra Grev Rosengatan 30, 70362 Örebro, Sweden.
E-mail addresses: anna-lena.flygare@liv.se (A.-L. Flygare), ingemar.engstrom@regionorebrolan.se (I. Engström), mikael.hasselgren@oru.se (M. Hasselgren),
markus.jansson-frojmark@ki.se (M. Jansson-Fröjmark), rikard.frejgrim@clarahalsan.se (R. Frejgrim), geran@ibv.liu.se (G. Andersson),
fredrik.hollandare@regionorebrolan.se (F. Holländare).
Internet Interventions 19 (2020) 100303
Available online 29 December 2019
2214-7829/ © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/BY-NC-ND/4.0/).
T
effective.
Much research on ICBT has been conducted in university settings
with nationwide recruitment (Andersson et al., 2013a) or via the media,
and several efficacy studies have found that guided ICBT for depression
is effective (Andersson et al., 2005;Johansson and Andersson, 2012;
Vernmark et al., 2010;Ruwaard et al., 2009;Carlbring et al., 2013;
Warmerdam et al., 2008;Perini et al., 2009;Robinson et al., 2010;
Hollandare et al., 2013;Carlbring et al., 2005;Andersson et al., 2006;
Berger et al., 2011a;Berger et al., 2011b;Botella et al., 2010;Titov
et al., 2011;Heinrich et al., 2016;Ivarsson et al., 2014;Zetterberg
et al., 2019) with a favorable outcome at the 3.5 year follow-up in one
study (Andersson et al., 2013b). Less research has been conducted in
representative clinical settings and few studies have had an active
control condition. However, some studies on ICBT in health care set-
tings have been carried out (Ruwaard et al., 2012;Watts et al., 2012;
Williams and Andrews, 2013;Kessler et al., 2009;Hedman et al., 2014;
Titov et al., 2015;Kivi et al., 2014;Johansson et al., 2019;Mathiasen
et al., 2018;Hadjistavropoulos et al., 2016;Nordgreen et al., 2018)Ina
large effectiveness study by Hedman et al. (2014) 1203 patients were
treated with ICBT for depression in routine psychiatric care in Stock-
holm, Sweden. The effect on depressive symptoms was large with a
within-group effect size (d) of 1.27. Some studies have used active
control conditions, often resulting in smaller between group effects
(Kampmann et al., 2016).
Comorbidity is common among depressed persons, and one meta-
analysis (Friborg et al., 2014) estimated that as many as 45% of the
patients with major depressive disorder (MDD) have a comorbid per-
sonality disorder (PD). There is a widespread perception within clinical
psychiatry that PD has an adverse effect on the outcome of many
treatments, and PD has been found to increase the risk of drop out
(Schindler et al., 2013) and fewer patients with a PD responded to CBT
compared to those without a PD in one study (Fournier et al., 2008).
Meta-analyses have shown that PD is a risk factor for poor outcome in
treatment for depression (Newton-Howes et al., 2006;Newton-Howes
et al., 2014). However, there are exceptions, i.e. RCTs showing that the
effect of CBT on patients with depression and PDs is comparable to that
for patients without a PD (Joyce et al., 2007;Lis and Myhr, 2016;van
Bronswijk et al., 2018).
It has also been found that at least 50% of depressed persons fulfill
the diagnostic criteria for a comorbid anxiety disorder (Kessler et al.,
2007). One study demonstrated that a high anxiety level increased the
risk of relapse after CBT for depression (Forand and Derubeis, 2013)
and that focusing on anxiety during sessions reduced the effect on de-
pressive symptoms (Gibbons and DeRubeis, 2008). However, there are
also studies indicating that a comorbid anxiety disorder does not affect
the outcome of CBT for depression (Smits et al., 2009), and in a study
by Kashdan and Roberts (2011), comorbid social anxiety actually in-
creased the effect of CBT for depression.
The aim of this study was to compare the effect of ICBT with the
effect of an active control condition in a clinical sample with a de-
pressive disorder recruited from primary and psychiatric care. An ad-
ditional aim was to investigate the impact of comorbidity.
We hypothesized that, in comparison to the active control condi-
tion, ICBT would lead to reduced depressive symptoms and that im-
provements would be sustained over time. The second hypothesis was
that patients suffering from a comorbid anxiety disorder or a person-
ality disorder would improve less.
2. Material and methods
2.1. Design
A randomized controlled trial (RCT) with repeated measurements
was conducted to compare the effects of ICBT with those of an active
control group in terms of changes in depressive symptoms and remis-
sion rates.
2.2. Participants
Participants were recruited in primary care and psychiatric care in
two similarly sized neighboring County Councils in Sweden, Örebro and
Värmland, each with around 280,000 inhabitants. Recruitment started
in 2007 and ended in 2012. Patients received information about the
study from posters in waiting rooms as well as being briefed by primary
and psychiatric care staff. Both referrals from a health professional and
self-referrals were accepted. To qualify, the participant should have
mild to moderate depressive symptoms (15–30 on the MADRS-S
(Svanborg and Asberg, 1994)), be at least 18 years old, have internet
access and be fluent in Swedish. Reasons for exclusion were ongoing
CBT, starting or adjusting antidepressant medication during the past
month, being suicidal, suffering from bipolar disorder, psychosis, and/
or seasonal affective disorder. A demographic description of the parti-
cipants is presented in Table 1.
2.3. Procedure
Patients who expressed an interest in the study received a letter
containing a formal invitation to participate, a written consent form,
and a postage prepaid envelope.
Patients who scored 15–30 on the MADRS-S (Svanborg and Asberg,
1994) were called for a SCID-I (First et al., 1998) and a SCID-II (First
et al., 1994) interview to assess psychopathology on the DSM-IV
(American Psychiatric Association, 2000) Axis I and comorbid PDs
(results are shown in Table 2). The face-to-face interview was con-
ducted by a licensed psychologist and lasted for about two hours. The
psychologists were unaware of the study hypotheses and not informed
about group allocation to ensure an independent diagnostic process
before and after treatment.
The participants were randomized to either ICBT or active control.
The randomization sequence had differing block sizes, unknown to the
researchers, which was prepared in advance by a statistician. A num-
bered opaque envelope was opened for each participant after the de-
cision of inclusion was made. The patients were recommended to
complete approximately one module per week, and hence the treatment
in eight weeks. At the start, the participants had access to one module
and subsequently to the other modules in sequence when the treating
psychologist considered that all homework had been completed sa-
tisfactorily. Each module contained questions to answer and return to
the psychologist via the online platform.
To identify deterioration or suicidal tendencies, patients were asked
to complete weekly self-ratings with the MADRS-S. In cases where a
participant rated 4 or higher on MADRS-S item 9 (zest for life) the
treating psychologist contacted the participant for an assessment of
current suicidal ideation and possible referral to further care.
After treatment, all patients were called for a second SCID-I face-to-
face interview. They were also asked to fill out self-ratings (MADRS-S
and BDI-II) on the website at post-treatment and after 6, 12 and
24 months. A flow chart of patient participation is presented in Fig. 1.
Table 1
Demographic description of the participants in the ICBT and active control
group.
ICBT
(n= 48)
Control
(n= 47)
Total
(n= 95)
Female; number (%) 38 (79.2) 34 (72.3) 72 (75.8)
Mean age (SD) 42.9 (11.26) 47.7 (12.74) 45.30 (12.20)
Age range 20–68 23–68 20–68
Patients with earlier depressive
episode(s); number (%)
28 (58.3) 27 (57.4) 55 (57.9)
Patients with ADM at recruitment;
number (%)
20 (41.7) 22 (46.8) 42 (44.2)
Note: ADM = antidepressant medication.
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
2
The protocol was approved by the Regional Ethics Committee in
Uppsala, Sweden (No. 2006:038). No trial registration was made.
2.4. Treatment content
A secure web-based platform was constructed for the study where
participants logged in with unique user names and passwords and
completed their self-ratings on the MADRS-S and BDI-II. The partici-
pants gained access to their treatment material, ICBT, or active control
(in accordance with their random allocation) on the platform. There
was also a messaging service (similar to email), by which the partici-
pants in both groups had most of their contact with the psychologist.
The participants could send messages to the psychologist without re-
strictions and typically did so once a week. Messages were normally
responded to within one working day. At the end of each module there
were mandatory questions that all participants answered in a message
to their psychologist. The psychologist gave feedback on completed
homework, answered questions and gave access to the next module in
their replies. Feedback on homework in the ICBT-group were typically
written to help the participant to understand and to use the strategies in
behavioral activation and cognitive restructuring. In the control group
however, the messages from the psychologist were restricted to support
and did not contain any suggestions about altering behavior or chal-
lenging negative thoughts.
The two treatment materials compared in the study differed in
content but were both divided into modules with questions at the end.
The guided self-help program was a modified version of the material
used in previous RCTs (Andersson et al., 2005;Vernmark et al., 2010).
It consisted of eight text modules including exercises and comprised
39,000 words (114 pages) in total. The material started with an in-
troduction to CBT followed by a module about depression from a CBT-
perspective with a behavioral focus (Martell et al., 2001). There were
two modules on behavioral activation and one on cognitive re-
structuring (Beck, 1979). There was one basic module on sleep and
relaxation (Morin, 1996) and one optional module on more advanced
strategies for improved sleep. The final module was designed to help
patients in defining long-term goals (Wilson and Murrell, 2004) and
relapse prevention (Gortner et al., 1998).
The material read by the active control group consisted of in-
formation about depression. It was divided into 9 modules without CBT
elements and comprised 23,000 words (49 pages). The material covered
information about the diagnosis of depression, epidemiology, gender
differences, comorbidity and short descriptions of different psycholo-
gical and pharmacological treatments.
2.5. Outcome measures
The self-reported Montgomery-Åsberg Depression Rating Scale
(MADRS-S) (Svanborg and Asberg, 1994) was used to measure de-
pressive symptoms and suicidal thoughts. MADRS-S is a self-adminis-
tered version of Montgomery-Åsberg Depression Rating Scale (MADRS)
(Montgomery and Asberg, 1979) and has been reported to have a high
or moderate correlation with MADRS (Svanborg and Asberg, 1994;
Svanborg and Asberg, 2001;Fantino and Moore, 2009).
MADRS-S is a 9-item measure where the patient is asked to answer
questions by scoring them from 0 to 6. There are four type-statements
for each item (representing scores 0, 2, 4 and 6) to make the scoring
easier. A higher score indicates more severe depression. The maximum
score for MADRS-S is 54 and the cutoffpoints are; 0–12 minimal, 13–19
mild, 20–34 moderate and > 34 severe(Montgomery and Asberg,
1979).
MADRS-S has been tested for reliability and validity, for instance by
Fantino and Moore (2009) who analyzed data from a large group of
patients diagnosed with major depressive disorder. In that study, the
construct validity was found to be satisfactory and Cronbach's alpha
was reported at 0.84. It has been shown that MADRS-S is sensitive to
change in depressive symptoms during treatment (Montgomery and
Asberg, 1979;Fantino and Moore, 2009).
In addition to MADRS-S, the Beck Depression Inventory –Second
edition (BDI-II) (Beck et al., 2005) was used to measure degree of de-
pressive symptoms. BDI-II is self-administered and it comprises 21
items. Each item yields a score of 0–3 which gives a maximum score of
63. Every item consists of a group of statements concerning a specific
symptom that is common in depressions. The patient chooses which
statement that best describes how he or she felt the last two weeks. BDI-
II cutoffscores are; 0–13 minimal, 14–19 mild, 20–28 moderate and
29–63 severe (Beck et al., 2005).
BDI-II is a revised version of BDI, created in 1996 to meet the cri-
teria of depression from Diagnostic and Statistical Manual of Mental
Disorders –Fourth Edition (American Psychiatric Association, 2000;
Beck et al., 1996). The reliability and validity of BDI-II have been tested
with good results in several studies, showing satisfactory internal con-
sistency and test-retest reliability, and was found to differentiate well
between grades of depression with sufficient sensitivity to change (Beck
et al., 2005;Garcia-Batista et al., 2018;Kuhner et al., 2007;Osman
et al., 2004;Storch et al., 2004). A comparative study of MADRS-S and
BDI-II applied within primary care was published by (Wikberg et al.,
2015), where the two instruments were found to correlate well, both
with sufficient reliability measures. Both the MADRS-S and BDI-II have
been validated for online use (Hollandare et al., 2010).
For the interviews, the SCID I (Structured Clinical Interview for
DSM-IV-Axis I) (First et al., 1998) and SCID II (Structured Clinical In-
terview for DSM-IV-Axis II) (First et al., 1994) were used to obtain a
good diagnostic picture of the participants.
Table 2
Diagnoses of the participants as reported in the SCID interviews at pre-treat-
ment.
ICBT
(n= 48)
Control
(n= 47)
Total
(n= 95)
Depressive disorders
Major depressive disorder 33 (68.8%) 35 (74.5%) 68 (71.6%)
Dysthymic disorder 6 (12.5%) 3 (6.4) 9 (9.5%)
Depressive disorder NOS 2 (4.2%) 2 (4.2%) 4 (4.2%)
MDD in partial remission 8 (16.7%) 7 (14.9%) 15 (15.8%)
Any depressive disorder 48 (100%) 47 (100%) 95 (100%)
Anxiety disorders
GAD 4 (8.3%) 3 (6.4%) 7 (7.4%)
Panic disorder with or without
agoraphobia
7 (14.6%) 7 (14.9%) 14 (14.7%)
Specific phobia 3 (6.3%) 5 (10.6%) 8 (8.4%)
Social phobia 4 (8.3%) 1 (2.1%) 5 (5.3%)
OCD 3 (6.3%) 2 (4.3%) 5 (5.3%)
PTSD 1 (2.1%) 2 (4.3%) 3 (3.2%)
Anxiety disorder NOS 4 (8.3%) 5 (10.6%) 9 (9.5%)
Any anxiety disorder 21 (43.8%) 24 (51.1%) 45 (47.4%)
Personality disorders
Borderline 1 (2.1%) 2 (4.3%) 3 (3.2%)
Obsessive-compulsive 1 (2.1%) 1 (2.1%) 2 (2.1%)
Antisocial 2 (4.2%) 0 (0.0%) 2 (2.1%)
Depressive 2 (4.2%) 2 (4.3%) 4 (4.2%)
Avoidant 0 (0.0%) 3 (6.4%) 3 (3.2%)
Passive-aggressive 0 (0.0%) 1 (2.1%) 1 (1.1%)
Personality disorder NOS 1 (2.1%) 1 (2.1%) 2 (2.1%)
Any personality disorder 7 (14.6%) 7 (14.9%) 14 (14.7%)
Other disorders
Maladaptive stress reaction 3 (6.3%) 2 (4.3%) 5 (5.3%)
Eating disorder 1 (2.1%) 0 (0.0%) 1 (1.1%)
Substance-related disorder 0 (0.0%) 3 (6.4%) 3 (3.2%)
Any other disorder 4 (8.3%) 5 (10.6%) 9 (9.5%)
Note: NOS = not otherwise specified; MDD = Major depressive disorder;
GAD = Generalized anxiety disorder; OCD = Obsessive compulsive disorder;
PTSD = Post traumatic stress disorder. A patient could have more than one
diagnosis.
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
3
2.6. Analyses
A group (ICBT versus control) × time (pre-, post-treatment, 6-, 12-,
and 24-month follow-ups) randomized design based on the intent-to-
treat model was used. Linear mixed models (Brown and Prescott, 1999)
with full information maximum likelihood estimation were employed to
handle missing data because repeated observations for the same in-
dividual are correlated which violates the assumption of independence
(Gueorguieva and Krystal, 2004;Brown and Prescott, 1999). Full in-
formation maximum likelihood estimation has the advantage of pro-
viding accurate estimates with missing data under fairly unrestrictive
missing data assumptions (Hesser, 2015) and is a recommended method
for handling missing data (Schafer and Graham, 2002). Estimated
parameters were obtained using a mixed-model approach employing an
unstructured covariance structure. The alpha level was set at 0.05.
Within and between-group effect sizes (Cohen's d)(Cohen, 1988) were
first calculated using the estimated means from the mixed model ana-
lysis and are shown in Table 3. We also calculated effect sizes based on
observed data (using the pooled standard deviations) for the ICBT and
control group (Table 4), as well as between ICBT-patients with or
without a PD (Table 5).
For remission rates, clinically significant change was determined for
BDI-II by using a combination of a lower limit cut-offscore (two stan-
dard deviations from the pretest mean) and a minimum improvement
Fig. 1. Participant flowchart (remaining participants are based on MADRS-S ratings).
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
4
score calculated by means of the Reliable Change Index (RCI) for each
individual. The formulas provided by Jacobson and Truax (1991) were
employed. The RCI was calculated using a test-retest reliability of 0.80
for both the MADRS-S (Svanborg and Asberg, 1994) and the BDI-II
(Beck et al., 1996). The RCI cut-offwas 1.96 (p< .05). For the
MADRS-S the change is considered reliable and clinically significant if
the score is below 15 points and has decreased by at least 5 points, and
for the BDI-II when the score is below 13 points and has decreased at
least 10 points. Cut-offs using these formulas have been applied in
previous ICBT research (Berger et al., 2011b). In the RCI-analyses,
missing data was replaced with the last known value. The second way
of determining clinical significance was whether the participants were
diagnosed with a depressive disorder in the SCID interview after
treatment. No assumptions were made about diagnostic status for par-
ticipants who we were unable to interview at post treatment. Chi
2
tests
were used to analyse the difference in proportions with diagnose be-
tween the groups.
Multiple regression analyses were performed to assess whether co-
morbidity predicted the improvement in depressive symptoms. The
outcome variables were standardized change scores on the MADRS-S
and the BDI-II between pre and post-treatment, calculated with the
formula Z
2
−(Z
1
∗R
12
)(Steketee and Chambless, 1992). The in-
dependent variables, i.e. the presence (or non-presence) of an anxiety
disorder or a personality disorder, were inserted simultaneously into
the model.
3. Results
3.1. Adherence
In the ICBT group, the mean number of modules completed was 5.9
(SD = 2.2) (range 1–8) out of a maximum of 8 (73.8%). In the control
condition, an average of 6.8 (SD = 3.1) (range 1–9) modules were
completed out of a maximum of 9 (75.6%).
3.2. Clinical significance
There was no significant difference between the proportions of pa-
tients with the diagnosis of major depression at pre-treatment in the
two groups (χ
2
= 0.38, df = 1, p= .537) nor any difference between
the proportions of those fulfilling the criteria for any depressive dis-
order at pre-treatment in the two groups (χ
2
= 0.56, df = 1, p= .81)
(see Table 2). In the ICBT group, the number diagnosed with MDD
dropped from 33/48 (68.8%) participants at pre-treatment to 9/38
(23.7%) after treatment. In the control group, the number diagnosed
with MDD decreased from 35/47 (74.5%) to 19/36 (52.7%). After
treatment, significantly fewer participants in the ICBT group fulfilled
the criteria for MDD compared to the control group (χ
2
= 6.653,
df = 1, p= .01). The proportions fulfilling the criteria for any de-
pressive disorder at post-treatment in the two groups did not differ
significantly (χ
2
= 1.9, df = 1, p= .168).
From pre- to post-treatment, 56% (n= 27) of the participants in the
ICBT group exhibited reliable change (Jacobson and Truax, 1991)on
the BDI-II compared to 36% (n= 17) in the control group. Although the
difference was not significant, a trend favoring ICBT could be observed
(χ
2
= 3.646, df = 1, p= .056). At the 6-month follow-up, 60%
(n= 29) in the ICBT group and 38% (n= 18) in the control group
demonstrated reliable change. At this stage the difference between the
two groups had decreased (χ
2
= 2.109, df = 1, p= .146). Similar
results were found at the 12-month follow-up; 58% (n= 28) in the
ICBT group and 30% (n= 14) in the control group exhibited reliable
change (χ
2
= 1.212, df = 1, p= .271).
3.3. Changes over time on the BDI-II: ICBT versus active control
The estimated means and statistics for the two groups across time
Table 3
Estimated means (standard deviations) for BDI-II and MADRS-S from pre-treatment to the 24-month assessment, effect sizes within and between groups.
Group Pre
[M (SD)]
Post
[M (SD)]
WG dBG d6-m
[M (SD)]
WG dBG d12-m
[M (SD)]
WG dBG d24-m
[M (SD)]
WG dBG d
BDI-II ICBT 29.5 (7.6) 18.5 (11.8) 1.11 0.23 14.7 (10.4) 1.62 0.18 15.3 (11.8) 1.43 0.08 13.5 (7.5) 2.11 0.00
CONT 27.0 (7.5) 21.2 (11.7) 0.59 16.6 (11.0) 1.10 16.3 (13.7) 0.97 13.5 (15.4) 1.11
MADRS-S ICBT 23.2 (4.2) 14.5 (8.3) 1.32 0.23 12.3 (7.6) 1.78 0.27 12.9 (9.7) 1.38 0.13 11.9 (10.6) 1.40 0.03
CONT 22.9 (4.1) 16.5 (8.9) 0.92 14.4 (8.2) 1.31 14.3 (11.7) 0.98 11.6 (12.0) 1.26
Q9 MADRS-S ICBT 2.1 (1.0) 1.4 (1.1) 0.67 0.09 1.1 (1.1) 0.95 0.09 1.2 (1.3) 0.78 0.07 1.0 (1.7) 0.81 0.06
CONT 2.2 (1.0) 1.5 (1.2) 0.63 1.2 (1.2) 0.91 1.3 (1.6) 0.67 1.1 (1.4) 0.92
Note. BDI-II = Beck Depression Inventory –Second Edition, BG = between-group, ICBT = internet-based cognitive behavior therapy, CONT = control group,
d= Cohen's d, M = mean, MADRS-S = Montgomery-Åsberg Depression Rating Scale –self-rated version, SD = standard deviation, WG = within-group. All the
within-group effect sizes were calculated based on the pre-treatment scores.
Table 4
Within and between-group effect size (Cohens' d), based on observed data, at
post-treatment and the 6-, 12- and 24-month follow-ups.
Within-group effect size Between-group effect size
ICBT Active control ICBT vs. Active control
MADRS-S
Post-treatment 1.57 1.00 0.31
6-month follow-up 2.02 1.37 0.30
12-month follow-up 1.42 1.25 0.15
24-month follow-up 1.80 1.55 0.01
BDI-II
Post-treatment 1.40 0.65 0.22
6-month follow-up 1.78 1.11 0.20
12-month follow-up 1.48 0.93 0.13
24-month follow-up 1.75 1.20 0.06
Table 5
Mean difference between participants with and without a personality disorder
(PD) on the MADRS-S and BDI-II as well as between-group (no PD vs. PD) effect
size (Cohens' d), based on observed data, at post-treatment and the 6-, 12- and
24-month follow-ups for the ICBT group.
Mean difference no PD vs. PD Between-group effect size
MADRS-S
Pre-treatment 0.07 0.02
Post-treatment 6.74 0.97
6-month follow-up 1.77 0.28
12-month follow-up 10.76 1.25
24-month follow-up 5.14 0.62
BDI-II
Pre-treatment −0.80 −0.11
Post-treatment 8.06 0.90
6-month follow-up 1.56 0.18
12-month follow-up 9.43 0.94
24-month follow-up 5.93 0.54
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
5
for the BDI-II are presented in Table 3. Mixed-effect models showed a
non-significant group effect (F = 0.01, df = 94.33, p= .991), a sig-
nificant time effect (F = 63.28, df = 78.06, p< .001), and a sig-
nificant group × time effect on the BDI-II (F = 5.52, df = 78.06,
p= .021) from pre to post-treatment in favor of the ICBT group.
Further analyses were carried out to examine changes from pre-
treatment to 6-, 12- and 24-month follow-ups. Mixed-effect models
showed a non-significant group effect (F = 0.08, df = 87.72,
p= .782), a significant time effect (F = 64.09, df = 75.86, p< .001),
and a non-significant group × time effect on the BDI-II (F = 3.01,
df = 75.86, p= .055) from pre-treatment to the 6-month follow-up.
Mixed effect models also demonstrated a non-significant group effect
(F = 0.19, df = 81.27, p= .661), a significant time effect (F = 42.68,
df = 70.61, p< .001), and a non-significant group × time effect on
the BDI-II (F = 2.44, df = 70.61, p= .071) from pre-treatment to the
12-month follow-up. There was a non-significant group effect
(F = 0.47, df = 73.61, p= .496), a significant time effect (F = 90.91,
df = 55.82, p< .001), and a non-significant group × time effect on
the BDI-II (F = 0.68, df = 55.82, p= .414) from pre-treatment to the
24-month follow-up.
Finally, analyses were carried out to examine changes from post-
treatment to the 6-, 12- and 24-month follow-ups. Mixed-effect models
showed a non-significant group effect (F = 0.88, df = 74.96, p= .351;
F = 0.36, df = 74.96, p= .553; F = 0.94, df = 68.82, p= .760), a
significant time effect (F = 10.17, df = 70.61, p= .002; F = 7.48,
df = 67.46 p= .008; F = 15.13, df = 54.75, p< .001), and a non-
significant group × time effect on the BDI-II (F = 0.18, df = 70.61,
p= .670; F = 0.92, df = 67.46, p= .341; F = 1.51, df = 54.75,
p= .225) from post-treatment to the 6-, 12- and 24-month follow-up
respectively.
3.4. Changes over time on the MADRS-S: ICBT versus active control
The estimated MADRS-S means and statistics for the two groups
over time are presented in Table 3. Mixed-effect models showed a non-
significant group effect (F = 0.55, df = 85.56, p=.460), a significant
effect of time (F = 83.11, df = 75.41, p< .001), and a non-significant
group × time effect on the MADRS-S (F = 1.99, df = 75.41, p= .163)
from pre- to post-treatment.
Further analyses were carried out to examine changes from pre-
treatment to the 6-, 12- and 24-month follow-ups. Mixed-effect models
showed a non-significant group effect (F = 0.99, df = 76.61,
p= .323), a significant time effect (F = 74.86, df = 70.63, p< .001),
and a non-significant group × time effect on the MADRS-S (F = 1.16,
df = 70.63, p= .318) from pre-treatment to the 6-month follow-up.
There was a non-significant group effect (F = 1.17, df = 71.28,
p= .282), a significant effect of time (F = 53.09, df = 65.53,
p<.001), and a non-significant group × time effect on the MADRS-S
(F = 1.09, df = 65.53, p= .358) from pre-treatment to the 12-month
follow-up. Likewise, there was a non-significant group effect (F = 0.06,
df = 60.65, p= .804), a significant time effect (F = 89.46, df = 54.04,
p< .001), and a non-significant group × time effect on the MADRS-S
(F = 0.01, df = 54.04, p= .974) from pre-treatment to the 24-month
follow-up.
Finally, analyses were carried out to examine changes from post-
treatment to the 6-, 12- and 24-month follow-ups. There was a non-
significant group effect (F = 2.03, df = 74.41, p= .159; F = 1.06,
df = 74.91, p= .306; F = 0.24, df = 68.08, p= .627), a significant
time effect (F = 4.67, df = 70.99, p= .034; F = 4.05, df = 65.53,
p= .048; F = 10.62, df = 55.02, p= .002), and a non-significant
group × time effect on the MADRS-S (F = 0.15, df = 70.99, p= .699;
F = 0.56, df = 65.53, p= .457; F = 2.23, df = 55.02, p= .141) from
post-treatment to the 6-, 12- and 24-month follow-ups.
3.5. Changes over time on suicidal thoughts (MADRS-S item 9): ICBT
versus active control
The estimated MADRS-S item 9 (zest for life) means and statistics
for the two groups across time are presented in Table 3. Mixed-effect
models revealed a non-significant group effect (F = 0.13, df = 91.23,
p= .724), a significant effect of time (F = 41.40, df = 80.24,
p< .001), and a non-significant group × time effect on item 9
(F = 0.01, df = 80.24, p= .999) from pre- to post-treatment.
There was a non-significant group effect (F = 0.05, df = 85.25,
p= .832), a significant time effect (F = 36.50, df = 74.02, p< .001),
and a non-significant group × time effect on item 9 (F = 0.07,
df = 74.02, p= .935) from pre-treatment to the 6-month follow-up.
Likewise, there was a non-significant group effect (F = 0.21,
df = 89.72, p= .649), a significant time effect (F = 26.20, df = 69.81,
p< .001), and a non-significant group × time effect on item 9
(F = 0.09, df = 69.81, p= .968) from pre-treatment to the 12-month
follow-up. There was also a non-significant group effect (F = 0.01,
df = 75.92, p= .932), a significant time effect (F = 46.07, df = 57.22,
p< .001), and a non-significant group × time effect on item 9
(F = 0.09, df = 57.22, p= .767) from pre-treatment to the 24-month
follow-up.
Finally, analyses were carried out to examine changes from post-
treatment to the 6-, 12- and 24-month follow-ups. There was a non-
significant group effect (F = 0.05, df = 75.60, p= .832; F = 0.28,
df = 77.82, p= .597; F = 0.01, df = 68.91, p= .990), a mixed effect
of time (F = 3.98, df = 72.02, p= .050; F = 2.06, df = 68.81,
p= .156; F = 5.85, df = 53.35, p= .019), and a non-significant group
× time effect on item 9 (F = 0.22, df = 72.02, p= .638; F = 0.01,
df = 68.81, p= .934; F = 0.86, df = 53.35, p= .358) from post-
treatment to the 6-, 12- and 24-month follow-ups.
3.6. Effects of comorbidity
The presence of a personality disorder (PD) was found to sig-
nificantly predict a lower level of improvement in depressive symptoms
between pre- and post-treatment when measured by the MADRS-S
(β=−0.342, t(39) = −2.213, p= .033), and when measured by the
BDI-II (β=−0.321, t(39) = −2.151, p= .038). The presence of an
anxiety disorder did not significantly predict the level of improvement
in depressive symptoms when measured by MADRS-S (β=−0.051, t
(39) = −0.327, p= .746), or when measured by the BDI-II
(β=−0.299, t(39) = −2.009, p= .052) although a trend was ob-
served.
3.7. Effect sizes based on observed data
The effect sizes (Cohen's d) calculated using observed data for both
MADRS-S and BDI-II were large within both the ICBT group and the
control group at post-treatment and small between groups (Table 4).
The between-groups effect sizes (Cohen's d) calculated using observed
data to compare participants with and without PDs in the ICBT group
were large at post-treatment (Table 5).
4. Discussion
The aim of this study was to compare ICBT to an active control
condition in a clinical sample of patients with a depressive disorder
recruited from primary and psychiatric care. Significantly more ICBT
patients had entered remission after treatment compared to the active
control patients. Internet-based CBT reduced depressive symptoms
(measured by BDI-II) significantly more than an active control condi-
tion between pre- and post-treatment in a clinical sample. During
follow-up, the difference in symptom levels between the groups de-
creased and disappeared after two years, however, the improvements
were sustained during two years. The effect size of ICBT in this study
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
6
can be considered large. The effect of ICBT on depressive symptoms was
not significantly moderated by the presence of a comorbid anxiety
disorder, but was clearly reduced in cases with a personality disorder.
We are not aware of any previous study that has investigated the
effect of comorbid personality pathology on ICBT for depression. In the
present study, we found that on average, patients without PD had al-
most a one standard deviation larger improvement at post-treatment
than those with PD. This is in line with previous research on psy-
chotherapy face-to-face (Newton-Howes et al., 2006;Newton-Howes
et al., 2014), however, we are not aware of any previous studies
showing that patients with PD may also benefit far less from internet
based CBT for depression. This result needs to be replicated, preferably
with a larger sample that enables examination of possible differences in
effect from specific PDs. If this result is replicated it raises the question
if ICBT can be adapted to become an effective treatment for depression
when comorbidity with a PD is present. Including elements from dia-
lectic behavior therapy (DBT) (Linehan, 1993) could be one way of
adapting the treatment for the needs of some patients with a PD since it
has been shown to be an effective face-to-face psychotherapy for pa-
tients with borderline personality disorder (Linehan et al., 2006;Cristea
et al., 2017) and has also shown an effect on depressive symptoms in
cases with comorbidity (Lynch et al., 2007).
The within-group effect size of ICBT in this study was (d) = 1.4,
which should be compared to (d) = 1.23 in the efficacy study by
Andersson et al. (2005), as the treatment content is almost identical.
However, the sample in the study by Andersson et al. (2005) was re-
cruited through the media and the study was conducted in a university
setting. This provides further evidence that ICBT can yield a similar
effect in a health care setting as in previous efficacy studies. The effect
in the present study was similar to the effects in previous effectiveness
studies, e.g. (d) = 1.27 (Hedman et al., 2014), (d) = 1.0 (calculated
with observed means) (Titov et al., 2015), and (d) = 1.09 (Kivi et al.,
2014).
The effect of the active control treatment was surprisingly large in
view of the fact that the modules did not contain any suggestions for
altering behavior or questioning dysfunctional thoughts. One possible
explanation is that therapist support seems to be responsible for a large
proportion of the effect in meta-analyses. The therapists were instructed
to only offer support to the control patients and not provide any gui-
dance resembling CBT techniques to ensure that no CBT elements
would seep into the control treatment. However, recent results reveal
that therapist guidance in ICBT is also mainly supportive (Holländare
et al., 2016) so perhaps the effect of supportive guidance explains why
the control patients also improved over time. Another possibility is that
merely reading the information material increased activity levels,
which could have reduced depressive symptoms.
The effect of ICBT in this study was not significantly reduced by the
presence of an anxiety disorder, which is in line with a recent meta-
analysis by Karyotaki et al. (2018) which found no difference in out-
come after ICBT for depression between patients with and without
comorbid anxiety. However, our result showing that the presence of a
PD did reduce the effect of ICBT raises questions about the effect of
other possible comorbidities, e.g., neuropsychiatric disorders or sub-
stance abuse, on the outcome of internet bases CBT. More knowledge
about the effect of comorbidities on ICBT outcome could help guide
clinical decisions and the development of adapted treatments for sub-
groups.
The drop-out rate was higher in the control condition compared to
the ICBT. Since CBT is a well-known form of therapy and the in-
formation condition is not an established intervention, a speculation is
that some of those randomized to information dropped out due to
disappointment with their allocation. Another speculation is that they
might have found the modules or the therapist guidance ineffective or
unappealing. The randomized design and long term follow-up are
strengths of this study. The inclusion of more than one depressive dis-
order might increase external validity because the case mix was derived
from a recruitment process within a regular clinical health care setting.
The aim of the recruitment process was that each participant should be
eligible for health care before being asked to participate, making the
sample representative of those who normally seek help for depression.
Additional strengths are the thorough diagnostic process using face-to-
face SCID-I interviews before and after treatment, which were con-
ducted by clinical psychologists who were not aware of the patients'
allocation. Another strength is the use of outcome measures that have
been validated for online use (Hollandare et al., 2010). The limited
sample size did not allow for analyses of the impact on outcome from
different personality disorders, and also makes the estimate of the size
of differences in outcome between patients with and without PD im-
precise (Kapur and Munafo, 2019). Another limitation was missing data
during the follow up which limits statistical power. We did not assess
PD after the intervention which limits our knowledge of diagnostic
status post treatment.
5. Conclusions
In Conclusion, although ICBT reduced depressive symptoms more
than an active control condition directly after treatment, the two in-
terventions seemed to have a similar effect over time. Depressed pa-
tients within primary and psychiatric care, with or without a comorbid
anxiety disorder, can benefit from ICBT. Patients with a PD seem to
benefit less from ICBT for depression, however this needs to be in-
vestigated further.
Funding
Financial support for the study was provided by two Swedish re-
search funds: Region Örebro County Research Committee [OLL-57/06],
and the Regional Research Council [RFR-73351]. No funding body took
part in designing the study, collecting or analysing the data, inter-
preting the results or writing the manuscript.
Declaration of competing interest
The authors have declared that no conflict of interest exists.
References
American Psychiatric Association, 2000. Diagnostic and Statistical Manual of Mental
Disorders, 4th ed. American Psychiatric Press, Washington DC Text Revision.
Andersson, G., 2016. Internet-delivered psychological treatments. Annu. Rev. Clin.
Psychol. 12, 157–179.
Andersson, G., Bergstrom, J., Hollandare, F., Carlbring, P., Kaldo, V., Ekselius, L., 2005.
Internet-based self-help for depression: randomised controlled trial. Br. J. Psychiatry
187, 456–461.
Andersson, G., Carlbring, P., Holmstrom, A., Sparthan, E., Furmark, T., Nilsson-Ihrfelt, E.,
Buhrman, M., Ekselius, L., 2006. Internet-based self-help with therapist feedback and
in vivo group exposure for social phobia: a randomized controlled trial. J. Consult.
Clin. Psychol. 74, 677–686.
Andersson, G., Carlbring, P., Ljótsson, B., Hedman, E., 2013a. Guided internet-based CBT
for common mental disorders. J. Contemp. Psychother. 43, 223–233.
Andersson, G., Hesser, H., Hummerdal, D., Bergman-Nordgren, L., Carlbring, P., 2013b. A
3.5-year follow-up of internet-delivered cognitive behavior therapy for major de-
pression. J. Ment. Health 22, 155–164. https://doi.org/10.3109/09638237.2011.
608747. (Epub 2011 Sep 29).
Andersson, G., Topooco, N., Havik, O., Nordgreen, T., 2016. Internet-supported versus
face-to-face cognitive behavior therapy for depression. Expert. Rev. Neurother. 16,
55–60. https://doi.org/10.1586/14737175.2015.1125783. (Epub 2015 Dec 15).
Beck, A.T., 1979. Cognitive Therapy of Depression. Guilford Press.
Beck, A.T., Steer, R.A., Brown, G.K., 1996. Manual for the Beck Depression Inventory-II.
Psychological Corporation, San Antonio, TX.
Beck, A.T., Steer, R.A., Brown, G.K., 2005. BDI-II, Beck Depression Inventory - Second
Edition. Manual - Svensk Version. Psykologiförlaget, Sandviken.
Berger, T., Caspar, F., Richardson, R., Kneubuhler, B., Sutter, D., Andersson, G., 2011a.
Internet-based treatment of social phobia: a randomized controlled trial comparing
unguided with two types of guided self-help. Behav. Res. Ther. 49, 158–169.
Berger, T., Hammerli, K., Gubser, N., Andersson, G., Caspar, F., 2011b. Internet-based
treatment of depression: a randomized controlled trial comparing guided with un-
guided self-help. Cogn. Behav. Ther. 40, 251–266.
Botella, C., Gallego, M.J., Garcia-Palacios, A., Guillen, V., Banos, R.M., Quero, S., Alcaniz,
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
7
M., 2010. An internet-based self-help treatment for fear of public speaking: a con-
trolled trial. Cyberpsychol. Behav. Soc. Netw. 13, 407–421.
Brown, H., Prescott, R., 1999. Applied Mixed Models in Medicine. John Wiley & Sons,
Chichester.
Carlbring, P., Nilsson-Ihrfelt, E., Waara, J., Kollenstam, C., Buhrman, M., Kaldo, V.,
Soderberg, M., Ekselius, L., Andersson, G., 2005. Treatment of panic disorder: live
therapy vs. self-help via the internet. Behav. Res. Ther. 43, 1321–1333.
Carlbring, P., Hägglund, M., Luthström, A., Dahlin, M., Kadowaki, Å., Vernmark, K.,
Andersson, G., 2013. Internet-based behavioral activation and acceptance-based
treatment for depression: a randomized controlled trial. J. Affect. Disord. 148 (2–3),
331–337. https://doi.org/10.1016/j.jad.2012.12.020.
Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence
Erlbaum Associates.
Cristea, I.A., Gentili, C., Cotet, C.D., Palomba, D., Barbui, C., Cuijpers, P., 2017. Efficacy
of psychotherapies for borderline personality disorder: a systematic review and meta-
analysis. JAMA Psychiat. 74, 319–328.
Dear, B.F., Fogliati, V.J., Fogliati, R., Johnson, B., Boyle, O., Karin, E., Gandy, M.,
Kayrouz, R., Staples, L.G., Titov, N., 2018. Treating anxiety and depression in young
adults: a randomised controlled trial comparing clinician-guided versus self-guided
internet-delivered cognitive behavioural therapy. Aust. N. Z. J. Psychiatry 52,
668–679.
Fantino, B., Moore, N., 2009. The self-reported Montgomery-Asberg Depression Rating
Scale is a useful evaluative tool in major depressive disorder. BMC Psychiatry 9, 26.
First, M., Spitzer, R., Gibbon, M., Williams, J., Benjamin, L., 1994. Structured Clinical
Interview for DSM-IV Axis II Personality Disorders (SCID II). Biometric Research
Department, New York.
First, M.B., Gibbon, M., Spitzer, R.L., Williams, J.B.W., 1998. Structured Clinical
Interview for DSM-IV Disorders (SCID-I) (Swedish Version). Pilgrim Press, Danderyd.
Forand, N.R., Derubeis, R.J., 2013. Pretreatment anxiety predicts patterns of change in
cognitive behavioral therapy and medications for depression. J. Consult. Clin.
Psychol. 81, 774–782. https://doi.org/10.1037/a0032985. (Epub 2013 May 6).
Fournier, J.C., Derubeis, R.J., Shelton, R.C., Gallop, R., Amsterdam, J.D., Hollon, S.D.,
2008. Antidepressant medications v. cognitive therapy in people with depression
with or without personality disorder. Br. J. Psychiatry 192, 124–129.
Friborg, O., Martinsen, E.W., Martinussen, M., Kaiser, S., Overgård, K.T., Rosenvinge,
J.H., 2014. Comorbidity of personality disorders in mood disorders: a meta-analytic
review of 122 studies from 1988 to 2010. J. Affect. Disord. 152–154, 1–11.
Garcia-Batista, Z.E., Guerra-Pena, K., Cano-Vindel, A., Herrera-Martinez, S.X., Medrano,
L.A., 2018. Validity and reliability of the Beck Depression Inventory (BDI-II) in
general and hospital population of Dominican Republic. PLoS One 13, e0199750.
Gibbons, C.J., Derubeis, R.J., 2008. Anxiety symptom focus in sessions of cognitive
therapy for depression. Behav. Ther. 39, 117–125. https://doi.org/10.1016/j.beth.
2007.05.006. (Epub 2007 Oct 31).
Gortner, E.T., Gollan, J.K., Dobson, K.S., Jacobson, N.S., 1998. Cognitive-behavioral
treatment for depression: relapse prevention. J. Consult. Clin. Psychol. 66, 377–384.
Gueorguieva, R., Krystal, J.H., 2004. Move over ANOVA: progress in analyzing repeated-
measures data and its reflection in papers published in the Archives of General
Psychiatry. Arch. Gen. Psychiatry 61, 310–317.
Hadjistavropoulos, H.D., Nugent, M.M., Alberts, N.M., Staples, L., Dear, B.F., Titov, N.,
2016. Transdiagnostic internet-delivered cognitive behaviour therapy in Canada: an
open trial comparing results of a specialized online clinic and nonspecialized com-
munity clinics. J. Anxiety Disord. 42, 19–29.
Hedman, E., Ljotsson, B., Lindefors, N., 2012. Cognitive behavior therapy via the internet:
a systematic review of applications, clinical efficacy and cost-effectiveness. Expert
Rev. Pharmacoecon. Outcomes Res. 12, 745–764.
Hedman, E., Ljotsson, B., Kaldo, V., Hesser, H., EL Alaoui, S., Kraepelien, M., Andersson,
E., Ruck, C., Svanborg, C., Andersson, G. & Lindefors, N. 2014. Effectiveness of in-
ternet-based cognitive behaviour therapy for depression in routine psychiatric care. J.
Affect. Disord.,155:49–58., https://doi.org/10.1016/j.jad.2013.10.023. (Epub 2013
Oct 26).
Heinrich, S., Rozental, A., Carlbring, P., Andersson, G., Cotter, K., Weise, C., 2016.
Treating tinnitus distress via the internet: a mixed methods approach of what makes
patients seek help and stay motivated during internet-based cognitive behavior
therapy. Internet Interv. 4, 120–130.
Hesser, H., 2015. Modeling individual differences in randomized experiments using
growth models: recommendations for design, statistical analysis and reporting of
results of internet interventions. Internet Interv. 2, 110–120.
Hollandare, F., Andersson, G., Engstrom, I., 2010. A comparison of psychometric prop-
erties between internet and paper versions of two depression instruments (BDI-II and
MADRS-S) administered to clinic patients. J. Med. Internet Res. 12, e49.
Hollandare, F., Anthony, S.A., Randestad, M., Tillfors, M., Carlbring, P., Andersson, G.,
Engstrom, I., 2013. Two-year outcome of internet-based relapse prevention for par-
tially remitted depression. Behav. Res. Ther. 51, 719–722. https://doi.org/10.1016/j.
brat.2013.08.002. (Epub 2013 Aug 16).
Holländare, F., Gustafsson, S., Berglind, M., Grape, F., Carlbring, P., Andersson, G.,
Hadjistavropoulos, H., Tillfors, M., 2016. Therapist behaviours in internet-based
cognitive behaviour therapy (ICBT) for depressive symptoms. Internet Interv. 3, 1–7.
Ivarsson, D., Blom, M., Hesser, H., Carlbring, P., Enderby, P., Nordberg, R., Andersson, G.,
2014. Guided internet-delivered cognitive behaviour therapy for post-traumatic
stress disorder: a randomized controlled trial. Internet Interv. 1, 33–40.
Jacobson, N.S., Truax, P., 1991. Clinical significance: a statistical approach to defining
meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 59, 12–19.
Johansson, R., Andersson, G., 2012. Internet-based psychological treatments for depres-
sion. Expert. Rev. Neurother. 12, 861–870.
Johansson, R., Bjorklund, M., Hornborg, C., Karlsson, S., Hesser, H., Ljotsson, B.,
Rousseau, A., Frederick, R.J., Andersson, G., 2013. Affect-focused psychodynamic
psychotherapy for depression and anxiety through the internet: a randomized con-
trolled trial. PeerJ 1, e102.
Johansson, O., Bjarehed, J., Andersson, G., Carlbring, P., Lundh, L.G., 2019. Effectiveness
of guided internet-delivered cognitive behavior therapy for depression in routine
psychiatry: a randomized controlled trial. Internet Interv. 17, 100247.
Joyce, P.R., Mckenzie, J.M., Carter, J.D., Rae, A.M., Luty, S.E., Frampton, C.M., Mulder,
R.T., 2007. Temperament, character and personality disorders as predictors of re-
sponse to interpersonal psychotherapy and cognitive-behavioural therapy for de-
pression. Br. J. Psychiatry 190, 503–508.
Kampmann, I.L., Emmelkamp, P.M., Morina, N., 2016. Meta-analysis of technology-as-
sisted interventions for social anxiety disorder. J. Anxiety Disord. 42, 71–84.
Kapur, S., Munafo, M., 2019. Small sample sizes and a false economy for psychiatric
clinical trials. JAMA Psychiatry 76, 676–677.
Karyotaki, E., Ebert, D.D., Donkin, L., Riper, H., Twisk, J., Burger, S., Rozental, A., Lange,
A., Williams, A.D., Zarski, A.C., Geraedts, A., Van Straten, A., Kleiboer, A., Meyer, B.,
Unlu Ince, B.B., Buntrock, C., Lehr, D., Snoek, F.J., Andrews, G., Andersson, G., Choi,
I., Ruwaard, J., Klein, J.P., Newby, J.M., Schroder, J., Laferton, J.A.C., Van Bastelaar,
K., Imamura, K., Vernmark, K., Boss, L., Sheeber, L.B., Kivi, M., Berking, M., Titov, N.,
Carlbring, P., Johansson, R., Kenter, R., Perini, S., Moritz, S., Nobis, S., Berger, T.,
Kaldo, V., Forsell, Y., Lindefors, N., Kraepelien, M., Bjorkelund, C., Kawakami, N.,
Cuijpers, P., 2018. Do guided internet-based interventions result in clinically relevant
changes for patients with depression? An individual participant data meta-analysis.
Clin. Psychol. Rev. 63, 80–92.
Kashdan, T.B., Roberts, J.E., 2011. Comorbid social anxiety disorder in clients with de-
pressive disorders: predicting changes in depressive symptoms, therapeutic re-
lationships, and focus of attention in group treatment. Behav. Res. Ther. 49, 875–884.
https://doi.org/10.1016/j.brat.2011.10.002. (Epub 2011 Oct 12).
Kessler, R.C., Merikangas, K.R., Wang, P.S., 2007. Prevalence, comorbidity, and service
utilization for mood disorders in the United States at the beginning of the twenty-first
century. Annu. Rev. Clin. Psychol. 3, 137–158.
Kessler, D., Lewis, G., Kaur, S., Wiles, N., King, M., Weich, S., Sharp, D.J., Araya, R.,
Hollinghurst, S., Peters, T.J., 2009. Therapist-delivered internet psychotherapy for
depression in primary care: a randomised controlled trial. Lancet 374, 628–634.
Kivi, M., Eriksson, M.C., Hange, D., Petersson, E.L., Vernmark, K., Johansson, B.,
Bjorkelund, C., 2014. Internet-based therapy for mild to moderate depression in
Swedish primary care: short term results from the PRIM-NET randomized controlled
trial. Cogn. Behav. Ther. 43, 289–298. https://doi.org/10.1080/16506073.2014.
921834. (Epub 2014 Jun 9).
Kuhner, C., Burger, C., Keller, F., Hautzinger, M., 2007. Reliability and validity of the
Revised Beck Depression Inventory (BDI-II). Results from German samples.
Nervenarzt 78, 651–656.
Linehan, M., 1993. Cognitive-behavioral Treatment of Borderline Personality Disorder.
The Guilford Press, New York.
Linehan, M.M., Comtois, K.A., Murray, A.M., Brown, M.Z., Gallop, R.J., Heard, H.L.,
Korslund, K.E., Tutek, D.A., Reynolds, S.K., Lindenboim, N., 2006. Two-year rando-
mized controlled trial and follow-up of dialectical behavior therapy vs therapy by
experts for suicidal behaviors and borderline personality disorder. Arch. Gen.
Psychiatry 63, 757–766.
Lis, E., Myhr, G., 2016. The effect of borderline personality pathology on outcome of
cognitive behavior therapy. J. Psychiatr. Pract. 22, 270–282. https://doi.org/10.
1097/PRA.0000000000000167.
Lynch, T.R., Cheavens, J.S., Cukrowicz, K.C., Thorp, S.R., Bronner, L., Beyer, J., 2007.
Treatment of older adults with co-morbid personality disorder and depression: a
dialectical behavior therapy approach. Int. J. Geriatr. Psychiatry 22, 131–143.
Martell, C.R., Addis, M.E., Jacobson, N.S., 2001. Depression in Context: Strategies for
Guided Action. W W Norton & Company Incorporated.
Mathiasen, K., Riper, H., Andersen, T.E., Roessler, K.K., 2018. Guided internet-based
cognitive behavioral therapy for adult depression and anxiety in routine secondary
care: observational study. J. Med. Internet Res. 20, e10927.
Montgomery, S.A., Asberg, M., 1979. A new depression scale designed to be sensitive to
change. Br. J. Psychiatry 134, 382–389.
Morin, C.M., 1996. Relief from Insomnia: Getting the Sleep of Your Dreams. Broadway
Books.
Newton-Howes, G., Tyrer, P., Johnson, T., 2006. Personality disorder and the outcome of
depression: meta-analysis of published studies. Br. J. Psychiatry 188, 13–20.
Newton-Howes, G., Tyrer, P., Johnson, T., Mulder, R., Kool, S., Dekker, J., Schoevers, R.,
2014. Influence of personality on the outcome of treatment in depression: systematic
review and meta-analysis. J. Personal. Disord. 28, 577–593.
Nordgreen, T., Gjestad, R., Andersson, G., Carlbring, P., Havik, O.E., 2018. The effec-
tiveness of guided internet-based cognitive behavioral therapy for social anxiety
disorder in a routine care setting. Internet Interv. 13, 24–29.
Osman, A., Kopper, B.A., Barrios, F., Gutierrez, P.M., Bagge, C.L., 2004. Reliability and
validity of the Beck depression inventory—II with adolescent psychiatric inpatients.
Psychol. Assess. 16, 120–132.
Paxling, B., Lundgren, S., Norman, A., Almlov, J., Carlbring, P., Cuijpers, P., Andersson,
G., 2013. Therapist behaviours in internet-delivered cognitive behaviour therapy:
analyses of e-mail correspondence in the treatment of generalized anxiety disorder.
Behav. Cogn. Psychother. 41, 280–289. https://doi.org/10.1017/
S1352465812000240. (Epub 2012 May 1).
Perini, S., Titov, N., Andrews, G., 2009. Clinician-assisted internet-based treatment is
effective for depression: randomized controlled trial. Aust. N. Z. J. Psychiatry 43,
571–578.
Richards, D., Richardson, T., 2012. Computer-based psychological treatments for de-
pression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329–342
(Epub 2012 Feb 28).
Robinson, E., Titov, N., Andrews, G., Mcintyre, K., Schwencke, G., Solley, K., 2010.
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
8
Internet treatment for generalized anxiety disorder: a randomized controlled trial
comparing clinician vs. technician assistance. PLoS ONE [Electronic Resource] 5,
e10942.
Ruwaard, J., Schrieken, B., Schrijver, M., Broeksteeg, J., Dekker, J., Vermeulen, H.,
Lange, A., 2009. Standardized web-based cognitive behavioural therapy of mild to
moderate depression: a randomized controlled trial with a long-term follow-up. Cogn.
Behav. Ther. 38, 206–221.
Ruwaard, J., Lange, A., Schrieken, B., Dolan, C.V., Emmelkamp, P., 2012. The effec-
tiveness of online cognitive behavioral treatment in routine clinical practice. PLoS
ONE [Electronic Resource] 7, e40089.
Sanchez-Ortiz, V.C., Munro, C., Startup, H., Treasure, J., Schmidt, U., 2011. The role of
email guidance in internet-based cognitive-behavioural self-care treatment for bu-
limia nervosa. Eur. Eat. Disord. Rev. 19, 342–348. https://doi.org/10.1002/erv.
1074. (Epub 2011 Mar 10).
Schafer, J.L., Graham, J.W., 2002. Missing data: our view of the state of the art. Psychol.
Methods 7, 147–177.
Schindler, A., Hiller, W., Witthoft, M., 2013. What predicts outcome, response, and drop-
out in CBT of depressive adults? A naturalistic study. Behav. Cogn. Psychother. 41,
365–370. https://doi.org/10.1017/S1352465812001063. (Epub 2012 Dec 5).
Smits, J.A., Minhajuddin, A., Jarrett, R.B., 2009. Cognitive therapy for depressed adults
with comorbid social phobia. J. Affect. Disord. 114, 271–278. https://doi.org/10.
1016/j.jad.2008.08.008. (Epub 2008 Sep 18).
Steketee, G., Chambless, D., 1992. Methodological issues in prediction of treatment
outcome. Clin. Psychol. Rev. 12, 387–400.
Storch, E.A., Roberti, J.W., Roth, D.A., 2004. Factor structure, concurrent validity, and
internal consistency of the Beck Depression Inventory-Second Edition in a sample of
college students. Depress. Anxiety 19, 187–189.
Svanborg, P., Asberg, M., 1994. A new self-rating scale for depression and anxiety states
based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr. Scand.
89, 21–28.
Svanborg, P., Asberg, M., 2001. A comparison between the Beck Depression Inventory
(BDI) and the self-rating version of the Montgomery Asberg Depression Rating Scale
(MADRS). J. Affect. Disord. 64, 203–216.
Titov, N., Dear, B.F., Schwencke, G., Andrews, G., Johnston, L., Craske, M.G., Mcevoy, P.,
2011. Transdiagnostic internet treatment for anxiety and depression: a randomised
controlled trial. Behav. Res. Ther. 49, 441–452.
Titov, N., Dear, B.F., Johnston, L., Mcevoy, P.M., Wootton, B., Terides, M.D., Gandy, M.,
Fogliati, V., Kayrouz, R., Rapee, R.M., 2014. Improving adherence and clinical
outcomes in self-guided internet treatment for anxiety and depression: a 12-month
follow-up of a randomised controlled trial. PLoS One 9.
Titov, N., Dear, B.F., Staples, L.G., Bennett-Levy, J., Klein, B., Rapee, R.M., Shann, C.,
Richards, D., Andersson, G., Ritterband, L., Purtell, C., Bezuidenhout, G., Johnston,
L., Nielssen, O.B., 2015. MindSpot clinic: an accessible, efficient, and effective online
treatment service for anxiety and depression. Psychiatr. Serv. 66, 1043–1050.
https://doi.org/10.1176/appi.ps.201400477. (Epub 2015 Jul 1).
Titov, N., Fogliati, V.J., Staples, L.G., Gandy, M., Johnston, L., Wootton, B., Nielssen, O.,
Dear, B.F., 2016. Treating anxiety and depression in older adults: randomised con-
trolled trial comparing guided v. self-guided internet-delivered cognitive-behavioural
therapy. BJPsych Open 2, 50–58.
Van Bronswijk, S.C., Lemmens, L., Viechtbauer, W., Huibers, M.J.H., Arntz, A., Peeters, F.,
2018. The impact of personality disorder pathology on the effectiveness of cognitive
therapy and interpersonal psychotherapy for major depressive disorder. J. Affect.
Disord. 225, 530–538.
Vernmark, K., Lenndin, J., Bjarehed, J., Carlsson, M., Karlsson, J., Oberg, J., Carlbring, P.,
Eriksson, T., Andersson, G., 2010. Internet administered guided self-help versus in-
dividualized e-mail therapy: a randomized trial of two versions of CBT for major
depression. Behav. Res. Ther. 48, 368–376.
Warmerdam, L., Van Straten, A., Twisk, J., Riper, H., Cuijpers, P., 2008. Internet-based
treatment for adults with depressive symptoms: randomized controlled trial. J. Med.
Internet Res. 10, e44.
Watts, S., Newby, J.M., Mewton, L., Andrews, G., 2012. A clinical audit of changes in
suicide ideas with internet treatment for depression. BMJ Open 2.
Wikberg, C., Nejati, S., Larsson, M.E., Petersson, E.L., Westman, J., Ariai, N., Kivi, M.,
Eriksson, M., Eggertsen, R., Hange, D., Baigi, A., Bjorkelund, C., 2015. Comparison
between the Montgomery-Asberg depression rating scale-self and the Beck depression
inventory II in primary care. Prim Care Companion CNS Disord 17.
Williams, A.D., Andrews, G., 2013. The effectiveness of Internet Cognitive Behavioural
Therapy (iCBT) for depression in primary care: a quality assurance study. PLoS ONE
[Electronic Resource] 8, e57447.
Wilson, K.G., Murrell, A.R., 2004. Values Work in Acceptance and Commitment Therapy.
Mindfulness and acceptance: Expanding the cognitive–behavioral tradition. Guilford, New
York, pp. 120–151.
Zetterberg, M., Carlbring, P., Andersson, G., Berg, M., Shafran, R., Rozental, A., 2019.
Internet-based cognitive behavioral therapy of perfectionism: comparing regular
therapist support and support upon request. Internet Interv. 17, 100237.
A.-L. Flygare, et al. Internet Interventions 19 (2020) 100303
9
