Content uploaded by Matthew R Bennion
Author content
All content in this area was uploaded by Matthew R Bennion on Oct 28, 2020
Content may be subject to copyright.
Review
Acceptability and Effectiveness of NHS-Recommended
e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis
Melanie Simmonds-Buckley1*, BSc, PhD; Matthew Russell Bennion1,2*, BEng, MSc, PhD; Stephen Kellett1,3, BSc,
MSc, ClinPsy; Abigail Millings1,4, BSc, PhD; Gillian E Hardy1, BA, MSc, PhD; Roger K Moore2, BA, MSc, PhD
1Department of Psychology, University of Sheffield, Sheffield, United Kingdom
2Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom
3Sheffield Health and Social Care NHS Foundation Trust, Sheffield, United Kingdom
4Centre for Behavioural Science and Applied Psychology, Sheffield Hallam University, Sheffield, United Kingdom
*these authors contributed equally
Corresponding Author:
Melanie Simmonds-Buckley, BSc, PhD
Department of Psychology
University of Sheffield
Cathedral Court, 1 Vicar Lane
S1 2LT
Sheffield, S1 2LT
United Kingdom
Phone: 44 01142226630
Email: m.simmonds-buckley@sheffield.ac.uk
Abstract
Background: There is a disconnect between the ability to swiftly develop e-therapies for the treatment of depression, anxiety,
and stress, and the scrupulous evaluation of their clinical utility. This creates a risk that the e-therapies routinely provided within
publicly funded psychological health care have evaded appropriate rigorous evaluation in their development.
Objective: This study aims to conduct a meta-analytic review of the gold standard evidence of the acceptability and clinical
effectiveness of e-therapies recommended for use in the National Health Service (NHS) in the United Kingdom.
Methods: Systematic searches identified appropriate randomized controlled trials (RCTs). Depression, anxiety, and stress
outcomes at the end of treatment and follow-up were synthesized using a random-effects meta-analysis. The grading of
recommendations assessment, development, and evaluation approach was used to assess the quality of each meta-analytic
comparison. Moderators of treatment effect were examined using subgroup and meta-regression analysis. Dropout rates for
e-therapies (as a proxy for acceptability) were compared against controls.
Results: A total of 24 studies evaluating 7 of 48 NHS-recommended e-therapies were qualitatively and quantitatively synthesized.
Depression, anxiety, and stress outcomes for e-therapies were superior to controls (depression: standardized mean difference
[SMD] 0.38, 95% CI 0.24 to 0.52, N=7075; anxiety and stress: SMD 0.43, 95% CI 0.24 to 0.63, n=4863), and these small effects
were maintained at follow-up. Average dropout rates for e-therapies (31%, SD 17.35) were significantly higher than those of
controls (17%, SD 13.31). Limited moderators of the treatment effect were found.
Conclusions: Many NHS-recommended e-therapies have not been through an RCT-style evaluation. The e-therapies that have
been appropriately evaluated generate small but significant, durable, beneficial treatment effects.
Trial Registration: International Prospective Register of Systematic Reviews (PROSPERO) registration CRD42019130184;
https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=130184
(J Med Internet Res 2020;22(10):e17049) doi: 10.2196/17049
KEYWORDS
e-therapy; anxiety; depression; treatment effectiveness; National Health Service; meta-analysis; mobile phone
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 1http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Introduction
The potential contribution of digital technology in enabling
access to evidenced-based psychological care for mental health
problems is high on national and international research, policy,
commissioning, and service management agendas [1]. In modern
life, as digital tools (eg, mobile phones, tablets, laptops, and
wearable devices) have become ubiquitous, psychological
interventions delivered by such devices (ie, e-therapies) offer
greater convenience and enable constant access to treatment
compared with traditional face-to-face therapy with health
professionals [2]. The increasing demand for primary care
psychological services globally has provided the context within
which e-therapies have been integrated into the offer of a suite
of low-intensity (LI) psychological interventions [3], often
delivered within stepped-care systems [4,5]. Although
technological innovation in methods of treatment delivery
usefully expands availability, it also creates the risk of
commercial promotion and availability of ineffective or possibly
harmful psychological interventions [6]. Therefore,
commissioners, clinicians, and patients need access to reliable
and contemporary guidance regarding the empirical status and
clinical utility of e-therapies.
The potential organizational, therapeutic, and health economic
benefits of e-therapies initially triggered a global wave of
investment and interest [7]. In the United Kingdom, for example,
the National Health Service (NHS) Commissioning Board
launched the NHS Health Apps Library in March 2013 and
NHS Mental Health Apps Library in March 2015. However,
the libraries were removed in 2015 after questions were raised
concerning e-therapy data security governance [8] and clinical
effectiveness [9]. NHS England launched 2 new digital platforms
in April 2017, a new beta of the NHS Digital Apps Library and
a mobile health space, in an effort to close the gap between
e-therapy development and thorough evaluation. Before the
removal of the initial NHS App Libraries, a list of 48
NHS-recommended e-therapies was compiled for the National
Institute for Health and Care Excellence (NICE) assessment of
digitally enabled psychological therapies for use in Improving
Access to Psychological Therapies (IAPT) services [10]. A
recent quality assessment of the development process of
NHS-recommended e-therapies strongly advocated developers
to routinely adopt clinical trial methods to test acceptability and
efficacy of e-therapies before wider dissemination [11]. NICE
has also recently published an evidence standards framework
for e-therapies providing guidance concerning efficacy and
effectiveness standards [12].
This review aims to quantitatively synthesize the evidence base
of e-therapies recommended for use in the NHS for depression,
anxiety, and stress in adults to better inform the commissioning
and use of e-therapies in clinical services. It was relevant to
restrict this review to adults as the NHS-recommended
e-therapies are intended for adults. Previously, an individual
participant meta-analysis of the e-therapy clinical trial evidence
base for depression showed that e-therapy was significantly
more effective than controls [13], and there is clinical trial
evidence for the efficacy of e-therapy as a treatment for anxiety
[14]. This study had 3 aims. First, we sought to quantify the
effect of NHS-recommended e-therapies (ie, the 48 e-therapies
identified by Bennion et al [10]), as no previous specific
meta-analysis of the efficacy of NHS-recommended e-therapies
has been attempted. As randomized controlled trials (RCTs)
are viewed as the gold standard evaluation [15], we sought to
only use RCT studies to increase the quality of the
meta-analysis. Second because e-therapies are criticized for
generating high dropout rates [16], we sought to compare
dropout rates in contrast to controls to appraise acceptability.
Finally, we sought to investigate the impact of potential
moderating factors (eg, gender, age, severity, treatment
approach, treatment duration, setting, focus problem, and risk
of bias) on e-therapy outcomes via subgroup and
meta-regression analyses.
Methods
The review was registered on the International Prospective
Register of Systematic Reviews (PROSPERO;
CRD42019130184). The PRISMA (Preferred Reporting
Guidelines for Systematic Reviews and Meta-Analyses) are
used throughout [17].
Study Selection
A 3-stage search strategy was developed to identify RCTs
evaluating all of the e-therapies recommended by the NHS for
the treatment of depression, anxiety, and stress. First, each of
the 48 NHS-recommended e-therapies identified by Bennion
et al [10] was used to determine those e-therapies to be included
in the search strategy. The name of each e-therapy and its
platform type (website or app) were combined to develop a
series of search terms (eg, “Beating the Blues” AND “Website”)
[18]. Electronic searches were conducted using PsycINFO, Web
of Science, and PubMed databases to identify relevant e-therapy
outcome studies published up until April 2019 (date of final
search was April 11, 2019; see Multimedia Appendix 1 for an
example search strategy). Second, reference lists of identified
studies and previous e-therapy reviews were also searched.
Third, as many e-therapies are not developed under their
commercial name, a survey was disseminated to the 48 app
developers of the identified NHS-recommended e-therapies to
identify additional gray literature not captured by the terms used
in the database searches [11]. This process was to supplement
the identification of all studies associated with any one
e-therapy, even when the commercial name was not used in the
reporting. A total of 36 out of 48 (75%) app developers
responded to the survey, and the full process was reported by
Bennion et al [11]. Titles and abstracts were screened initially
(MB), with the full texts of identified studies then screened
against inclusion and exclusion eligibility criteria (MB). Queries
regarding study eligibility were resolved through discussion
among reviewers (MB, SK, and AM).
Eligibility Criteria
Studies were included if the web-based or smartphone app
intervention used was one of the 48 NHS-recommended
e-therapies [10] for depression, anxiety, and stress; therefore,
all studies of other types of e-therapies and for other clinical
conditions were excluded. Studies were eligible for inclusion
if, and only if, they used an RCT design to examine the efficacy
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 2http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
of e-therapy with an adult population (ie, aged >18 years). To
be included, the developer of the e-therapy had to be locatable
via a Google search when entering the app name as the search
term, and the app had to reference the targeted condition (ie,
depression, anxiety, or stress) in its marketing literature or be
based on a therapeutic tool known to benefit the targeted
condition. Posttreatment outcomes were required to have been
assessed using a validated measure of anxiety and/or depression
symptoms. Comparators included any control condition,
comprising a wait list or no treatment, placebo or
attention-control activity, or treatment as usual (TAU). Only
English language articles were included.
Outcomes
The 2 main outcomes of interest were participant-reported
outcomes of (1) depression and/or (2) anxiety and stress taken
at posttreatment and at follow-up (where available, to assess
the durability of e-therapy effectiveness). Where multiple
measures of one outcome were used (ie, 2 measures of
depression), the most frequently used measure across the
included studies was prioritized. Therefore, each study only
contributed one effect size per outcome. Dropout (as a proxy
for acceptability) was classified as the percentage of e-therapy
and comparator condition noncompleters, as determined by the
definition applied in the original study.
Data Extraction
A priority data extraction tool was designed for the purpose of
the review. MB extracted data from the original studies and
then reviewers (SK and AM) independently verified the findings.
Data were coded according to the following criteria: (1) study
information—sample size, trial design, context, comparator
type, study length, analytic approach (intention to treat [ITT]
or completers), and trial quality; (2) participant
characteristics—mean age, percentage of males, population
sample, presenting problem, and diagnostic information or
relevant inclusion criteria; (3) outcome
characteristics—outcome measure and, if applicable, length of
follow-up; and (4) intervention features—e-therapy program,
regularity of instructed use, duration, intervention component
details of the comparator condition, and self-help typology. The
self-help typology for each e-therapy was coded based on the
framework by Newman et al [19]: minimal contact therapy,
predominantly self-help, predominantly therapist-administered
treatment, or self-administered therapy. This was selected to
provide an assessment of the level and extent of therapist support
within the e-therapies. Outcome data on depression, anxiety,
and stress symptoms and dropout rates were extracted at
treatment completion and follow-up (ie, at 6 months or the
closest assessment point available).
Study and Evidence Quality
The Cochrane risk of bias tool [20] was used to assess the
methodological quality of the original studies using the
Cochrane Review Manager (RevMan) program [21]. All
included studies were assessed on 7 elements: (1) randomization,
(2) allocation concealment, (3) blinding of participants and
personnel, (4) blinding of outcome assessment, (5) data attrition,
(6) selective outcome reporting, and (7) other threats to validity.
Elements were rated as having low risk, unclear, or high risk
of bias. One rater assessed all the included studies, with all
studies double rated by 2 other raters (rater 1 assessed 63%
[15/24] and rater 2 assessed 37% [9/24]). Cohen kappa
coefficient (k) was used to assess the interrater agreement on
risk of bias overall scores between the primary rater and 2
second raters [22], and these were interpreted using the Landis
and Koch [23] categories: <0 as indicating no agreement, 0 to
0.20 as slight, 0.21 to 0.40 as fair, 0.41 to 0.60 as moderate,
0.61 to 0.80 as substantial, and 0.81 to 1 as almost perfect
agreement. There was substantial agreement between the
primary rater and rater 1 (k=.63) and moderate agreement
between the primary rater and rater 2 (k=.54). Any differences
in rating were discussed by the raters to reach a consensus on
the overall risk of bias rating for each included study. The
grading of recommendations assessment, development, and
evaluation (GRADE) approach was used to rate the quality of
the evidence included in each meta-analysis conducted [24].
The quality of evidence was assessed on 5 domains: (1) risk of
bias in the individual included studies, (2) publication bias, (3)
inconsistency, (4) imprecision, and (5) indirectness of treatment
estimate effects. The meta-analysis was graded by 2 reviewers
(SK and MS) and a consensus agreed (rated as high, moderate,
low, or very low quality).
Effect Sizes
Standardized mean differences (SMDs) were used to assess
differences in outcome between e-therapy and the comparator
conditions at posttreatment and follow-up. SMDs were
computed by calculating Cohen d(mean outcome score of the
comparator condition subtracted from the mean outcome score
of the e-therapy and dividing by the pooled standard deviation).
Where available, effect sizes were computed using ITT outcome
data. To account for potential biases in studies with small sample
sizes, SMDs were converted to Hedges gusing the Jadjustment
[25]. Effect sizes were calculated so that a beneficial effect of
e-therapy was represented by a positive SMD and vice versa.
Interpretations of effect size magnitude were classified as 0.20
to 0.49=small, 0.50 to 0.79 = medium, and >0.80=large [26].
When studies had multiple treatment arms delivering e-therapies
that could be considered comparable (ie, the same e-therapy
with different component combinations, such as reminders and
telephone support), the data were collapsed into a single group
using Cochrane guidelines [20]. When studies had multiple
treatment arms that could not be collapsed (ie, three-arm trial
comparing 2 different types of recommended e-therapy to a
control), the treatment arms were included independently. The
sample size of the shared comparator condition was split evenly
across independent treatment arm comparisons to avoid
participant data being included twice.
Data Synthesis
Meta-Essentials workbooks were used to synthesize e-therapy
treatment effects in a random-effects meta-analysis to account
for the extent of expected study heterogeneity [27]. Individual
study effect sizes were weighted using the inverse of the
variance to produce overall pooled treatment effect estimates
and 95% CIs. The threshold for statistical significance was set
at an αvalue of .05. The I2statistic was employed as an indicator
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 3http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
of the percentage of between-study heterogeneity, whereas the
Qstatistic provided a test of the statistical significance of the
presence of study variation. Thresholds of heterogeneity were
interpreted as <40% may not be relevant, 30% to 60%
representing moderate heterogeneity, 50% to 90% representing
substantial heterogeneity, and 75% to 100% representing
considerable heterogeneity [28]. As recommended by Cochrane,
the magnitude and direction of effect sizes were used to interpret
the implications of I2percentages. The overall pooled effect
sizes of e-therapy were translated into numbers needed to treat
(NNTs) [29]. NNT is an approximation of how many patients
would need treatment with e-therapy to generate an additional
outcome of benefit when compared with another intervention
(ie, the comparator condition). A Mann-Whitney U test was
used to assess for differences in dropout rates between e-therapy
and controls.
Moderator and Sensitivity Analyses
Preplanned random-effects moderator analyses were performed
using the Meta-Essentials workbooks to evaluate between-study
variation in treatment effects in posttreatment comparisons with
a minimum of 10 studies [20]. Moderators were selected based
on methodological, clinical, and intervention features that were
likely to vary between studies. Meta-regressions were applied
to 5 continuous variables: mean age, mean number of sessions
completed, percentage of males, baseline symptom severity
(standardized Z scores), and risk of bias (number of items
meeting criteria for low risk of bias: 0-7). Subgroup analyses
were applied to 6 categorical variables: 4 of them were specified
a priori (control type, e-therapy type, self-help typology, and
recruitment setting) and 2 were conducted post hoc (focus
problem and analysis method). Owing to multiple testing, the
αthreshold for significance of the meta-regression
beta-coefficients and the between-subgroup differences was
lowered to P<.01. A series of sensitivity analyses were
performed to assess the impact of outliers on the pooled effect
sizes (with extreme outliers removed) and to further explore
treatment effect durability (comparisons of follow-up effects
separately at short-term [1-2 months], medium-term [6 months],
and long-term [>8 months] follow-up).
Publication Bias
Several methods were employed to assess for the presence of
publication bias in the posttreatment comparisons that had a
sufficient number of studies (k>10). Visual inspection of the
asymmetry of a funnel plot (SE plotted against effect sizes)
gave an indication of the extent of potential publication bias,
whereas the accompanying Trim and Fill imputation [30]
accounted for any reporting bias to provide an adjusted treatment
estimate. Finally, additional statistical testing of asymmetrical
study distribution was undertaken using Egger regression [31].
Results
Study Selection
The electronic searches returned a total of 944 records. This
was combined with the 152 records collected by surveying app
developers and 7 records from a manual reference list and
review searches, giving a combined total of 1103 records (Figure
1). Duplicates were removed, leaving a total of 910 records to
be screened. After excluding records that did not meet the
inclusion criteria based on abstracts, 159 full-text articles were
retrieved and assessed. Overall, 26 trials were considered
eligible, and 2 were excluded because they contained duplicate
data from another trial. Thus, a total of 24 studies that tested
the efficacy of 7 of the 48 NHS-recommended e-therapies
(Beating the Blues, FearFighter, MoodGYM, IESO, Headspace,
Silver Cloud, and Work Guru) in an RCT design were included
in the meta-analysis. Details of the included studies can be found
in Multimedia Appendix 2 [32-55].
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 4http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Figure 1. PRISMA (Preferred Reporting Guidelines for Systematic Reviews and Meta-Analyses) flowchart of study selection. NHS: National Health
Service; RCT: randomized controlled trial.
The risk of bias ratings are presented in Table 1. Of the 24
included studies, quality ranged between 1 and 7 quality items
meeting low risk of bias criteria (maximum of 7). The overall
study quality was moderate to good, with 13 studies meeting
low risk of bias criteria on at least five items. A lack of or
unclear blinding of participants and personnel or outcome
assessment and incomplete outcome data were the most common
reasons for risk of bias. For the most poorly rated item across
studies, only 3 trials demonstrated suitable blinding of
participants and personnel.
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 5http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Table 1. Risk of bias assessment of the included studies.
Risk of bias itemsStudy
7g
6f
5e
4d
3c
2b
1a
+++
?h
−h
+
+h
Proudfoot et al (2003) [32]
+????++Grime (2004) [33]
+++?−++Proudfoot et al (2004) [34]
?+?+−++Marks et al (2004) [35]
++++−++Schneider et al (2005) [36]
+++?−??Mackinnon et al (2008) [37]
+++++++Kessler et al (2009) [38]
+???−??Ellis et al (2011) [39]
+?+??++Farrer et al (2011) [40]
+++?−++Høifødt et al (2013) [41]
+++?−++Lintvedt et al (2013) [42]
+++?−++Powell et al (2013) [43]
+++−−++Sethi (2013) [44]
+?−?+++Howells et al (2016) [45]
+++++++Phillips et al (2014) [46]
?+−−−++Twomey et al (2014) [47]
+++−−++Gilbody et al (2015) [48]
+++??++Richards et al (2015) [49]
+++??++Richards et al (2016) [50]
+++?−++Carolan et al (2017) [51]
+++?−?+Flett et al (2018) [52]
+++−−?+Forand et al (2018) [53]
+?+?−?+Bostock et al (2019) [54]
?++?−++Löbner et al (2019) [55]
aRandom sequence generation (selection bias).
bAllocation concealment (selection bias).
cBlinding of participants and personnel (performance bias).
dBlinding of outcome assessment (performance bias).
eIncomplete outcome data (attrition bias).
fSelective outcome reporting.
gOther potential threats to validity.
h+=low risk; −=high risk; ?=unclear risk.
Study Characteristics
Out of the 48 NHS e-therapies identified by Bennion et al [10],
a total of 7 (15%) were based on RCT evidence of efficacy,
which comprised 6 web-based e-therapies and 1
smartphone-based e-therapy (Table 2). MoodGYM was the
e-therapy with the greatest degree of evaluation (k=11 studies),
with 2 of the e-therapies having a single RCT evaluation (ie,
Ieso and WorkGuru). All 6 web-based e-therapies had both
clinical and academic personnel adding expertise during
technological development, but the smartphone-based e-therapy
had no clinical or academic personnel being involved in its
technological development phase [11]. A summary of e-therapy
version numbers used in each study and whether a
CONSORT-EHEALTH (Consolidated Standards of Reporting
Trials of Electronic and Mobile Health Applications and Online
Telehealth) checklist [56] was provided (for studies published
post-2011 after the checklist was developed) is reported in
Multimedia Appendix 3 [32-55]. Reporting of version numbers
was generally inconsistent, meaning establishing whether the
e-therapies had been updated between studies was difficult.
Beating the Blues had been updated between studies, with
version 1.0 used in the early studies (2003-2004) [32,34] and
version 2.5 used in the most recent study (2018) [53]. Updates
to MoodGYM were unable to be established because of
inconsistent reporting of version numbers, but there was an
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 6http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
indication that the studies between 2011 and 2018 used version
III [41,42,55]. It appeared that Headspace was updated from
version 1.0 or above in 2014 to a version equal to or above 2.0
in studies from 2019. Studies of FearFighter, SilverCloud, IESO,
and WorkGuru either did not refer to version numbers or were
only evaluated in 1 RCT, so updates could not be conclusively
determined.
All but one of the-therapies were based on the cognitive
behavioral theory (CBT) [11]. E-therapy treatments lasted
between 10 and 70 days (mean 44.52, SD 16.11), comprising
between 3 and 45 sessions (mean 8.37, SD 7.98) lasting 10 to
60 min each (mean 48.21, SD 15.26). The majority of
e-therapies were administered weekly (k=19), whereas 3 of the
trials required daily e-therapy usage (2 trials did not report the
instructed frequency of usage). Self-help typology was
characterized as self-administered therapy (k=7 studies),
predominantly self-help (k=11 studies), minimal contact therapy
(k=5 studies), and predominantly therapist-delivered treatment
(k=1 study).
The control conditions employed in the studies were waitlist or
no treatment (k=13), TAU (k=5), and placebo or
attention-control tasks (k=9; note: k=3 studies had multiple
control conditions). TAU comprised usual general practitioner
(GP) care, allowing access to any treatment prescribed or
referred to by a GP. Placebo or attention-control conditions
included depression information websites (eg, Bluepages; k=2),
online peer support forums (eg, MoodGarden; k=1), tracking
or structured weekly phone calls (k=2), neutral tasks or
note-taking organization apps (eg, Catch notes software or
Evernote; k=2), or online self-relaxation (without exposure, ie,
a sham treatment; eg, managing anxiety or de-STRESS; k=2).
In k=12 trials, clinical participants were recruited from primary
care (k=7), psychiatric outpatients (k=2), a university counseling
center (k=1), public sector employees (k=1), and a telephone
counseling service (k=1). In the remaining 12 trials, community
participants were recruited from university students (k=3),
occupational health attendees (k=3), the internet (k=2), electoral
role (k=1), youth center (k=1), charity users (k=1), and
treatment-seeking adults (k=1). Mean ages across the samples
ranged from 20 to 45 years (mean 35.71, SD 7.76).
E-therapies were delivered for symptoms of depression (k=10),
anxiety or panic and phobia (k=3), stress (k=2), or a combination
of anxiety and depression symptoms (k=6). Three of the trials
did not require participants to have any symptoms or indicators
of poor mental health. The Beck Depression Inventory (I or II)
was the most commonly used depression outcome measure
(k=7), followed by the Centre for Epidemiologic Studies
Depression Scale (CES-d; k=6). The most commonly employed
anxiety outcome measures were the Generalized Anxiety
Disorder-7 (k=4) and the Depression Anxiety Stress
Scales—anxiety subscale (k=4). Follow-up assessments were
conducted in 18 trials (k=2 had insufficient data to be included
in the follow-up analysis). The duration of follow-up ranged
between 1 and 20 months (mean 5 months). Dropout rates
ranged from 0% to 64%. The average e-therapy dropout rate
was 31% (SD 17.35), and the average dropout rate for controls
was 17% (SD 13.31). Therefore, significantly more participants
dropped out during e-therapies compared with controls
(U=181.000; Z=−3.026; P=.002).
Table 2. Types of e-therapies used in included studies.
Evidence of updates be-
tween studies
Psychological theory or
clinical approach used
Academic involve-
ment
Clinical involve-
ment
Delivery
platform
Number of
trialsa
E-therapy
Yes
CBTc
Y
Yb
Web-based5Beating the Blues
Could not be determinedCBTYYWeb-based2Fear Fighter
YesMindfulnessN
Nd
Phone-based3Headspace
N/Ae
CBTYYWeb-based1IESO
Could not be determinedCBTYYWeb-based11MoodGYM
N/ACBTYYWeb-based2SilverCloud Health
N/A
CBT, mindfulness, and PPf
YYWeb-based1WorkGuru
aA total of 2 e-therapies were evaluated in one trial; therefore, the total number of trials exceeded the overall number of included studies.
bY: yes.
cCBT: cognitive behavioral therapy.
dN: no.
eN/A: not applicable, as e-therapy content was not assessed in multiple studies.
fPP: positive psychology.
Meta-Analysis of E-Therapy Versus Controls
Meta-analytic comparisons were performed to aggregate the
effect of e-therapy vs controls on (1) depression and (2) anxiety
and stress symptoms at posttreatment and follow-up. GRADE
assessments are reported for each comparison, indicating the
quality of evidence. All comparisons were based on RCT
evidence so they started as high-quality evidence. Across the
meta-analyses, limited issues were found in terms of study
limitations or publication bias, but some limitations were found
for heterogeneity, treatment comparisons, and imprecision. As
a result, the level of evidence was downgraded for all
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 7http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
comparisons, with the majority demonstrating moderate quality.
Comparisons were downgraded one level specifically due to
significant and considerable I2statistic indicating marked
heterogeneity in the original studies, variability in primary
outcome measure, differing control groups, and varied effects
based on lower and upper bounds of confidence intervals. One
comparison was downgraded 2 levels to low-quality evidence
because of additional limitations created by the small number
of studies restricting subsequent moderator analyses and
variability in follow-up time.
Effect of E-Therapy on Depression Outcomes
Posttreatment and Follow-Up Comparisons
Overall, 26 treatment arm comparisons (extracted from 22
studies) totaling 7075 participants evaluated posttreatment
e-therapy depression outcomes in comparison with a control
condition (e-therapy, n=3545; control, n=3530). The pooled
SMD presented in Figure 2 signified a small, significant
treatment effect in favor of greater depression reductions
following e-therapy (SMD 0.38; 95% CI 0.24 to 0.52; Z=5.78;
P<.001; GRADE=moderate). The NNT was 4.72, indicating
that for every 5 patients who received e-therapy, there was one
additional beneficial depression outcome compared with if they
had received a control condition. Between-study variation was
significant, indicating substantial heterogeneity between studies
(I2=73%; 95% CI 60% to 82%; Q=92.30; P<.001). Furthermore,
16 follow-up treatment arm comparisons (extracted from 13
studies) provided follow-up data on depression outcomes for
e-therapies versus control conditions for 5709 participants
(e-therapy, n=2850; control, n=2859). There was a small
significant pooled SMD in favor of depression outcomes at
follow-up compared with controls (Figure 2; SMD 0.25; 95%
CI 0.08 to 0.41; Z=3.23; P=.001; NNT=7.12;
GRADE=moderate). The between-study variation was
significant, indicating moderate-to-substantial heterogeneity
(I2=69%; 95% CI 48% to 81%; Q=48.11; P<.001).
Figure 2. Forest plot of post-treatment and follow-up depression outcome effect sizes (ES) for e-therapy versus controls.
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 8http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Moderator and Sensitivity Analyses
The significant heterogeneity between studies at posttreatment
and follow-up was investigated using meta-regression (Table
3) and subgroup moderator analyses (Table 4). Meta-regression
analyses found that variations in e-therapy treatment effects
were not explained by gender, age, number of sessions, or study
quality at posttreatment or follow-up. Although initial depression
severity was not significantly associated with effect size at
posttreatment, higher levels of depression severity were
associated with larger beneficial effects of e-therapy at
follow-up. Subgroup analyses showed that variation in
posttreatment effect size was associated with the type of control
condition (although the effect fell short of significance after
accounting for multiple testing). A moderate effect was observed
in favor of e-therapy vs wait list controls, whereas the effects
for e-therapy compared with placebo conditions and TAU were
small. At follow-up, e-therapy effect sizes did not significantly
differ according to the control type with e-therapy, showing a
small significant beneficial effect compared with placebo and
TAU controls and a small nonsignificant effect compared with
wait list. Posttreatment and follow-up effects were not
significantly affected by the e-therapy type, self-help typology,
recruitment setting, focus problem, or analysis method.
Substantial significant heterogeneity was evident in
approximately half of the subgroups.
Table 3. Meta-regression analyses of effect e-therapy vs controls on depression and anxiety outcomes (posttreatment and follow-up).
R2(%)c
Pvalueb
SE95% CIB coefficientka
Time point and outcome, variable
Posttreatment
Depression
4.15.260.06−0.06 to 0.210.0726Initial severity
8.30.090.01−0.02 to 0.00-0.0126Percentage of males
0.95.580.01−0.02 to 0.010.0026Mean age (years)
10.23.080.010.00 to 0.050.0217Mean number of sessions completed
0.28.770.05−0.11 to 0.08-0.0126Risk of bias
Follow-upc
Depression
53.17<.0010.060.12 to 0.390.2516Initial severity
11.64.130.01−0.03 to 0.01-0.0116Percentage of males
3.88.380.01−0.01 to 0.040.0116Mean age (years)
0.44.780.03−0.06 to 0.080.0111Mean number of sessions completed
0.40.780.06−0.11 to 0.140.0216Risk of bias
Posttreatment
Anxietyd
8.84.170.09−0.07 to 0.310.1217Initial severity
5.85.240.01−0.03 to 0.01-0.0117Percentage of males
3.03.430.01−0.03 to 0.01-0.0117Mean age (years)
23.93.070.010.00 to 0.050.0211Mean number of sessions completed
0.18.850.06−0.14 to 0.12-0.0117Risk of bias
ak: number of comparisons.
bAlpha threshold Bonferroni adjusted to P<.01 for multiple testing.
cInsufficient number of comparisons and limited between-study heterogeneity to warrant moderator analyses of anxiety outcomes at follow-up.
dR2: percentage of variance explained by the moderator.
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 9http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Table 4. Subgroup analysis of effect e-therapy versus controls on depression outcomes (posttreatment and follow-up).
NNTg
R2(%)f
Pvalue (between sub-
groups)e
I2(%)d
95% CI
SMDb(Hedges
g)c
ka
Time point and variable, Subgroup
Posttreatment
Control type
3.368.00.02
79h
0.34 to
0.75
0.54h
12Wait list
5.58—
—j
79h
0.06 to
0.58
0.32h
7
TAUi
8.89——20.06 to
0.34
0.20h
7Placebo
E-therapy type
6.153.94.30
57h
0.15 to
0.43
0.29h
14MoodGYM
3.30——
89h
0.00 to
1.10
0.55h
5Beating the Blues
4.97——00.22 to
0.49
0.36h
3Headspace
3.61——20.32 to
0.68
0.50h
4Other
Self-help typology
5.955.87.08
65h
0.15 to
0.45
0.30h
8Self-administered
4.60——
76h
0.16 to
0.62
0.39h
14Predominantly self-help
3.42——00.39 to
0.67
0.53h
3Minimal contact
2.95————0.61
1k
Predominantly therapist delivered
Setting
4.600.01.91
68h
0.22 to
0.57
0.39h
12Clinical
4.72——
76h
0.18 to
0.58
0.38h
14Community
Focus problem
4.600.79.74
84h
0.13 to
0.64
0.39h
12Depression
4.72——00.25 to
0.52
0.38h
3Anxiety or stress
3.84——00.29 to
0.65
0.47h
7Both
Analysis method
4.600.49.50
76h
0.24 to
0.54
0.39h
9
ITTl
5.42——00.21 to
0.44
0.33h
3Completers
Follow-up
Control type
6.151.19.75
71h
−0.15 to
0.73
0.294Wait list
6.15——
79h
0.03 to
0.54
0.29h
7TAU
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 10http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
NNTg
R2(%)f
Pvalue (between sub-
groups)e
I2(%)d
95% CI
SMDb(Hedges
g)c
ka
Time point and variable, Subgroup
9.87——00.00 to
0.36
0.18h
5Placebo
E-therapy type
8.470.96.79
73h
−0.01 to
0.43
0.219MoodGYM
5.76——
73h
−0.03 to
0.64
0.314Beating the Blues
5.58——510.05 to
0.59
0.32h
3Other
Self-help typology
11.101.29.46
80h
−0.10 to
0.41
0.164Self-administered
6.15——
65h
0.07 to
0.51
0.29h
10Predominantly self-help
44.32————0.04
1k
Minimal contact
3.25————0.56
1k
Predominantly therapist delivered
Setting
5.424.68.13
77h
0.09 to
0.57
0.33h
10Clinical
12.68——00.07 to
0.21
0.14h
6Community
Focus problem
8.087.42.07
77h
−0.01 to
0.46
0.2210Depression
11.83————0.15
1k
Anxiety or stress
3.69——00.32 to
0.66
0.49h
3Both
Analysis method
6.60——
71h
0.09 to
0.45
0.27h
3ITT
10.45————0.17
1k
Completers
ak: number of comparisons.
bSMD: standardized mean difference.
cPositive effect size indicates in favor of e-therapy.
dSignificance of associated Qstatistic.
eAlpha threshold Bonferroni adjusted to P<.01 for multiple testing.
fR2: percentage of variance explained by moderator.
gNNT: number needed to treat.
hSignificant at P<.05.
iTAU: treatment as usual.
jOne between-groups Pvalue and R2value are provided for each subgroup comparison, reported on the row of the first subgroup category.
kWhere there is only one comparison within a subgroup, 95% confidence intervals and I2values are not reported.
lITT: intention to treat.
Sensitivity analyses explored the impact of the extreme outliers
and length of follow-up on the pooled depression effect sizes.
Although the removal of outlier effects resulted in a slight
reduction in the effect of e-therapy on depression from 0.38 to
0.34 at posttreatment and from 0.25 to 0.22 at follow-up,
outcomes still indicated small, significant benefits of e-therapy
compared with controls. E-therapy demonstrated a small,
beneficial effect compared with controls at short-term and
medium-term follow-up, which diminished at long-term
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 11http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
follow-up. The full sensitivity analysis results are reported in
Multimedia Appendix 4.
Assessment of Publication Bias
Visual inspection of the posttreatment funnel plot (Figure 3)
suggested that there was some asymmetry in the distribution of
studies, indicating that the smaller included studies were more
likely to report larger effects for e-therapy interventions. Trim
and fill imputed missing data to represent 4 smaller studies with
effects more in favor of controls, producing a slightly reduced
adjusted effect size in favor of e-therapy (SMD 0.31; 95% CI
0.15 to 0.46). Statistical testing of publication bias using Egger’s
regression did not detect significant asymmetry in the study
distribution for posttreatment outcomes (B=−0.15; t25=1.49;
P=.15). Assessment of study distribution for follow-up
depression outcomes also did not detect a significant influence
of publication bias (B=0.31; t15=1.34; P=.20). Taken together,
the multiple assessments of publication bias suggest a
minimal-to-small influence of bias on the overall e-therapy
treatment effect for depression outcomes.
Figure 3. Funnel plot for distribution of studies reporting e-therapy versus controls post-treatment depression outcomes.
Effect of E-Therapy on Anxiety and Stress Outcomes
Posttreatment and Follow-Up Comparisons
Overall, 17 treatment arm comparisons (extracted from 16
studies) totaling 4863 participants evaluated posttreatment
e-therapy anxiety and stress outcomes alongside a control
condition (e-therapy, n=2443; control, n=2420). The pooled
SMD presented in Figure 4 signified a small-to-moderate,
significant treatment effect in favor of greater anxiety reductions
following e-therapy (SMD=0.43; 95% CI 0.24 to 0.63; Z=4.63;
P<.001; GRADE=moderate). The NNT was 4.18, indicating
that for approximately every 4 patients who received e-therapy,
there was one additional beneficial anxiety and stress outcome
compared with if they had received a control condition. The
between-study variation was significant, indicating substantial
heterogeneity (I2=73% [95% CI 56% to 83%]; Q=59.13;
P<.001). Furthermore, 10 studies provided follow-up data on
anxiety and stress outcomes for e-therapies vs control conditions
for 3983 participants (e-therapy, n=2000; control, n=1983). At
follow-up, there was a small, significant pooled SMD in favor
of e-therapy compared with controls (Figure 4; SMD=0.23;
95% CI 0.17 to 0.29; Z=8.30; P<.001; NNT=7.74;
GRADE=low). The between-study variation was minimal and
not significant (I2=0% [95% CI 0% to 46%]; Q=6.31; P=.71).
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 12http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Figure 4. Forest plot of post-treatment and follow-up stress/anxiety outcome effect sizes (ES) for e-therapy versus controls.
Moderator and Sensitivity Analyses
The significant heterogeneity between studies at posttreatment
was investigated with meta-regression (Table 3) and subgroup
moderator analyses. Minimal heterogeneity and an insufficient
number of studies (k<10) negated the need for moderator
analysis of follow-up effects. Meta-regression analyses found
variations in e-therapy posttreatment anxiety and stress effects
were not explained by initial severity, gender, age, number of
sessions, or study quality. Subgroup analyses showed that
posttreatment effect sizes for anxiety and stress symptoms did
not significantly differ for different control conditions. However,
e-therapy vs wait list produced a moderate, significant effect
compared with the small effects observed for TAU and placebo
controls (placebo effect not significant). Posttreatment effects
were not significantly affected by the e-therapy type, recruitment
setting, focus problem, or analysis method. Self-help typology
indicated larger effects were observed for therapies with greater
therapist involvement (P=.02); however, the effect did not
remain significant when applying a Bonferroni correction.
Substantial significant heterogeneity was evident in about a
quarter of the subgroups.
Sensitivity analyses explored the impact of extreme outliers and
length of follow-up on the pooled anxiety and stress effect sizes.
Although the removal of outlier effects resulted in a slight
reduction in the e-therapy treatment effect on anxiety from 0.43
to 0.37 at posttreatment and from 0.23 to 0.22 at follow-up, the
outcomes still indicated small, significant benefits of e-therapy
compared with controls. E-therapy demonstrated a small,
beneficial effect compared with controls at both short-term and
medium-term follow-up (insufficient studies of long-term
follow-up were available). The full sensitivity analysis results
are reported in Multimedia Appendix 4.
The significant heterogeneity between studies at posttreatment
was investigated with subgroup moderator analyses (Table 5)
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 13http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Table 5. Subgroup analysis of effect e-therapy versus controls on anxiety and stress outcomes (posttreatment).
NNTh
R2(%)g
Pvalue (between sub-
groups)f
I2(%)e
95% CI
SMDc(Hedges
g)d
kb
Time pointaand variable, Subgroup
Posttreatment
Control type
3.042.99.41
84i
0.24 to 0.86
0.55i
9Wait list
4.49—
—k
00.35 to 0.45
0.40i
3
TAUj
6.86——28−0.02 to
0.55
0.265Placebo
E-therapy type
4.090.50.86
80i
0.01 to 0.86
0.44i
7MoodGYM
4.49——00.35 to 0.45
0.40i
3Beating the Blues
3.92——
61i
0.24 to 0.68
0.46i
7Other
Self-help typology
7.7413.38.0280.09 to 0.36
0.23i
4Self-administered
3.84——
74i
0.11 to 0.83
0.47i
8Predominantly self-help
3.04——450.36 to 0.83
0.60i
5Minimal contact
——————
0l
Predominantly therapist deliv-
ered
Setting
4.090.00.9900.33 to 0.54
0.44i
8Clinical
4.09——
84i
0.10 to 0.78
0.44i
9Community
Focus problem
3.690.82.85
88i
0.05 to 0.93
0.49i
3Depression
4.09——00.27 to 0.62
0.44i
5Anxiety or stress
3.14——
70i
0.14 to 1.02
0.58i
7Anxiety or depression
Analysis method
3.845.76.06
75i
0.27 to 0.68
0.47i
7
ITTm
9.87——0−0.05 to
0.42
0.182Completers
aInsufficient number of comparisons and limited between-study heterogeneity to warrant moderator analyses of anxiety outcomes at follow-up.
bk: number of comparisons.
cSMD: standardized mean difference.
dPositive effect size indicates in favor of e-therapy.
eSignificance of associated Qstatistic.
fAlpha threshold Bonferroni adjusted to P<.01 for multiple testing.
gR2: percentage of variance explained by moderator.
hNNT: number needed to treat.
iSignificant at P<.05.
jTAU: treatment as usual.
kOne between-groups Pvalue and R2value are provided for each subgroup comparison, reported on the row of the first subgroup category.
lWhere there are no comparisons within a subgroup, SMD, 95% confidence intervals and I2values are not reported.
mITT: intention to treat.
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 14http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Assessment of Publication Bias
Visual inspection of the funnel plot in Figure 5 suggested that
there was some asymmetry in the distribution of studies
reporting posttreatment anxiety and stress outcomes. However,
the trim and fill imputation did not impute any missing data in
relation to smaller studies in favor of controls or minimal
differences between groups producing an adjusted effect size
identical to the initial pooled SMD. The Egger regression failed
to detect sufficient asymmetry in the study distribution of
posttreatment anxiety and stress outcomes (B=–0.35; t16=1.82;
P=.09). Taken together, the multiple assessments of publication
bias imply a minimal-to-small influence of reporting bias on
the overall e-therapy treatment effect for anxiety and stress
outcomes. There were insufficient studies (k<10) to enable
accurate assessment of publication bias on comparisons of
follow-up anxiety and stress outcomes.
Figure 5. Funnel plot for distribution of studies reporting e-therapy versus controls post-treatment anxiety/stress outcomes.
Discussion
Principal Findings
This study has been the first attempt to assess the breadth and
quality of the evidence base for NHS-recommended e-therapies
and to quantify the efficacy of this health technology through
a meta-analysis of the clinical trial evidence base. Only 15%
(7/48) of the NHS-recommended e-therapies had eligible RCT
studies underpinning their clinical evaluation. Of the 7
e-therapies with RCT evidence, 2 contributed a single RCT
study to the meta-analysis, and there was poor and variable
reporting of version numbers across studies. These findings are
at odds with the philosophy of evidenced-based practice,
whereby clinical guidelines are underpinned by gold standard
evidence of efficacy. Overall, however, the available good
quality evidence shows that the e-therapies tested do benefit
adult participants in better managing anxiety, stress and
depression compared with controls, and this appears to be a
durable effect in the short to medium term. The magnitude of
the e-therapy treatment effects found here mirrors the effect
sizes seen in the overall LI intervention evidence base
(g=0.2-0.5) [5]. The NNT analysis suggests that for every 5
patients treated with an e-therapy, one has a good outcome. The
acceptability and efficacy of the e-therapies without RCT
evidence (ie, 85%, 41/48) of those actually recommended for
use in the NHS) remains open to question. It would be premature
to clinically champion any single e-therapy as being the most
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 15http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
effective at this point in time. MoodGYM has been exposed to
most evaluation and scrutiny, but it was unclear whether
differing versions were being tested.
The acceptability of e-therapies can be called into question
because of the higher dropout rates compared with controls
reported here. Criticisms of LI psychological interventions, and
e-therapies in particular, have been previously made concerning
their high dropout rates being an index for poor patient
acceptability, because of the low therapist contact and time
approach [13,16,57,58]. Dropout rates may also have been
influenced by multiple (unmeasured) factors such as the poor
face validity of the CBT theoretical approach [59], low readiness
to change, poor attitudes to the delivery of eHealth [60], and
the usability or characteristics of the web or app design itself
[61,62]. Ongoing issues with poor acceptability will remain an
obstacle in the commissioning and delivery of e-therapies as
frontline LI psychological interventions. Clearly, the clinical
utility of any e-therapies needs to be considered in a matrix of
cost, safety, acceptability, feasibility, and efficacy evidence
[63].
Comparison of study characteristics highlighted noteworthy
commonalities and differences across and between e-therapies.
First, 5 of the 7 e-therapies evaluated were based on CBT (one
other was based on CBT alongside other approaches). This
mirrors that LI interventions as a whole tend to be based and
focused on variants of CBT [64]. Recent innovations in
e-therapies have included acceptance and commitment therapy
[65], interpersonal psychotherapy [66], mindfulness [67], and
psychodynamic psychotherapy [68]. Second, 6 of the 7
e-therapies were web based, so the clinical utility of
smartphone-based app delivery of NHS-recommended
e-therapies has not been appropriately empirically evaluated.
Variations in e-therapy treatment effects were explored with
moderator analyses, as a previous individual participant
meta-analysis of e-therapies for depression found few significant
moderators [13]. Significantly larger e-therapy effects were
apparent when compared with wait list controls (for
posttreatment depression outcomes), for patients with greater
baseline severity (for follow-up depression outcomes), and when
there was a greater amount of therapist input (for end of
treatment anxiety and stress outcomes). However, the effects
of control type and amount of therapist input did not remain
significant after accounting for multiple testing, so caution
should be taken with any conclusions. Larger wait list
comparison effects are commonly observed in psychotherapy
trials and when taken in isolation can lead to overestimated
treatment effects [69]. E-therapy effects shrunk as the activeness
of comparators increased. In this review, baseline severity was
only a significant moderator at follow-up. Greater e-therapy
benefits for higher baseline depression severity have previously
been shown to predict better outcomes for internet-based CBT
[70]. The trend for e-therapies with a greater amount of therapist
input generating better outcomes has been widely reported
[71-73]. It is worth noting that e-therapy typologies in this
meta-analysis emphasized some therapist contact, but that
contact time was still relatively brief because of the LI approach.
Furthermore, 75% (18/24 studies of 4 different apps) had less
than 30 min of real-time person-to-person support. The efficacy
of LI interventions appears to be better enabled when supported
by even brief interpersonal contact [72,73].
Limitations
This review has several limitations, which also highlight how
the e-therapy evidence base could be further developed. First,
although the included studies were restricted to high-quality
RCT evidence, the GRADE approach highlighted issues with
inconsistency across results, treatment comparisons, and some
imprecision resulting in meta-analytic comparisons of
moderate-to-low quality. Second, there are limitations
concerning the generalizability of the findings. This review was
limited to the treatment of depression, anxiety, and stress with
e-therapies and so cannot comment on applicability to other
clinical presentations. Services in the United Kingdom use the
NICE guidelines to organize the delivery of treatments for
anxiety and depression via stepped-care principles. Therefore,
the generalizability of results from this meta-analysis is less
applicable for different approaches to mental health delivery,
for example, via stratified care [74]. The inclusion of only those
e-therapies recommended by the NHS excluded those e-therapies
very similar in technical format and content.
Third, there were some methodological weaknesses that may
have introduced bias, and the conclusions should be treated with
caution. The lack of formal screening and selection of articles
by a second reviewer is a major limitation that may have led to
bias in terms of which studies were selected for inclusion and
therefore influenced the results. Similarly, the quality ratings
of the studies were made by raters that were not independent
from the meta-analysis, and levels of agreement were not
optimal [75]. In addition, restrictions in the search strategy may
have missed eligible studies or excluded studies evaluating an
NHS e-therapy for other clinical presentations or outcomes [76].
Given that eHealth is a rapidly expanding area that makes
reviews outdated relatively quickly, the duration since the final
searches were conducted (April 2019) means there will
undoubtedly be additional relevant e-therapy trials now
available. Since the final searches, trials of 3 NHS e-therapies
(all with existing trial evidence) have been published; an RCT
of SilverCloud used in IAPT [77], evaluations of MoodGYM
[78], and Headspace in student samples [79,80].
Finally, synthesis and analysis were restricted by the data from
the available studies. The number of trials conducted was small,
and thus restricted the power and range of possible moderator
analyses. The original studies had the common methodological
flaws of limited diagnostic assessments of participants,
inconsistent reporting of e-therapy version numbers, overuse
of self-reported measures rather than independent assessment,
lack of reporting of adverse event rates [63], lack of measures
of e-therapy adherence, and lack of true long-term follow-up.
The frequent use of passive controls risked inflating treatment
effect sizes in meta-analyses [81], and there were insufficient
active comparators to establish efficacy of e-therapies vs other
therapies. There was no standard definition of dropout or
treatment completion across the studies, and therefore, we were
forced to adopt the definition used by each study. It is
acknowledged that dropout is a limited proxy for acceptability
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 16http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
[82] and that wider indices of acceptability also include
understanding barriers to e-therapy engagement.
Research and Service Implications
Finding studies relating to a specific e-therapy by searching for
its name in academic databases proved difficult. This was
because before commercialization, many e-therapy platforms
were known by their initial project name and not their eventual
product name. A solution to this problem would be to ensure
that e-therapy developers and researchers register their software
on a public database with a unique identifier to be referenced
in any subsequent publications. Trials of e-therapies should also
be reported according to the CONSORT-EHEALTH checklist
[56], and the e-therapy version should be indicated using
semantic versioning to clarify whether the e-therapy program
being evaluated has been updated (ie, reporting the major, minor,
and patch version [eg, version 2.1.1]).
Several e-therapies included in this review were developed to
be available without clinical support or guidance (eg,
MoodGYM and Headspace). Given that e-therapies outperform
controls (with moderate effects compared with wait list),
e-therapies may offer particular promise as a waitlist
intervention. Although unguided e-therapy may be beneficial
to patients waiting for face-to-face psychological interventions,
the trend observed in this review and findings from previous
studies imply that some clinician involvement is important for
ensuring good outcomes if an e-therapy is the sole intervention
[72,73]. The manner in which e-therapies can be effectively
blended with face-to-face psychological therapies is currently
poorly understood and demands more research. Studies also
need to be conducted on the utility of e-therapies as wait list
interventions.
Given the recent availability of differing theoretical approaches,
patient choice for e-therapy can now be offered and researched.
Treatment completion rates need to be consistently reported,
and trials adopt the ITT approach to reduce biasing treatment
effects. Consistent reporting of safety issues (eg, via untoward
incident rates) is needed for e-therapies. Health economic
evaluations that are embedded in clinical trials need to be
increased. A dropout meta-analysis (with independent study
quality ratings of all studies using the latest version of the
Cochrane risk of bias tool) of this evidence base is now also
indicated to better index e-therapy acceptability issues [83].
Little is known about why patients’drop out of e-therapies, and
qualitative investigations would be useful here. Treatment
adherence (ie, how much time is spent and how many modules
of eHealth are completed by participants) needs to be more
consistently reported. The role of moderating factors of
treatment outcome in e-therapies needs to be better researched,
particularly the role of variables such as blended vs pure
e-therapy approaches, time spent on the app, and theoretical
approach. E-therapies potentially still play an important role in
clinical services, regardless of the organizational system used
to coordinate delivery of care [84], particularly when the
approach has been well evaluated.
Conclusions
In this meta-analysis of gold standardclinical trials, e-therapies
have been found to be efficacious as LI psychological
interventions that produce small beneficial effects for adults
with depression, anxiety, and stress compared with controls.
However, only a relatively small proportion of
NHS-recommended e-therapies had been subjected to such gold
standard evaluation. Although these conclusions should be
considered in light of the methodological limitations, the
targeted nature of this review to NHS-recommended e-therapies
still has relevance to the global field of e-therapies. This is
particularly through highlighting the need to consistently
integrate high quality and controlled evaluation into the
technological development of e-therapies. This is to ensure
eventual safe and evidence-based e-therapy practice in routine
clinical services. Technological development and scrupulous
evaluation of e-therapies need to be conducted in parallel and
considered in equipoise.
Acknowledgments
This study was supported by a PhD studentship awarded by the University of Sheffield to MB and by the Economic and Social
Research Council grant number ES/L001365/1.
Conflicts of Interest
AM was an employee of Ultrasis PLC (no longer trading), the original distributor of Beating the Blues, from September 2010 to
December 2012.
Multimedia Appendix 1
Example search strategy.
[PDF File (Adobe PDF File), 83 KB-Multimedia Appendix 1]
Multimedia Appendix 2
Characteristics of included studies.
[PDF File (Adobe PDF File), 154 KB-Multimedia Appendix 2]
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 17http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
Multimedia Appendix 3
Summary of e-therapy version numbers.
[PDF File (Adobe PDF File), 154 KB-Multimedia Appendix 3]
Multimedia Appendix 4
Sensitivity analyses.
[PDF File (Adobe PDF File), 33 KB-Multimedia Appendix 4]
References
1. Torous J, Andersson G, Bertagnoli A, Christensen H, Cuijpers P, Firth J, et al. Towards a consensus around standards for
smartphone apps and digital mental health. World Psychiatry 2019 Feb;18(1):97-98 [FREE Full text] [doi:
10.1002/wps.20592] [Medline: 30600619]
2. Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported Internet cognitive behavioural therapy for
anxiety disorders in adults. Cochrane Database Syst Rev 2015 Mar 5(3):CD011565. [doi: 10.1002/14651858.CD011565]
[Medline: 25742186]
3. Orman J, O'Dea B. e-Therapy in primary care mental health. Aust J Gen Pract 2018 Apr;47(4):168-172 [FREE Full text]
[doi: 10.31128/AJGP-11-17-4410] [Medline: 29621855]
4. Bower P, Gilbody S. Stepped care in psychological therapies: access, effectiveness and efficiency. Narrative literature
review. Br J Psychiatry 2005 Jan;186:11-17. [doi: 10.1192/bjp.186.1.11] [Medline: 15630118]
5. Delgadillo J. Guided self-help in a brave new world. Br J Psychiatry 2018 Feb;212(2):65-66. [doi: 10.1192/bjp.2017.17]
[Medline: 29436327]
6. Hill C, Martin JL, Thomson S, Scott-Ram N, Penfold H, Creswell C. Navigating the challenges of digital health innovation:
considerations and solutions in developing online and smartphone-application-based interventions for mental health disorders.
Br J Psychiatry 2017 Aug;211(2):65-69. [doi: 10.1192/bjp.bp.115.180372] [Medline: 28522435]
7. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression
disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78
[FREE Full text] [doi: 10.1016/j.janxdis.2018.01.001] [Medline: 29422409]
8. Huckvale K, Prieto JT, Tilney M, Benghozi P, Car J. Unaddressed privacy risks in accredited health and wellness apps: a
cross-sectional systematic assessment. BMC Med 2015 Sep 7;13:214 [FREE Full text] [doi: 10.1186/s12916-015-0444-y]
[Medline: 26404673]
9. Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evid Based Ment Health 2015 Nov;18(4):97-99.
[doi: 10.1136/eb-2015-102203] [Medline: 26459466]
10. Bennion MR, Hardy G, Moore RK, Millings A. E-therapies in England for stress, anxiety or depression: what is being used
in the NHS? a survey of mental health services. BMJ Open 2017 Jan 23;7(1):e014844. [doi: 10.1136/bmjopen-2016-014844]
[Medline: 28115336]
11. Bennion M, Hardy G, Moore R, Kellett S, Millings A. e-Therapies in England for stress, anxiety or depression: how are
apps developed? a survey of NHS e-therapy developers. BMJ Health Care Inform 2019 Jun;26(1):e100027. [doi:
10.1136/bmjhci-2019-100027] [Medline: 31171556]
12. Cowie MR. National institute for health and care excellence. Eur Heart J 2015 Jan 21;36(4):195. [Medline: 25756143]
13. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive
behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. J Am Med
Assoc Psychiatry 2017 Apr 1;74(4):351-359. [doi: 10.1001/jamapsychiatry.2017.0044] [Medline: 28241179]
14. Bell CJ, Colhoun HC, Carter FA, Frampton CM. Effectiveness of computerised cognitive behaviour therapy for anxiety
disorders in secondary care. Aust N Z J Psychiatry 2012 Jul;46(7):630-640. [doi: 10.1177/0004867412437345] [Medline:
22327097]
15. Bothwell LE, Greene JA, Podolsky SH, Jones DS. Assessing the gold standard: lessons from the history of RCTs. N Engl
J Med 2016 Jun 2;374(22):2175-2181. [doi: 10.1056/NEJMms1604593] [Medline: 27248626]
16. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the
quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [doi: 10.1017/S0033291708004224] [Medline:
18812006]
17. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and
meta-analyses: the PRISMA statement. Ann Intern Med 2009 Aug 18;151(4):264-9, W64. [doi:
10.7326/0003-4819-151-4-200908180-00135] [Medline: 19622511]
18. Bennion M. An Investigation Into E-therapies for Older Adults. White Rose eTheses Online. 2019. URL: http://etheses.
whiterose.ac.uk/23347/ [accessed 2020-09-28]
19. Newman M, Szkodny L, Llera S, Przeworski A. A review of technology-assisted self-help and minimal contact therapies
for anxiety and depression: is human contact necessary for therapeutic efficacy? Clin Psychol Rev 2011 Feb;31(1):89-103.
[doi: 10.1016/j.cpr.2010.09.008] [Medline: 21130939]
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 18http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
20. Cochrane Handbook for Systematic Reviews of Interventions. New York, USA: Cochrane; 2011.
21. RevMan. Cochrane Training. 2014. URL: https://training.cochrane.org/online-learning/core-software-cochrane-reviews/
revman [accessed 2020-09-28]
22. Cohen J. A Coefficient of Agreement for Nominal Scales. Sage Journals: Your Gateway to World-class Research Journals.
1960. URL: https://doi.org/10.1177/001316446002000104 [accessed 2018-03-23]
23. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159-174.
[Medline: 843571]
24. Dijkers M. Introducing GRADE: a systematic approach to rating evidence in systematic reviews and to guideline development.
KT Update Aug 2013;1(5).
25. Hedges LV. Statistical Methods Meta-analysis. Orlando, FL: Academic Press; 1985.
26. Cohen J. A power primer. Psychol Bull 1992 Jul;112(1):155-159. [doi: 10.1037//0033-2909.112.1.155] [Medline: 19565683]
27. Suurmond R, van Rhee H, Hak T. Introduction, comparison, and validation of meta-essentials: a free and simple tool for
meta-analysis. Res Synth Methods 2017 Dec;8(4):537-553 [FREE Full text] [doi: 10.1002/jrsm.1260] [Medline: 28801932]
28. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Br Med J 2003 Sep
6;327(7414):557-560 [FREE Full text] [doi: 10.1136/bmj.327.7414.557] [Medline: 12958120]
29. Kraemer HC, Kupfer DJ. Size of treatment effects and their importance to clinical research and practice. Biol Psychiatry
2006 Jun 1;59(11):990-996. [doi: 10.1016/j.biopsych.2005.09.014] [Medline: 16368078]
30. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in
meta-analysis. Biometrics 2000 Jun;56(2):455-463. [doi: 10.1111/j.0006-341x.2000.00455.x] [Medline: 10877304]
31. Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997
Sep 13;315(7109):629-634 [FREE Full text] [doi: 10.1136/bmj.315.7109.629] [Medline: 9310563]
32. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural
program for anxiety and depression in general practice. Psychol Med 2003 Feb;33(2):217-227. [doi:
10.1017/s0033291702007225] [Medline: 12622301]
33. Grime PR. Computerized cognitive behavioural therapy at work: a randomized controlled trial in employees with recent
stress-related absenteeism. Occup Med (Lond) 2004 Aug;54(5):353-359. [doi: 10.1093/occmed/kqh077] [Medline: 15289593]
34. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, et al. Clinical efficacy of computerised
cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004
Jul;185:46-54. [doi: 10.1192/bjp.185.1.46] [Medline: 15231555]
35. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine
aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan;34(1):9-17.
[doi: 10.1017/s003329170300878x] [Medline: 14971623]
36. Schneider AJ, Mataix-Cols D, Marks IM, Bachofen M. Internet-guided self-help with or without exposure therapy for
phobic and panic disorders. Psychother Psychosom 2005;74(3):154-164. [doi: 10.1159/000084000] [Medline: 15832066]
37. Mackinnon A, Griffiths KM, Christensen H. Comparative randomised trial of online cognitive-behavioural therapy and an
information website for depression: 12-month outcomes. Br J Psychiatry 2008 Feb;192(2):130-134. [doi:
10.1192/bjp.bp.106.032078] [Medline: 18245031]
38. Kessler D, Lewis G, Kaur S, Wiles N, King M, Weich S, et al. Therapist-delivered Internet psychotherapy for depression
in primary care: a randomised controlled trial. Lancet 2009 Aug 22;374(9690):628-634. [doi:
10.1016/S0140-6736(09)61257-5] [Medline: 19700005]
39. Ellis L, Campbell A, Sethi S, O'Dea B. Comparative randomized trial of an online cognitive-behavioral therapy program
and an online support group for depression and anxiety. J Cybertherapy Rehabil 2011;4(4):467.
40. Farrer L, Christensen H, Griffiths KM, Mackinnon A. Internet-based CBT for depression with and without telephone
tracking in a national helpline: randomised controlled trial. PLoS One 2011;6(11):e28099 [FREE Full text] [doi:
10.1371/journal.pone.0028099] [Medline: 22140514]
41. Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, et al. The clinical effectiveness of web-based
cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled
trial. J Med Internet Res 2013 Aug 5;15(8):e153 [FREE Full text] [doi: 10.2196/jmir.2714] [Medline: 23916965]
42. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy
of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin
Psychol Psychother 2013;20(1):10-27. [doi: 10.1002/cpp.770] [Medline: 21887811]
43. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral
tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2012 Dec
31;15(1):e2 [FREE Full text] [doi: 10.2196/jmir.2240] [Medline: 23302475]
44. Sethi S. Treating Youth Depression and Anxiety: A Randomised Controlled Trial Examining the Efficacy of Computerised
versus Face-to-face Cognitive Behaviour Therapy. Australian Psychologist 2013 Feb 14;48(4):249-257. [doi:
10.1111/ap.12006]
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 19http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
45. Howells A, Ivtzan I, Eiroa-Orosa FJ. Putting the ‘app’in Happiness: A Randomised Controlled Trial of a Smartphone-Based
Mindfulness Intervention to Enhance Wellbeing. J Happiness Stud 2014 Oct 29;17(1):163-185. [doi:
10.1007/s10902-014-9589-1]
46. Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized
cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med
2014 Mar;44(4):741-752 [FREE Full text] [doi: 10.1017/S0033291713001323] [Medline: 23795621]
47. Twomey C, O'Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized
CBT programme, MoodGYM, for public mental health service users waiting for interventions. Br J Clin Psychol 2014
Nov;53(4):433-450. [doi: 10.1111/bjc.12055] [Medline: 24831119]
48. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, REEACT Team. Computerised cognitive
behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised
controlled trial. Br Med J 2015 Nov 11;351:h5627. [doi: 10.1136/bmj.h5627] [Medline: 26559241]
49. Richards D, Timulak L, O'Brien E, Hayes C, Vigano N, Sharry J, et al. A randomized controlled trial of an internet-delivered
treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav Res Ther
2015 Dec;75:20-31. [doi: 10.1016/j.brat.2015.10.005] [Medline: 26523885]
50. Richards D, Timulak L, Rashleigh C, McLoughlin O, Colla A, Joyce C, et al. Effectiveness of an internet-delivered
intervention for generalized anxiety disorder in routine care: a randomised controlled trial in a student population. Internet
Interv 2016 Nov;6:80-88 [FREE Full text] [doi: 10.1016/j.invent.2016.10.003] [Medline: 30135817]
51. Carolan S, Harris PR, Greenwood K, Cavanagh K. Increasing engagement with an occupational digital stress management
program through the use of an online facilitated discussion group: results of a pilot randomised controlled trial. Internet
Interv 2017 Dec;10:1-11 [FREE Full text] [doi: 10.1016/j.invent.2017.08.001] [Medline: 30135747]
52. Flett JAM, Hayne H, Riordan BC, Thompson LM, Conner TS. Mobile Mindfulness Meditation: a Randomised Controlled
Trial of the Effect of Two Popular Apps on Mental Health. Mindfulness 2018 Oct 31;10(5):863-876. [doi:
10.1007/s12671-018-1050-9]
53. Forand NR, Barnett JG, Strunk DR, Hindiyeh MU, Feinberg JE, Keefe JR. Efficacy of guided iCBT for depression and
mediation of change by cognitive skill acquisition. Behav Ther 2018 Mar;49(2):295-307 [FREE Full text] [doi:
10.1016/j.beth.2017.04.004] [Medline: 29530267]
54. Bostock S, Crosswell AD, Prather AA, Steptoe A. Mindfulness on-the-go: effects of a mindfulness meditation app on work
stress and well-being. J Occup Health Psychol 2019 Feb;24(1):127-138 [FREE Full text] [doi: 10.1037/ocp0000118]
[Medline: 29723001]
55. Löbner M, Pabst A, Stein J, Dorow M, Matschinger H, Luppa M, et al. Computerized cognitive behavior therapy for patients
with mild to moderately severe depression in primary care: a pragmatic cluster randomized controlled trial (@ktiv). J Affect
Disord 2018 Oct 1;238:317-326. [doi: 10.1016/j.jad.2018.06.008] [Medline: 29902736]
56. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of
web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [doi:
10.2196/jmir.1923] [Medline: 22209829]
57. Postel MG, de Haan HA, ter Huurne ED, Becker ES, de Jong CA. Effectiveness of a web-based intervention for problem
drinkers and reasons for dropout: randomized controlled trial. J Med Internet Res 2010 Dec 16;12(4):e68 [FREE Full text]
[doi: 10.2196/jmir.1642] [Medline: 21163776]
58. Melville KM, Casey LM, Kavanagh DJ. Dropout from internet-based treatment for psychological disorders. Br J Clin
Psychol 2010 Nov;49(Pt 4):455-471. [doi: 10.1348/014466509X472138] [Medline: 19799804]
59. Hind D, O'Cathain A, Cooper CL, Parry GD, Isaac CL, Rose A, et al. The acceptability of computerised cognitive behavioural
therapy for the treatment of depression in people with chronic physical disease: a qualitative study of people with multiple
sclerosis. Psychol Health 2010 Jul;25(6):699-712. [doi: 10.1080/08870440902842739] [Medline: 20204955]
60. Currie M, Philip LJ, Roberts A. Attitudes towards the use and acceptance of eHealth technologies: a case study of older
adults living with chronic pain and implications for rural healthcare. BMC Health Serv Res 2015 Apr 16;15:162 [FREE
Full text] [doi: 10.1186/s12913-015-0825-0] [Medline: 25888988]
61. Bennion MR, Hardy GE, Moore RK, Kellett S, Millings A. Usability, acceptability, and effectiveness of web-based
conversational agents to facilitate problem solving in older adults: controlled study. J Med Internet Res 2020 May
27;22(5):e16794 [FREE Full text] [doi: 10.2196/16794] [Medline: 32384055]
62. Wozney L, Huguet A, Bennett K, Radomski AD, Hartling L, Dyson M, et al. How do ehealth programs for adolescents
with depression work: a realist review of persuasive system design components in internet-based psychological therapies.
J Med Internet Res 2017 Aug 9;19(8):e266 [FREE Full text] [doi: 10.2196/jmir.7573] [Medline: 28793983]
63. Smart A. A multi-dimensional model of clinical utility. Int J Qual Health Care 2006 Oct;18(5):377-382. [doi:
10.1093/intqhc/mzl034] [Medline: 16951425]
64. The Improving Access to Psychological Therapies Manual. NHS England. 2020. URL: https://www.england.nhs.uk/
publication/the-improving-access-to-psychological-therapies-manual/ [accessed 2020-09-28]
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 20http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
65. Buhrman M, Skoglund A, Husell J, Bergström K, Gordh T, Hursti T, et al. Guided internet-delivered acceptance and
commitment therapy for chronic pain patients: a randomized controlled trial. Behav Res Ther 2013 Jun;51(6):307-315.
[doi: 10.1016/j.brat.2013.02.010] [Medline: 23548250]
66. Dagöö J, Asplund RP, Bsenko HA, Hjerling S, Holmberg A, Westh S, et al. Cognitive behavior therapy versus interpersonal
psychotherapy for social anxiety disorder delivered via smartphone and computer: a randomized controlled trial. J Anxiety
Disord 2014 May;28(4):410-417. [doi: 10.1016/j.janxdis.2014.02.003] [Medline: 24731441]
67. Boettcher J, Aström V, Påhlsson D, Schenström O, Andersson G, Carlbring P. Internet-based mindfulness treatment for
anxiety disorders: a randomized controlled trial. Behav Ther 2014 Mar;45(2):241-253 [FREE Full text] [doi:
10.1016/j.beth.2013.11.003] [Medline: 24491199]
68. Andersson G, Paxling B, Roch-Norlund P, Östman G, Norgren A, Almlöv J, et al. Internet-based psychodynamic versus
cognitive behavioral guided self-help for generalized anxiety disorder: a randomized controlled trial. Psychother Psychosom
2012;81(6):344-355 [FREE Full text] [doi: 10.1159/000339371] [Medline: 22964540]
69. Cuijpers P, van Straten A, Bohlmeijer E, Hollon S, Andersson G. The effects of psychotherapy for adult depression are
overestimated: a meta-analysis of study quality and effect size. Psychol Med 2010 Feb;40(2):211-223. [doi:
10.1017/S0033291709006114] [Medline: 19490745]
70. Spek V, Nyklícek I, Cuijpers P, Pop V. Predictors of outcome of group and internet-based cognitive behavior therapy. J
Affect Disord 2008 Jan;105(1-3):137-145. [doi: 10.1016/j.jad.2007.05.001] [Medline: 17543392]
71. Spek V, Nyklícek I, Smits N, Cuijpers P, Riper H, Keyzer J, et al. Internet-based cognitive behavioural therapy for
subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med 2007
Dec;37(12):1797-1806. [doi: 10.1017/S0033291707000542] [Medline: 17466110]
72. Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapist
factor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res 2007 Jun;7(3):291-297. [doi:
10.1586/14737167.7.3.291] [Medline: 20528315]
73. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety
disorders: a meta-analytic review. Cogn Behav Ther 2009;38(2):66-82. [doi: 10.1080/16506070802694776] [Medline:
20183688]
74. Richards DA, Bower P, Pagel C, Weaver A, Utley M, Cape J, et al. Delivering stepped care: an analysis of implementation
in routine practice. Implement Sci 2012 Jan 16;7:3 [FREE Full text] [doi: 10.1186/1748-5908-7-3] [Medline: 22248385]
75. Russo MW. How to review a meta-analysis. Gastroenterol Hepatol (N Y) 2007 Aug;3(8):637-642 [FREE Full text] [Medline:
21960873]
76. Bush NE, Smolenski DJ, Denneson LM, Williams HB, Thomas EK, Dobscha SK. A virtual hope box: randomized controlled
trial of a smartphone app for emotional regulation and coping with distress. Psychiatr Serv 2017 Apr 1;68(4):330-336. [doi:
10.1176/appi.ps.201600283] [Medline: 27842473]
77. Richards D, Enrique A, Eilert N, Franklin M, Palacios J, Duffy D, et al. A pragmatic randomized waitlist-controlled
effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. NPJ Digit Med 2020;3:85
[FREE Full text] [doi: 10.1038/s41746-020-0293-8] [Medline: 32566763]
78. Howell AN, Rheingold AA, Uhde TW, Guille C. Web-based CBT for the prevention of anxiety symptoms among medical
and health science graduate students. Cogn Behav Ther 2019 Sep;48(5):385-405 [FREE Full text] [doi:
10.1080/16506073.2018.1533575] [Medline: 30526364]
79. Flett JA, Conner TS, Riordan BC, Patterson T, Hayne H. App-based mindfulness meditation for psychological distress and
adjustment to college in incoming university students: a pragmatic, randomised, waitlist-controlled trial. Psychol Health
2020 Sep;35(9):1049-1074. [doi: 10.1080/08870446.2019.1711089] [Medline: 32046499]
80. Huberty J, Green J, Glissmann C, Larkey L, Puzia M, Lee C. Efficacy of the mindfulness meditation mobile app "Calm"
to reduce stress among college students: randomized controlled trial. J Med Internet Res Mhealth Uhealth 2019 Jun
25;7(6):e14273. [doi: 10.2196/14273] [Medline: 31237569]
81. Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition
in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiatr Scand 2014 Sep;130(3):181-192. [doi:
10.1111/acps.12275] [Medline: 24697518]
82. Simon N, McGillivray L, Roberts NP, Barawi K, Lewis CE, Bisson JI. Acceptability of internet-based cognitive behavioural
therapy (i-CBT) for post-traumatic stress disorder (PTSD): a systematic review. Eur J Psychotraumatol 2019;10(1):1646092
[FREE Full text] [doi: 10.1080/20008198.2019.1646092] [Medline: 31497259]
83. Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias
in randomised trials. Br Med J 2019 Aug 28;366:l4898. [doi: 10.1136/bmj.l4898] [Medline: 31462531]
84. Firth N, Barkham M, Kellett S. The clinical effectiveness of stepped care systems for depression in working age adults: a
systematic review. J Affect Disord 2015 Jan 01;170:119-130. [doi: 10.1016/j.jad.2014.08.030] [Medline: 25240141]
Abbreviations
GP: general practitioner
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 21http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX
IAPT: Improving Access to Psychological Therapies
ITT: intention to treat
LI: low intensity
NHS: National Health Service
NICE: National Institute for Health and Care Excellence
NNT: number needed to treat
RCT: randomized controlled trial
SMD: standardized mean difference
TAU: treatment as usual
Edited by G Eysenbach; submitted 13.11.19; peer-reviewed by L Donkin, P Romero-Sanchiz, R Nogueira-Arjona, C Christ, K Kaipainen;
comments to author 23.03.20; revised version received 18.05.20; accepted 24.07.20; published 28.10.20
Please cite as:
Simmonds-Buckley M, Bennion MR, Kellett S, Millings A, Hardy GE, Moore RK
Acceptability and Effectiveness of NHS-Recommended e-Therapies for Depression, Anxiety, and Stress: Meta-Analysis
J Med Internet Res 2020;22(10):e17049
URL: http://www.jmir.org/2020/10/e17049/
doi: 10.2196/17049
PMID:
©Melanie Simmonds-Buckley, Matthew Russell Bennion, Stephen Kellett, Abigail Millings, Gillian E Hardy, Roger K Moore.
Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 28.10.2020. This is an open-access article
distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which
permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal
of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on
http://www.jmir.org/, as well as this copyright and license information must be included.
J Med Internet Res 2020 | vol. 22 | iss. 10 | e17049 | p. 22http://www.jmir.org/2020/10/e17049/ (page number not for citation purposes)
Simmonds-Buckley et alJOURNAL OF MEDICAL INTERNET RESEARCH
XSL
•
FO
RenderX