Mobile Health (mHealth) vs Clinic-Based Group Intervention for People with Serious Mental Illness: A Randomized Controlled Trial

Abstract

Objective: mHealth approaches that use mobile phones to deliver interventions can help improve access to care for people with serious mental illness. The goal was to evaluate how mHealth performs against more traditional treatment. Methods: A three-month randomized controlled trial was conducted of a smartphone-delivered intervention (FOCUS) versus a clinic-based group intervention (Wellness Recovery Action Plan [WRAP]). Participants were 163 clients, mostly from racial minority groups and with long-term, serious mental illness (schizophrenia or schizoaffective disorder, 49%; bipolar disorder, 28%; and major depressive disorder,
23%). Outcomes were engagement throughout the intervention;satisfaction post treatment (three months); and improvement in clinical symptoms, recovery, and quality of life (assessed at baseline, posttreatment, and six months). Results: Participants assigned to FOCUS were more likely than those assigned to WRAP to commence treatment (90% versus 58%) and remain fully engaged in eight weeks of care (56% versus 40%). Satisfaction ratings were comparably high for both interventions. Participants in both groups improved
significantly and did not differ in clinical outcomes, including general psychopathology and depression. Significant improvements in recovery were seen for the WRAP group posttreatment, and significant improvements in recovery and quality of life were seen for the FOCUS group at six months. Conclusions: Both interventions produced significant gains among clients with serious and persistent mental illnesses who were mostly from racial minority groups. The mHealth intervention showed superior patient engagement and produced patient satisfaction and clinical and recovery outcomes that were comparable to those from a widely used clinic-based group intervention for illness management.

Full text

Mobile Health (mHealth) Versus Clinic-Based Group
Intervention for People With Serious Mental Illness:
A Randomized Controlled Trial
Dror Ben-Zeev, Ph.D., Rachel M. Brian, M.P.H., Geneva Jonathan, B.A., Lisa Razzano, Ph.D., C.P.R.P., Nicole Pashka, M.S.,
Elizabeth Carpenter-Song, Ph.D., Robert E. Drake, M.D., Ph.D., Emily A. Scherer, Ph.D.
Objective: mHealth approaches that use mobile phones to
deliver interventions can help improve access to care for
people with serious mental illness. The goal was to evaluate
how mHealth performs against more traditional treatment.
Methods: A three-month randomized controlled trial was
conducted of a smartphone-delivered intervention (FOCUS)
versus a clinic-based group intervention (Wellness Recovery
Action Plan [WRAP]). Participants were 163 clients, mostly
from racial minority groups and with long-term, serious
mental illness (schizophrenia or schizoaffective disorder,
49%; bipolar disorder, 28%; and major depressive disorder,
23%). Outcomes were engagement throughout the inter-
vention; satisfaction posttreatment (three months); and im-
provement in clinical symptoms, recovery, and quality of life
(assessed at baseline, posttreatment, and six months).
Results: Participants assigned to FOCUS were more likely
than those assigned to WRAP to commence treatment (90%
versus 58%) and remain fully engaged in eight weeks of care
(56% versus 40%). Satisfaction ratings were comparably high
for both interventions. Participants in both groups improved
significantly and did not differ in clinical outcomes, including
general psychopathology and depression. Significant im-
provements in recovery were seen for the WRAP group
posttreatment, and significant improvements in recovery
and quality of life were seen for the FOCUS group at six
months.
Conclusions: Both interventions produced significant gains
among clients with serious and persistent mental illnesses
who were mostly from racial minority groups. The mHealth
intervention showed superior patient engagement and pro-
duced patient satisfaction and clinical and recovery outcomes
that were comparable to those from a widely used clinic-based
group intervention for illness management.
Psychiatric Services in Advance (doi: 10.1176/appi.ps.201800063)
Serious mental illnesses affect approximately 4% of the
population (1). Functional impairments related to serious
mental illness interfere with life activities, such as work,
independent living, and self-care (2). Individuals with seri-
ous mental illness typically experience periods of illness
exacerbation characterized by greater impairment inter-
spersed with periods of partial or complete remission (3,4).
With appropriate supports, people with serious mental ill-
ness can lead rewarding and productive lives, even in the
context of ongoing symptoms (5,6).
Self-management interventions that increase and lengthen
the periods in which people with serious mental illness re-
main healthier are popular and increasingly offered at clinics
(7–12). Self-management interventions can help people ad-
here to treatment regimens, reduce the severity and distress
associated with symptoms, avoid hospitalization, increase
self-esteem, and improve perceived recovery (13). However,
the barriers associated with clinic-based care may limit the
benefits of these interventions. When individuals experience
symptom exacerbations (arguably, when they need illness
management support the most), they may avoid going to a
clinic or interacting with others, perhaps because of the clinic’s
distance from their residence or hours of operation (14,15), the
stigma associated with seeking care (16), or dissatisfaction with
services (17–19).
Mobile health (mHealth) approaches that use mobile
phones in support of health care can help overcome some
of the barriers associated with clinic-based care. Mobile
phones are ubiquitous, even among people with serious
mental illness, who often have limited access to resources
(20–22). Research across continents has shown that the
majority of adults with serious mental illness are interested
in using their mobile phones as instruments for self-
management (23). Early mHealth efforts have produced
promising outcomes in terms of feasibility, acceptability, and
preliminary efficacy in this population (24–29). Whether
mHealth interventions can serve as stand-alone treatments,
effectively engage individuals with serious mental illness in
PS in Advance ps.psychiatryonline.org 1
ARTICLES

remote care, and produce clinical outcomes that are com-
parable to those of clinic-based interventions is unknown. As
more mHealth and clinic-based interventions are made ac-
cessible in real-world practice, patients and their providers
will have more options to choose from when deciding on
their preferred model of care. Direct comparison of the
strengths and weaknesses of existing interventions for a
designated clinical problem is at the core of comparative
effectiveness research.
Our objective was to compare smartphone-delivered
mHealth to a clinic-based, group self-management inter-
vention for people with serious mental illness. We evaluated
differences between treatment groups in patient engage-
ment, satisfaction, and clinical outcomes. To our knowledge,
this article reports on the first comparative effectiveness
trial with a head-to-head comparison of mHealth and a
clinic-based intervention for people with serious mental
illness.
METHODS
We conducted an assessor-blind, two-arm, randomized
controlled trial between June 2015 and September 2017 in
partnership with Thresholds, a large agency that provides
services to people with serious mental illness living in the
midwestern United Sates. The study was approved by the
Institutional Review Boards of the University of Washington
and Dartmouth College and monitored by an independent
safety monitoring board at Dartmouth’s Department of
Psychiatry. All study participants completed informed con-
sent. Individuals were randomly assigned (1:1 ratio) into one
of two treatment arms: an mHealth intervention (FOCUS)
or a clinic-based group intervention (Wellness Recovery Ac-
tion Plan [WRAP]). Interventions were deployed for a period
of 12 weeks, using cycles of eight cohorts of participants
assigned to individual FOCUS or group-based WRAP over
parallel periods. We conducted assessments at baseline
(zero months), posttrial (three months), and follow-up (six
months). Participants were not monetarily incentivized to
engage in interventions but were compensated for com-
pleting assessments ($30 per assessment). [A CONSORT
diagram of the study is available in an online supplement to
this article.]
Participants
Participants were identified by using the electronic health
record and then recruited by 20 clinical teams at three
centers. Clinical staff approached candidates to describe the
project and provide informational handouts with a contact
number for the research team. Interested clients called study
staff to learn more and undergo a brief phone screening.
Suitable candidates were invited to attend a more compre-
hensive in-person evaluation meeting. Inclusion criteria
were chart diagnosis of schizophrenia, schizoaffective dis-
order, bipolar disorder, or major depressive disorder;
age $18 years; and a rating #3ononeofthreeitems
constituting the domination by symptoms factor from the
Recovery Assessment Scale (RAS) (30,31), indicating a need
for the type of resources both interventions may offer. Ex-
clusion criteria were hearing, vision, or motor impairment
affecting operation of a smartphone (determined by using a
demonstration device); less than fifth-grade English reading
ability (determined with the reading section of the WRAT-4
[32]); and exposure to WRAP or FOCUS in the past three
years.
Randomization and Blinding
The study statistician created a computer-generated ran-
domization list. Individual intervention allocations were
placed in sequentially numbered envelopes containing in-
structions to contact an mHealth support specialist (for
FOCUS) or a group facilitator (for WRAP) to schedule an
appointment. Study assessors were excluded from study
meetings in which procedures that would jeopardize blind-
ing were discussed. Participants were instructed not to dis-
close their treatment allocation during assessments.
Interventions
FOCUS (25,33,34) is a multimodal, smartphone-delivered
intervention for people with serious mental illness that in-
cludes three components: FOCUS application (app), clini-
cian dashboard, and mHealth support specialist. The system
includes preprogrammed daily self-assessment prompts and
on-demand functions that can be accessed 24 hours a day.
Self-management content targets five broad domains: voices
(coping with auditory hallucinations via cognitive restruc-
turing, distraction, and guided hypothesis testing), mood
(managing depression and anxiety via behavioral activation,
relaxation techniques, and supportive content), sleep (sleep
hygiene, relaxation, and health and wellness psycho-
education), social functioning (cognitive restructuring of
persecutory ideation, anger management, activity schedul-
ing, and skills training), and medication (behavioral tailor-
ing, reminders, and psychoeducation). Content can be
accessed as either brief video or audio clips or sequences of
digital screens with written material coupled with images
(34). FOCUS users’responses to daily self-assessments are
displayed on a digital dashboard. Participants received brief
weekly calls from an mHealth support specialist who assis-
ted them in all technical and clinical aspects of the intervention
(35). [A complete description of the interventions is available in
theonlinesupplement.]
WRAP (12) is a widely used (36) group self-management
intervention led by trained facilitators with lived experi-
ence of mental illness. Sessions follow a sequenced cur-
riculum, and specific group discussion topics and examples
draw from the personal experiences of the participants and
cofacilitators. The model emphasizes individuals’equip-
ping themselves with “personal wellness tools,”each fo-
cusing on recovery concepts (for example, hope, personal
responsibility, and self-advocacy), language (for example,
person-first recovery language), development of a WRAP
2ps.psychiatryonline.org PS in Advance
mHEALTH VERSUS CLINIC INTERVENTION FOR SERIOUS MENTAL ILLNESS

(for example, establishing a daily maintenance plan and
identifying and responding to triggers and early warning
signs), and encouraging positive thinking (for example,
changing negative thoughts to positive thoughts, build-
ing self-esteem, suicide prevention, and journaling). Fa-
cilitators incorporate these tools into a written plan,
which includes daily maintenance, identification of trig-
gers and methods to avoid them, identification of warn-
ing signs and response options, and a crisis management
plan.
FOCUS and WRAP are similar in that both are recovery
oriented, use an array of empowerment and self-management
techniques, and involve similar intervention periods; em-
pirical findings suggest that both interventions are engag-
ing and beneficial to people with serious mental illness
(25,34,37–40). The differences between these approaches
represent core distinctions between mHealth and clinic-
based models of care (that is, accessed in one’s own envi-
ronment versus administered in a center, largely automated
versus person delivered, and on demand versus scheduled).
Measures
Engagement. We considered participants as commencing
treatment if they used FOCUS once or attended one WRAP
session. We calculated engagement in treatment for each
participantweeklybyusinghisorherFOCUSusedata
(logged by the software) or WRAP session attendance
(logged by WRAP facilitators). Participants in the FOCUS
arm were considered engaged if they used the app on at
least five of seven days a week (that is, approximately 70%).
AFOCUS“use”event is recorded as such only if, following
a prompt, participants elect to engage in a clinical status
assessment or if they self-initiate one of the FOCUS
on-demand tools. Participants in the WRAP arm were
considered engaged if they attended at least 60 minutes of
the scheduled 90-minute group session (that is, approxi-
mately 70%) or completed a makeup session in the same
week.
Satisfaction. We measured satisfaction as the sum of five
self-report items completed during the three-month, post-
intervention assessment. Participants rated the following
statements with a 7-point rating scale (1, strongly disagree, to
7, strongly agree): I am satisfied with the treatment program,
the treatment program helped me feel better, the treatment
program was not interactive enough (reverse scored), I
enjoyed the treatment program, and I would recommend the
treatment program to a friend.
Clinical outcomes. Our primary clinical outcome was gen-
eral psychopathology, which is most appropriate for the
different clinical groups represented in the sample of
people with serious mental illness. General psychopathol-
ogy was measured with the Symptom Checklist–9(SCL-9)
(41), a brief version of the Symptom Checklist–90R (42)
that captures several domains of mental health (for
example, anxiety, somatization, hostility, paranoid think-
ing, and psychoticism) and provides a single global rat-
ing of severity (43,44). Secondary clinical outcomes
included depression, psychosis, recovery, and quality of
life. Depression was assessedwiththeBeckDepression
Inventory–Second Edition (BDI-II) (45), which includes
21 items rated on a 4-point scale, summed for a total de-
pression severity score. Psychosis was assessed with the
Psychotic Symptom Rating Scales (PSYRATS) (46). PSY-
RATS includes dimensions of auditory hallucinations (for
example, frequency, duration, loudness, and distress) and
delusions (for example, preoccupation, conviction, and
disruption). Recovery was assessed with the RAS (30,31), a
24-item measure assessing five recovery factors with a
5-point Likert scale: personal confidence and hope, will-
ingnesstoaskforhelp,goaland success orientation, re-
liance on others, and domination by symptoms. Quality of
life was assessed as the total of six items focusing on one’s
personal evaluation of one’s life, self, family, time spent
with family, time spent with others, and participation
in activities. Participants respond on a 7-point delighted-
terrible scale. Clinical outcome measures were adminis-
tered by assessors who were trained and supervised
by licensed clinical psychologists with extensive experi-
ence in their administration among individuals with
serious mental illness. Challenges in administration or
scoring were discussed and resolved during weekly project
team meetings.
Sample Size
We designed the study to detect a medium effect (defined as
f=.24) in difference between groups in change in clinical
outcome from baseline to three months, with 80% power
(a=.05). This power is achieved with 72 participants per
group. To allow for 10% attrition, 80 participants were
randomly assigned to each group.
Analytic Approach
We used an intent-to-treat analysis that included all ran-
domly assigned individuals. For treatment comparisons
among clinical outcomes, we used mixed-effects models,
including treatment condition, time of assessment, and an
interaction term for treatment condition 3time. Linear
mixed models (47) were fit for all outcomes except PSY-
RATS, which was modeled via nonlinear Poisson hurdle
mixed model, which estimates a logistic model for proba-
bility of a count .0 (likelihood of experiencing symptoms)
as well as a Poisson model for mean symptom ratings if any
symptoms were experienced (48). PSYRATS was modeled in
this way because of the skewed nature and zero-inflation
observed for this outcome (for example, 64% of individuals
had a score of 0 at baseline), which made linear models
inappropriate for this outcome. We evaluated engagement
by using chi-square tests, and treatment satisfaction was
assessed with t tests. [The online supplement includes a
complete description of the analyses.]
PS in Advance ps.psychiatryonline.org 3
BEN-ZEEV ET AL.

RESULTS
The study enrolled 163 participants, whose mean age was
49. Most participants were male (N=96, 59%) and African
American (N=106, 65%). Diagnoses were as follows:
schizophrenia or schizoaffective disorder, 49% (N=80); bi-
polar disorder, 28% (N=46); and major depressive disorder,
23% (N=37). Participants differed between treatment groups
only in that significantly more participants randomly assigned
to FOCUS had previously used a smartphone (73% versus
57%, x
2
=4.81, df=1, p=.03). Table 1 summarizes participant
characteristics by group.
Engagement
Following randomization, 90% (N=74) of participants as-
signed to FOCUS commenced use of the mHealth app, and
58% (N=47) of those assigned to WRAP attended at least one
group session (x
2
=22.11, df=1, p,.001) (Figure 1). Averaging
across all participants assigned to FOCUS, participants used
the app on 5.462.4 days in the first week of the intervention,
on 4.662.7 days in the third week, on 4.362.7 days in the
sixth week, on 3.962.7 days in the ninth week, and on 3.86
2.9 days in the last week. In the first week of WRAP, 48%
(N=39) of assigned participants attended the weekly meet-
ing; 42% (N=34) attended in the third and sixth weeks, 36%
(N=29) attended in the ninth week, and 28% (N=23) atten-
ded in the last week. FOCUS group participants were more
likely than WRAP participants to fully engage in treatment
for at least eight weeks (56% versus 40%) (x
2
=4.50, df=1,
p=.03) (Figure 1). The groups did not differ in the propor-
tions of participants fully engaging in all weeks of treatment.
Satisfaction
Mean posttreatment satisfaction ratings were similar be-
tween groups: overall ratings of 25.763.8 for FOCUS and
25.563.6 for WRAP (t=–.31, df=1, p=.76). Figure 2 shows
average responses for the individual satisfaction items. No
adverse events were reported for participants in either
intervention arm.
Clinical Outcomes
Treatment groups did not differ in change from baseline
to three months postintervention on primary and secondary
clinical outcomes. Primary analyses found no significant
differences in clinical outcomes between diagnostic groups.
Exploratory analyses found within-group changes. Table 2
summarizes the mean scores for clinical outcomes. Im-
provement between baseline and three months (end of
treatment) in the primary clinical outcome, general psy-
chopathology as measured by the SCL-9, was seen for
FOCUS (mean6SE of the estimated mean difference=
22.736.75, t=23.64, df=289, p,.001) and WRAP (22.146.76,
t=22.84, df=289, p=.005). Similar improvements in SCL-9
scores were seen between baseline and six months for
FOCUS (22.516.75, t=23.33, df=289, p=.001) and for WRAP
(21.936.77, t=22.51, df=289, p=.01). Improvement between
baseline and three months in BDI-II scores was seen
for FOCUS (22.7661.09, t=22.54, df=289, p=.01) and WRAP
(22.3361.10, t=22.13, df=289, p=.03). Similar improvements
in BDI-II were seen between baseline and six months for
FOCUS (24.2161.09, t=23.85, df=289, p,.001) and for WRAP
(23.7461.12, t=23.34, df=289, p,.001). Improvements
between baseline and three months in RAS scores were
seen for WRAP (2.4461.10, t=2.21, df=288, p=.03). Between
baseline and six months, improvements in RAS scores were
seen for FOCUS (4.5661.10, t=4.16, df=289, p,.001) and for
WRAP (2.8661.12, t=2.55, df=289, p=.01). No significant
within-group differences in PSYRATS or quality-of-life scores
were noted between baseline and three months. Between
baseline and six months, significant improvements were seen
in quality of life among the FOCUS participants (1.586.62,
t=2.55, df=289, p=.01).
Treatment groups did not differ on changes from three
months postintervention to the six-month follow-up. Within
treatment group, improvement in RAS scores was seen for
FOCUS (2.7461.11, t=2.46, df=288, p=.01) (Table 2).
FOCUS Effects
In the FOCUS group, education and treatment engagement
were significantly associated with secondary clinical out-
comes. Having more than a high school education was as-
sociated with larger increases in RAS scores (mean6SE of
the estimated mean difference from baseline to three
months=4.3162.15, t=1.71, df=145, p=.05), as was more weeks
of engagement (five or more days per week of FOCUS use)
TABLE 1. Baseline characteristics of participants in Wellness
Recovery Action Plan (WRAP) and FOCUS
WRAP
(N=81)
FOCUS
(N=82)
Characteristic N % N %
Age (M6SD) 4969.8 49610.1
Male 47 58 49 60
Previously used smartphone 46 57 60 73
Race
White 22 28 22 27
African American 53 66 53 65
Other or more than 1 race 5 6 7 9
Education
High school or less 48 59 51 63
More than high school 33 41 31 38
Diagnosis
Schizophrenia or
schizoaffective disorder
42 52 38 46
Bipolar disorder 25 31 21 26
Major depressive disorder 14 17 23 28
Lifetime psychiatric
hospitalizations
04556
1–525323441
6–10 18 23 15 18
11–15 10 13 12 15
16–20 5 6 4 5
$20 17 22 12 15
4ps.psychiatryonline.org PS in Advance
mHEALTH VERSUS CLINIC INTERVENTION FOR SERIOUS MENTAL ILLNESS

(.576.28, t=2.07, df=133, p=.04). Age, gender, race, past use of
a smartphone, and number of hospitalizations were not as-
sociated with mHealth intervention outcomes.
DISCUSSION
The FOCUS mHealth intervention produced clinical out-
comes and patient satisfaction ratings that were comparable
to those of WRAP, an evidence-based, self-management in-
tervention. No adverse events were reported for participants
in either intervention arm. FOCUS had significantly higher
treatment commencement rates after random assignment
(90%), compared with WRAP (58%). These findings suggest
that FOCUS was easier to initiate or more accessible. Sig-
nificantly more FOCUS participants fully completed eight
or more weeks of treatment (56%), compared with WRAP
(40%). Groups did not differ significantly in the percentage
of participants who fully completed 12 weeks of treatment.
Taken together, participants were exposed to treatment
content more often and over longer intervention periods
(dose) via FOCUS than via clinic-based WRAP.
Satisfaction with treatment did not differ across groups.
Participants provided high satisfaction ratings for FOCUS
and WRAP, and participants in each approach reported that
it was enjoyable and interactive and helped them feel better.
We draw several conclusions. First, people with serious
mental illness can be satisfied with mHealth treatments that
are largely automated, involving weekly remote check-in
calls but minimal in-person contact. Second, although bar-
riers related to clinic-based care might have affected treat-
ment commencement and engagement in WRAP, such
barriers did not negatively affect participants’overall im-
pressions of the intervention. Participants in each arm were
not exposed to the other treatment, and thus their satis-
faction ratings were not grounded in familiarity with an
alternative. A cross-over design in which participants ex-
perienced both interventions would enable more direct
comparison of satisfaction.
Changes in primary (general psychopathology) and sec-
ondary (depression, psychosis, recovery, and quality of life)
clinical outcomes did not differ by intervention. No differ-
ences were seen between groups in retention of gains three
months after the conclusion of the intervention (at six-month
follow-up). Exploratory analyses within treatment arms
showed significant and comparable reductions in psychopa-
thology and depression in both treatment groups. Significant
improvements in recovery were seen in the WRAP group at
the end of treatment (three months) and in the FOCUS group
at six-month follow-up.
Among FOCUS participants, a higher level of education
and greater treatment engagement (that is, weeks of daily
smartphone app use) were both linked to greater recovery
postintervention. Notably, age, gender, race, having previous
experience with smartphones, and number of previous
FIGURE 1. Percentage of patients fully engaged in Wellness
Recovery Action Plan (WRAP) and FOCUS, by stage of
intervention
Commenced
treatment after
randomization
Fully engaged
in 8 weeks of
treatment
Fully engaged
in 12 weeks of
treatment
80
70
60
50
40
30
20
10
0
N of participants
74
90%
47
58%
46
56% 32
40% 21
26%
18
22%
FOCUS
WRAP
FIGURE 2. Mean satisfaction ratings after treatment among participants in Wellness Recovery Action Plan (WRAP) and FOCUS
a
7
6
5
4
3
2
1
0
Rating
I am satisfied with
the treatment program.
The treatment program
helped me feel better.
The treatment program
was not interactive
enou
g
h.
I enjoyed the
treatment program.
I would recommend
the treatment program
to a friend.
FOCUS
WRAP
a
Responses range from 1, strongly disagree, to 7, strongly agree.
PS in Advance ps.psychiatryonline.org 5
BEN-ZEEV ET AL.

psychiatric hospitalizations (an indicator of baseline illness
severity) were not associated with clinical outcomes.
This study had notable strengths. To our knowledge, it
is the first randomized controlled trial examining the effects
of a smartphone intervention involving individuals with
schizophrenia spectrum disorders. The comparator in-
tervention (WRAP) is an active evidence-based treatment.
Both interventions were introduced at the study site at the
same time, ensuring equal levels of enthusiasm among study
staff and clinical personnel. Additional methodological
strengths included use of psychometrically sound outcomes
measures, random assignment, maintenance of assessor
blindness throughout the study, and deployment of inter-
ventions as parallel cohorts to control for historical effects.
Both interventions were delivered with guidance from treat-
ment experts; the mHealth support specialist was trained and
supervised via phone by the lead FOCUS developer (DBZ).
WRAP facilitators were trained by the Copeland Center for
Wellness and Recovery (the premiere WRAP training and
education center) and supervised on site by a certified
advanced-level WRAP facilitator.
The study had several limitations. We did not include a
treatment-as-usual comparator arm. We thus cannot conclude
that the clinical outcomes reported in the study are a direct
result of the interventions deployed rather than of the passage
of time or of artifacts related to involvement in research.
FOCUS participants received a smartphone with an active data
plan, which may be less likely in standard care. However, once
they were randomly assigned to the mHealth arm, participants’
access to the device and data were not contingent upon t heir
use of the intervention app. This noncontingent access
suggests that ongoing engagement in FOCUS was not for
secondary gains. In addition to receiving new interventions
in the context of research, participants continued to receive
various services from the community agency, which may
have influenced the results. The measures of engagement
and satisfaction were developed for this study and have not
been validated in previous research. Because the study was
powered to detect differences in the full sample between
treatment groups, exploratory analyses had reduced power,
and thus the results should be interpreted cautiously.
The findings support the notion that mHealth can play an
important role in 21st century mental health care (49,50).
Contemporary mobile phone “smart”functionalities enable
these devices to serve as much more than static information
repositories (51). Audio and video media players, graphic
displays, interactive capabilities, bidirectional calling and
texting, and Internet connectivity create new opportunities to
engage patients with both automated resources and human
supports. The portability of smartphones enables patients to
take them wherever they go. FOCUS users in this study co uld
read, hear, or view self-management skills, suggestions,
and demonstrations that were relevant to the challenges
they encountered as they went about their daily life. Instead
of having to retain and recall clinicians’suggestions, they
accessed their “pocket therapist”on demand—an experience
akin to a friend checking in on them (34). FOCUS users’daily
self-assessments were relayed to their mHealth support spe-
cialist, who reviewed the data to better understand what
they were experiencing. This information was brought up
in their weekly check-in calls, which likely strengthened
the feeling that a caring individual was paying attention to
their status (35). The combination of automated functions
andliveremotehumansupport facilitated a therapeutic
model unlike any they had encountered. For some, this
proved to be engaging and helpful.
TABLE 2. Clinical outcomes over time among participants in Wellness Recovery Action Plan (WRAP) and FOCUS
(intent-to-treat sample)
Baseline 3 months (end of treatment) 6-month follow-up
FOCUS WRAP FOCUS WRAP FOCUS WRAP
Measure N M SD N M SD N M SD N M SD N M SD N M SD
SCL-9
a
82 12.71 7.24 81 11.93 8.06 75 10.0
b
6.52 74 9.53
b
7.33 74 10.38
c
8.08 70 9.99
c
7.76
BDI-II
d
82 22.00 11.20 81 19.53 12.09 75 19.08
b
12.57 74 16.80
b
11.66 74 17.85
c
12.79 70 16.03
c
11.61
PSYRATS
e
Score .0 823138 8128357525 337418 247419 266911 16
Among those
with score
.0
31 21.6 12.1 28 23.0 13.9 25 30.6 12.1 18 25.0 15.1 19 27.7 14.0 11 29.5 14.0
RAS
f
82 90.34 13.33 81 91.72 13.23 75 92.39 11.91 73 94.89
b
15.89 74 94.59
c,g
13.02 70 94.40
c
13.74
Quality of life
h
81 26.54 7.10 82 26.69 7.46 75 26.87 7.44 74 27.80 7.48 74 28.11
c
7.30 70 27.60 7.91
a
SCL-9, Symptom Checklist–9. Possible scores range from 0 to 36, with higher scores indicating greater symptom severity.
b
Significant within-group change from baseline to end of treatment.
c
Significant within-group change from baseline to six-month follow-up.
d
BDI-II, Beck Depression Inventory–Second Edition. Possible scores range from 0 to 63; minimal (0–13), mild (14–19), moderate (20–28), severe (29–63) depression.
e
PSYRATS, Psychotic Symptom Rating Scales. Values in row for score .0 are Ns and percentages. Possible scores (in row for among those with score .0)
range from 0 to 68, with higher scores indicating greater symptom severity.
f
RAS, Recovery Assessment Scale. Possible scores range from 24 to 120, with higher scores indicating greater recovery.
g
Significant within-group change from end of treatment to six-month follow-up.
h
Possible quality-of-life ratings range from 6 to 42, with higher scores indicating greater quality of life.
6ps.psychiatryonline.org PS in Advance
mHEALTH VERSUS CLINIC INTERVENTION FOR SERIOUS MENTAL ILLNESS

CONCLUSIONS
The FOCUS model is one of several promising mHealth
approaches. Additional paradigms include using mHealth
applications to augment clinic-based services to improve
their efficacy (52,53) and leveraging multimodal smartphone
embedded sensors (for example, GPS, accelerometers, and
microphone) to unobtrusively collect information about
patients’behavior, context, and functioning (54,55). These
mobile data can be shared with clinicians to enhance their
patient-monitoring capabilities and inform more tailored
in-person care (56,57).
Evidence from mHealth research will accumulate over
the upcoming years, including additional promising results
from randomized controlled trials. As more mHealth inter-
ventions prove to be engaging and clinically useful to pa-
tients with serious mental illness, enthusiasm for their use in
clinical practice will grow (15). In the future, it will be im-
portant to ensure that these mHealth technologies are
deployed ethically and responsibly (58).
AUTHOR AND ARTICLE INFORMATION
Dr. Ben-Zeev and Ms. Brian are with the Department of Psychiatry and
Behavioral Sciences, University of Washington, Seattle. Ms. Jonathan is
with the Department of Psychiatry and Behavioral Sciences,
Northwestern University, Evanston, Illinois. Dr. Razzano is with the
Department of Psychiatry, University of Illinois at Chicago and with
Thresholds, Chicago. Ms. Pashka is with Thresholds, Chicago. Dr.
Carpenter-Song is with the Department of Anthropology, Dartmouth
College, Hanover, New Hampshire. Dr. Drake is with the Dartmouth
Institute for Health Policy and Clinical Practice and Dr. Scherer is with
the Department of Biomedical Data Science and the Department of
Community and Family Medicine, Geisel School of Medicine at Dart-
mouth, Lebanon, New Hampshire. Dr. Drake is also with Westat,
Rockville, Maryland. Send correspondence to Dr. Ben-Zeev (e-mail:
dbenzeev@uw.edu).
Research reported here was supported by award CER-1403-11403 from
the Patient-Centered Outcomes Research Institute (PCORI). The study
was registered in clinicaltrials.gov (https://clinicaltrials.gov/ct2/show/
NCT02421965). The authors gratefully acknowledge the contributions
of staff and service recipients at Thresholds, Chicago. The views and
statements in this article are solely the responsibility of the authors and
do not necessarily represent the views of PCORI or its Board of Gov-
ernors or Methodology Committee.
Dr. Ben-Zeev has an intervention content licensing and consulting
agreement with Pear Therapeutics. The other authors report no financial
relationships with commercial interests.
Received February 6, 2018; revisions received March 27 and April 20,
2018; accepted April 24, 2018; published online 25 May, 2018.
REFERENCES
1. Key Substance Use and Mental Health Indicators in the United
States: Results From the 2015 National Survey on Drug Use and
Health. SMA 16-4984, NSDUH Series H-51. Rockville, MD, Sub-
stance Abuse and Mental Health Services Administration, Center
for Behavioral Health Statistics and Quality, 2016
2. Substance Abuse and Mental Health Services Administration, Cen-
ter for Mental Health Services: Final notice. Federal Register 58:
29422–29425, 1993
3. Strauss GP, Harrow M, Grossman LS, et al: Periods of recovery in
deficit syndrome schizophrenia: a 20-year multi-follow-up longi-
tudinal study. Schizophrenia Bulletin 36:788–799, 2010
4. Judd LL, Schettler PJ, Solomon DA, et al: Psychosocial disability
and work role function compared across the long-term course of
bipolar I, bipolar II and unipolar major depressive disorders.
Journal of Affective Disorders 108:49–58, 2008
5. Davidson L, Schmutte T, Dinzeo T, et al: Remission and recovery
in schizophrenia: practitioner and patient perspectives. Schizo-
phrenia Bulletin 34:5–8, 2008
6. Deegan PE: Recovery: the lived experience of rehabilitation. Psy-
chosocial Rehabilitation Journal 11:11–19, 1988
7. Mueser KT, Cook JA: Introduction to the special issue on illness
self-management. Psychiatric Rehabilitation Journal 36:229–230,
2013
8. Ridgway P, McDiarmid D, Davidson L, et al: Pathways to Re-
covery: A Strengths Recovery Self-Help Workbook. Lawrence, KS,
University of Kansas School of Social Welfare, 2002
9. Spaniol L, Koehler M, Hutchinson D: The Recovery Workbook:
Practical Coping and Empowerment Strategies for People With
Psychiatric Disability. Leader’s Guide. Boston, Boston University,
Sargent College of Health and Rehabilitation Sciences, Center for
Psychiatric Rehabilitation, 2000
10. Diehl SM, Baxter EA, Talley CL, et al: Building Recovery of In-
dividual Dreams and Goals Through Education and Support: A
Peer-Taught Curriculum on Recovery From Mental Illness. Chi-
cago, University of Illinois, 2006
11. Illness Management and Recovery: Practitioner Guides and
Handouts. Rockville, MD, Substance Abuse and Mental Health
Services Administration, 2009
12. Copeland ME, Mead S: Wellness Recovery Action Plan and Peer
Support: Personal, Group and Program Development. Brattleboro,
VT, Peach Press, 2004
13. Petros R, Solomon P: Reviewing illness self-management programs:
a selection guide for consumers, practitioners, and administrators.
Psychiatric Services 66:1180–1193, 2015
14. Kreyenbuhl J, Nossel IR, Dixon LB: Disengagement from mental
health treatment among individuals with schizophrenia and
strategies for facilitating connections to care: a review of the lit-
erature. Schizophrenia Bulletin 35:696–703, 2009
15. Ben-Zeev D: Technology in mental health: creating new knowl-
edge and inventing the future of services. Psychiatric Services 68:
107–108, 2017
16. Corrigan P: How stigma interferes with mental health care.
American Psychologist 59:614–625, 2004
17. Mojtabai R, Fochtmann L, Chang S-W, et al: Unmet need for
mental health care in schizophrenia: an overview of literature and
new data from a first-admission study. Schizophrenia Bulletin 35:
679–695, 2009
18. Drake RE, Bond GR, Essock SM: Implementing evidence-based
practices for people with schizophrenia. Schizophrenia Bulletin
35:704–713, 2009
19. McGuire AB, Bartholomew T, Anderson AI, et al: Illness
management and recovery in community practice. Psychiatric
Rehabilitation Journal 39:343–351, 2016
20. Ben-Zeev D, Davis KE, Kaiser S, et al: Mobile technologies among
people with serious mental illness: opportunities for future ser-
vices. Administration and Policy in Mental Health and Mental
Health Services Research 40:340–343, 2013
21. Torous J, Chan SR, Yee-Marie Tan S, et al: Patient smartphone
ownership and interest in mobile apps to monitor symptoms of
mental health conditions: a survey in four geographically distinct
psychiatric clinics. JMIR Mental Health 1:e5, 2014
22. Naslund JA, Aschbrenner KA, Bartels SJ: How people with serious
mental illness use smartphones, mobile apps, and social media.
Psychiatric Rehabilitation Journal 39:364–367, 2016
23. Firth J, Cotter J, Torous J, et al: Mobile phone ownership and
endorsement of “mHealth”among people with psychosis: a meta-
analysis of cross-sectional studies. Schizophrenia Bulletin 42:
448–455, 2016
PS in Advance ps.psychiatryonline.org 7
BEN-ZEEV ET AL.

24. Granholm E, Ben-Zeev D, Link PC, et al: Mobile Assessment and
Treatment for Schizophrenia (MATS): a pilot trial of an interactive
text-messaging intervention for medication adherence, socializa-
tion, and auditory hallucinations. Schizophrenia Bulletin 38:
414–425, 2012
25. Ben-Zeev D, Brenner CJ, Begale M, et al: Feasibility, acceptability,
and preliminary efficacy of a smartphone intervention for schizo-
phrenia. Schizophrenia Bulletin 40:1244–1253, 2014
26. Pijnenborg GH, Withaar FK, Brouwer WH, et al: The efficacy of
SMS text messages to compensate for the effects of cognitive
impairments in schizophrenia. British Journal of Clinical Psy-
chology 49:259–274, 2010
27. Kannisto KA, Adams CE, Koivunen M, et al: Feedback on SMS
reminders to encourage adherence among patients taking anti-
psychotic medication: a cross-sectional survey nested within a
randomised trial. BMJ Open 5:e008574, 2015
28. Kane JM, Perlis RH, DiCarlo LA, et al: First experience with a
wireless system incorporating physiologic assessments and direct
confirmation of digital tablet ingestions in ambulatory patients
with schizophrenia or bipolar disorder. Journal of Clinical Psy-
chiatry 74:e533–e540, 2013
29. Faurholt-Jepsen M, Frost M, Vinberg M, et al: Smartphone data as
objective measures of bipolar disorder symptoms. Psychiatry Re-
search 217:124–127, 2014
30. Corrigan PW, Giffort D, Rashid F, et al: Recovery as a psycho-
logical construct. Community Mental Health Journal 35:231–239,
1999
31. Corrigan PW, Salzer M, Ralph RO, et al: Examining the factor
structure of the Recovery Assessment Scale. Schizophrenia Bul-
letin 30:1035–1041, 2004
32. Wilkinson GS, Robertson GJ: Wide Range Achievement Test,
Fourth Edition (WRAT–4) Professional Manual. Lutz, FL, Psy-
chological Assessment Resources, Inc, 2004
33. Ben-Zeev D, Kaiser SM, Brenner CJ, et al: Development and
usability testing of FOCUS: a smartphone system for self-
management of schizophrenia. Psychiatric Rehabilitation Journal
36:289–296, 2013
34. Ben-Zeev D, Brian RM, Aschbrenner KA, et al: Video-based mobile
health interventions for people with schizophrenia: bringing the
“pocket therapist”to life. Psychiatric Rehabilitation Journal 41:
39–45, 2018
35. Jonathan GK, Pivaral L, Ben-Zeev D: Augmenting mHealth with
human support: notes from community care of people with serious
mental illnesses. Psychiatric Rehabilitation Journal 40:336–338,
2017
36. Roberts G, Wolfson P: The rediscovery of recovery: open to all.
Advances in Psychiatric Treatment 10:37–48, 2004
37. Ben-Zeev D, Scherer EA, Gottlieb JD, et al: mHealth for schizo-
phrenia: patient engagement with a mobile phone intervention
following hospital discharge. JMIR Mental Health 3:e34, 2016
38. Cook JA, Copeland ME, Jonikas JA, et al: Results of a randomized
controlled trial of mental illness self-management using Wellness
Recovery Action Planning. Schizophrenia Bulletin 38:881–891,
2012
39. Cook JA, Copeland ME, Floyd CB, et al: A randomized controlled
trial of effects of Wellness Recovery Action Planning on de-
pression, anxiety, and recovery. Psychiatric Services 63:541–547,
2012
40. Cook JA, Copeland ME, Corey L, et al: Developing the evidence
base for peer-led services: changes among participants following
Wellness Recovery Action Planning (WRAP) education in two
statewide initiatives. Psychiatric Rehabilitation Journal 34:113–120,
2010
41. Klaghofer R, Brähler E: Construction and statistical test of a short
form of the SCL-90-R [in German]. Zeitschrift für Klinische Psy-
chologie, Psychiatrie und Psychotherapie 49:115–124, 2001
42. Derogatis LR, Unger R: Symptom Checklist‐90‐Revised. New York,
Wiley, 2010
43. Prinz U, Nutzinger DO, Schulz H, et al: Comparative psychometric
analyses of the SCL-90-R and its short versions in patients with
affective disorders. BMC Psychiatry 13:104, 2013
44. Müller JM, Postert C, Beyer T, et al: Comparison of eleven short
versions of the Symptom Checklist 90-Revised (SCL-90-R) for use
in the assessment of general psychopathology. Journal of Psy-
chopathology and Behavioral Assessment 32:246–254, 2010
45. Beck AT, Steer RA, Brown GK: Beck Depression Inventory–II. San
Antonio, TX, Psychological Corp, 1996
46. Haddock G, McCarron J, Tarrier N, et al: Scales to measure di-
mensions of hallucinations and delusions: the Psychotic Symp-
tom Rating Scales (PSYRATS). Psychological Medicine 29:
879–889, 1999
47. Fitzmaurice GM, Laird NM, Ware JH: Applied Longitudinal
Analysis. New York, Wiley, 2012
48. Atkins DC, Baldwin SA, Zheng C, et al: A tutorial on count re-
gression and zero-altered count models for longitudinal substance
use data. Psychology of Addictive Behaviors 27:166–177, 2013
49. Marzano L, Bardill A, Fields B, et al: The application of mHealth to
mental health: opportunities and challenges. Lancet Psychiatry 2:
942–948, 2015
50. Proudfoot J: The future is in our hands: the role of mobile phones
in the prevention and management of mental disorders. Australian
and New Zealand Journal of Psychiatry 47:111–113, 2013
51. Ben-Zeev D, Drake RE, Corrigan PW, et al: Using contemporary
technologies in the assessment and treatment of serious mental
illness. American Journal of Psychiatric Rehabilitation 15:357–376,
2012
52. Depp CA, Ceglowski J, Wang VC, et al: Augmenting psycho-
education with a mobile intervention for bipolar disorder: a ran-
domized controlled trial. Journal of Affective Disorders 174:23–30,
2015
53. Torous J, Roux S: Patient-driven innovation for mobile mental
health technology: case report of symptom tracking in schizo-
phrenia. JMIR Mental Health 4:e27, 2017
54. Harari GM, Lane ND, Wang R, et al: Using smartphones to collect
behavioral data in psychological science: opportunities, practical
considerations, and challenges. Perspectives on Psychological
Science 11:838–854, 2016
55. Mohr DC, Zhang M, Schueller SM: Personal sensing: un-
derstanding mental health using ubiquitous sensors and machine
learning. Annual Review of Clinical Psychology 13:23–47, 2017
56. Ben-Zeev D, Brian R, Wang R, et al: CrossCheck: integrating self-
report, behavioral sensing, and smartphone use to identify digital
indicators of psychotic relapse. Psychiatric Rehabilitation Journal
40:266–275, 2017
57. Matthews M, Abdullah S, Murnane E, et al: Development and
evaluation of a smartphone-based measure of social rhythms for
bipolar disorder. Assessment 23:472–483, 2016
58. Torous J, Roberts LW: The ethical use of mobile health technology
in clinical psychiatry. Journal of Nervous and Mental Disease 205:
4–8, 2017
8ps.psychiatryonline.org PS in Advance
mHEALTH VERSUS CLINIC INTERVENTION FOR SERIOUS MENTAL ILLNESS