Content uploaded by Tobias Kowatsch
Author content
All content in this area was uploaded by Tobias Kowatsch on Sep 28, 2017
Content may be subject to copyright.
Design and Evaluation of a Mobile Chat App
for the Open Source Behavioral Health
Intervention Platform MobileCoach
Tobias Kowatsch
1(&)
, Dirk Volland
2
, Iris Shih
2
, Dominik Rüegger
2
,
Florian Künzler
2
, Filipe Barata
2
, Andreas Filler
1,3
, Dirk Büchter
4
,
Björn Brogle
4
, Katrin Heldt
4
, Pauline Gindrat
5
,
Nathalie Farpour-Lambert
6
, and Dagmar l’Allemand
4
1
Institute of Technology Management,
University of St. Gallen, St. Gallen, Switzerland
tobias.kowatsch@unisg.ch
2
Department of Management, Technology and Economics,
ETH Zurich, Zurich, Switzerland
3
Energy Efficient Systems Group, University of Bamberg, Bamberg, Germany
4
Children’s Hospital of Eastern Switzerland, St. Gallen, Switzerland
5
Fondation SportSmile, Nyon, Switzerland
6
Department of Community Medicine, Primary Care and Emergency,
University Hospital of Geneva/University of Geneva, Geneva, Switzerland
Abstract. The open source platform MobileCoach (mobile-coach.eu) has been
used for various behavioral health interventions in the public health context.
However, so far, MobileCoach is limited to text message-based interactions.
That is, participants use error-prone and laborious text-input fields and have to
bear the SMS costs. Moreover, MobileCoach does not provide a dedicated chat
channel for individual requests beyond the processing capabilities of its chatbot.
Intervention designers are also limited to text-based self-report data. In this
paper, we thus present a mobile chat app with pre-defined answer options, a
dedicated chat channel for patients and health professionals and sensor data
integration for the MobileCoach platform. Results of a pretest (N= 11) and
preliminary findings of a randomized controlled clinical trial (N= 14) with
young patients, who participate in an intervention for the treatment of obesity,
are promising with respect to the utility of the chat app.
Keywords: Health intervention Digital coaching Chat-based interaction
1 Introduction
Non communicable diseases (NCDs) such as heart diseases, asthma, obesity, diabetes
or chronic kidney disease impose the greatest burden on global health [14]. According
to WHO’s NCD global monitoring framework, many of these diseases are conse-
quences of adverse health behaviors, for example, harmful use of alcohol and tobacco
or physical inactivity [15]. However, health personnel is strongly limited [2]. Conse-
quently, scalable behavioral health interventions are required.
©Springer International Publishing AG 2017
A. Maedche et al. (Eds.): DESRIST 2017, LNCS 10243, pp. 485–489, 2017.
DOI: 10.1007/978-3-319-59144-5_36
Innovative digital health interventions (DHIs) have not only the potential to
improve the efficacy of preventive or therapeutic behavioral health interventions but
also to reduce their costs [1]. With the goal to provide an open source platform that
allows health professionals to design scalable, low-cost and evidence-based DHIs,
MobileCoach (mobile-coach.eu) was developed [5] and evaluated [9].
However, it uses the short message service (SMS) for delivering behavioral health
interventions, and thus comes with various shortcomings as outlined in the next
section. We therefore present in this paper the first mobile chat app for the Mobile-
Coach platform that addresses these shortcomings and thus, complements existing
communication such as personal exchange, SMS-based, phone-based or video-based
interactions.
The remainder of this paper is structured as follows. Next, we describe the design of
the chat app. Then, the app’s significance to research and practice is outlined. Finally,
we present results from an empirical study with 11 obese children who assessed the
new chat app as the first target group.
2 Design of the Chat App
Hands-on experience with several MobileCoach-based interventions [7–9] has revealed
four major shortcomings related to its text messaging approach. First, participants have
to bear the SMS costs which may be an entry barrier if the caregiver does not provide a
monetary compensation. Second, participants are always requested to manually type in
text to answer even Likert-scale type questions. These answers are then parsed by the
MobileCoach, which is error-prone in case the answer does not perfectly fit to the
question. Processing these answers is a time-consuming process for the caregiver, too.
Third, participant-initiated requests usually require an individual answer from a care-
giver instead of a scripted answer by a chatbot. A rule-based chatbot does therefore not
always fit to the communication needs of the participants. Fourth, text-messaging is
limited to self-report data, i.e. health professionals cannot use objective sensor data
from a smartphone (e.g., accelerometer data used to measure physical activity) or
sensor data from devices connected to that smartphone (e.g., Bluetooth-enabled blood
glucose or peak flow meters) for the design of their DHIs.
Regarding these shortcomings and against the background of smartphone perva-
siveness [4], the following requirements have been defined: (R1) The app must not rely
on the short message service for communication purposes; (R2) the app must imple-
ment pre-defined answer sets for efficient and error-free chat interaction; (R3) the app
must implement a chat channel for individual communication needs that complements
the scalable chatbot channel; (R4) the app must be able to access sensor data from the
smartphone or smartphone-connected devices.
By considering these four requirements, we built a first mock-up of a mobile app
and evaluated it with six behavioural health experts. As a result of that assessment, a
generic dashboard view was designed. Its purpose is to summarize key statistics of the
envisioned behavioural health interventions for self-monitoring purposes (e.g. steps
achieved per day, intervention progress or goals achieved). Based on this generic
486 T. Kowatsch et al.
mock-up, we implemented a native chat app for Android smartphones for the Mobi-
leCoach platform. Figures 1,2and 3show the graphical user interface of the chat app.
3 Significance to Research and Practice
The mobile chat app presented in this paper allows behavioral scientists and health
professionals to enrich self-report data with objective sensor data in the everyday life of
their clients. This paves the way for a better understanding of whether psychological/
self-report data and physiological/objective data are rather complement or alternative
measures, a recent research question in the field of NeuroIS [13].
Moreover, it is by far not clear how to design and frame chatbots for DHIs (e.g., as
an expert or a “patient like me”) and its interplay with a “physical”caregiver such that
they have a positive effect on the bond between caregiver and their clients and thus,
also on therapeutic outcomes [6]. In contrast to general purpose chat agents such as Siri
(Apple), Alexa (Amazon) or Cortana (Microsoft) and agents with a health focus such as
Florence (getflorence.co.uk), Molly (sense.ly) or Lark (lark.com), our chat app allows
full control of personal health data and a generic framework to manipulate the design
and communication style of chatbots in lab and field settings.
Finally and consistent with the MobileCoach platform, the chat app will be made
open source under the Apache 2.0 license to enable a community-driven design such
that research teams and (business) organizations interested in chat-based digital
coaching approaches do not have to start from scratch but can re-use, revise and
improve the existing code together with the MobileCoach platform.
Fig. 1. Dashboard view, indi-
vidual caregiver chat channel
PathMate and channel with
chatbot Anna
Fig. 2. Chatbot Anna, pre-
defined answer options and
sensor integration; steps are
tracked and used in the chat
Fig. 3. Chat channel with
the caregiver; the PathMate
study team of the children’s
hospital
Mobile Chat App for the Open Source Platform MobileCoach 487
4 Evaluation of the Artifact
Based on prior work demonstrating the acceptance of chat apps by adolescents [11], the
first test of the novel chat app was conducted in a children’s hospital in December 2016
with 11 patients (age
Mean
= 12.6 years, SD = 2.4; 8 girls), who participated in an
intervention for the treatment of obesity. The goal of this test was (1) to assess
enjoyment, ease of use, usefulness and the intention to use the app [10], and (2) to
identify and address major usability problems with the app [12] prior to a randomized
controlled trial (RCT), in which the efficacy of a chat-based six-month DHI for the
treatment of childhood obesity will be compared to a control group.
First, a chat-based DHI was collaboratively designed by computer scientists,
physicians, a psychotherapist, diet and sport experts. The patients were then asked to
select a chatbot of their liking, i.e. they could choose between a female and male
chatbot (Anna or Lukas). Then, they interacted with the bot for 10 min including
various chat-based photo, physical activity and quiz interactions. The patients were
observed during these interactions by a computer scientist and physician. Afterwards,
patients were asked to fill out a questionnaire to assess the app and to provide quali-
tative feedback on their experience with the app. Similar to prior work [10], we
assessed technology perceptions and behavioral intentions with seven-point Likert
scales anchored from strongly disagree (1) to strongly agree (7). As young patients
deserve special consideration, we used single-item measures to reduce the burden of
evaluation [3].
The descriptive statistics of the evaluation are shown in Table 1. Results indicate
that the chat app was perceived positive regarding all four constructs. A sign test
against the neutral Likert-scale median of 4 supports this observation. Finally, we
found no major usability problems based on the observations and the qualitative
feedback.
First findings of the aforementioned RCT show that new young patients assigned to
the chat-based DHI (N= 14) completed successfully approx. 61% of the daily inter-
vention tasks over the first two months. The efficacy of this DHI will be finally
measured by the Body Mass Index after the six-month RCT. We hypothesize that the
chat-based DHI is more effective as the chatbot can provide everyday support on
therapy goals and tasks, thus increasing therapy adherence compared to patients of the
treatment-as-usual control group without everyday support.
Table 1. Descriptive statistics and results of a sign test against the neutral value 4 on a 7-point
Likert-scale (N= 11). Note: Perceived ease of use (PEU), Perceived enjoyment (PEN), Perceived
usefulness (PU) and Intention to use (IU); Significance */**/*** p < .05 /.01 /.001
# Item Mean Median SD p-value
PEU I found the chat easy to use 6.7 7.0 0.7 ***
PEN I enjoyed chatting 6.2 7.0 1.5 **
PU Chatting with Lukas/Anna could motivate me
to accomplish my intervention tasks
5.9 6.0 1.1 *
IU I could imagine chatting daily that way 5.6 6.0 1.4 *
488 T. Kowatsch et al.
In our future work, we will test chat-based DHIs with older patient populations and
different therapies to assess the degree to which our findings can be generalized.
Acknowledgements. We would like to thank the CSS Insurance and the Swiss National Science
Foundation for their support through grants 159289 and 162724.
References
1. Agarwal, R., Gao, G., DesRoches, C., et al.: The digital transformation of healthcare: current
status and the road ahead. Inf. Syst. Res. 21, 796–809 (2010)
2. Aluttis, C., Bishaw, T., Frank, M.W.: The workforce for health in a globalized context –
global shortages and international migration. Glob. Health Action 7(2014). https://www.
ncbi.nlm.nih.gov/pmc/articles/PMC3926986/
3. Bergkvist, L., Rossiter, J.R.: The predictive validity of multiple-item versus single-item
measures of the same constructs. J. Mark. Res. 44, 175–184 (2007)
4. Chaffey, D.: Mobile Marketing Statistics compilation (2016). http://www.smartinsights.com/
. Accessed Feb 2017
5. Filler, A., Kowatsch, T., Haug, S., et al.: MobileCoach: a novel open source platform for the
design of evidence-based, scalable and low-cost behavioral health interventions. In: 14th
Annual Wireless Telecommunications Symposium (WTS 2015), New York (2015)
6. Flückiger, C., Del Re, A.C., Wampold, B.E., et al.: How central is the alliance in
psychotherapy? A multilevel longitudinal meta-analysis. J. Couns. Psychol. 59,10–17
(2012)
7. Haug, S., Kowatsch, T., Paz Castro, R., et al.: Efficacy of a web- and text messaging-based
intervention to reduce problem drinking in young people: study protocol of a
cluster-randomised controlled trial. BMC Public Health 14,1–8 (2014)
8. Haug, S., Paz Castro, R., Filler, A., et al.: Efficacy of an internet and SMS-based integrated
smoking cessation and alcohol intervention for smoking cessation in young people: study
protocol of a two-arm cluster randomised controlled trial. BMC Public Health 14, 1140
(2014)
9. Haug, S., Paz, R., Kowatsch, T., et al.: Efficacy of a web- and text messaging-based
intervention to reduce problem drinking in adolescents: Results of a cluster-randomised
controlled trial. J. Consult. Clin. Psychol. 85, 147–159 (2017)
10. Kamis, A., Koufaris, M., Stern, T.: Using an attribute-based decision support system for
user-customized products online: an experimental investigation. MIS Q. 32, 159–177 (2008)
11. Lilian, S., Gregor, W., Sarah, G., et al.: MIKE - Medien, Interaktion, Kinder, Eltern. Zürcher
Hochschule für Angewandte Wissenschaften, Zürich (2015)
12. Nielsen, J., Landauer, T.K.: A mathematical model of the finding of usability problems. In:
CHI 1993 Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human
Factors in Computing Systems, pp. 206–213. ACM, Amsterdam (1993)
13. Tams, S., Hill, K., de Guinea, A.O., et al.: NeuroIS - alternative or complement to existing
methods? Illustrating the holistic effects of neuroscience and self-reported data in the context
of technostress research. J. Assoc. Inf. Syst. 15, Article 1 (2014)
14. WHO: Global Health and Aging. World Health Organization, Genéve, Switzerland (2011)
15. WHO: Draft comprehensive global monitoring framework and targets for the prevention and
control of noncommunicable diseases. World Health Organization, Genéve, Switzerland
(2013)
Mobile Chat App for the Open Source Platform MobileCoach 489