ArticlePDF Available

Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies for Mood- and Anxiety-Related Problems: MoodMission


Abstract and Figures

Given the ubiquity and interactive power of smartphones, there are opportunities to develop smartphone applications (apps) that provide novel, highly accessible mental health supports. This paper details the development of a smartphone app, “MoodMission,” that aims to provide evidence-based Cognitive Behavior Therapy (CBT) strategies for mood- and anxiety-related problems, contributing to the prevention of clinically significant depression and anxiety disorders and serving as an adjunct to therapeutic interventions delivered by trained health professionals. MoodMission was designed to deliver strategies in the form of real-time, momentary responses to user-reported low moods and anxiety. The development process involved: (a) construction of a battery of strategies, (b) empirical evaluation, (c) a software and behavioral plan design and testing process, (d) user feedback, and (e) a public launch. A pilot study of 44 participants completed the Mobile Application Rating Scale (MARS; Hides et al., 2014) for usability testing and feedback. MoodMission was rated significantly higher than standardized health app norms on the majority of the domains, including Entertainment, Interest, Customization, Target Group, Graphics, Visual Appeal, Quality of Information, Quantity of Information, Visual Information, Credibility of Source, Recommendation to Use, Estimated Frequency of Use, and Overall Rating (Hedges's g range 0.57–1.97, p < .006). Case examples illustrate the practical uses of the app. In addition to clinical applications, MoodMission holds promise as a research tool either as an augmentation to clinician-delivered therapy, or as a vehicle for standardizing client access to specific CBT strategies (e.g., in studies intending to study different change processes).
Content may be subject to copyright.
Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior
Therapy Strategies for Mood- and Anxiety-Related Problems: MoodMission
David Bakker and Nikolaos Kazantzis, Monash University
Debra Rickwood, University of Canberra
Nikki Rickard, Monash University and University of Melbourne
Given the ubiquity and interactive power of smartphones, there are opportunities to develop smartphone applications (apps) that
provide novel, highly accessible mental health supports. This paper details the development of a smartphone app, MoodMission,that
aims to provide evidence-based Cognitive Behavior Therapy (CBT) strategies for mood- and anxiety-related problems, contributing to the
prevention of clinically significant depression and anxiety disorders and serving as an adjunct to therapeutic interventions delivered by
trained health professionals. MoodMission was designed to deliver strategies in the form of real-time, momentary responses to user-
reported low moods and anxiety. The development process involved: (a) construction of a battery of strategies, (b) empirical evaluation,
(c) a software and behavioral plan design and testing process, (d) user feedback, and (e) a public launch. A pilot study of 44
participants completed the Mobile Application Rating Scale (MARS; Hides et al., 2014) for usability testing and feedback.
MoodMission was rated significantly higher than standardized health app norms on the majority of the domains, including
Entertainment, Interest, Customization, Target Group, Graphics, Visual Appeal, Quality of Information, Quantity of Information,
Visual Information, Credibility of Source, Recommendation to Use, Estimated Frequency of Use, and Overall Rating (Hedgess g range
0.571.97, p b.006). Case examples illustrate the practical uses of the app. In addition to clinical applications, MoodMission holds
promise as a research tool either as an augmentation to clinician-delivered therapy, or as a vehicle for standardizing client access to
specific CBT strategies (e.g., in studies intending to study different change processes).
GIVEN the exponential growth in smartphone use
(Deloitte, 2016b), there is a potential to increase
access to and create novel delivery of mental health
interventions. Data collection capability for the expressed
purpose of evaluating the stated health aims and objectives
of smartphone applications (apps) designed for mental
health (MHapps) is an important ethical and practice
consideration (Luxton, McCann, Bush, Mishkind,& Reger,
2011). Despite this opportunity to significantly transform
behavioral health care, recent reviews have found a
significant number of MHapps have been developed
without an empirical base or evaluation capacity (Bakker,
Kazantzis, Rickwood, & Rickard, 2016; Donker et al., 2013;
Jones & Moffitt, 2016). For the continued advancement of
MHapps, there is a need for careful evaluation that includes
user feedback on intended benefits, as well as the overall
interface and design of the platform.
Cognitive behavior therapy (CBT) is an effective
treatment for depression (Cuijpers et al., 2013) and
anxiety (Bolognesi, Baldwin, & Ruini, 2014), and has been
translated successfully for delivery via the Internet for a
range of clinical disorders (Andrews & Williams, 2014;
Dèttore, Pozza, & Andersson, 2015; Kuester, Niemeyer, &
Knaevelsrud, 2016; Newby, Twomey, Yuan Li, & Andrews,
2016). Many Internet CBT (iCBT) programs have been
designed to include techniques that can be flexibly
applied to a range of disorder groups (Păsărelu,
Andersson, Nordgren, & Dobrean, 2017), through their
emphasis on core dimensions in psychopathology and
treatment processes, including: attention and other
processes of cognition (e.g., acceptance, tolerance),
cognitive reappraisal (e.g., decentering, defusion), be-
havior change (e.g., activation, exposure), and emotional
dysregulation (Aldao, Nolen-Hoeksema, & Schweizer,
2010; Hayes & Hofmann, 2017, 2018; and see Kazantzis,
2018, for outline of technique-process links). For exam-
ple, common treatment processes in iCBT for anxiety and
depression (Ellard, Fairholme, Boisseau, Farchione, &
Barlow, 2010) can be reliably facilitated through psychoe-
ducation, self-monitoring of thoughts and emotions,
emotion regulation skills, and relapse prevention (Newby
Keywords: mobile; app; self-guided; depression; anxiety; cognitive
behavior therapy
1077-7229/18/© 2018 Association for Behavioral and Cognitive
Therapies. Published by Elsevier Ltd. All rights reserved.
CBPRA-00724; No of Pages 19: 4C
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Available online at
Cognitive and Behavioral Practice xx (2018) xxx-xxx
et al., 2016). By enabling individuals to learn broadly
relevant skills, such as the ability to identify emotions and
evaluate unhelpful thinking patterns, iCBT has the potential
to be broadly beneficial, both as an augmentation to
therapy, and standalone prevention and therapeutic
In traditional delivery of CBT, a therapist would guide a
patient to a shared understanding of their problems using a
generic cognitive model illustrating the interactive patterns
of cognition, behavior, emotions, and physiology in
problematic situations (Layard & Clark, 2014; Westbrook,
Kennerley, & Kirk, 2011). The basic CBT components
involve the construction of individualized emotion rating
scales (e.g., SUDS) for the evaluation of specific interven-
tions focused on behavior change, and interventions
focused on cognitive change. Cognitive change is posited
as the main change mechanism within standard CBT
(Beck, 2011), traditionally facilitated by techniques that
involve cognitive reappraisal or reframing, but can also
include techniques that focus on acceptance, building
tolerance, decentering, and defusion, among others
(Mennin, Ellard, Fresco, & Gross, 2013;Petrik, Kazantzis,
& Hofmann, 2013). Interventions focused on behavioral
exposure (e.g., in various anxiety disorder treatments) and
activation (e.g., in various mood disorder treatments) can
also facilitate cognitive change processes. Similarly, there is
attention to the process of cognition, such as noticing
themes in content of underlying assumptions and core
beliefs, as well as the information processes that strengthen
maladaptive beliefs and accompanying behavioral strate-
gies. Between sessions, clients practice strategies to consol-
idate cognitive and behavioral changes, but ensuring
engagement can pose unique motivational and practical
challenges (Kazantzis, Deane, & Ronan, 2005).
Despite the conceptual clarity of and empirical support
for CBT, delivery in community settings can often suffer from
low levels of engagement and high treatment dropout
(Fernandez, Salem, Swift, & Ramtahal, 2015). These factors
may relate to the significant lifestyle and other behavioral
changes required by CBT. Thus, maximizing engagement is
necessary to reach the full potential of CBT (Ballegooijen
et al., 2014). Technology can play a helpful role in enhancing
client experience of treatment (Andrews & Williams, 2014),
improving engagement and, in turn, accelerating treatment
response. For example, the between-session practice of
therapeutic skills (or homework) can be recorded, tracked,
symptom severity and improvement (Reger et al., 2013),
though the evidence suggests there is currently an untapped
potential for apps to support monitoring (Kazantzis,
Brownfield, Mosely, Usatoff, & Flighty, 2017). When apps
are used, they provide both a memory aid to complete
homework and have the potential to improve motivation to
experience the benefits of task completion.
A growing literature demonstrates that mobile applica-
tions may be useful adjuncts or modes for the delivery of
psychological interventions (Firth et al., 2017a, 2017b). For
example, Titov et al. (2015) compared four different
variants of iCBT for depression, including either self-
guided or clinician-guided, and transdiagnostic or
disorder-specific. All variants were effective at reducing
depressive symptoms and comorbid anxiety, and there were
no significant differences in effectiveness between the
variants, suggesting that self-guided, transdiagnostic iCBT
can be just as effective as clinician-guided diagnosis-specific
iCBT. Dear et al. (2016) replicated these findings for
participants with social anxiety disorder, noting effects on
comorbid depression, generalized anxiety disorder, and
panic disorder. Meta-analyses have found strong effect sizes
conditions (i.e., self-monitoring, discussion groups) with an
overall superiority in effect size across anxiety and depressive
disorders (i.e., g = .88, 95% CI = .76.99 in Andrews,
Cuijpers, Craske, McEvoy, & Titov, 2010), and in transdiag-
nostic iCBT programs (i.e., g = .84, 95% CI = .671.01 for
depression; g = .78, 95% CI = .57-398 for anxiety, : and g =
.48, 95% CI = .35.61 for quality of life in Newby et al., 2016).
A meta-analysis of 13 studies comparing iCBT to face-to-face
CBT found equivalence between the two modes of therapy
typically designed for use on personal computers, and may
not be well suited to delivery via smartphones.
Individuals interact with smartphones and with personal
computers in different ways. For example, a persons
interaction with a therapy intervention delivered by
computer in the privacy of theirownhomeisverydifferent
from their public use of a smartphone in transit from one
location to another. Smartphone use generally involves a
greater number of shorter, more momentary interactions
than personal computer use, across a greater number of
situations and settings (Deloitte, 2016c). Available data
suggest Americans may check their phone 46 times a day
(Deloitte, 2016a) and engage with their smartphone more
than computers (i.e., in Google, 2016170 min through
the day; 12 min in the evening). Accordingly, there are
design and feasibility issues for both the therapeutic
intervention being delivered via Internet or smartphone,
and the way the user interface is designed (Wendel, 2013).
Although some evidence suggests that CBT programs can be
delivered effectively and equivalently via either computer or
smartphone (Watts et al., 2013), further evaluating the
extent to which CBT interventions can be effectively
delivered via smartphone applications is one of the
augmented and -delivered CBT can be enhanced.
Few available MHapps have been specifically evaluated
for their effectiveness as a mode of delivery, even if their
techniques and strategies are based on a body of evidence
2Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
(Bakker et al., 2016; Donker et al., 2013; Van Ameringen,
Turna, Khalesi, Pullia, & Patterson, 2017). It is possible,
for example, that some CBT strategies are effective when
completed under the guidance of a therapist, but when
incorporated into an MHapp, unique challenges to
smartphone apps may limit their utility. Similarly,
smartphone apps may afford some CBT strategies greater
flexibility and enhance their effects. For example,
behavioral activation strategies and scheduling pleasant
activities may be more accurately self-monitored when
tracked using a self-guided MHapp, but recording and
reappraising negative automatic thoughts may require
initial introduction and skill acquisition in a private
learning environment or therapy setting. Thus, as with
iCBT, there is a need for research to address the gap that
exists regarding CBT delivered via MHapps.
This paper will outline the development and pilot
evaluation of a new MHapp, called MoodMission,for the
delivery of CBT strategies for managing mood- and anxiety-
related problems. Details are given to aid practitioners in
their understanding and use of the app with clients, and to
inform the development of future apps by practitioners.
Current Evidence for Smartphone Delivered CBT
Compared to other modes of delivery, such as group,
phone, or Internet-delivered CBT, there is currently a
scarcity of published experimental evidence investigating
the outcomes of MHapps using CBT strategies (Donker
et al., 2013; Grist, Porter, & Stallard, 2017; Olff, 2015). While
several studies have found evidence for the efficacy of
MHapp interventions in the acquisition of specific CBT skills
(e.g. Franklin et al., 2016; Kauer et al., 2012; Roepke et al.,
2015), their limitations suggest the need for more research.
Studies that have investigated MHapps have tended to
focus on relatively narrow clinical applications, or have used
methods that do not represent typical smartphone use. For
example, Roepke et al. (2015) found that the use of two
different versions of the MHapp SuperBetterdecreased
depression symptoms in participants experiencing depres-
sion, compared with a waitlist control condition. However,
the exclusion of participants who were not experiencing
clinical distress obstructed investigation of preventative
utility. Similarly, Kauer et al. (2012) found that the use of
an app to track mental-health-related variables increased
emotional self-awareness (ESA) and reduced depressive
symptoms in depressed participants when compared to a
control group. However, participants did not download the
app to their own phones, but instead were given a device
with it installed, and they reviewed the self-monitoring data
they had collected with their doctor at several time points.
While this suggests that MHapps can have impacts on
depression and ESA, it does not demonstrate the effective-
ness of MHapps in naturalistic self-guided applications.
Franklin et al. (2016) designed a therapeutic evaluative
conditioning (TEC) MHapp which was effective at reducing
self-cutting episodes, suicidal behaviors, and suicide plans.
While this suggests an efficacious intervention, it was for a
narrow clinical purpose and 76% of participants reported a
history of psychiatric treatment. A publicly available MHapp
like this would have significant barriers to access, as the vast
majority of smartphone users would not consider down-
loading it, let alone using it outside of a research study
context. There is a need to study MHapps designed for
nonclinical populations, as these represent the target of the
majority of MHapps available. The present study aimed to
fill this gap in the literature by developing a publicly, freely
available MHapp that was useful for individuals of all mental
health statuses, adhered to latest evidence-based guidelines,
and was capable of collecting data for further experimental
Many existing MHapps have been designed for adoles-
cent and young adult users (e.g., RaysNightOut[Hides
et al., 2015], WorryTime [ReachOut, 2016]). This is
understandable given the high ownership rates in these
demographics, with up to 98% of 18- to 24-year-olds owning
asmartphone(Nielsen, 2016). However, high ownership
rates are observed in other age demographics, with 97%
of 25- to 34-year-olds, 96% of 35- to 44-year-olds, and 89% of
45- to 54-year-olds owning a smartphone. Furthermore, in a
survey of 100 psychiatric outpatients, those aged 3045 were
more likely to want to use MHapps (81%) than those under
30 years of age (78%; Torous, Friedman, & Keshavan, 2014),
and a significant proportion of patients aged 4560 years
also expressed interest in using a MHapp (71%). To meet
this demand, future MHapps should be designed for
smartphone owners of all ages.
It is important for MHapps to undergo usability testing
before public release (Dubad, Winsper, Meyer, Livanou,
& Marwaha, 2017). The technical justification for usability
testing is to ensure that the intervention works reliably
across multiple unique devices and under a variety of
usage scenarios (Jaspers, 2009). Furthermore, novel self-
guided CBT interventions require prerelease testing to
ensure that their therapeutic aims are being achieved
through the hypothesized mechanisms (Kinderman et al.,
2016). This is to mitigate against the risk that the MHapp
is used in an unintended manner and gives the developers
the opportunity to adjust the intervention to realign its
usage patterns with the therapeutic aims.
The MoodMission Smartphone Application
Reviewing the MHapp marketplace and literature, a gap
was noted that could be filled by an app that would meet all
16 of Bakker et al.s (2016) recommendations. This app
and help give direction to the question, Im low/anxious,
what can I do right now to help?The designed app was
3Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
intended to provide individuals with discrete, mission-
based coping solutions for mood problems, so was given the
name MoodMission.MoodMission was designed to be an
easy-to-use and engaging application to enhance mental
health and well-being for smartphone users of all ages,
adolescents and older, and all mental health statuses. It was a
crowd-funded project specifically designed to be a noncom-
mercial product and included comprehensive data collec-
tion strategies to facilitate evaluation. Research using this app
could investigate the utility of various, discrete strategies for
reducing specific types of distress, and, more broadly, the
utility of similar MHapps across a range of clinical and
nonclinical contexts.
CBT Strategies
MoodMission was designed with three primary aims:
(a) to provide self-administered prevention and self-help
strategies to reduce the risk of clinically significant mood and
anxiety disorders; (b) to support stepped-care interventions
2010) as a platform for access to low-intensity intervention for
low-level clinical symptoms or subclinical symptoms of
depression and anxiety; and (c) as an adjunct to psychother-
apy or other face-to-face treatments for mood and anxiety
disorders. The strategies, called Missions,contained within
MoodMission originate from behavioral activation (Dahne,
Kustanowitz, & Lejuez, 2017; Mazzucchelli, Kane, & Rees,
2009), relaxation (Manzoni, Pagnini, Castelnuovo, &
Molinari, 2008), mindfulness (Hofmann, Sawyer, Witt, &
Oh, 2010; Shipherd & Fordiani, 2015), physical exercise
(Cooney et al., 2013), cognitive reframing (Butler,
Chapman, Forman, & Beck, 2006), and other activities
promoted or supported by CBT. Missions are designed
specifically to be appropriate for a self-guided smartphone
app, and due to the user-friendly interface, do not require
introduction by a therapist. Time and environment con-
straints are also considered, with each Mission achievable
within 510 minutes and in most public or private spaces.
Missions were only included if they were suitable for
adolescents as well as older users. Example Missions can be
seen in screenshots presented in Figure 2.
Researchers aim to pursue controlled trials for empirical
validation of MoodMission, so the app has built-in data
collection, which will be collated and analyzed in time. The
current paper reports on the development process and
usability testing, presenting app functionality and data on
user experience.
Interface Design and Engagement
A central consideration to the design of an MHapp is its
appeal to the individual using it, which may transfer to how
engaged they are in the intervention. Engagement may be
defined in different ways; for example, many iCBT studies
report retention rates as a measure of how many participants
did not complete all the interventionsstages. Studies in
face-to-face CBT have assessed the amount of therapeutic
interventions completed by clients, also referred to as
homework(Kazantzis et al., 2016). However, engagement
is a broader concept than adherence or compliance and
takes into consideration the degree of difficulty and
obstacles experienced by the individual in attempting the
intervention (Holdsworth,Bowen,Brown,&Howat,2014).
MHapps may facilitate engagement, not only through ease
of access to information, automated recorded and tallying of
responses, but through their interface, and the potential for
improved assessment of engagement that would not be
possible through paper-and-pencil worksheet completion
(Kazantzis et al., 2017).
Game-inspired mechanics, sometimes referred to as
gamification, can improve user engagement and under-
standing in eHealth interventions (Comello et al., 2016).
This can be as simple as tracking the number of minutes
spent meditating (example apps include Headspace,
2015; Smiling Mind, 2015). Gamification can be under-
stood using self-determination theory (SDT), which
emphasizes the roles of perceived autonomy and mastery
on intrinsic motivation (Ryan & Deci, 2000). For example,
the MHapp SuperBetter (SuperBetter Inc, 2014) awards
users resiliencepoints for completing short activities,
helping users quantify and reflect on achievements
(Roepke et al., 2015). SDT principles have previously
been considered when improving client engagement and
therapeutic outcomes in CBT (Tee & Kazantzis, 2011).
Gamification harnesses the same principles to improve
engagement with an app, but of the 27 MHapps in the
Bakker et al. (2016) review, only 19% included gamification.
Recently published recommendations (Bakker et al.,
2016) informed the development of MoodMission.
Care was taken to keep MoodMissions design simple,
easy to use, and with a distinct purpose so smartphone
users, including adolescents and older adults, would
be able to understand how to engage with it. Inclusive
design cues were taken from broadly accessible apps, such
as those that come standard on smartphone operating
systems, as these apps are designed to be used by all
potential users.
The formation of a behavioral plan for MoodMission
was a dynamic process to accommodate as many
recommendations as possible without overcomplicating
the usersengagement. The Hook model of user-centered
design proposed by Eyal (2014) was used to establish
triggers for engagement, the actions involved in engage-
ment, variable rewards, and generation of investment. As
outlined in Table 1, a pre-intervention engagement plan
was formulated to cover the pathway towards engagement
with the intervention. Table 2 lists how MoodMissions
behavioral plan accommodated Bakker et al.s (2016)
4Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
recommendations, and the phases of the behavioral plan
are displayed in Table 3. The end goal of the behavioral
plan was to encourage repeated use of MoodMission so a
positive habit of use forms. This is important for
MoodMission to achieve its goal of enhancing an
individuals repertoire of useful strategies for overcoming
low moods and anxiety in a variety of contexts. It is
expected that repeated use will lead to more learning
opportunities. The pre-intervention surveys mentioned in
Table 1 are detailed in Table 4.
Following formulation of a behavioral plan, the plan
was converted into a series of diagrammatic wireframes
that set out exactly how the individual would interact with
the app through the various screens. Figure 1 shows one
of the initial wireframes generated for MoodMission.
These were used to approach app development firms and
collect proposals and quotes. Spark Digital was the firm
chosen to build and launch the app, based on their
proposal and experience developing the MHapp Smiling
Mind (2015).
Table 1
Preintervention Engagement Plan for MoodMission
Stage Details
Promotionuser learns of MoodMission Potential avenues included:
1. Web search is likely to be used by individuals looking for self-guided mental
health support. A well-designed website, search engine optimization, and options
such as Google AdWords (Dirmaier, Liebherz, Sänger, Härter, & Tlach, 2016) may
increase the visibility of MoodMission in these searches.
2. Healthcare providers such as general practitioners, psychologists, social
workers, or community health organisations, may suggest self-guided support
options when they have contact with clients or patients suffering from preclinical
or clinical psychological disorders (Bower & Gilbody, 2005). MoodMission
can be promoted to healthcare providers, who can then make professional
recommendations to suitable clients and patients.
3. Users promoting via their own online social networks on sites like Facebook and
Twitter can provide dissemination of products and ideas, especially ones that are
highly viral (Weng, Menczer, & Ahn, 2013). Curation of official Facebook and
Twitter pages can enable sharing.
4. Crowdfunding supporters pledged money in return for rewards to raise the
funds for the initial development of MoodMission. Pledgers are invested in the
success of the projects they support and are likely to promote them among their
own communities (Belleflamme, Lambert, & Schwienbacher, 2014).
Accessuser downloads MoodMission Considerations included:
1. Platform: iPhone development was more efficient to complete, so Android
development was postponed until it could be financed.
2. App Store Category: The vast majority of MHapps can be found in Health &
Fitness rather than Medical,Lifestyle, or others.
3. Price: Apps that are free to download are more likely to be accessed than ones
that require an initial payment, even if it is a very small fee (Garg & Telang, 2014).
There are models of revenue that do not rely on payments for downloads,
including the subscription models, freemium models, and in-app purchases
(Lambrecht et al., 2014). Adopting these free-to-download revenue models at a
later date can help cover ongoing development and maintenance costs, while
keeping accessibility high.
Onboardingintroducing MoodMissions interface
and purpose
The onboarding process can enhance the users understanding of the app and
hence their engagement (Sian Morson, 2015). Five screens of images and text
were devised to orient users to the triggers, actions, and rewards associated with
MoodMissions use, and emphasize that MoodMission does not replace
professional help and users should consult a GP, psychologist, or mental health
professional for more support.
Completion of pre-intervention surveys While compulsory completion of thesesurveyswas predicted to be a potential barrier
to further engagement with the app, collection of the survey data was necessary for
experimental validation. A rationale was provided to users to encourage persistence
through the surveys, emphasizing the short time that each survey would take, and the
contribution the user would be making to important research.
5Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
The researcherswireframes were clarified and expand-
ed upon by the developers, and once confirmed by all
parties, MoodMissions screens were graphically designed
to be attractive and engaging. Design inspiration was taken
from several other successful apps, such as the bright color
gradients of Vent (2017), and the simple home screen of
Table 2
How MoodMissions Features Adhere to the Recommendations Made by Bakker et al. (2016)
Recommendation from Bakker et al. (2016) Use in MoodMission
1 Cognitive behavior
therapy (CBT) based
Start with an evidence-based framework to
maximize effectiveness.
Uses a CBT-based system for categorization of
Missions. Many Missions have origins in CBT.
2 Address both anxiety
and low mood
Increases accessibility and addresses
comorbidity between anxiety and depression.
Also compatible with transdiagnostic
theories of anxiety and depression.
Trigger for access is when users feel low or
anxious. Treats anxiety and low mood as two
different types of emotional distress.
3 Designed for use by
nonclinical populations
Avoiding diagnostic labels reduces stigma,
increases accessibility, and enables
preventative use.
No diagnostic labels used. Emphasizes the
normality of low moods and anxiety.
4 Automated tailoring Tailored interventions are more efficacious
than is rigid self-help.
MoodMission learns userscoping styles by noting
which categories of Missions reduce distress for
each category of problems.
5 Reporting of thoughts,
feelings, or behaviors
Self-monitoring and self-reflection to
promote psychological growth and enable
progress evaluation.
Users select whether they are experiencing
distressing thoughts, feelings, behaviors, or
physiological responses. They then rate their
distress on a scale 0-10.
6 Recommend activities Behavioral activation to boost self-efficacy
and repertoire of coping skills.
Five activities are suggested based on the users
7 Mental health
Develop mental health literacy. Missions contain a Why This Helpssection,
providing psychoeducation and a rationale for
doing the Mission.
8 Real-time engagement Allows users to use in moments in which
they are experiencing distress for optimum
benefits of coping behaviors and
relaxation techniques.
Trigger for engagement is real-time distress.
Missions are designed to be real-time coping
strategies, achievable in a wide variety of
9 Activities explicitly
linked to specific
reported mood
Enhances understanding of cause-and-effect
relationship between actions and emotions.
Missions are selected for specific mood problems
and rationale is explained in Why This Helps
10 Encourage nontechnology-
based activities
Helps to avoid potential problems with
attention, increase opportunities for
mindfulness, and limit time spent on devices.
Missions are designed to be nontechnology-based.
11 Gamification and
intrinsic motivation
to engage
Encourage use of the app via rewards and
internal triggers, and positive reinforcement
and behavioral conditioning. Also links
with flourishing.
Badge-based rewards structure for completing
certain achievements. Rank-based rewards for
completing more Missions.
12 Log of past app use Encourage use of the app through personal
investment. Internal triggers for
repeated engagement.
Mission Log documents all past Missions
attempted in detail.
13 Reminders to engage External triggers for engagement. Push notifications alert users when they have
incomplete Missions or when they have not
engaged with MoodMission recently.
14 Simple and intuitive
interface and interactions
Reduce confusion and disengagement
in users.
Behavioral plan designed to be linear and intuitive.
Clean graphic design reduces confusion.
15 Links to crisis support
Helps users who are in crisis to seek help. Link to Lifeline and other supports available
throughout app.
16 Experimental trials It is important to establish the apps own
efficacy and effectiveness before
recommending it as an intervention
Randomized controlled trial planned to compare
MoodMission against waitlist and other MHapps.
6Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Table 3
Outline of MoodMissions Behavioral Plan
Phase Details MoodMission key information
Trigger How the user will be motivated to open
MoodMission for mental health purposes
Triggered by distress associated with low mood
or anxiety
Action What the user will do within MoodMission once
they open it and start engagement
User inputs information about their distress and
is provided with a list of coping activities
(Missions) to choose from
Rewards What reinforcements will incline the user to
maintain their engagement with MoodMission
Gamified rewards are issued based on
completion of Missions. User gains a sense of
accomplishment and autonomy, and their overall
distress is decreased
Generation of investment How the reinforcements will lead to repeated
engagements with MoodMission over time
Only five Missions are presented with every
engagement, drawn from a much larger
database. This keeps each engagement fresh
and the user is constantly discovering new
MoodMission learns a users coping style so
more engagements will lead to better Mission
Users pair MoodMissions use with trigger and
will seek to engage with MoodMission under
future episodes of distress.
Table 4
Outcome Surveys Administered in MoodMission
Construct Measured Name of Measure Reference Rationale
Emotional Self-Awareness Emotional Self-Awareness
(Kauer et al., 2012) Assess how reflection-focused
MoodMission is (see Bakker
et al., 2016)
Mental Health Literacy Mental Health Literacy
No appropriate standardized MHL
measure exists, so this
questionnaire has been developed
by the researchers.
Assess quality of psychoeducation
and how education-focused
MoodMission is (see Bakker et al.,
Coping Self-Efficacy Coping Self-Efficacy Scale (Chesney, Neilands, Chambers,
Taylor, & Folkman, 2006)
Assess how goal-focused
MoodMission is (see Bakker et al.,
Emotional Mental Health GAD-7
(Spitzer, Kroenke, Williams, &
Löwe, 2006; Kroenke, Spitzer,
& Williams, 2001)
Assess anxiety and depression
Positive Well-being Warwick-Edinburgh Mental
Well-being Scale
(Tennant et al., 2007) Assess positive psychological
functioning and flourishing
(self-devised) (self-devised) Collect information about users
gender, age, education, and
employment status for the benefit
of analyses
App Feedback and
Feedback Questionnaire Adapted from the Mobile
Application Rating Scale (MARS;
Hides et al., 2014)byRickard,
Arjmand, Bakker, & Seabrook
Assess how engaged users are
with the app, enabling analyses
that correlate engagement with
mental health and wellbeing
Administered only in the initial surveys.
Administered only in the 30 day follow-up surveys.
7Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Pacifica (2016). Several design iterations were made before
the researchers and developers confirmed each screen for
the prototyping stage. Figure 2 illustrates several screens
from MoodMission that demonstrate these designs.
MoodMission Usability Evaluation
Once the apps designs were confirmed, a prototype of
the app was coded and made available to the developers
for initial testing. This process included ensuring that all
components from the designs were included, that the
interactions proved pleasant and congruent, and that no
significant bugs or errors were present. To test the app
before it was launched on the App Store, a group of 60
beta-testerswere recruited to use a prerelease version of
MoodMission and provide feedback.
MoodMission was designed to be used in-vivo as
participants experienced low moods or anxious feelings
throughout their daily lives. This limited the utility of any
laboratory-based usability testing methods, in which users
engage with the intervention under studied laboratory
settings (Jaspers, 2009). Collecting qualitative user reflec-
tions and responses on validated self-report usability
measures about their experiences using the app was a
highly scalable and intervention-appropriate option for
informing MoodMissions initial development and ongo-
ing improvements.
The Mobile Application Rating Scale (MARS; Hides
et al., 2014; Stoyanov, Hides, Kavanagh, & Wilson, 2016),
which was developed to rate mobile health applications,
was used as a guide and foundation to determine usability.
The guidelines for mobile health (mHealth) evidence
reporting and assessment (mERA) checklist (Agarwal
et al., 2016) was consulted to ensure that the apps
implementation was rigorous and transparently reported.
Participants and Recruitment
A total of 44 participants provided feedback about their
use of MoodMission. Of these, 13 were beta-testers who were
given access to the app before its public release, and 31 were
participants in a randomized controlled trial (RCT) who had
downloaded MoodMission from the iTunes Store. The beta-
testers consisted of individuals who had pledged funds to the
crowd-funding campaign that supported the initial develop-
ment of the iOS app. The RCT participants had voluntarily
opted in to a study on MHapps by providing their email
address on an online form that had been advertised widely
on social media. Like the beta-testers, RCT participants were
asked to use MoodMission for the next 30 days before
providing feedback as part of an online survey assessment.
Participants were not drawn from clinical sources. Ages
ranged between 18 and 62 years (M=36,SD = 13), and 82%
were female.
The User Version of the MARS (uMARS; Stoyanov et al.,
2016) is a 26-item measure designed to rate mobile health
applications in a standardized, multidimensional way, and is
designed for end users rather than experts. Items are rated
on a 5-point scale from Inadequate to Excellent, and are
classified under six subscales, including Engagement (e.g., is
the app interesting to use? Does it present its information in
an interesting way compared to other similar apps?),
Figure 1. MoodMission early development wireframe
8Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Functionality (e.g., How easy is it to learn how to use theapp?
How clear are the menu labels, icons and instructions?),
Aesthetics (e.g., How good does the app look?), Information
(e.g., is the information within the app comprehensive but
concise?), Subjective Quality, and App-Specific. Norms for
the MARS were developed by analyzing ratings for 50 mental
health and well-being apps from two expert raters (Hides
et al., 2014). Comparing obtained MARS ratings to these
scores enables comparison to existing standards for MHapps.
The uMARS has high internal consistency, Cronbachsα=
.90, and good test-retest reliability, interclass correlation
coefficient = .70 after 3 months.
The Homework Rating ScaleMobile Application
Version (HRS-MA; Bakker & Kazantzis, 2017) is a 12-
item self-report scale designed to assess engagement and
theoretically derived appraisals of CBT strategies used or
recommended by MHapps. The HRS-MA contains 12
items (e.g., Quantity: I was able to do the activities;
Rationale: The reasons for doing the activities were clear
to me), closely modeled on the original HRS (Kazantzis
et al., 2005), rated on a 5-point Likert scale from 0 (not at
all)to4(completely/extensive/ extremely). In the present
study, the HRS-MA achieved acceptable levels of internal
consistency, Cronbachsα= .77, comparable to the
original (i.e., Cronbachsα= .85; Hara, Aviram,
Constantino, Westra, & Antony, 2017).
App Design and ContentmERA Checklist
Access of Individual Participants
The use of nonclinical language in MoodMission is
designed to increase the accessibility of the app to individuals
Figure 2. MoodMission sample screens displaying the apps behavioral plan
9Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
who do not identify with having a diagnosed mental illness.
However, to be motivated to download the app individuals
still have to identify that they have occasional low moods or
anxious feelings, and that strategies can help. These are
potential barriers to access, so promotional efforts are aimed
at reducing them by conveying the normalization message
that everyone has low moods and anxious feelingsand
there are interventions that can help. For example, flyers for
the app featured the slogan change the way you feel,and
social media posts used inclusive, normalizing language to
encourage a me tooreaction and sharing of posts within
individualssocial networks.
Cost Assessment
MoodMission is free to download. The costs of
maintaining the app, including server fees and developer
updates, equate to about AUDD150 per month. Zero
equity funding from a start-up accelerator program
has been secured to cover these costs for the next
1224 months. This funding will also support the
development of additional features, which will be released
as discrete, affordable in-app purchases to secure a self-
sustaining revenue stream.
Adoption Inputs/Program Entry
MoodMission is designed to be used by novice,
untrained users after downloading directly from the
App Store. A series of onboardingscreens educates
the individual on the uses of the app. The app was
promoted through social media channels, featured online
articles, radio interviews, communications from Monash
University, and blog posts. Care was taken to ensure that
MoodMissions website was well designed and made
downloading the app very simple, as this was the site
linked to from other online promotions. Sending users
notifications based on depression-screening measures can
enable help-seeking in individuals who would not
otherwise seek help (BinDhim et al., 2016), so a
notification system was used to suggest mental health
contacts and services to individuals who scored above
clinical cut-offs on the depression and anxiety measures.
These notifications were also delivered when individuals
attempted three Missions at high distress and their
distress did not significantly decrease following the
Limitations for Delivery at Scale
As a completely automated platform, MoodMission is
highly accessible at scale. The main limitation to truly
global scale is the use of written language and currently
only English is supported. As the intervention garners
support, the developers and researchers hope that
translations can be achieved and multiple languages
Contextual Adaptability
The Missions recommended by MoodMission are
designed to be achievable across many settings and
contexts, and they each take about 510 minutes to
complete. However, many Missions may not be suitable
for contexts where behavior is restricted; for example,
when an individual is unable to practice a quick yoga
move or go for a walk around the block. Offering a choice
of 510 Missions overcomes this.
Please refer to Figures 1 and 2 for a detailed account of
the intervention, which may aid in replicability.
Data Security
All data collected by MoodMission are deidentified.
Login details, including an email address and password,
are stored unlinked to other user data, including survey
answers and Mission data. User data are stored using a
Firebase backend and hosted on Googles infrastructure.
The app WordPress backend is hosted on an Amazon
Web Services ec2 server with Linux.
Compliance With National Guidelines or Regulatory Statutes
At the time of writing, there is no regulatory system for
MHapps. MoodMissions design has endeavored to follow
all current evidence-based recommendations (e.g.,
Bakker et al., 2016).
Fidelity of the Intervention. Dummy accounts were
created throughout the testing process to ensure that
interactions with the app were being accurately recorded
in the backend database. The results presented in this
article provide evidence for the utility of MoodMission.
Infrastructure (Population Level). MoodMission is aimed
at engaging typical smartphone users over the age of 12,
and 77% of the U.S. population (comScore, 2015), 79%
of Australians (Deloitte, 2016b), and 81% of adults in the
U.K. (Deloitte, 2016c) use a smartphone. A survey of
Australians revealed that 76% of adult smartphone
owners were interested in using MHapps if they are free
to download (Proudfoot et al., 2010).
Technology Platform. MoodMission was initially devel-
oped as an iPhone app for iOS 9 and above. It was coded
using hybrid mobile app development and uses a
WordPress backend, enabling more streamlined cross-
platform development than using native coding. Devel-
opment of an Android version of the app occurred after
the successful launch of the iOS version. MoodMission is
now available on both iOS and Android platforms.
Interoperability/ Health Information Systems (HIS)
Context. The current version of MoodMission offers no
10 Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
direct integration into existing health systems. However,
future proposed developments for the app include a
platform for psychologists and mental health practitioners
to engage with patients and clients through the app.
Intervention Delivery. Individuals access MoodMission
when they identify that they are feeling low or anxious.
They report how they are feeling and are supplied with
510 Mission options. They can review the objectives and
rationale for each Mission before accepting it. Following
completing the Mission, they again rate how they feel. See
Figure 2 for an illustration of this process. While it is
possible that individuals may not experience a reduction
in distress following a Mission, several design choices were
made to help prevent a loss of confidence and subsequent
disengagement. Firstly, Missions are not framed as
definitive solutions, and are instead suggested as activities
that may help out. Second, the large diversity of Missions is
intended to give individuals hope that there are many
options for coping. Third, care was taken to avoid
impressions of expected results, so for example, badges
and ranks are awarded for completing Missions ratherthan
experiencing decreases in distress. Finally, push notifica-
tions are sent to individuals who have stopped using the app
after a few days to encourage them to reengage.
Intervention Content. All Missions included in
MoodMission have been taken from evidence-based
psychotherapies, including but not limited to CBT,
acceptance and commitment therapy (ACT; Brown,
Glendenning, Hoon, & John, 2016), and dialectical
behavior therapy (DBT; Kliem, Kroger, & Kosfelder,
2010). For a Mission to be included, it was required to
have at least two good quality sources that established it as
an effective strategy for decreasing anxious or depressive
symptoms. For example, some Missions are drawn from
behavioral activation, which has substantial evidence as an
effective treatment for depression from a meta-analysis of
34 studies (Pooled effect size = 0.78, Mazzucchelli et al.,
2009). Another meta-analysis of 20 studies found
significant effects on improving psychological well-
being (Pooled effect size = 0.52; Mazzucchelli, Kane, &
Rees, 2010). Behavioral activation is made up of many
strategies which could be appropriately translated into
the MoodMission format, so reliable therapy resources
were consulted to extract individual Missions from lists of
behavioral activation strategies (e.g., G. Bakker, 2008;
Dobson & Dobson, 2009). Missions in the database were
classified as either anxious or depressive, depending on
what evidence was available. Classification of Missions
determined which were offered when individuals select-
ed their current problem; e.g. only Missions classified as
anxious were offered when the individual reported that
they were anxious, nervous, or worried.
Beta-testing participants downloaded MoodMission to
their personal iPhones using app-testing software, before
the public release of the app. The other RCT participants
downloaded MoodMission from the iTunes Store. Partici-
pants were not instructed how to use MoodMission by the
researchers, in order to replicate the circumstances by
which individuals would naturally access the app through-
out their daily routines. They were encouraged to email the
researchers if they encountered technical difficulties, such
as the app freezing or buttons being unresponsive. Care was
taken to be responsive to these emails, and participants
were updated about their reported issues and the
subsequent software updates that fixed them. Several
updates were released for the app throughout both beta-
testing and public release phases, but these fixed small
technical errors and none altered overall design or
functionality. Thirty days after downloading MoodMission
participants were emailed with a link to complete a
feedback survey, administered by online platform Qual-
trics. No identifying information was provided in this survey
and because it was administered separately to the app-based
surveys, data collected via MoodMission was unable to be
paired to online survey responses. The online survey
contained the uMARS, the HRS-MA, and several optional
text-entry questions relating to the apps specific features.
Following completion of the survey, participants were
thanked for their time and were encouraged to provide
any additional feedback via online form or email.
Quantitative Results for Usability Analysis
Scores obtained on the MARS from 44 users, including
13 beta-testers and 31 study participants, were compared
to the established norms (Hides et al., 2014). MoodMis-
sion scored higher than the MARS norms across multiple
items, as seen in Figure 3 and Table 5.
Twenty Bonferroni-corrected two-way independent
samples t-tests (α= .0025) were performed for each
comparison to establish significance, and Table 6 displays
the results. Hedges's gwas used as a measure of effect size,
given the unequal sample sizes between the normative
data and the collected data (Cumming, 2011). MoodMis-
sion scored significantly higher than the norms on the
following items: Entertainment (g= 0.89), Interest (g=
1.26), Customization (g= 0.57), Target Group (g= 0.75),
Graphics (g= 0.97), Visual Appeal (g= 1.02), Quality of
Information (g= 0.94), Quantity of Information (g=
1.14), Visual Information (g= 1.93), Credibility of Source
(g= 1.97), Recommendation to Use (g= 1.19), Estimated
Frequency of Use (g= 0.63), and Overall Rating (g= 1.10).
No significant differences from the norm ratings were
observed in the remaining items.
11Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Twenty-three participants completed the HRS-MA
following the MARS: 96% reported that they were able
to do some or more of the activities; 91% reported that
they were able do the activities moderately well or better;
65% found the activities not at all or somewhat difficult,
and 26% rated them as moderately difficult; 70%
reported that they had no or little obstacles in doing the
activities, and 22% had some obstacles; 96% reported that
they understood the activities a lot or completely; 61%
understood the rationale for the activities very or
completely, with the remaining 39% understanding the
rationale moderately; 78% reported that they had some, a
lot, or extensive collaboration in planning the activities;
61% reported that the guidelines for carrying out the
activities were very or extremely specific, and a further
30% reported that they were moderately specific; 43%
agreed a lot or completely that the activities matched their
goals for using the app, 30% agreed somewhat, and 22%
agreed a little; 43% enjoyed the activities a lot or
extremely, 30% enjoyed them somewhat, and 13%
enjoyed them a little; 66% reported that the activities
helped them somewhat, a lot, or extensively gain a sense
of control over their problems, and 22% reported a little
gain in control over their problems; and 78% reported
that the activities helped somewhat, a lot, or extremely
with their progress in using the app.
Qualitative Results
Of the 44 participants, 20 (12 beta-testers and 8 RCT
participants) provided qualitative feedback via text-entry
responses on the online surveys or email. Several themes
from these messages were noted, and feedback-informed
improvements were made to the app.
Participants commented on the length of the surveys
and the difficulty they had in feeling motivated to
complete them. Some participants also had problems
with the interface, as sliders and buttons behaved
unpredictably. The interface issues were tuned and
priority was given to removal of the surveys following
the research phase of the apps development.
Participants reported that they enjoyed the Missions,
particularly their speed, ease, and emotional relevance.
Figure 3. MoodMission MARS scores compared to norms. Error bars represent pooled SD, giving an estimation of 95% confidence intervals
with within-subject variance removed. *pb.0025 (Bonferroni corrected pvalue)
12 Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Participants who had experience with mental health
services reported that the app helped them discover new
mental health strategies or reinforced existing ones.
Suggestions were noted to make the Missions more
interactive and illustrative (e.g., on-screen animations to
guide breathing exercises).
Participants reported that they liked the visual and
interface design of the app. The Achievements and Stats
features were not well understood, so plans were made to
make these features more accessible. Some persisting
bugs were noted and plans made to rectify them in future
Overall Impressions
Participants reported that using the app made them
more conscious of their mental health, and using
nonclinical language was helpful in making the apps
messages accessible. However, others suggested that the
lack of clinical language avoided addressing the stigma
surrounding psychiatric or medication terms, and includ-
ing psychoeducation about biological and chemical
components of mental health problems may help
individuals. One participant left the following feedback:
It is really comforting to know that I have MoodMission
on my phone as a resource when things get tough. I know
its not connecting me with a real professional/someone
to actually talk to in real time but it serves the equally
valuable purpose of just being able to find some helpful
strategies to approach challenging times without bother-
ing friends at 1am in the morning; an app doesnt have
bed timenor is it judgmental. Oddly, in that respect,
you can trust an app and feel free to use it anytime. I want
to keep using it and finish my goals.
Case Examples
Two constructed case examples are presented below,
based on participant reports and the vignettes that
informed the apps design process. Each reflects how
the app can be used in a different context, but it should be
noted that the app can be used in many more contexts
than just these listed. For example, the app could be used
in group therapy contexts, in hospitals, or in workplaces.
Case 1: Use in Individual Therapy
Jakeis a 25-year-old male undergoing CBT for
anxiety. Jake and his therapist decide that it would be
useful for Jake to practice progressive muscle relaxation,
and Jakes therapist coaches him through the exercise in
therapy so Jake can continue practicing at home.
However, at the next session, Jake admits that he did
not practice the relaxation strategy for a number of
different reasons, including his lack of alone time, feeling
too anxious to begin the exercise, and ultimately his
Table 6
Results of t-Tests Comparing MoodMission MARS Scores to
MARS Norms
tp Hedgessg
Entertainment 4.39 b0.001 0.89
Interest 6.27 b0.001 1.26
Customization 2.81 0.006 0.57
Interactivity 1.33 0.187 0.27
Target group 3.65 b0.001 0.75
Performance 1.61 0.110 0.33
Ease of use 1.58 0.117 0.32
Navigation -0.27 0.787 -0.06
Gestural design -0.82 0.416 -0.17
Layout 1.70 0.092 0.35
Graphics 4.75 b0.001 0.97
Visual appeal 4.93 b0.001 1.02
Quality of information 4.74 b0.001 0.94
Quantity of information 5.72 b0.001 1.14
Visual information 9.75 b0.001 1.93
Credibility of source 9.73 b0.001 1.97
Would you recommend the app? 5.82 b0.001 1.19
Estimated frequency of use 3.02 0.003 0.63
Would you pay for the app? 2.08 0.040 0.44
Overall rating 5.48 b0.001 1.10
Note.df = 92 for all comparisons.
Table 5
Means (SD) of MARS scores for MoodMission and Norms
MARS Item and
Entertainment 3.43 (0.81) 2.49 (0.81)
Interest 3.82 (0.78) 2.52 (0.78)
Customization 2.84 (0.80) 2.27 (0.80)
Interactivity 3.00 (0.96) 2.70 (0.96)
Target group 4.06 (0.81) 3.41 (0.81)
Total (Engagement) 3.43 (0.83) 2.67 (0.83)
Performance 4.30 (0.88) 4.00 (0.88)
Ease of use 4.18 (0.69) 3.93 (0.69)
Navigation 3.95 (0.72) 4.00 (0.72)
Gestural design 3.97 (0.67) 4.10 (0.67)
Total (Functionality) 4.10 (0.74) 4.00 (0.74)
Layout 4.18 (0.69) 3.91 (0.69)
Graphics 4.20 (0.70) 3.41 (0.70)
Visual appeal 4.04 (0.87) 3.14 (0.87)
Total (Aesthetics) 4.14 (0.75) 3.48 (0.75)
Quality of information 4.27 (0.70) 3.18 (0.70)
Quantity of information 4.27 (0.76) 2.87 (0.76)
Visual information 4.25 (0.87) 1.35 (0.87)
Credibility of source 4.41 (0.66) 2.79 (0.66)
Total (Information) 4.3 (0.75) 2.54 (0.75)
Recommend? 3.62 (1.02) 2.31 (1.02)
Estimated frequency 3.18 (1.20) 2.46 (1.20)
Would you pay? 1.6 (0.77) 1.31 (0.77)
Overall rating 3.69 (0.70) 2.69 (0.70)
13Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
skepticism that it would work. The therapist could
troubleshoot these issues with Jake and develop a new
plan for relaxation, or they could coach him through the
downloading and use of MoodMission.
Jake uses apps on his phone regularly so is receptive to
the idea of using MoodMission. While in the therapy
session, he downloads the app and his therapist points out
the apps features and uses. Leaving the session, Jake
opens MoodMission when hes feeling anxious. He rates
his anxiety 7/10, chooses I cant stop thinking about
somethingto indicate his anxiety is mainly thought-
based, and the app provides a tailored list of suggested
strategies. The list gives Jake a choice, so he chooses the
one that he feels is most achievable and suits his
circumstances. This is the This situation wont last
foreverMission, which involves him repeating the phrase
in his head, applying it to his current situation, and
writing it down as a reminder. He feels like this is a little
helpful, and when finished rates his distress as slightly
lower at 6/10.
The second time Jake uses MoodMission he reports
feeling anxious at 8/10, again with anxious thoughts, but
this time chooses the Sit upsMission, which involves
him doing 20 sit ups. After completing this he rates his
distress as 5/10, as performing a short burst of physical
exercise was particularly helpful for him to shake his
anxious thoughts. The third time Jake uses MoodMission
he reports 7/10 thought-based anxiety. The app suggests
a few more physical-based Missions than the other
categories, as past success indicates that these were
helpful for Jake.
When Jake returns to his therapist, he opens
MoodMission and shows his therapist his Mission Log.
They review each Mission completed and Jakestherapist
asks him a few questions about his experiences performing
each Mission. Jake has discovered new coping options for
him to reduce his anxiety, and his therapist has discovered
that physical strategies, such as exercise, will be particularly
useful for Jake progressing in therapy.
For Jake, using an app like MoodMission helped him
discover a range of alternative coping options that were
more achievable than progressive muscle relaxation.
Jakes failure in completing the progressive muscle
relaxation homework may have increased his sense of
hopelessness, reinforcing the belief that relaxing is too
hard.Even if some of the Missions attempted by Jake
were not successful at reducing his anxiety, the broad
range of strategies on offer increase the likelihood that
Jake will be hopeful to find something that works for him.
MoodMission provided him psychoeducation about each
strategy, giving him a rationale to engage. Reviewing his
progress in the Mission Log, alone and with his therapist,
enabled him to gain more self-insight and motivation to
continue making therapeutic change.
Case 2: Use Outside of Therapy
Annabelis a 19-year-old female who has just started
an undergraduate university degree. She has no history of
mental health issues and does not know when or how she
would seek help if she had a mental health related
problem. She has moved away from her hometown and is
now living in a residential college on campus. As she
settles in to her new routine, she realizes that some of the
things she used to enjoy doing are no longer available to
her. She used to look forward to dinner table discussions
with her siblings and parents at the end of the day, and
she used to play sport on the weekends. Combined with
the new stresses of college social life, starting a degree,
and being in an unfamiliar place, she starts to feel quite
down on herself. A flyer in her orientation pack mentions
MoodMission as something that may help, so she
downloads the app and reports feeling low. MoodMission
suggests a list of strategies that she tries out and after a few
she discovers that doing a short productive chore, like a
load of laundry or cleaning her room, helps her feel
better about herself. In this example, MoodMission has
engaged Annabel in light mastery-based behavioral
As the semester progresses Annabel is doing OK, but
she experiences a particularly bad episode of low mood
and anxiety in the week following exams. She attempts a
few Missions on MoodMission, and due to her persistently
high distress scores the app suggests that she visit her
doctor about getting support. She sees her doctor who
refers her to a therapist for treatment. With this therapist
she is able to review her progress with MoodMission, so
therapy can be efficiently tailored to her demonstrated
strengths. In this example, Annabel is introduced to
concepts of mental health self-care, she discovers new
context-appropriate strategies for improving her mood,
she is prompted to seek clinical support when it would be
useful, and the start of her therapy is enhanced by her
Mission Log data.
For Annabel, the mere promotional flyer for
MoodMission serves as an acknowledgment that mental
health is real,and self-care is a helpful skill to
develop. Using the app, she is able to reframe boring
chores as important self-care achievements. This
improves her understanding of the relationship be-
tween mood and activity levels. During her depressive
episode, she struggles to experience improvements
from the Missions, but is encouraged to seek help early,
providing an earlier intervention therapeutic advan-
tage. The app improves her help seeking by reducing
barriers of uncertainty (e.g., Should I see a therapist?
How do I even arrange therapy?), and reviewing her
Mission Log improves the efficiency of the therapists
14 Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
The purpose of this paper was to outline the
development phases of a CBT-based MHapp for managing
low moods and anxiety, and recommend future applica-
tions of this tool. The app development drew on recently
published recommendations (Bakker et al., 2016)to
optimize tailoring for individual users, incorporation of
data collection capacity, and the use of validated CBT
principles. The resulting app, MoodMission, successfully
achieved this aim, with the delivery of an engaging CBT-
based MHapp based in evidence-based principles. This app
was successfully developed, tested, and released on the
Apple iTunes Store. Preliminary testing revealed that
MoodMission was rated superior to other health apps in
terms of entertainment, aesthetics, and information.
MoodMission has several anticipated applications across
both clinical and research domains.
MoodMission could have many applications in both
public and private health sectors as a clinical and
preventative tool. MoodMission is an easily accessible,
intuitive tool that does not require introduction by a health
practitioner. General practitioners, counselors, social
workers, and other professionals are able to recommend
the use of MoodMission to at-risk populations. The flexibility
of MoodMission allows it to cover most kinds of low mood
and anxious distress, and built-in alerts recommend
accessing professional mental health support if an
individuals responses indicate sufficient severity.
Tang and Kreindler (2017) outline features of MHapps
that can be used for the tracking, encouragement, and
compliance of homework activities in CBT. MoodMission has
features that meet each of the six recommendations:
congruency to therapy, fostering learning, guiding therapy,
building connections, emphasizing completion, and popu-
lation specificity. Future work will focus on ways of integrating
MoodMission into the practices of CBT clinicians so the app
can be used collaboratively to set and review homework.
A firm evidence base establishing the effectiveness and
efficacy of MoodMission is required to enable adoption by
health professionals. At the time of writing, MoodMission
has one RCT in progress investigating its efficacy in
improving mental health, positive well-being, emotional
self-awareness, mental health literacy, and coping self-
efficacy. MoodMission also collects effectiveness data
from every individual who completes the in-built assess-
ment surveys. Using frequency of use as a variable allows
controlled investigation of the apps effects on mental
health and well-being, as described by Carpenter et al.
(2016) for analysis of data from the well-being app
Happify. It is anticipated that future publication of the
RCT and the effectiveness data will support MoodMission
and other similar MHapps as efficacious and effective self-
guided mental health tools.
MoodMission also holds promise as a way of investi-
gating research questions beyond the effects of the app
itself. In addition to outcome data collected via surveys,
MoodMission collects data that can be used to investigate
the effectiveness of specific Missions. Comparing the pre-
mission and post-mission distress scores for specific
Missions may reveal how effective they are at reducing
specific types of distress. This would enable objective
comparison between two similar psychological techniques
for the same type of distress. Findings could inform
therapeutic work done by clinical psychologists and guide
them to encourage the most effective evidence-based
techniques for their clientsunique distress.
A distinct benefit that MoodMission can afford re-
searchers ofdifferent psychological therapies is component
analysis. Therapies are made up of collections of techniques
and strategies (Mennin, Ellard, Fresco, & Gross, 2013). For
example, CBT interventions for panic disorder may include
components of exposure, relaxation, and cognitive restruc-
turing (Craske & Barlow, 2014). However, evidence for
specific strategies within evidence-based interventions is
lacking. This is in part dueto methodological limitations, as
most studies of CBT treat large groups of participants with
different intervention variants, and variation is on the
macro level rather than the micro, strategy-based level.
MoodMission enables ecologically valid comparisons be-
tween each of these components to identify those with the
greatest impact on distress and anxiety symptomatology.
Furthermore, strategies can be assessed on their preventa-
tive power, and their effects on individuals without
identifiable clinical disorders. In the case of panic disorder,
data from usersanxious thought-based problems could be
parceled out and distress score differences could be
compared between a mindfulness body scan meditation
and a similar simple relaxation exercise. Component
analysis research is often confined to strictly controlled
situations in which participants receive slightly adjusted
versions of the same therapy (e.g., Borkovec, Newman,
Pincus, & Lytle, 2002; Jacobson et al., 1996; Linehan et al.,
2015). Such research is resource intensive and is only able
to compare two or three different techniques at a time.
MoodMission allows easier component analysis across a
wide range of very different therapeutic techniques to find
the most efficient, effective treatments of low mood and
The artificial or controlled settings of many laboratory-
or clinic-based studies may reduce the ecological validity
of findings concerning the effectiveness of certain
distress-reduction strategies. For example, recall biases
may limit participantsability to accurately recall levels of
distress or the effectiveness of coping strategies hours
or days after experiencing the distress (Shiffman, Stone,
& Hufford, 2008). MoodMission allows participants
to record their distress in the moment that they are
15Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
experiencing it and rate the effectiveness of strategies
immediately after they are used. This reduces recall biases
and provides participants with immediate feedback on the
helpfulness of the strategies.
Most studies that use therapists to deliver different
modes of therapy are inherently influenced by a
practitioners skill level for particular modes of therapy
(Lambert, 1989; Westen, Novotny, & Thompson-Brenner,
2004). For example, one therapist may prefer cognitive
restructuring over imaginal exposure in their own
practice, so when they are involved in a study that
compares the two therapeutic modes, they are likely to
present cognitive restructuring with more skill than other
experimental conditions. MoodMission bypasses these
presentation biases by presenting all therapeutic tech-
niques in a consistent fashion.
While MoodMission was designed to be appropriate for
all smartphone users over the age of 12, some specific age
groups and demographics may derive more benefit than
others. For example, the needs of undergraduate
university students aged 1825, like Annabel from Case
2, are particularly compatible with MoodMissions fea-
tures. These young adults are often exposed to new,
stressful situations, and relied upon coping mechanisms
may have recently become inconvenient or inappropri-
ate. Teaching flexible, evidence-based coping strategies
may have a protective effect for this demographic,
potentially preventing the onset of mental health issues
at this critical life stage. This age group also has a high rate
of smartphone usage, making MoodMission suitable for
both clinical and research applications.
Research that investigates models of therapy often uses
clinical samples made up of individuals who qualify for some
sort of diagnosis or achieve a certain score on a measure of
symptoms (Henry & Crawford, 2005). MoodMission allows
for inclusion of both clinical and nonclinical samples, as
individuals do not need a diagnosis to engage with the
intervention. MoodMission is therefore capable of investi-
gating both the clinical and preventative utility of mental
health interventions, as well as how different individuals
engage with these interventions.
This study was limited by the relatively small sample size
and the noninvasive assessment methods used. However,
this was a pilot evaluation to prepare MoodMission for
public release, and planned future studies aim to collect data
from much larger and more representative samples of
participants who are using the app in a wider variety of
situations. Future studies will also aim to collect in-depth
user cases, increasing the resolution and comprehensiveness
of the assessments to collect richer data about individuals
experiences using the app.
There is a balance in app development between:
(a) releasing a prototype app and making fixes as
user feedback is received, and (b) conducting extensive
in-house testing before releasing it to the public. The risk
with option (a) in the above is that provoking frustration
in users who are expecting a fully functional product,
while the risk with option (b) is the public may disagree
with many of the choices of the in-house team, and the
app may need major adjustment. Traditionally, psychol-
ogists have designed CBT interventions with patient
feedback informing changes after implementation. How-
ever, the self-guided nature of MHapps increases the
utility of user feedback. This project aimed to achieve a
compromise between options (a) and (b), but there is a
possibility that more user involvement at the very early
stages of design and development may have increased the
utility of MoodMission.
In conclusion, MoodMission is a mental health
smartphone app, built using evidence-based therapeutic
techniques, that aims to build its own evidence base as an
effective intervention. It was designed using recommen-
dations from Bakker et al. (2016) informing a behavioral
plan using a trigger, action, reward, investment loop
(Eyal, 2014). There are many potential therapeutic and
research uses for MoodMission, and it is anticipated that
these applications will be developed and evaluated over
the coming years.
Conflict of Interest Statement
The authors declare that there are no conflicts of
We would like to acknowledge Spark Digital for their
technical development of MoodMission, the pledgers to
both crowdfunding campaigns for funding this develop-
ment, and all participants involved in the development
and pilot testing process.
Agarwal, S., LeFevre, A. E., Lee, J., LEngle, K., Mehl, G., Sinha, C., &
Labrique, A. (2016). Guidelinesfor reporting of health interventions
using mobile phones: mobile health (mHealth) evidence reporting
and assessment (mERA) checklist. BMJ,352, i1174. https://doi.
Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010). Emotion-
regulation strategies across psychopathology: A meta-analytic
review. Clinical Psychology Review,30(2), 217237. https://doi.
Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010).
Computer therapy for the anxiety and depressive disorders is
effective, acceptable and practical health care: A meta-analysis.
PLoS One,5, e13196.
Andrews, G., & Williams, A. D. (2014). Internet psychotherapy and the
future of personalized treatment. Depression and Anxiety,31(11),
Bakker, D., & Kazantzis, N. (2017). Homework rating scale - mobile
application version [Measurement Instrument].
health smartphone apps: Review and evidence-based recommenda-
tions for future developments. JMIR Mental Health,3,e7.https://doi.
16 Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Bakker, G. (2008). Practical CBT: Using functional analysis and
standardised homework in everyday therapy. Bowen Hills, Qld.:
Australian Academic Press.
Ballegooijen, W. v., Cuijpers, P., Straten, A. v., Karyotaki, E., Andersson,
G., Smit, J. H., & Riper, H. (2014). Adherence to internet-based
and face-to-face cognitive behavioural therapy for depression: A
meta-analysis. PLoS One,9, e100674.
Beck, J. S. (2011). Cognitive behavior therapy: Basics and beyond (2nd ed.).
New York: The Guilford Press.
Belleflamme, P., Lambert, T., & Schwienbacher, A. (2014). Crowdfunding:
Tapping the right crowd. Journalof Business Venturing,29(5), 585609.
BinDhim, N. F., Alanazi, E. M., Aljadhey, H., Basyouni, M. H., Kowalski,
S. R., Pont, L. G., Alhawassi, T. M. (2016). Does a mobile phone
depression-screening app motivate mobile phone users with high
depressive symptoms to seek a health care professionalshelp?Journal
of Medical Internet Research,18(6), e156.
Bolognesi, F., Baldwin, D. S., & Ruini, C. (2014). Psychological
interventions in the treatment of generalized anxiety disorder: A
structured review. Journal of Psychopathology,20(2), 111126.
Borkovec, T. D., Newman, M. G., Pincus, A. L., & Lytle, R. (2002). A
component analysis of cognitive-behavioral therapy for generalized
anxiety disorder and the role of interpersonal problems. Journal of
Consulting and Clinical Psychology,70(2), 288298. https://doi.
Bower, P., & Gilbody, S. (2005). Stepped care in psychological
therapies: access, effectiveness and efficiency. The British
Journal of Psychiatry,186(1), 1117.
Brown, M., Glendenning, A., Hoon, A. E., & John, A. (2016).
Effectiveness of web-delivered acceptance and commitment
therapy in relation to mental health and well-being: A systematic
review and meta-analysis. Journal of Medical Internet Research,18(8).
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The
empirical status of cognitive-behavioral therapy: A review of meta-
analyses. Clinical Psychology Review,26(1), 1731. https://doi.
Cobb, A. M., & Parks, A. C. (2016). Seeing the bigpicture: Big
data methods for exploring relationships between usage,
language, and outcome in internet intervention data. Journal of
Medical Internet Research,18(8), e241.
Chesney, M. A., Neilands, T. B., Chambers, D. B., Taylor, J. M., &
Folkman, S. (2006). A validity and reliability study of the coping
self-efficacy scale. British Journal of Health Psychology,11, 421437.
Comello, M. L. G., Qian, X., Deal, A. M., Ribisl, K. M., Linnan, L. A., &
Tate, D. F. (2016). Impact of game-inspired infographics
on user engagement and information processing in an
eHealth program. Journal of Medical Internet Research,18(9), e237.
comScore. (2015). comScore Reports March 2015 U.S. Smartphone
Subscriber Market Share. Retrieved from http://www.comscore.
Cooney, G. M., Dwan, K., Greig, C. A., Lawlor, D. A., Rimer, J., Waugh,
F. R., Mead, G. E. (2013). Exercise for depression. Cochrane
Database of Systematic Reviews,9, CD004366.
Craske, M. G., & Barlow, D. H. (2014). Panic disorder and agoraphobia.
In D. H. Barlow (Ed.),Clinical handbook of psychologicaldisorders: A step-
by-step treatment manual (5th ed., pp. 161). New York: Guilford
Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., &
Dobson, K. S. (2013). A meta-analysis of cognitive-behavioural
therapy for adult depression, alone and in comparison with
other treatments. Canadian Journal of Psychiatry,58(7), 376385.
Cumming, G. (2011). Understanding the new statistics: Effect sizes,
confidence intervals, and meta-analysis. New York: Routledge.
Dahne, J., Kustanowitz, J., & Lejuez, C. W. (2017). Development
and preliminary feasibility study of a brief behavioral activation
mobile application (Behavioral Apptivation) to be used in
conjunction with ongoing therapy. Cognitive and Behavioral
Dear, B. F., Staples, L. G., Terides, M. D., Fogliati, V. J., Sheehan, J.,
Johnston, L., Titov, N. (2016). Transdiagnostic versus disorder-
specific and clinician-guided versus self-guided internet-delivered
treatment for Social Anxiety Disorder and comorbid disorders: A
randomized controlled trial. Journal of Anxiety Disorders,42,3044.
Deloitte. (2016a). 2016 Global mobile consumer survey: US edition |
Deloitte US. Retrieved March 5, 2017, from
Deloitte. (2016b). Mobile Consumer Survey 2015 The Australian Cut.
Retrieved from
Deloitte. (2016c). Theres no place like phone: Consumer usage patterns in the era
of peak smartphone. Retrieved from
Dèttore, D., Pozza, A., & Andersson, G. (2015). Efficacy of technology-
delivered cognitive behavioural therapy for OCD versus
control conditions, and in comparison with therapist-administered
CBT: Meta-analysis of randomized controlled trials. Cognitive
Behaviour Therapy,44(3), 190211.
Dirmaier, J., Liebherz, S., Sänger, S., Härter, M., & Tlach, L. (2016). Development and process evaluation of an e-mental
health portal. Informatics for Health and Social Care,41(3), 267285.
Dobson, D., & Dobson, K. S. (2009). Evidence-based practice of cognitive-
behavioral therapy. New York: The Guilford Press.
Christensen, H. (2013). Smartphones for smarter delivery
of mental health programs: A systematic review. Journal of
Medical Internet Research,15, e247.
Dubad, M., Winsper, C., Meyer, C., Livanou, M., & Marwaha, S. (2017).
A systematic review of the psychometric properties, usability and
clinical impacts of mobile mood-monitoring applications in young
people. Psychological Medicine,121.
Ellard, K. K., Fairholme, C. P., Boisseau, C. L., Farchione, T. J., &
Barlow, D. H. (2010). Unified protocol for the transdiagnostic
treatment of emotional disorders: Protocol development and
initial outcome data. Cognitive and Behavioral Practice,17(1),
Eyal, N. (2014). Hooked: How to build habit-forming products. New York:
Penguin Group.
Fernandez,E., Salem, D., Swift,J. K., & Ramtahal, N. (2015). Meta-analysis
of dropout from cognitive behavioral therapy: Magnitude, timing,
and moderators. Journal of Consulting and Clinical Psychology,83(6),
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S.,
& Sarris, J. (2017a). The efficacy of smartphone-based mental
health interventions for depressive symptoms: A meta-analysis of
randomized controlled trials. World Psychiatry,16(3), 287298.
Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J.
(2017b). Can smartphone mental health interventions reduce
symptoms of anxiety? A meta-analysis of randomized controlled
trials. Journal of Affective Disorders,218(Supplement C), 1522.
Franklin, J. C., Fox, K. R., Franklin, C. R., Kleiman, E. M., Ribeiro, J. D.,
Jaroszewski, A. C., Nock, M. K. (2016). A brief mobile app
reduces nonsuicidal and suicidal self-injury: Evidence from three
randomized controlled trials. Journal of Consulting and Clinical
Psychology,84(6), 544557.
17Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Garg, R., & Telang, R. (2014). Inferring app demand from publicly
available data. MIS Quarterly,37(4), 12531264. https://doi.
Google. (2016). How people use their devices: What marketers need to know.
Retrieved from
Grist, R., Porter, J., & Stallard, P. (2017). Mental health mobile apps for
preadolescents and adolescents: A systematic review. Journal of
Medical Internet Research,19(5), e176.
(2017). Therapist empathy, homework compliance, and outcome
in cognitive behavioral therapy for generalized anxiety disorder:
Partitioning within- and between-therapist effects. Cognitive
Behaviour Therapy,46(5), 375390.
Hayes, S. C., & Hofmann, S. G. (2017). The third wave of
cognitive behavioral therapy and the rise of process-based
care. World Psychiatry,16(3), 245246.
Hayes, S. C., & Hofmann, S. G. (Eds.). (2018). Process-based CBT: The
science and core clinical competencies of Cognitive Behavioral Therapy
(1st ed.). Oakland CA: New Harbinger.
Headspace. (2015). Headspace: Meditation techniques for mindfulness,
stress relief & peace of mind (Version 2.4) [Mobile application
software]. Retrieved from
Henry, J. D., & Crawford, J. R. (2005). The short-form version of the
Depression Anxiety Stress Scales (DASS-21): Construct validity
and normative data in a large non-clinical sample. British
Journal of Clinical Psychology,44(2), 227239.
Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D.,
Stoyanov, S. R., & Cockshaw, W. (2015). Rays Night Out: A
new iPhone app targeting alcohol use in young people. Retrieved
Hides, L., Kavanagh, D., Stoyanov, S., Zelenko, O., Tjondronegoro,
D., & Mani, M. (2014). Mobile application rating scale (MARS).
Abbotsford, Victoria, AUS: Young and Well Cooperative Research
Hofmann, S. G., Sawyer, A. T., Witt, A. A., & Oh, D. (2010). The
effect of mindfulness-based therapy on anxiety and depres-
sion: A meta-analytic review. Journal of Consulting,78,169183.
Holdsworth, E., Bowen, E., Brown, S., & Howat, D. (2014).
Client engagement in psychotherapeutic treatment and
associations with client characteristics, therapist characteristics,
and treatment factors. Clinical Psychology Review,34(5), 428450.
Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K.,
Gollan, J. K., Prince, S. E. (1996). A component analysis of
cognitive-behavioral treatment for depression. Journal of Consulting
and Clinical Psychology,64(2), 295304.
Jaspers, M. W. M. (2009). A comparison of usability methods for testing
interactive health technologies: Methodological aspects and
empirical evidence. International Journal of Medical Informatics,78
(5), 340353.
Jones, N., & Moffitt, M. (2016). Ethical guidelines for mobile app
development within health and mental health fields. Professional
Psychology: Research and Practice,47(2), 155162.
Kauer, S. D., Reid, S. C., Crooke, A. H. D., Khor, A., Hearps, S. J. C.,
Jorm, A. F., Patton, G. (2012). Self-monitoring using mobile
phones in the early stages of adolescent depression: Randomized
controlled trial. Journal of Medical Internet Research,14, e67.
Kazantzis, N. (2018). Introduction to the special issue on processes of
Cognitive Behavioral Therapy: Does necessary, but not suffi-
cientstill capture it? Cognitive Therapy and Research,42(2),
Kazantzis,N., Brownfield, N. R., Mosely,L., Usatoff, A. S., & Flighty, A. J.
(2017). Homework in cognitive behavioral therapy: A systematic
review of adherence assessment in anxiety and depression
(20112016). Psychiatric Clinics of North America,40(4), 625639.
Kazantzis, N., Deane, F. P., & Ronan, K. R. (2005). Assessment of
homework completion. In N. Kazantzis, F. P. Deane, K. R. Ronan,
&L.LAbate (Eds.), Using homework assignments in cognitive behavior
therapy (pp. 103122). New York: Routledge.
Kazantzis, N., Whittington, C., Zelencich, L., Kyrios, M., Norton, P. J.,
& Hofmann, S. G. (2016). Quantity and quality of homework
compliance: A meta-analysis of relations with outcome in
cognitive behavior therapy. Behavior Therapy,47(5), 755772.
Kinderman, P., Hagan, P., King, S., Bowman, J., Chahal,J., Gan, L.,
Tai, S. (2016). The feasibility and effectiveness of Catch It, an
innovative CBT smartphone app. British Journal of Psychiatry
Open,2(3), 204209.
Kliem, S., Kroger, C., & Kosfelder, J. (2010). Dialectical behavior
therapy for borderline personality disorder: A meta-analysis using
mixed-effects modeling. Journal of Consulting and Clinical Psychology,
78(6), 936951.
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9:
Validity of a brief depression severity measure. Journal of General
Internal Medicine,16, 606613.
Kuester, A., Niemeyer, H., & Knaevelsrud, C. (2016). Internet-based
interventions for posttraumatic stress: A meta-analysis of
randomized controlled trials. Clinical Psychology Review,43,116.
Lambert, M. J. (1989). The individual therapistscontribution
to psychotherapy process and outcome. Clinical Psychology
Review,9(4), 469485.
Lambrecht, A., Goldfarb, A., Bonatti, A., Ghose, A., Goldstein, D. G.,
Lewis, R., Yao, S. (2014). How do firms make money selling
digital goods online? Marketing Letters,25(3), 331341.
Layard, R., & Clark, D. (2014). Thrive: The power of evidence-based
psychological therapies. London: Penguin UK.
Linehan, M. M., Korslund, K. E., Harned, M. S., Gallop, R. J., Lungu,
A., Neacsiu, A. D., Murray-Gregory, A. M. (2015). Dialectical
behavior therapy for high suicide risk in individuals with
borderline personality disorder: A randomized clinical trial
and component analysis. JAMA Psychiatry,72(5), 475482.
Luxton, D. D., McCann, R. A., Bush, N. E., Mishkind, M. C., & Reger,
G. M. (2011). mHealth for mental health: Integrating smartphone
technology in behavioral healthcare. Professional Psychology:
Research and Practice,42(6), 505512.
Manzoni, G., Pagnini, F., Castelnuovo, G., & Molinari, E. (2008).
Relaxation training for anxiety: A ten-year systematic review
with meta-analysis. BMC Psychiatry,8(1), 41.
Mazzucchelli, T., Kane, R., & Rees, C. (2009). Behavioral activation
treatments for depression in adults: A meta-analysis and
review. Clinical Psychology: Science and Practice,16(4), 383411.
Mazzucchelli, T., Kane, R. T., & Rees, C. S. (2010). Behavioral
activation interventions for well-being: A meta-analysis. The Journal of
Positive Psychology,5(2), 105121.
Mennin, D. S., Ellard, K. K., Fresco, D. M., & Gross, J. J. (2013).
United we stand: Emphasizing commonalities across
cognitive-behavioral therapies. Behavior Therapy,44, 234248.
Newby, J. M., Twomey, C., Yuan Li, S. S., & Andrews, G. (2016).
Transdiagnostic computerised cognitive behavioural therapy for
depression and anxiety: A systematic review and meta-analysis.
Journal of Affective Disorders,199,3041.
18 Bakker et al.
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
Nielsen (2016). Millennials Are Top Smartphone Users. Retrieved
January 23, 2018, from
Olff, M. (2015). Mobile mental health: A challenging research agenda.
European Journal of Psychotraumatology,6, 27882.
Pacifica Labs Inc (2016). Pacifica - Anxiety, Stress, & Depression
relief based on CBT & Mindfulness on the App Store. Retrieved
March 6, 2016, from
Păsărelu, C. R., Andersson, G., Nordgren, L. B., & Dobrean, A.
(2017). Internet-delivered transdiagnostic and tailored cognitive
behavioral therapy for anxiety and depression: A systematic
review and meta-analysis of randomized controlled trials.
Cognitive Behaviour Therapy,46(1), 128.
Petrik, A. M., Kazantzis, N., & Hofmann, S. G. (2013). Distinguishing
integrative from eclectic practice in cognitive behavioral thera-
pies. Psychotherapy: Theory, Research, Practice, Training,50(3),
Proudfoot,J., Parker, G., Hadzi Pavlovic, D., Manicavasagar, V., Adler,
E., & Whitton, A. (2010). Community attitudes to the appropriation
of mobile phonesfor monitoring and managingdepression, anxiety,
and stress. Journal of Medical Internet Research,12,e64.
ReachOut. (2016). ReachOut WorryTime. Retrieved March 7, 2016,
Reger, G., Hoffman, J., Riggs, D., Rothbaum, B., Ruzek, J., Holloway,
K., & Kuhn, E. (2013). The PE coachsmartphone application:
An innovative approach to improving implementation,
fidelity, and homework adherence during prolonged exposure.
[References]. Psychological Services,10(3), 342349. https://doi.
Rickard, N., Arjmand, H. -A., Bakker, D., & Seabrook, E. (2016).
Development of a mobile phone app to support self-monitoring of
emotional well-being: A mental health digital innovation. JMIR
Mental Health,3(4), e49.
Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R.,
& Maxwell, B. (2015). Randomized controlled trial of SuperB-
etter, a smartphone-based/internet-based self-help tool to reduce
depressive symptoms. Games for Health Journal,4(3), 235246.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and
the facilitation of intrinsic motivation, social development, and
well-being. American Psychologist,55,6878.
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological
momentary assessment. Annual Review of Clinical Psychology,4,
Shipherd, J. C., & Fordiani, J. M. (2015). The application of
mindfulness in coping with intrusive thoughts. Cognitive and
Behavioral Practice,22(4), 439446.
Sian Morson. (2015). Designing for iOS with Sketch. Berkeley, CA: Apress.
Smiling Mind. (2015). Smiling Mind (Version 2.1.6) [Mobile application
software]. Retrieved January 16, 2015, from
Spitzer, R. L., Kroenke, K., Williams, J. W., & Löwe, B. (2006). A brief
measure for assessing generalized anxiety disorder: The GAD-7.
Archives of Internal Medicine,166, 10921097.
Stoyanov, S. R., Hides, L., Kavanagh, D. J., & Wilson, H. (2016).
Development and validation of the user version of the Mobile
Application Rating Scale (uMARS). JMIR MHealth and UHealth,4
(2), e72.
SuperBetter Inc. (2014). SuperBetter (Version 1.1) [Mobile applica-
tion software]. Retrieved January 13, 2015, from https://itunes.
Tang, W., & Kreindler, D. (2017). Supporting homework compliance
in cognitive behavioural therapy: Essential features of mobile
apps. JMIR Mental Health,4(2), e20.
Tee, J., & Kazantzis, N. (2011). Collaborative empiricism in cognitive
therapy: A definition and theory for the relationship construct.
Clinical Psychology: Science and Practice,18,4761.
Tennant, R., Hiller, L., Fishwick, R., Platt, S., Joseph, S., Weich, S.,
Stewart-Brown, S. (2007). The Warwick-Edinburgh Mental Well-being
Scale (WEMWBS): Development and UK validation. Health and
Quality of Life Outcomes,5,63.
Titov, N., Dear, B. F., Staples, L. G., Terides, M. D., Karin, E.,
Sheehan, J., McEvoy, P. M. (2015). Disorder-specific versus
transdiagnostic and clinician-guided versus self-guided treatment
for major depressive disorder and comorbid anxiety disorders: A
randomized controlled trial. Journal of Anxiety Disorders,35,
Torous, J., Friedman, R., & Keshavan, M. (2014). Smartphone
ownership and interest in mobile applications to monitor
symptoms of mental health conditions. JMIR MHealth and UHealth,
2(1), e2.
Van Ameringen, M., Turna, J., Khalesi, Z., Pullia, K., & Patterson, B.
(2017). There is an app for that! The current state of mobile
applications (apps) for DSM-5 obsessive-compulsive disorder,
posttraumatic stress disorder, anxietyand mood disorders. Depression
and Anxiety,34(6), 526539.
Vent. (2017). Vent (Version 5.7) [Mobile application software].
Retrieved from
Vogl, G., Ratnaike, D., Ivancic, L., Rowley, A., & Chandy, V. (2016).
One click away? Insights into mental health digital self-help by young
Australians. Sydney: EY and ReachOut Australia.
Watts, S., Mackenzie, A., Thomas, C., Griskaitis, A., Mewton, L.,
Williams, A., & Andrews, G. (2013). CBT for depression: A pilot
RCT comparing mobile phone vs. computer. BMC Psychiatry,13,
Wendel, S. (2013). Designing for behavior change: Applying psychology and
behavioral economics. Sebastopol, CA: OReilly Media.
Weng, L., Menczer, F., & Ahn, Y. -Y. (2013). Virality Prediction
and Community Structure in Social Networks. Scientific Reports,3.
Westbrook, D., Kennerley, H., & Kirk, J. (2011). An introduction to
cognitive behaviour therapy: Skills and applications (2nd ed.). Los
Angeles: SAGE Publications.
Westen,D., Novotny,C.M.,& Thompson-Brenner,H.(2004).The
empirical status of empirically supported psychotherapies: Assump-
tions, findings, and reporting in controlled clinical trials. Psychological
Bulletin,130(4), 631663.
White, J. (2010). The STEPS model: A high volume, multi-level, multi-
purpose approach to address common mental health problems.
In J. Bennett-Levy, D. Richards, P. Farrand, H. Christensen, K.
Griffiths, D. Kavanagh, J. Proudfoot (Eds.), Oxford Guide to Low
Intensity CBT Interventions (1st ed., pp. 3552). Oxford: Oxford
University Press.
Address correspondence to David Bakker, DPsych(Clin), School of
Psychological Sciences and Monash Institute of Cognitive and Clinical
Neurosciences, 18 Innovation Walk, Clayton, 3800 Australia; e-mail:
Received: September 28, 2017
Accepted: July 14, 2018
Available online xxxx
19Smartphone Delivered CBT Strategies
Please cite this article as: Bakker et al., Development and Pilot Evaluation of Smartphone-Delivered Cognitive Behavior Therapy Strategies
for Mood- and Anxiety-Related Problems: ..., Cognitive and Behavioral Practice (2018),
... Implementation elements (e.g., maternal depression and anxiety) will be examined to estimate effect size and variance for a future sample size calculations and will be compared to a Standard of Care (SOC), i.e., MoodMission. MoodMission was chosen as the SOC as this program has been rigorously studied using RCTs, which is rare for a mHealth app and shows a decrease in moderate depressive and anxiety symptoms post-intervention (27)(28)(29)(30). ...
... The SOC arm will receive MoodMission, an App-based program effective in the treatment of moderate depression and anxiety in adults and increasing their well-being (27)(28)(29)(30). ...
... Compared to BEAM, this selfpaced generalized service is not tailored to the complex needs faced by parents and lacks coaching, social support, and parenting skills development. MoodMission was chosen as the SOC because it has been rigorously studied using RCTs (27)(28)(29)(30). Those in the MoodMission group will work through program content on the App at their own pace. ...
Full-text available
Background: Drastic increases in the rates of maternal depression and anxiety have been reported since the COVID-19 pandemic began. Most programs aim to improve maternal mental health or parenting skills separately, despite it being more effective to target both concurrently. The Building Emotional Awareness and Mental health (BEAM) program was developed to address this gap. BEAM is a mobile health program aiming to mitigate the impacts of pandemic stress on family well-being. Since many family agencies lack infrastructure and personnel to adequately treat maternal mental health concerns, a partnership will occur with Family Dynamics (a local family agency) to address this unmet need. The study’s objective is to examine the feasibility of the BEAM program when delivered with a community partner to inform a larger randomized controlled trial (RCT). Methods: A pilot randomized controlled trial will be conducted with mothers who have depression and/or anxiety with a child 6-18 months old living in Manitoba, Canada. Mothers will be randomized to the 10 weeks of the BEAM program or a standard of care. Back-end App data (collected via Google Analytics and Firebase) will be used to examine feasibility, engagement, and accessibility of the BEAM program; cost-effectiveness will also be examined. Implementation elements (e.g., maternal depression [Patient Health Questionnaire-9] and anxiety [Generalized Anxiety Disorder-7]) will be piloted to estimate the effect size and variance for future sample size calculations. Discussion: In partnership with a local family agency, BEAM holds the potential to promote maternal-child health via a cost-effective and easily accessible program designed to scale. Results will provide insight into the feasibility of the BEAM program and will inform future RCTs of the BEAM program. Trial Registration: This trial was retrospectively registered with (NCT05398107) on May 31st, 2022.
... These operations were consistent with the study of applications that care for mental health in foreign countries. It found that most developed applications provided mental health information (Dorian and Erhardt, 2016;Bakker et al., 2018;Lovejoy et al., 2019). As well as some application designed to provide psychological counseling services including a comfortable application developed by the Department of Mental Health of Thailand. ...
... Bakker et al. (2018) studied the experimentation of 3 mental health-enhancing applications: Mood Mission, Mood Kit, Mood Prism and found the following information: Mood Mission developed 5 activities based on CBT theory called Missions. Each mission developed self-confidence from stress management efforts. ...
Full-text available
Depression and suicide rates among youths tended to increase. From reviews, many applications and online counseling could reduce depression and anxiety to promote well-being of youths and university students effectively. This study was conducted to develop "Friend from heart" application based on LINE system to promote well-being for undergraduate students of faculty of education, Kasetsart University. The research method included the survey of basic data for developing the application and evaluation of the application by specialists. A total of 72 voluntary students were invited to join an online survey. It was found that most of the students (81.94%) wanted applications that provide physical health information such as exercise, eating healthy food, and health care. However, about 16.66% of students needed an application that can speak or listen problems with video calls. Then, researchers took the services that students were interested more than 50% to develop the applications. It worked through the application, consisting of chatbot, physical health, mental health, and appointment with counselor. The index of item-objective congruence was 0.66-1.00 with additional specialists commenting that the application had an interesting design with good structure to help students. For ethical approval, it was obtained from the Kasetsart University Research Ethics Committee.
... This aligns with clinicians' reluctance about using mental health apps in their practice, with many reporting concerns about data security and clinical safety (22,43). In the context of apps being used to aid health service provision from promotion to prevention to treatment, health service providers need to be aware of the core indicators of quality and integrity of such apps and provide informed recommendations that are specific to their patients and clients' health support needs (44)(45)(46). Development of mental health apps would benefit from an application of APA quality assessment to improve their capacity and potential place in the public health support system. Future research could examine to what degree APA's quality check is FIGURE 6 Association between apps visibility ranking (x-Axis, with rank of 1 being the most visible app on app stores) and app's quality score (out of maximum 14). ...
Full-text available
Mobile phones are playing an increasingly important role in supporting mental health, by providing confidential, accessible and scalable support for individuals who may not seek or have means of accessing professional help. There are concerns, however, that many apps claiming to support mental health do not meet professional, ethical or evidence-based standards. App store search algorithms favour popularity (reviews and downloads) and commercial factors (in-app purchases), with what appears to be low prioritisation of safety or effectiveness features. In this paper, the most visible 100 apps for “depression”, “anxiety” and/or “mood” on the Google Play and Apple App stores were selected for assessment using the American Psychiatric Association App Evaluation model. This model systematically assesses apps across five broad steps: accessibility, integrity, clinical and research evidence base, user engagement and interoperability. Assessment is hierarchical, with the most fundamental requirements of apps assessed first, with apps excluded at each step if they do not meet the criteria. The relationship between app quality and app store visibility was first analysed. App quality was also compared across four different app function types: mental health promotion or psychoeducation; monitoring or tracking; assessment or prevention; and intervention or treatment. Of the 92 apps assessed (after eight failed to meet inclusion criteria), half failed to meet the first criterion step of accessibility, and a further 20% of the remaining apps failed to meet the second criterion step of security and privacy. Only three of the 10 apps most visible on app stores met the criteria for research/clinical base and engagement/ease of use, and only one app fulfilled all five criterion steps of the evaluation model. Quality did not differ significantly across app function type. There was no significant correlation between app quality and app store visibility, which presents a potential risk to vulnerable consumers. The findings of this review highlight the need for greater accountability of app developers to meet, and report, at least minimum quality and integrity standards for their apps. Recommendations are also provided to assist users and clinicians to make informed choices in their selection of reputable and effective mental health apps.
... It is an application that uses CBT strategies for users (patients) reporting anxiety and depressed mood. Bakker et al., (2018b) write that MoodMission was created with the intention of users learning how to use strategies themselves to reduce depressed mood and anxiety, especially at low, subclinical intensity and also as an addition to psychotherapy. Users estimate their mood, which is some kind of self-diagnosis, and then the software provides a list of five CBT-based tasks which have been called "Missions". ...
... It is an application that uses CBT strategies for users (patients) reporting anxiety and depressed mood. Bakker et al., (2018b) write that MoodMission was created with the intention of users learning how to use strategies themselves to reduce depressed mood and anxiety, especially at low, subclinical intensity and also as an addition to psychotherapy. Users estimate their mood, which is some kind of self-diagnosis, and then the software provides a list of five CBT-based tasks which have been called "Missions". ...
... It is an application that uses CBT strategies for users (patients) reporting anxiety and depressed mood. Bakker et al., (2018b) write that MoodMission was created with the intention of users learning how to use strategies themselves to reduce depressed mood and anxiety, especially at low, subclinical intensity and also as an addition to psychotherapy. Users estimate their mood, which is some kind of self-diagnosis, and then the software provides a list of five CBT-based tasks which have been called "Missions". ...
... It is an application that uses CBT strategies for users (patients) reporting anxiety and depressed mood. Bakker et al., (2018b) write that MoodMission was created with the intention of users learning how to use strategies themselves to reduce depressed mood and anxiety, especially at low, subclinical intensity and also as an addition to psychotherapy. Users estimate their mood, which is some kind of self-diagnosis, and then the software provides a list of five CBT-based tasks which have been called "Missions". ...
Full-text available
This book examines the use of modern technologies in clinical psychological practice. It considers how we define attachment in an age where changes in technology and the COVID-19 pandemic have increased the prevalence of online contact in the process of diagnosis and psychological treatment. Based on an attachment paradigm that is relatively unexplored, the book outlines how modern online contact influences mental health and development, along with the therapeutic relationship between client and professional. It discusses people’s relationships with new technologies, how relationships can be established using these technologies, and how these technologies affect professional relationships between psychologists and their clients, which they define as e-attachment. In the context of new technologies, the book draws on neurobiology and clinical psychology to consider mental health, social functioning, and emotional regulation. Presenting both theory and examples from case studies, this cutting-edge book will be of great interest to researchers, academics, and post-graduate students in the fields of clinical psychology, psychotherapy, and mental health. Those also carrying out research into digital and online learning within the field of mental health will also benefit from this text. Link:
Background: Millions of workers experience work-related ill health every year. The loss of working days often accounts for poor well-being because of discomfort and stress caused by the workplace. The ongoing pandemic and postpandemic shift in socioeconomic and work culture can continue to contribute to adverse work-related sentiments. Critically investigating state-of-the-art technologies, this study identifies the research gaps in recognizing workers’ need for well-being support, and we aspire to understand how such evidence can be collected to transform the workforce and workplace. Objective: Building on recent advances in sentiment analysis, this study aims to closely examine the potential of social media as a tool to assess workers’ emotions toward the workplace. Methods: This study collected a large Twitter data set comprising both pandemic and prepandemic tweets facilitated through a human-in-the-loop approach in combination with unsupervised learning and meta-heuristic optimization algorithms. The raw data preprocessed through natural language processing techniques were assessed using a generative statistical model and a lexicon-assisted rule-based model, mapping lexical features to emotion intensities. This study also assigned human annotations and performed work-related sentiment analysis. Results: A mixed methods approach, including topic modeling using latent Dirichlet allocation, identified the top topics from the corpus to understand how Twitter users engage with discussions on work-related sentiments. The sorted aspects were portrayed through overlapped clusters and low intertopic distances. However, further analysis comprising the Valence Aware Dictionary for Sentiment Reasoner suggested a smaller number of negative polarities among diverse subjects. By contrast, the human-annotated data set created for this study contained more negative sentiments. In this study, sentimental juxtaposition revealed through the labeled data set was supported by the n-gram analysis as well. Conclusions: The developed data set demonstrates that work-related sentiments are projected onto social media, which offers an opportunity to better support workers. The infrastructure of the workplace, the nature of the work, the culture within the industry and the particular organization, employers, colleagues, person-specific habits, and upbringing all play a part in the health and well-being of any working adult who contributes to the productivity of the organization. Therefore, understanding the origin and influence of the complex underlying factors both qualitatively and quantitatively can inform the next generation of workplaces to drive positive change by relying on empirically grounded evidence. Therefore, this study outlines a comprehensive approach to capture deeper insights into work-related health.
Mental health conditions in children and adolescents have wide-ranging effects, limiting opportunities for future productive lives. While there has been an upsurge of interest in using games for learning, gamification to optimize health outcomes is worth reviewing when this approach to address mental health conditions is in its infancy. A literature search was conducted with two hundred and fifteen articles involving participants up to 19 years of age with diagnosable mental health conditions, involving the concept of game design elements to motivate health in the context of comparing gamification and non-gamification approaches were retrieved after employing Boolean operators and pre-determined search strategies. Literature of participants with addiction to alcohol or substance use and organic brain issues were excluded as their mental health recovery differed regardless of gamification impact. Findings, extracted from 8 included articles following rigorous screening and critical appraisal, showed that gamification to relieve mental health symptoms were conducted via mobile devices or computers. Key features of the games involved applications or video where players with mental health conditions took on roles in a virtual world with narratives. Real-life knowledge and skills to manage the symptoms of mental health conditions were learned in the process as players leveled up in the game. Only one study utilized gamification platforms that could detect breathing changes but it was shown to be only helpful towards relief of anxiety symptoms. Nevertheless, the potential for gamification for mental health outcomes remains promising.
Full-text available
Objectives To evaluate the efficacy and acceptability of ‘Whitu: seven ways in seven days’, a well-being application (app) for young people. Design Prospective randomised controlled trial of Whitu against waitlist control, with 45 participants in each arm. Participants 90 New Zealand young people aged 16–30 recruited via a social media advertising campaign. Setting Participants’ homes. Interventions Developed during the COVID-19 pandemic, and refined from a prototype version that was evaluated during a smaller qualitative study, ‘Whitu: seven ways in seven days’ is a well-being app that, as its name suggests, contains seven modules to help young people (1) recognise and rate emotions, (2) learn relaxation and mindfulness, (3) practice self-compassion and (4) gratitude, (5) connect with others, (6) care for their physical health and (7) engage in goal-setting. It can be completed within a week or as desired. Main outcome measures Primary outcomes were changes in well-being on the WHO 5-item Well-Being Index and Short Warwick-Edinburgh Mental Well-Being Scale. Secondary outcomes were changes in depression on the Centre for Epidemiological Studies Depression Scale, anxiety on the Generalised Anxiety Disorder 7-item Scale, self-compassion on the Self Compassion Scale-Short Form, stress on the 10-item Perceived Stress Scale, sleep on the single-item Sleep Quality Scale and user engagement on the end-user version of the Mobile Application Rating Scale and via qualitative feedback during an online survey. Outcomes were evaluated at baseline, 4 weeks (primary study endpoint) and 3 months, and analysed using linear mixed models with group, time and a group–time interaction. Results At 4 weeks, participants in the Whitu group experienced significantly higher emotional (Mean difference (md) 13.19 (3.96 to 22.42); p=0.005) and mental (md 2.44 (0.27 to 4.61); p=0.027) well-being, self-compassion (md 0.56 (0.28 to 0.83); p<0.001) and sleep (md 1.13 (0.24 to 2.02); p=0.018), and significantly lower stress (md −4.69 (−7.61 to –1.76); p=0.002) and depression (md −5.34 (−10.14 to –0.53); p=0.030), compared with the waitlist controls. Group differences remained statistically significant at 3 months for all outcomes. Symptoms of anxiety were also lower in the intervention group at 4 weeks (p=0.096), with statistically significant differences at 3 months (md −2.31 (−4.54 to –0.08); p=0.042). Usability of Whitu was high (subjective ratings of 4.45 (0.72) and 4.38 (0.79) out of 5 at 4 weeks and 3 months, respectively) and qualitative feedback indicated individual and cultural acceptability of the app. Conclusions Given the evolving psychological burden of the COVID-19 pandemic, Whitu could provide a clinically effective and scalable means of improving the well-being, mental health and resilience of young people. Replication of current findings with younger individuals and in other settings is planned. Trial registration number Australian New Zealand Clinical Trials Registry (ACTRN12620000516987).
Full-text available
Research over the last four decades on cognitive behavioral therapy (CBT) is leading to a more comprehensive understanding of the mechanisms that underpin its efficacy. This introduction to the next generation of research on process offers a brief account of the evolution in the scientific foundations of CBT, to our current focus on treatment and in-session processes. It also provides a generic model for linking techniques with their target, uses, and treatment processes. In addition, how each component can be adapted according to CBT-specific elements of the client-therapist relationship (collaboration, empiricism, and Socratic dialogue) in a manner that is guided by the case conceptualization provides a more complex and comprehensive understanding of treatment delivery. The various research studies included within this special issue make important contributions to our understanding of the different ways in which both treatment and in-session processes are important to CBT.
Full-text available
The rapid advances and adoption of smartphone technology presents a novel opportunity for delivering mental health interventions on a population scale. Despite multi-sector investment along with wide-scale advertising and availability to the general population, the evidence supporting the use of smartphone apps in the treatment of depression has not been empirically evaluated. Thus, we conducted the first meta-analysis of smartphone apps for depressive symptoms. An electronic database search in May 2017 identified 18 eligible randomized controlled trials of 22 smartphone apps, with outcome data from 3,414 participants. Depressive symptoms were reduced significantly more from smartphone apps than control conditions (g=0.38, 95% CI: 0.24-0.52, p<0.001), with no evidence of publication bias. Smartphone interventions had a moderate positive effect in comparison to inactive controls (g=0.56, 95% CI: 0.38-0.74), but only a small effect in comparison to active control conditions (g=0.22, 95% CI: 0.10-0.33). Effects from smartphone-only interventions were greater than from interventions which incorporated other human/computerized aspects along the smartphone component, although the difference was not statistically significant. The studies of cognitive training apps had a significantly smaller effect size on depression outcomes (p=0.004) than those of apps focusing on mental health. The use of mood monitoring softwares, or interventions based on cognitive behavioral therapy, or apps incorporating aspects of mindfulness training, did not affect significantly study effect sizes. Overall, these results indicate that smartphone devices are a promising self-management tool for depression. Future research should aim to distil which aspects of these technologies produce beneficial effects, and for which populations.