User involvement in digital mental health: approaches, potential and the need for guidelines

Full text

EDITED BY
Uffe Kock Wiil,
University of Southern Denmark, Denmark
REVIEWED BY
Daniel Z. Q. Gan,
The University of Melbourne, Australia
*CORRESPONDENCE
Sylvie Bernaerts
sylvie.bernaerts@thomasmore.be
RECEIVED 29 May 2024
ACCEPTED 24 July 2024
PUBLISHED 22 August 2024
CITATION
Bernaerts S, Van Daele T, Carlsen CK,
Nielsen SL, Schaap J and Roke Y (2024) User
involvement in digital mental health:
approaches, potential and the need for
guidelines.
Front. Digit. Health 6:1440660.
doi: 10.3389/fdgth.2024.1440660
COPYRIGHT
© 2024 Bernaerts, Van Daele, Carlsen, Nielsen,
Schaap and Roke. This is an open-access
article distributed under the terms of the
Creative Commons Attribution License (CC
BY). The use, distribution or reproduction in
other forums is permitted, provided the
original author(s) and the copyright owner(s)
are credited and that the original publication in
this journal is cited, in accordance with
accepted academic practice. No use,
distribution or reproduction is permitted
which does not comply with these terms.
User involvement in
digital mental health:
approaches, potential and the
need for guidelines
Sylvie Bernaerts1*, Tom Van Daele1,2, Christian Korthé Carlsen3,
Søren Lange Nielsen3, Jolanda Schaap4and Yvette Roke4
1
Psychology and Technology, Centre of Expertise Care and Well-Being, Thomas More University of
Applied Sciences, Antwerp, Belgium,
2
Centre for Technological Innovation, Mental Health and
Education, Queen’s University Belfast, Belfast, United Kingdom,
3
Centre for Digital Psychiatry, Region of
Southern Denmark, Odense, Denmark,
4
Expertise Center for Autism Spectrum Disorder, GGZ Centraal,
Almere, Netherlands
KEYWORDS
digital mental health, user participation, end-user involvement, guidelines, adaptation
Introduction
Over the past decades, the importance of mental health is increasingly being
acknowledged, with more people reaching out for help. However, mental healthcare
struggles to help all in need. Those finding their way to formal services face long
waiting lists, while for others, the associated stigma is still too large to reach out. Both
cases result in unmet needs, which remain a pressing issue. One attempt to overcome
these challenges is to rely on digital mental health, the use of technology for mental
health interventions, ranging from promotion, prevention, and treatment to
maintenance. Technologies can for example include computers and smartphones,
extended reality, wearables, social media, chatbots (1) and may or may not make use of
artificial intelligence. A wealth of evidence already supports the efficacy and
effectiveness of online interventions for common mental health conditions, such as
depression and anxiety in older adults (2), and in adolescents and young people (3)as
well as –to a lesser extent - their cost-effectiveness (4). Despite this potential,
successful implementation of these interventions and other forms of digital mental
health has proven to be challenging, particularly concerning adaptation, uptake and
adherence (5). As this is a multifaceted challenge a single solution is non-existent.
Addressing this challenge requires taking into account many aspects and perspectives,
with implementation sciences gaining increased attention as a result. In this opinion
paper, we highlight user involvement as one important aspect in the development,
implementation and international adaptation of digital mental health interventions
which, to date, still often seems to be overlooked. In the following paragraphs we will
define the concept, highlight the potential of involving users to facilitate uptake of
digital mental health, and argue for the need for clear guidelines on how to do so, not
only for initial development, but also for subsequent international adaptation.
Approaches
In principle, user involvement means that different stakeholders or users are included
in one or more steps of the design process. In a context of digital mental health
TYPE Opinion
PUBLISHED 22 August 2024
|
DOI 10.3389/fdgth.2024.1440660
Frontiers in Digital Health 01 frontiersin.org

interventions (DMHIs), users might entail patients or clients and
their friends and family, clinicians, mental health organizations,
and developers. There are, however, multiple approaches to
involving these users, of which we highlight three methods: user-
centred design, participatory design (or co-design), and user
innovation. Within these approaches, the extent to which users
have control over design decisions varies: low in user-centred
design, higher in participatory design (or co-design), and the
highest in user innovation (6). According to Mao et al. (7), user-
centred design is a multidisciplinary design approach that
actively involves users to improve the understanding of both user
requirements and task requirements, as well as the iteration of
design and evaluation. In co-design, (potential) users are invited
to cooperate with designers, researchers and developers in an
innovation process starting from idea generation to decision
making (8). Although both approaches are very similar, the
difference lies in the starting point and in the extent of the user
involvement. In user-centred design, users act as consultants for
the designers (after a design idea has already been formulated)
and provide feedback throughout the design process. In co-
design, users are considered as partners throughout the process
from need exploration and idea generation onwards, which
ensures that user’s’needs and preferences are met and that
technologies are acceptable and helpful (9,10). Finally, in user
innovation, the design and development of new products or
services is started by end-users, either individual end- users or
intermediate users (e.g., organisations) (11). According to a
systematic review by Moore et al. (12) user-centred design has
been the most reported approach to involve end- users in digital
health innovations. However, co-design can be put forward as
the more sensible approach to involve vulnerable populations,
such as children (9) and older adults (13), but also individuals
with mental health conditions (14). Specific methods range from
brief user consultation through a review process all the way up
to true collaboration. There is not one clear method or process
of user involvement, yet common methods include focus groups,
surveys, interviews, prototype/storyboards, think-aloud exercises
and literature search (12). In particular, Sanz et al. (15) have
shown that studies using co-design methods, mostly rely on
interviews and workshops, followed by meetings and surveys. In
sum, although there is not a single method to involve users, the
most promising and sensible approach seems to be co- design:
involving users as partners rather than mere consultants from
the onset of the design process (9,13,16).
Potential
Throughout the years, multiple reviews highlighting the
relevance of user involvement during the development of DMHIs
have been conducted and the results have shown both similarities
and differences.
Torous et al. (14) theorized that low DMHI engagement could
be due to poor usability, lack of user-centric design and/or a lack of
trust (among other reasons), suggesting co-design with users as a
potential solution. Indeed, involving different users in the
development and implementation of DMHIs can limit known
barriers to uptake and engagement. For example, Liverpool et al.
(17) have shown that child and youth engagement with DMHIs
is influenced by intervention- specific factors, such as suitability,
usability and acceptability on the one hand, and person- specific
factors, such as motivation, opportunity and capability.
Similarly, a review by Borghouts et al. (18) has also identified
barriers and facilitators to user engagement with DMHIs, user-
related (e.g., beliefs, experience and skills), as well as
intervention-related (e.g., content, perceived fit and usefulness)
barriers and facilitators to user engagement with DMHIs. All of
these can be enhanced or tackled, respectively, by involving users
in a co-design process.
Orlowski et al. (19) found that user involvement (named
consumer consultation in their study) helped to shape specific
DMHIs for youth, but they also stated that the effects of user
involvement in intervention design are unclear due to limited
evidence on specific outcomes and insufficient implementation
after piloting in research. In addition, Fischer et al. (13)revealed
that involving older adults in technology design (not limited to
DMH) leads to better learning of the user’s needs, designs adjusted
to these needs and better quality of the design, but also showed
that effects on acceptance and uptake are unclear. In line with
these reviews, the findings of Bevan Jones et al. (9) corroborate the
notion that there is little evidence on the impact of user
involvement on uptake, adherence and intervention effectiveness.
In contrast, a more recent review has shown positive effects of
user involvement, in particular, to enhance cultural sensitivity,
enrich ideas, increase acceptance of the DMHIs, better
engagement and a sense of community (16). Taken together,
while research is unclear on specific outcomes concerning uptake
and efficacy, findings do support the importance of involving
users in a co-design process to tackle barriers and enhance
facilitators to uptake and engagement.
International adaptation of digital mental
health interventions
One promising application for user involvement is in the context
of international adaptation. Considering the vast number of available
DMHIs, in particular mental health apps (14), it is more sensible to
use resources to adapt existing, evidence-based DMHIs for use in
other countries, rather than to reinvent the wheel. In this respect,
adapting interventions to the proposed target population has been
a longstanding recommendation (20). Developing or designing
technologies to be used beyond a country’s borders, however,
requires particular considerations further than mere translation of
the particular intervention’s content. Involving users in the
adaptation process can, for example, help to inform about
potential user characteristics that may be associated with lower
adherence and/or higher drop-out rates (21).
One’s approach should therefore take into consideration the
target population’s cultural, clinical and regulatory aspects, to
name only a few. For example, the US has been dominating the
app market for smartphone-based mental health apps (22). This
Bernaerts et al. 10.3389/fdgth.2024.1440660
Frontiers in Digital Health 02 frontiersin.org

means that most apps, evidence-based or not, are primarily in
English and developed within the US context, adhering to local
regulations and referencing local services. Given that user
engagement with a DMHI is enhanced by perceived fit–how
well users feel the intervention has culturally appropriate content
and understandable language (18)–potential users from Europe,
Africa, or Asia might be less interested due to language barriers
and lack of cultural sensitivity. To the best of our knowledge,
however, evidence on best practices for international adaptation
of digital mental health interventions is limited, more so since
clear methods for developing and implementing apps in broader
international contexts are scarce. In one example, Storm et al.
(23) conducted usability tests of an American prototype app
called PeerTECH, a peer support app for individuals with a
serious mental health condition, with Norwegian users, including
clients, clinicians and peer support workers. By doing so,
researchers learned that app’s adaptation to the Norwegian
context would be viable and useful. However, no information on
concrete development steps was provided. In another example,
Bartlett et al. (24) assessed how well an Australian company
involved Arabic-speaking refugees, refugee advocates and
healthcare workers during a design thinking process. Their goal
was to develop a web-based application to deliver local, evidence-
based and culturally relevant health information to its non-
English speaking users. Based on their results, relevant
recommendations were suggested concerning key communication
principles to take into account. Nevertheless, a structured
approach for practitionersor researchers to involveusers from
different cultures was not discussed. We therefore argue that
more research assessing international user involvement is
necessary to inspire concrete guidelines for development and
international adaptation of digital mental health interventions.
Need for guidelines
Although researchers have provided frameworks,
recommendations and specific methods to involve users, this
information seems to be insufficiently specific, nor easily
retrievable for entrepreneurs and mental health organizations to
use. There is, therefore, a clear need for practice- oriented
guidelines aimed at stakeholders on different levels, such as
policy makers, entrepreneurs and mental health organizations, on
how to involve these different users and mental health
professionals in the development, implementation and adaptation
of mental health technology.
The formulation of these guidelines entails the consideration of
multiple critical factors, of which we will highlight four. As a first
point, guidelines require more consistency in terminology.
Literature on user involvement mentions the concepts of co-
design, co-production, co-creation, participatory design, user
involvement, etc, seemingly interchangeably. Although these
concepts each have their own definitions, and their
operationalisation sometimes also differs, the underlying notion
is the same, namely (the importance of) involving different
stakeholders, specifically users. A second point involves the
need for a comprehensive framework. Similar to the
terminology, there are multiple frameworks or theories
describing how to involve users, for example, the British
Design’s Double Diamond model (25),ortheGenerativeCo-
Design Framework for Healthcare Innovation (26). No study
to date, however, has appeared to have described the use of
the Double Diamond model for involving users to design a
DMHI. In addition, citation analysis shows that the latter has
mainly been referenced for its description of co-design
principles rather than for following the steps of the framework
itself. In one example, the StigmaBeat project has adopted the
framework to involve marginalized youngsters to develop short
films for reducing mental health stigma (27). In another
example, parents of children with cancer were involved in the
co-design process of a paediatric cancer pain management app
(28). However, in both cases, no information is provided on
resulting adoption or user engagement. A third point of
attention constitutes the gaps in evidence concerning
development of digital mental health interventions. A recent
review by Brotherdale et al. (16) on co-production for digital
mental health interventions revealed that there is considerable
variability concerning which users to involve, the stage and
role of their involvement, which methods are used, which
frameworks are implemented and how to deal with power
dynamics between designers or producers and users, making it
difficult to provide evidence-based guidelines. Notwithstanding
these gaps, Brotherdale et al. (16)havealsoidentified several
commonalities among studies. Successful involvement of users
is often hindered by resource constraints, recruitment
challenges, conflicting views within the stakeholders and
power imbalances between users and designers. It is, therefore,
important to suggest potential (evidence-based) solutions and
clearly defined steps on how to tackle these barriers. As a
fourth and final point, there are, to the best of our knowledge,
no evidence-based recommendations for international
adaptation of available digital mental health interventions. It
is,however,essentialtoinvolvelocalstakeholdersascultural
and regulatory variations between nations are plausible. In
light of these evolutions, one initiative that aims to contribute
to the aforementioned challenges is the “Successful User
Participation Examples and Recommendations”-project
(SUPER). Funded by Interreg North Sea Region, it aims to
develop guidelines for entrepreneurs and mental health
organizations on how to involve different stakeholders, in
particular users such as patients and mental health professionals,
in the (transnational) development, implementation and
adaptation of mental health technology.
Conclusion
Successful implementation of digital mental health
interventions has proven challenging, and in this opinion paper
we wanted to argue that user involvement has the potential to
provide at least part of the solution. Although evidence on the
impact of user involvement on intervention effectiveness is
Bernaerts et al. 10.3389/fdgth.2024.1440660
Frontiers in Digital Health 03 frontiersin.org

lacking, its added value for increasing cultural sensitivity, enriching
ideas, and increasing acceptance of the digital mental health
interventions, and improve engagement is clear (16).
Nevertheless, translation to practice is hampered by the fact that
clear user involvement steps are rarely properly documented and
reported in research. Moreover, concrete evidence-based (or even
evidence-inspired) guidelines and steps are lacking, making it
difficult for practitioners, developers, and healthcare
organizations to adequately involve relevant stakeholders in the
design and development process, as well in the increasingly
common international adaptation of digital mental health
applications. Initiatives, such as the SUPER project, are currently
underway to help offer a concrete framework and guidelines.
Nevertheless, this will still require uptake in research, as well as
practice, to lead to improved user involvement and, ideally, also
better digital mental health.
Author contributions
SB: Conceptualization, Writing –original draft, Writing –
review & editing. TV: Conceptualization, Writing –review &
editing. CC: Writing –review & editing. SN: Writing –review
&editing.JS:Writing–review & editing. YR: Writing –
review & editing.
Funding
The author(s) declare financial support was received for the
research, authorship, and/or publication of this article.
The Successful User Participation Examples and
Recommendations (SUPER) project received funding from the
European Union’s Interreg North Sea programme.
Conflict of interest
The authors declare that the research was conducted in the
absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board
member of Frontiers, at the time of submission. This had no
impact on the peer review process and the final decision.
Publisher’s note
All claims expressed in this article are solely those of the authors
and do not necessarily represent those of their affiliated organizations,
or those of the publisher, the editors and the reviewers. Any product
that may be evaluated in this article, or claim that may be made by its
manufacturer, is not guaranteed or endorsed by the publisher.
References
1. De Witte NAJ, Joris S, Van Assche E, Van Daele T. Technological and digital
interventions for mental health and wellbeing: an overview of systematic reviews.
Front Digit Health. (2021) 3:754337. doi: 10.3389/FDGTH.2021.754337/BIBTEX
2. Dworschak C, Heim E, Maercker A. Efficacy of internet-based interventions for
common mental disorder symptoms and psychosocial problems in older adults: a
systematic review and meta-analysis. Internet Interv. (2022) 27. doi: 10.1016/J.
INVENT.2022.100498
3. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental
health interventions for adolescents and young people: systematic overview. JMIR
Ment Health. (2021) 8(4):e25847. doi: 10.2196/25847
4. Rohrbach PJ, Dingemans AE, Evers C, Van Furth EF, Spinhoven P, Aardoom JJ,
et al. Cost-effectiveness of internet interventions compared with treatment as usual for
people with mental disorders: systematic review and meta-analysis of randomized
controlled trials. J Med Internet Res. (2023) 25:e38204. doi: 10.2196/38204
5. Berardi C, Antonini M, Jordan Z, Wechtler H, Paolucci F, Hinwood M. Barriers
and facilitators to the implementation of digital technologies in mental health systems:
a qualitative systematic review to inform a policy framework. BMC Health Serv Res.
(2024) 24(1):1–19. doi: 10.1186/S12913-023-10536-1
6. Jarke J. Co-creating digital public services. Public Admin Inf Technol. (2021)
6:15–52. doi: 10.1007/978-3-030-52873-7_3/TABLES/4
7. Mao JY, Vredenburg K, Smith PW, Carey T. The state of user-centered design
practice. Commun ACM. (2005) 48(3):105–9. doi: 10.1145/1047671.1047677
8. Sanders EBN, Stappers PJ. Co-creation and the new landscapes of design. Co-
Design. (2008) 4(1):5–18. doi: 10.1080/15710880701875068
9. Bevan Jones R, Stallard P, Agha SS, Rice S, Werner-Seidler A, Stasiak K, et al.
Practitioner review: co-design of digital mental health technologies with children
and young people. J Child Psychol Psychiatry Allied Discip. (2020) 61(8):928–40.
doi: 10.1111/JCPP.13258
10. Hodson E, Dadashi N, Delgado R, Chisholm C, Sgrignoli R, Swaine R. Co-
design in mental health; mellow: a self-help holistic crisis planning mobile
application by youth, for youth. Design J. (2019) 22(sup1):1529–42. doi: 10.1080/
14606925.2019.1594975
11. Bogers M, Afuah A, Bastian B. Users as innovators: a review, critique, and future
research directions. J Manage. (2010) 36(4):857–75. doi: 10.1177/0149206309353944
12. Moore G, Wilding H, Gray K, Castle D. Participatory methods to engage health
service users in the development of electronic health resources: systematic review.
J Particip Med. (2019) 11(1):e11474. doi: 10.2196/11474
13. Fischer B, Peine A, Östlund B. The importance of user involvement: a systematic
review of involving older users in technology design. Gerontologist. (2020) 60
(7):513–23. doi: 10.1093/geront/gnz163
14. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user
engagement with mental health smartphone apps: evidence, theory and
improvements. BMJ Ment Health. (2018) 21(3):116–9. doi: 10.1136/eb-2018-102891
15. Sanz MF, Acha BV, García MF. Co-design for people-centred care digital
solutions: a literature review. Int J Integr Care. (2021) 21(2):16. doi: 10.5334/IJIC.5573
16. Brotherdale R, Berry K, Branitsky A, Bucci S. Co-producing digital mental health
interventions: a systematic review. Digital Health. (2024) 10. doi: 10.1177/
20552076241239172
17. Liverpool S, Mota CP, Sales CMD, ČušA, Carletto S, Hancheva C, et al.
Engaging children and young people in digital mental health interventions:
systematic review of modes of delivery, facilitators, and barriers. J Med Internet Res.
(2020) 22(6):e16317. doi: 10.2196/16317
18. Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, et al.
Barriers to and facilitators of user engagement with digital mental health interventions:
systematic review. J Med Internet Res. (2021) 23(3):e24387. doi: 10.2196/24387
19.OrlowskiSK,LawnS,VenningA,WinsallM,JonesGM,WyldK,etal.Participatory
research as one piece of the puzzle: a systematic review of consumer involvement in
design of technology-based youth mental health and well-being interventions. JMIR
Human Factors. (2015) 2(2):e12. doi: 10.2196/HUMANFACTORS.4361
20. Van Daele T, Karekla M, Kassianos AP, Compare A, Haddouk L, Salgado J, et al.
Recommendations for policy and practice of telepsychotherapy and e-mental health in
Europe and beyond. J Psychother Integr. (2020) 30(2):160–73. doi: 10.1037/int0000218
21. Karekla M, Kasinopoulos O, Neto DD, Ebert DD, Van Daele T, Nordgreen T,
et al. Best practices and recommendations for digital interventions to improve
engagement and adherence in chronic illness sufferers. Eur Psychol. (2019) 24
(1):49–67. doi: 10.1027/1016-9040/A000349
22. Grand View Research. Mental Health Apps Market Size & Trends (2022).
Available online at: https://www.grandviewresearch.com/industry-analysis/mental-
health-apps-market-report (accessed July 5, 2024).
Bernaerts et al. 10.3389/fdgth.2024.1440660
Frontiers in Digital Health 04 frontiersin.org

23. Storm M, Fjellså HMH, Skjærpe JN, Myers AL, Bartels SJ, Fortuna KL. Usability
testing of a Mobile health application for self-management of serious mental illness in
a Norwegian community mental health setting. Int J Environ Res Public Health. (2021)
18(16):8667. doi: 10.3390/IJERPH18168667
24. Bartlett R, Boyle JA, Simons Smith J, Khan N, Robinson T, Ramaswamy R.
Evaluating human-centred design for public health: a case study on developing a
healthcare app with refugee communities. Res Involv Engagem. (2021) 7(1):1–13.
doi: 10.1186/S40900-021-00273-2/TABLES/4
25. The Double Diamond - Design Council (n.d.). Available online at: https://
www.designcouncil.org.uk/our-resources/the-double-diamond/ (accessed July 15,
2024).
26. Bird M, McGillion M, Chambers EM, Dix J, Fajardo CJ, Gilmour M, et al. A
generative co-design framework for healthcare innovation: development and
application of an end-user engagement framework. Res Involv Engagem. (2021) 7
(1):1–12. doi: 10.1186/S40900-021-00252-7/FIGURES/1
27. Hine R, Gladstone B, Reupert A, O’Dea L, Cuff R, Yates S, et al. Stigmabeat:
collaborating with rural young people to co-design films aimed at reducing mental
health stigma. Qual Health Res. (2024) 34(6):491–506. doi: 10.1177/10497323231211454
28. Jibb LA, Sivaratnam S, Hashemi E, Chu CH, Nathan PC, Chartrand J, et al.
Parent and clinician perceptions and recommendations on a pediatric cancer pain
management app: a qualitative co-design study. PLoS Digit Health. (2023) 2(11):
e0000169. doi: 10.1371/journal.pdig.0000169
Bernaerts et al. 10.3389/fdgth.2024.1440660
Frontiers in Digital Health 05 frontiersin.org