Access to this full-text is provided by SAGE Publications Inc.
Content available from Digital Health
This content is subject to copyright.
mHealth App to improve medication adherence
among older adult stroke survivors:
Development and usability study
Wenjing Cao
1,2
, Juan Wang
3
, Yuhui Wang
4
, Intan Idiana Hassan
2
and Azidah Abdul Kadir
5
Abstract
Background: Effective medication adherence is vital for older adult stroke survivors, yet 20–33% cease treatment within a
year post-discharge, increasing risks of recurrent strokes and mortality. A mobile health (mHealth) app could be a novel
tool to improve medication adherence among stroke survivors because of its potential to increase patient empowerment.
A few stroke-related apps provide information and support to stroke survivors. However, none have focused on medication
adherence and documented their development and evaluation process, particularly those focused on this older population.
Objective: This study aims to design and develop a smartphone app called OASapp to improve medication adherence among
older adult stroke survivors and evaluate its usability.
Methods: OASapp was developed in a three-phase development process. Phase 1 is the exploration phase (including a cross-
sectional survey, a systematic review, a search for stroke apps on the app stores of Apple App Store and Google Play Store,
and a nominal group technique). In phase 2, a prototype was designed based on the Health Belief Model and Technology
Acceptance Model. In phase 3, Alpha and Beta testing was conducted to validate the app.
Results: Twenty-five features for inclusion in the app were collected in round one, and 14 features remained and were ranked by
the participants during nominal group technique. OASapp included five core components (medication management, risk factor
management, health information, communication, and stroke map). Users of OASapp were satisfiedbasedonreportsfrom
Alpha and Beta testing. The mean Usability Metric for User Experience (UMUX) score was 71.4 points (SD 14.6 points).
Conclusion: OASapp was successfully developed using comprehensive, robust, and theory-based methods and was found to
be highly accepted by users. Further research is needed to establish the clinical efficacy of the app so that it can be utilized to
improve clinically relevant outcomes.
Keywords
mHealth app, mHealth development process, medication adherence, eHealth, mHealth, mobile apps, usability, older adult
stroke survivors
Submission date: 5 August 2023; Acceptance date: 14 February 2024
1
Xiangnan University, Chenzhou, Hunan Province, China
2
School of Health Sciences, Universiti Sains Malaysia, Kota Baru, Kelantan, Malaysia
3
Guangzhou University of Chinese Medicine, Guangzhou, Guangdong Province, China
4
Central South University, Changsha, Hunan Province, China
5
Department of Family Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kubang Kerian, Kelantan, Malaysia
Corresponding author:
Azidah Abdul Kadir, Department of Family Medicine, School of Medical Sciences, Health Campus, Universiti Sains Malaysia, 16150, Kubang Kerian, Kelantan,
Malaysia.
Email: azidahkb@usm.my
Creative Commons NonCommercial-NoDerivs CC BY-NC-ND: This article is distributed under the terms of the Creative Commons Attribution-
NoDerivs 4.0 License (https://creativecommons.org/licenses/by-nc-nd/4.0/) which permits any use, reproduction and distribution of the work as
published without adaptation or alteration, provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/
en-us/nam/open-access-at-sage).
Original Research Article
DIGITAL HEALTH
Volume 10: 1–15
© The Author(s) 2024
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/20552076241236291
journals.sagepub.com/home/dhj
Introduction
Stroke remains the second leading cause of death and a
major contributor to disability worldwide. The estimated
cases of stroke would continuously increase due to the sub-
optimal management of stroke-related diseases (e.g. hyper-
tension, atrial fibrillation, hyperlipidemia, and diabetes).
1
Medication adherence is considered a substantial and
crucial attempt to prevent stroke.
2
However, there is a
large volume of published studies describing medication
adherence among stroke survivors is problematic.
3,4
Adhering to medication can be challenging for stroke
patients due to concerns about long-term use, memory chal-
lenges, financial constraints, side effects, and regimen com-
plexity.
5–7
A consensus among social scientists indicates
that medication non-adherence is associated with increased
risk of stroke events and all-cause mortality.
8,9
As such,
measures should be implemented to enhance medication
adherence among this population. Brown et al.
10
demon-
strated that improving medication adherence may have
greater effect on the population’s health than any improve-
ments in new therapies.
Several attempts have been implemented to improve
medication adherence among stroke survivors. However,
Wessol et al.
11
conducted randomized controlled trial inter-
ventions to increase medication adherence in adult stroke
survivors and concluded that despite some isolated
success, most interventions had shown limited effective-
ness. The worldwide development and adoption of afford-
able smartphones in the last several years has brought
enormous advantages toward influencing the behavior of
patients. The advancement of apps is unavoidable and has
the potential to facilitate interventions because they can
improve engagement with established strategies for preven-
tion and treatment of diseases through individualized
dosing reminders, personalized goal setting, and gamifica-
tion.
12
Zeng et al.
13
indicated that the development and
testing of mHealth intervention programs for stroke survi-
vors is still at an early stage, and evidence of their effects
are limited. Furthermore, apps are increasingly used to
assist older people in staying connected to friends and
family, remaining active, and self-managing chronic ill-
nesses.
14,15
Many medication adherence apps are currently
available given the accessibility and widespread use of
mobile phones
16
; however, these apps are not specificto
diseases, such as stroke. Piran et al.
17
pointed out that
most apps exist to specifically support stroke survivors/
caregivers and primarily focus on language and communi-
cation difficulties instead of medication adherence. Our
recent review found that a feature related to medication is
not one of the primary features of stroke-related apps.
18
A few stroke-related apps provide information and
support to stroke survivors, but none have comprehensively
documented their development process and usability, which
will allow future researchers to develop similar mHealth
apps and may be especially useful for the more innovative
aspects of mHealth (e.g. design principles, technical fea-
tures, and the presentation of information). Moreover, a
theory-based app could stress and sustain health promotion
behavior change; otherwise, the app’s effectiveness could
be limited.
19,20
The Health Belief Model (HBM) and
Technology Acceptance Model (TAM) provide a well-
defined framework for the development of apps.
21,22
Accordingly, this study reports on the design and develop-
ment of a mHealth app, namely, OAS, based on HBM and
TAM to improve medication adherence among older adult
stroke survivors.
Methods
Overview
The app was developed in three phases (Figure 1). The
researcher conducted the development process with a technol-
ogy team and an expert advisory panel (EAP). The EAP con-
sisted of three stroke experts, a nurse, a doctor, and a
pharmacist, each with abundant experience in clinic care tar-
geting stroke survivors. The experts were regarded as
project consultants, and they provided consultation or feed-
back throughout the prototype design and development
process. The technology team was responsible for the proto-
type’s design and programming and consisted of four people
(e.g. a technology director, an app developer, a user experi-
ence and graphic designer, and a database engineer).
Phase 1 involves three steps. In step 1, a cross-sectional
study was conducted to determine the prevalence of medi-
cation non-adherence and its associated factors among older
adult stroke survivors in China. This step is important in
creating an appropriate strategy to improve medication
adherence. In step 2, a systematic review of stroke apps
Figure 1. App development process.
2DIGITAL HEALTH
in the literature, Apple App Store, and Google Play Store was
conducted to gather ideas for the development and content of
the app. In step 3, a nominal group technique (NGT) was
applied to develop the framework and content of the app.
Phase 2 included the design of the app and development of
its overall structure. In phase 3, Alpha and Beta testing was
conducted to validate the relevance, appropriateness, practical-
ity, and clear aspects of the mobile app.
Phase 1: exploration phase
Step 1: survey of medication non-adherence and associated
factors among older adult stroke survivors in China. A cross-
sectional study on 402 older adult stroke survivors was con-
ducted from June 2022 to August 2022 by using the follow-
ing questionnaires: the General Medication Adherence
Scale,
23
Beliefs about Medicines Questionnaire,
24
and
Health Literacy Scale for Stroke Patients.
25
The details of
the methodology have been described in a previously pub-
lished study.
3
The points generated from the survey regard-
ing app development were as follows: the app should
provide information on the necessity of treatments as well
as the known side effects of prescribed drugs and help par-
ticipants to share their concerns; older adult stroke survivors
are more adherence to medications if it meets their broader
health needs medical conditions (e.g. hypertension, history
of cardiac-related comorbidities, diabetes, etc.); the app
should utilize a reminder function to facilitate adherence
to medications; and additional information on hypertension
and diabetes would be necessary.
Step 2: systematic review of stroke apps in the literature, Apple
App Store and Google Play Store
Overview. We conducted two reviews, namely, a litera-
ture review and research in commercial application stores.
The study design for review of commercial apps has been
reported elsewhere.
18
The details of the literature can be
found in our previously published paper.
26
Some important
conclusions can be obtained from the literature review: an
increasing number of apps targeting stroke survivors are
released and mostly geared toward young and middle-aged
patients. Developers must address the needs of older age
groups and offer enhanced accessibility features. When
developing the app, enhanced accessibility features
should be offered to increase to older adults groups. For
example, the app could launch a care mode, which has
large and clear words, vivid colors, good recognition, and
large buttons for ease of use.
Step 3: NGT. NGT was conducted to explore three elements:
1. Identify the main issue of medication compliance in
elderly diagnosed with stroke,
2. Present the literature review and available apps in the
play store, and
3. Identify and prioritize a list of key app features suitable
for older adult stroke survivors.
NGT involves a highly structured face-to-face meeting that
provides an orderly procedure for obtaining relevant and reli-
able qualitative information from target groups that are related
to the problem area.
27
This technique has been applied in
numerous studies to produce ideas and identify solutions
within groups and generate recommendations for best prac-
tices.
28,29
NGT comprises four key stages: silent generation,
round robin, clarification, and voting (ranking or rating).
30
The facilitators presented the content and features of the pro-
posed apps and the techniques and elements suitable for older
adult stroke survivors were used for discussion.
Two weeks before the session, the researcher selected
and invited the participants for a group session. The
researcher briefly discussed the topic and the expected
outcome of the session.
Nine participants were included in the discussion group:
1. One neurologist
2. One nursing specialist
3. One education specialist
4. Two pharmacy specialists
5. One public health physician
6. Three representatives from intended users (all of them
are older adult stroke survivors above 60 years old).
Two moderators from the research team conducted the
session. NGT consisted of six phases (e.g. introduction to
the purpose and procedure of the meeting, silent generation
of ideas, sharing ideas, discussion of ideas, voting and
ranking, and conclusion) which was adopted from the
studies of Potter et al.
31
and McMillan et al.
32
:
To be specific, facilitators conducted a literature review
on the features of a mobile application (e.g. emergency
information collection, risk assessment, reminders, educa-
tional information, self-monitoring of blood pressure/
lipids/glucose, rehabilitation guidance, communication
with healthcare providers and others, and psychological
support) expected by users from published research; they
shared the findings to the participants and asked them to
rank the answers accordingly. The flow of the whole discus-
sion in NGT is shown in Figure 2.
Phase 2: design of app prototype
Inspired by the conception of storyboard in the movie
industry,
32
we intended to generate a storyboard to map
out and organize ideas based on the findings from the app
store review, literature search, cross-sectional study, and
NGT in the exploration phase before actually developing
the app. The storyboard is the blueprint for the app and
Cao et al. 3
describes all the elements that would appear on a single
screen. Figure 3 shows the storyboard of the app.
The development of the app prototype can be defined as
translating the storyboard into a whole app design. Two
project researchers underwent two mobile application devel-
opment workshops to develop the prototype. The develop-
ment was based on a framework of guiding interface design
for older adults adopted from the study of Liu & Joines
33
as
well as guidelines for developing apps for elderly people
adopted from the work of Phiriyapokanon.
34
Furthermore,
the app was developed based on HBM and TAM
(Figure 4). During the prototype development, the researchers
considered all inputs from the NGT session. The prototype
was developed on the Android platform.
Phase 3: validation of app
Alpha and Beta testing was conducted to validate the rele-
vance, appropriateness, practicality, and clear aspects of the
mobile app.
Alpha testing. Alpha testing is known as simulated oper-
ational testing on apps that have been developed to identify
and solve problems that have arisen. Alpha testing was con-
ducted by involving content experts and an app expert. A
stroke specialist and a senior lecturer, who has PhD qualifi-
cation and is an expert in medication adherence, were
involved as content experts in the Alpha testing. The
content experts were requested to evaluate the app by
filling out an evaluation form (Appendix 1), which was
adapted from a previous study.
35
An app expert, whose
research interests are usability assessment and product
design, was also involved in the Alpha testing. He was
responsible for evaluating the usability of the app. After
evaluating the app, the app expert was asked to fill out
the heuristic evaluation form (Appendix 2) adapted from
a previous work.
36
The experts assessed the comprehensibility, clarity, and
technical errors encountered when using the app. Feedback
from the content and app experts involved was recorded to
improve the quality of the app. Based on the Alpha testing
results, the researcher carried out modifications to improve
the quality of the app.
Beta testing. After all amendments based on the comments
of the experts involved, Beta testing was carried out on the
target group, namely, older adults stroke survivors. Beta
testing aims to determine (in conjunction with data from
Alpha testing) whether the developed prototype meets
users’needs in ways that would lead to sustained adop-
tion.
37
According to Mohd & Shahbodin,
38
Beta test is
carried out to serve as a trial before conducting a real test.
Beta testing determines the usability of the prototype. It is
formal process used to understand the usefulness and
usability of an application that has been developed.
39
Usefulness is related to how useful the developed applica-
tion is. Usability refers to the technical operation of the
application, which includes two aspects, namely, interface
and interaction. Through this test, the researcher determined
the effectiveness of the improvements after the Alpha
testing. The researcher was also able to identify any weak-
ness or problem about the use of the developed software
from the perspective of the target user group.
From the viewpoint of Allesi and Trollip,
39
there are
seven steps that must be executed in Beta testing, namely:
(i) Select the participant –in our study, older adults stroke
survivors were selected for Beta testing. We enrolled
patients from the neurology department of three tertiary
hospitals in Chenzhou City of Hunan Province, China
(Affiliated Hospital of Xiangnan University,
Chenzhou No. 1 People’s Hospital, and Chenzhou
Third People’s Hospital) through purposeful sampling
to obtain a maximum variation sample of key stake-
holders. The inclusion criteria were as follows: age of
60 years or older; history of stroke; modified Rankin
score of three or less; taken at least one medication in
the previous month, such as (but not limited to) anti-
platelets, statins, and anti-hypertensives to control risk
factors for strokes; last stroke episode that occurred
more than a month; and ability to read Chinese and
communicate in Mandarin Chinese or the local
Chenzhou dialect. Patients who had the following
were excluded: psychiatric illness or deafness,
aphasia, or other language barriers; and cognitive
impairment (Mini-Mental State Examination score
≤17 [for illiterate] or ≤20 [individuals with 1–6years
of education] or ≤24 [individuals with 7 or more
years of education]). The information sheet and
Figure 2. Flow of nominal group technique (adopted from the
studies of McMillan et al.
30
and Potter et al.
31
).
4DIGITAL HEALTH
consent form were given to the patients who indicated
willingness to participate.
(ii) Explain the procedure and the purpose –the
researcher explained the procedure and purpose of
the Beta test to the participants.
(iii) Determine prior knowledge –the researcher determined
how much knowledge the patient has about stroke.
(iv) Observe them throughout the whole process –the
researcher observed the participants and recorded pat-
terns that emerged from their interactions with the
prototype through the program.
(v) Interview them afterward –after the participants
interacted with the app prototype, the researcher
interviewed them to provide feedback on its usability
and content.
(vi) Assess their learning –after the participants inter-
acted with the app prototype, the researcher
evaluated how much of the learning process
they had.
(vii) Revise the program –the researcher created a plan
that covers information about revisions conducted,
time needed, and deadline for each revision.
Figure 3. Storyboard of the app.
Cao et al. 5
Beta testing was divided into two levels, namely, (1)
one-to-one evaluation and (2) small group evaluation.
One-to-one evaluation. Three older adults stroke survi-
vors were involved in the evaluation. The participants
were given iPads with a pre-loaded app prototype. The
researcher kept the provision of guidance on how to use
the app to a minimum to allow the participants to explore
the prototype freely and ensure the identification of any
potentially unclear features or instructions in the prototype.
The participants were with the researcher individually and
asked to explore the app under the supervision of the
researcher. The researcher observed the participants
without interrupting them and recorded patterns that
emerged from their reaction towards the app. The partici-
pants were then asked to provide detailed feedback on the
app, particularly on its content and operation.
Small group evaluation. The researchers helped the
patients to download the app, discussed the purpose of
the study, and conducted a brief demonstration of the fea-
tures of the app (e.g. logging into the app on a mobile
device, setting medication lists and reminding, using the
stroke map, making a BP/ blood glucose entry, viewing a
BP graph and sharing it with friends by WeChat, sharing
Figure 4. Relationship between the constitutive definitions of HBM and the contents generated in the conceptual modeling of the
application (top) and TAM (bottom).
6DIGITAL HEALTH
experience with others, and reviewing the entered health
information). The participants were asked to complete pre-
specified tasks with the mobile app independently to ensure
that they could use all areas/functions of the app. They were
then provided with the details of using the app and were
asked to test the app for 7 days.
All the participants were given the opportunity to
explore the developed app freely, and they were asked to
evaluate the app in terms of usability by filling out the
form called the Chinese version of the Usability Metric
for User Experience (UMUX) adapted from the study of
Wang et al.
40
UMUX is a standard usability questionnaire
designed to measure perceived usability consistent with
the System Usability Scale (SUS) but using only four
items (rather than 10 items).
41
The questionnaire is
compact enough to serve as a usability module in a
broader user experience metric.
41
A reliability analysis of
the merged data set resulted in a Cronbach’s alpha value
of 0.83 for UMUX.
42
Lewis et al.
43
indicated that UMUX
provides an alternative metric for perceived usability for
situations where reducing the number of items is critical
while obtaining accurate usability assessment. The items
vary in tone; odd-numbered items have a positive tone,
and even-numbered items have a negative tone.
44
UMUX
adopts a seven-point Likert scale (1 =Strongly Disagree
to 7 =Strongly Agree). It is scored as [score-1] for items
one and three and [7-score] for items two and four. The
scores for each item are summed, divided by 24, and multi-
plied by 100.
45
The reliability of the Chinese UMUX is
0.824, and the construct validity KMO of UMUX is
0.787.
40
The correlation between UMUX and SUS is r
(173) =0.829 (p< 0.01).
40
Results
NGT
The NGT results present the various features of the app that
were ranked according to their importance. In total, 25 fea-
tures were collected for inclusion in the app in round one.
After selecting, prioritizing, and discussing these features
in rounds two and three, 14 features remained and were
ranked by the participants. The results of the voting cards
were reviewed to determine the features that received the
most votes and the ranking of the votes. Of the 14 features
identified by the participants, medication reminder was the
top feature voted as essential for inclusion in the app. It was
followed by track BP/lipids/glucose, disease information,
pharmacy information, and communication (Table 1).
Furthermore, the NGT findings were used to build a
Table 1. Number of participants who chose specific features.
Feature Number of votes Top 5 highest priority
Medication reminders 9 (100%) 1
A drug–drug interaction check 2 (22.2%)
Risk assessment 5 (55.6%)
Record medical history 2 (22.2%)
Pharmacy information 8 (88.9%) 2
Disease information 8 (88.9%) 2
Export information from app 3 (33.3%)
Hospital information 4 (44.4%)
Track blood pressure/lipids/glucose 8 (88.9%) 2
Versatility of medication information input and display 4 (44.4%)
Communication 7 (77.8%) 3
Work with wearables 2 (22.2%)
Important contacts 3 (33.3%)
Symptom management 5 (55.6%)
Cao et al. 7
framework of the app (Table 2). The prototype version of
the app is registered for national copyright from the
China Government (Appendix 3).
Prototype features
OASapp is composed of five components (Figure 5):
- Medication management component
- Risk factor management component
- Health information component
- Communication component
- Stroke map component
Figure 5 shows a screenshot of the app.
Table 2. Framework of the app.
No. Items Description
1. Name of the app OASapp, the abbreviation of older adult stroke application
2. Objective To improve medication adherence among older adults stroke survivors
3. Platform for publishing
the app
Android platform
4. Main functions Section “Medication management”
This section has a medication list with reminders to take appropriate doses. It can generate adherence
reports and allow a patient to visualize their adherence. The medication lists include indications to
ensure that patients are educated. Patients could involve family members or healthcare providers
by sharing their medication lists via WeChat.
Section “Risk factor management”
This section allows for the self-reporting and storage of BP, capillary glycemia level, blood lipid
profile, and BMI readings, producing feedback in the form of graphs and numerical data logs. An
export function allows users to share these data via WeChat.
Section “Health information”
This section lists patients’knowledge of stroke, such as its definition, signs, and symptoms. It is
presented in short videos and textual graphic presentations to improve the effectiveness of
information dissemination. Voice Broadcast is applied.
Section “Communication”
This section allows patients to chat with health care professionals (HCPs) and other patients. Patients
can also browse a list of frequently asked questions.
Section “Stroke map”
This section helps patients to identify stroke signs and symptoms with the so-called “Stroke 1-2-0”
interface, where 1 represents “First, look for an uneven face,”2 refers to “Second, examine for arm
weakness,”and 0 refers to “Zero (absence of) clear speech.”If a stroke sign is suspected or
identified through the three-step procedure, the patient can link the quick recognition to the
immediate activation of emergency medical services (EMS) by dialing 1-2-0 to reduce stroke
prehospital delay. In addition, nearby hospitals are provided in map and list views, incorporating
proximity to the user’s location by using the Global Positioning System.
5. Other ancillary functions Patients can store their health data in the app (e.g. blood test results, imaging examinations).
This section allows users to store the emergency contact numbers of family and provides emergency
access to them through the app.
Patients can access their favorite health information from the “Favorites”screen.
8DIGITAL HEALTH
Each component is described in detail below.
The medication management component encourages
patients to upload their medication schedule (medication
name, medication appearance, dosage, unit, and frequency).
Patients can receive customized reminders to take their
tablets. The medication lists include indications, so patients
are educated and make associations with each medication.
Participants can add a picture to provide visual clues (e.g.
to ensure correct medication is taken). Medication adher-
ence reports can be generated and shared with a healthcare
provider or family member by WeChat.
The risk factor management component allows users to
input BP, capillary glycemia level, blood lipid profile, and
BMI measurements; and view numerical logs as well as
1-week graphs of BP, capillary glycemia level, blood
lipid profile, and BMI data. These data can be shared with
a medical team or family member by WeChat; as such, the
user has easy access to the stored information for future
acute or chronic care visits to healthcare providers. If a partici-
pant records a BP with systolic reading greater than 180
mmHg or less than 100 mmHg or a diastolic reading of
greater than 110 mmHg, then the user will be guided by auto-
mated notifications within the app at the time of the elevated
reading regarding what to do next. Figure 6 shows an
example of a push notification in the app.
The health information component provides users with
knowledge on stroke. Topics include the definition, epi-
demiology, harmful effects, and risk factors of stroke as
well as how to identify it. Topics specific to hypertension
and diabetes are also provided. All topics were created
based on literature review, multidisciplinary expert group
meeting, and user needs analysis. All health knowledge
were broadcasted by voice.
The communication component provides a platform for
active collaboration between healthcare professionals and
patients. Users can consult health care professionals about
stroke-related questions and share their feelings with other
patients.
The stroke map component helps educate users on the key
symptoms to look out for and allows them to directly dial 120
(Emergency Medical Services contact all over China). The
component adopts a stroke screening tool based on the
“Stroke 1-2-0”program, which was developed by adapting
Face, Arm, Speech, and Time (FAST) and is regarded as a
rapid response program for stroke in China. Stroke 1-2-0 is
transformed into three stroke recognition actions, where 1
represents “First, look for an uneven face,”2represents
“Second, examine for arm weakness,”and 0 represents
“Zero (absence of) clear speech.”
46
If a user taps an icon repre-
senting a symptom, then a pop-up window shows that the user
may be having a stroke, informs the user that it may be a stroke,
and directs the user to call emergency services immediately.
The user also has access to view the nearest hospitals and
his/her location by the help of the Global Positioning System
(a built-in function of smartphones).
Figure 5. Screenshots of OASapp, including its (A) home page, (B)
medication management component, (C) risk factor management
component, (D) health information component, (E) communication
component, and (F) stroke map component.
Cao et al. 9
The app launches the care mode, which has large and clear
words, vivid colors, good recognition, and large buttons for
ease of use of older adult stroke survivors (Figure 7).
Alpha testing
Several minor flaws (e.g. wrong spelling) were identified by
the content expert. In addition, the content expert suggested
the addition of more information on hypertension and diabetes.
The app expert gave several pieces of advice, such as using the
same font size and color throughout the app. The issues
encountered in the Alpha testing are summarized in Table 3.
Beta testing
One-to-one evaluation. All the three participants were satis-
fied with the app during one-to-one survey and provided
positive feedback. The application usability fulfilled the cri-
teria; for example, it is easy to use and has simple interface
design and appropriate color schemes as well as high-
resolution graphics. No programming or technical errors
undermined the use of the app based on the evaluation.
Small group evaluation. A total of 15 older adult stroke survi-
vors participated including 10 men (66.7%) and 5 women
(33.3%). The participants were aged 60 to 87 years, with an
average age of 71.33 years. Eight (53.3%) participants had
a primary school education or less; 46.7% lived with
spouse; the majority had comorbidities (86.7%); and 66.7%
of them were married. The mean total types of prescription
medications per day was 3.73 (1.62), and the mean duration
of stroke disease was 40.8 months (range 12–240 months).
An overall UMUX score of 71.4 (SD 14.6) was obtained,
which was higher than 68 on the SUS score, which is the
minimum cut-off point for a usable system; this finding the
good usability of OASapp as a mobile app. Based on Beta
testing, older adult stroke survivors provided some sugges-
tions regarding adjustments: information should be stepwise,
brief, and short (e.g. presented as maximum three screens of
text); choosing whether one would like to read or listen
should be easily conducted; the language used in the app
should be changed to reflect everyday user language,
instead of academic or medical terminology; animations and
illustrations should be used to create visual aids and substan-
tiate the information in clear and engaging manner.
Discussion
Main final findings
The development of apps by using comprehensive, robust,
and theory-based methods is critical to their success.
Figure 7. Care mode effectiveness of the app.
Figure 6. Example of a push notification in the app.
10 DIGITAL HEALTH
OASapp was well accepted by the content expert (Alpha
testing) and consumers (Beta testing). In this context, the
usability of the app is critical in older adults with stroke
because they have major setbacks in using smartphone
apps compared with other adult consumers. Wildenbos
et al.
47
reported that age is associated with normal physical
decline (e.g. hearing or visual impairments), which poses a
barrier to effective mobile device use. Four key aging
barrier categories, namely, cognition, physical ability,
motivation, and perception, can influence the usability of
mHealth.
47
Operational and technical literacy are the
major challenges for many older adults because they are
still not as tech-savvy as the younger generation.
48
If the
design accommodates age-related perceptual, cognitive,
and motor changes, then the app may be more likely to
be acceptable for older adults. A recent observational
study underscored the pivotal role of a simplistic design
and the incorporation of features catering to neurologic def-
icits when developing a mHealth app for post-stroke care
derived from usability testing.
49
Previous research indicated
that the complexities of older adults (with chronic condi-
tions) and their disabilities are often overlooked and that
many of the developed mHealth apps consist of many diffi-
culties to understand features.
50,51
App functions that are
difficult to use or understand might result in a significant
decrease in usability. These factors are common problems
and shortcomings of currently available stroke mobile
apps. Based on our review of existing apps and systematic
review of literature, most apps are not designed to cater to
older adults.
18
Limited literature is available on the devel-
opment of apps that improve medication adherence
among older adult stroke survivors, and the present study
should inform future researchers in this evolving field.
HBM and TAM provide a well-defined framework for app
development. Based on the HBM constructs, perceived bene-
fits, perceived barriers, self-efficacy, and cues to action are
associated with app uptake intention.
52
During the develop-
ment process, all these elements were addressed by health
information in different formats (e.g. textual information,
graphic presentation, or video). The stroke map component
analyzes the user’s risk for stroke and provides advice or
recommended action. This component will strengthen the per-
ceived susceptibility that an individual might have the risk of
suffering from stroke. The app content and functionality were
refined and focused using HBM. Furthermore, the perceived
ease of use and usefulness can pose barriers to technology
acceptance and universal access among older adults.
53
Additionally, Walrave et al.
52
reported a negative relationship
between age and perceived benefits; thus, older potential users
need to be more convinced of the app’sbenefits. TAM posits
that higher perceived usefulness and ease of use results in
more greater behavioral intention and actual use of the tech-
nology.
54
Therefore, we applied TAM to facilitate the devel-
opment of OASapp. For example, we launched care mode,
voice input mode, voice broadcast, and share mode through
WeChat to ensure that the app is easy to use. HBM and
TAM contributed to the successful development of the app.
However, Mohr et al.
38
commented on the lack of theory
and evidence to inform the design of development of most
apps available to consumers. In general, our study provides
an example on how the theory component was incorporated
into designing a mHealth app. The results will have potential
use and implication for developers in terms of incorporating
theory in the development of health-related apps.
NGT was used to develop OASapp. In contrast to other
NGTs,
30,55
where participants generate their list of attributes
before the discussion and rank them afterward, the NGT in the
present work compiled attributes from the literature review
and presented them for discussion with the participants.
Although the participants were not allowed to brainstorm
before the discussion because of time constraints, they were
provided with opportunities to voice their opinion on what
the components should be and how to implement the
program. NGT allows everyone in the group to contribute to
the discussion. The NGT results showed that medication
reminders, track BP/lipids/glucose, disease information, phar-
macy information, and communication were the top five fea-
tures voted by the participants. The ranking of these features
Table 3. Issues with Alpha testing.
Issue Changes made to the app
Delay in medication reminding Fast sync capability added
Dosage unit mg is not available Put “mg”in dosage unit
Wrong spelling Amending errors
Font size and color are not uniform Using the same fonts in size and color throughout the app
Line spacing cannot be set in the body of health information Glitch in system amended immediately
There is no information on hypertension and diabetes Add information on hypertension and diabetes
Cao et al. 11
highlighted that the group had a consensus on what is the
important issue and the prioritization of app features. Our
study confirms that the use of NGT is feasible and valuable
for identifying important features about apps.
Despite the plethora of medication adherence apps, the
majority of them were considered low quality.
56
Haase
et al.
57
identified 30 medication-related apps that were
written in English and summarized few ideal features from
the top five applications (e.g. xNetwork, Mango Health,
MyMeds, C3HealthLink, and HuCare). In the present study,
we incorporated ideal features that could help patients take
medications as prescribed. First, OASapp delivers medication
reminders to promote medication adherence by using push
notifications, alarms, and short-messaging service (SMS)
reminders. Ahmed et al.
58
conducted a review of medication
adherence apps available in repositories and found that very
few of them (1.4%, 6/420) incorporated SMS reminders.
Recent evidence suggests the significant effect of SMS remin-
ders on improving medication adherence.
59,60
Additionally,
OASapp can generate medication reminders for different med-
ications for more than one user, including family members,
pharmacists, and other healthcare providers. By involving dif-
ferent stakeholders in the patient’s care journey, OASapp
creates a more holistic and interconnected support system.
This collaborative feature enhances communication and
coordination among those responsible for stroke survivors’
well-being. Several studies have revealed that support for
stroke survivors from family members, pharmacists, and
other healthcare providers plays a significant role in lowering
the risk of stroke.
61,62
OASapp also has a convenient feature that allows medi-
cation lists to include indications so patients are educated
and make associations with each medication. Educating
stroke patients about their medications is crucial, especially
considering the complexity of their medication routines.
OASapp empowers patients by offering information about
the purpose of each drug, enabling them to make informed
decisions and fostering a deeper understanding of their
treatment plan. Finally, OASapp provides users the oppor-
tunity to add a picture to provide visual clues (e.g. to ensure
correct medication is taken). For stroke survivors who may
face cognitive challenges, visual cues serve as valuable aids
in ensuring that the correct medication is taken. This feature
goes beyond the typical functionalities of medication adher-
ence apps and caters specifically to the unique needs of
stroke patients. In conclusion, OASapp stands out from
other medication apps with its SMS reminders, support
for multiple users, medication indications, and visual
cues. These features together offer a more personalized
and comprehensive solution for stroke patients, addressing
their unique challenges and promoting better medication
adherence in this group. However, further research should
be conducted to establish the clinical efficacy of the app.
Alpha testing showed a positive result, that is, the
content developed met the requirement of the app.
Moreover, the app expert involved in the testing assumed
that the app was good and met the usability requirement.
Overall, the Beta testers had positive feedback with
respect to the usability of the mobile application. The
usability score of the app was significantly higher than
the minimum cut-off point for a usable mobile app; as
such, OASapp is a usable mobile app for intended users.
However, open-ended questions may be applied while util-
izing UMUX on its own if unique problems should be
refined with mobile applications. Mummah et al.
19
men-
tioned that without qualitative measures, one could not
easily explain why people assigned a low, or even a high,
score with respect to the usability of mobile applications.
According to Sauro,
63
seasoned users continued to have
higher SUS scores relative to first-time consumers. Thus,
if the respondents were given more time and exposure to
explore mobile applications, then the UMUX score may
be higher. Further evaluation of different methodologies
for usability testing could be included such as by those
who have utilized the mobile application for some time.
Limitations and future work
One of the limitations of this project is the lack of a web
portal for sharing patient-entered information with health
care providers. Currently, patients have to share informa-
tion to health care providers via WeChat. In the future, a
web portal will be created to make data readily available
to providers. We plan to take a series of measures to
improve the next version of the app, including but not
limited to, making and publishing OASapp operation
guidelines on the official WeChat account platform, creat-
ing databases and decision rules to help users determine
the normality of laboratory and diagnostic test results, and
adding informative tips to the operation steps.
Another limitation is the limited number of participants
who tested the usability of the current version of the app.
About 80–90% of usability problems would be uncovered
if 7 to 10 participants are involved.
64
Once the users have
used the app for a period of time, they will have a better
idea of what they want. We will create and distribute ques-
tionnaires to understand the specific needs of actual users
and update the app in the next phase of this project.
We will further examine the effects of OASapp. More
work is done in the commercial field than in the research
field, which is quite logical due to the mercenary nature
of businessmen. Parati et al.
65
suggested that the rapid
growth of the commercial market has led to overabundance
of apps that lack readily available evidence of their effect-
iveness. Ahmed et al.
58
indicated that robust evidence sup-
porting the use of app-based interventions is necessary.
Conclusions
We developed and tested OASapp through mixed methods
and iterative design to improve the medication adherence of
12 DIGITAL HEALTH
older stroke survivors. The development and evaluation of
OASapp highlights the important aspects of the creation
process, which may be beneficial for researchers and
medical professionals who aim to develop similar mHealth
apps in the future. As a way forward, OASapp will be used
in clinic trial to determine the effectiveness of it.
Summary table
What was already known on the topic
•Medication adherence among stroke survivors is
problematic.
•Many medication adherence apps are currently available
given the accessibility and widespread use of mobile
phones; however, these apps are not specific to diseases,
such as stroke.
•A theory-based app could stress and sustain health pro-
motion behavior change; otherwise, the app’s effective-
ness could be limited.
What is this study added to our knowledge
•Reports on the design and development of a mHealth app,
namely, OAS, based on the Health Belief Model (HBM)
and Technology Acceptance Model (TAM) to improve
medication adherence among older adult stroke survivors.
•mHealth apps, such as OASapp, are promising digital
technologies for stroke survivors.
•Further research is needed to establish the clinical effi-
cacy of OASapp so that it can be utilized to improve
clinically relevant outcomes.
Acknowledgments:The authors would like to extend their
gratitude for the time and effort of the stroke survivors who
provided input and feedback on their app in usability testing.
They would also like to thank Yong Huang for his assistance in
designing the mobile app. They express their special thanks to
KGSupport for their professional assistance in proofreading and
editing the manuscript.
Availability of data and materials: The original contributions
presented in the study are included in the article, further
inquiries can be directed to the corresponding author on
reasonable request.
Contributorship: Wenjing Cao, Juan Wang, Intan Idiana Binti
Hassan, and Azidah Abdul Kadir had a significant share in the
app design phase. Yuhui Wang had a significant contribution in
the app evaluation phase. Wenjing Cao wrote the initial draft of
the article. Azidah Abdul Kadir and Intan Idiana Binti Hassan
contributed to the design of the study and revisions to the
article. All authors read and approved the final manuscript.
Declaration of conflicting interests: The authors declared no
potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.
Ethics approval and consent to participate: This study follows
the principles of the Helsinki Declaration 2013. The entire protocol
was reviewed and approved by the ethical committees of Universiti
Sains Malaysia (USM/JEPeM/22080534), The Affiliated hospital of
Xiangnan University (Linyan K2022-003-01), Chenzhou No. 1
People’s Hospital (Yu2022033), and Third People’s Hospital
(Lunshen 2022-10). Written informed consent is obtained from all
participants before they are included in the trial.
Funding: The authors disclosed receipt of the following financial
support for the research, authorship, and/or publication of this
article: This work was supported by the School-level scientific
research project of Xiangnan University, the Xiangnan
University Students’Innovation and Entrepreneurship Training
Project, young key teachers in Hunan Province, China, open
Experimental Project of Xiangnan University.
Guarantor: Azidah Abdul Kadir
ORCID iDs: Wenjing Cao https://orcid.org/0000-0002-3488-
2430
Intan Idiana Hassan https://orcid.org/0000-0001-6907-8920
Supplemental material: Supplemental material for this article is
available online.
References
1. Feigin VL, Stark BA, Johnson CO, et al. Global, regional, and
national burden of stroke and its risk factors, 1990–2019: A
systematic analysis for the Global Burden of Disease Study
2019. Lancet Neurol 2021; 20: 1–26.
2. Mafruhah OR, Huang Y-M and Lin H-W. Impacts of medica-
tion non-adherence to major modifiable stroke-related dis-
eases on stroke prevention and mortality: A meta-analysis.
J Neurol 2023; 270: 2504–2516.
3. Cao W, Kadir AA, Wang J, et al. Medication non-adherence
and associated factors among older adult stroke survivors in
China. Front Pharmacol 2022; 13: 1–11.
4. Yang M, Cheng H, Wang X, et al. Antithrombotics prescrip-
tion and adherence among stroke survivors: A systematic
review and meta-analysis. Brain Behav 2022; 12: e2752.
5. Shani SD, Sylaja PN, Sankara Sarma P, et al. Facilitators and
barriers to medication adherence among stroke survivors in
India. J Clin Neurosci 2021; 88: 185–190.
6. Velusamy S, Varatharaj K, Kathirvel R, et al. Determination
of barriers in medication adherence for secondary prevention
in stroke patients. Indian J Phar Pract 2022; 15: 83–89.
https://doi:10.5530/ijopp.15.2.16
7. Jamison J, Graffy J, Mullis R, et al. Barriers to medication
adherence for the secondary prevention of stroke: A qualita-
tive interview study in primary care. Br J Gen Pract 2016;
66: e568–e576.
Cao et al. 13
8. Liu M, Zheng G, Cao X, et al. Better medications adherence
lowers cardiovascular events, stroke, and all-cause mortality
risk: A dose-response meta-analysis. J Cardiovasc Dev Dis
2021; 8: 146.
9. Xu T, Yu X, Ou S, et al. Adherence to antihypertensive med-
ications and stroke risk: a dose-response meta-analysis. JAm
Heart Assoc 2017; 6: e006371.
10. Brown MT, Bussell J, Dutta S, et al. Medication adherence:
truth and consequences. Am J Med Sci 2016; 351: 387–399.
11. Wessol JL, Russell CL and Cheng AL. A systematic review of
randomized controlled trials of medication adherence inter-
ventions in adult stroke survivors. J Neurosci Nurs 2017;
49: 120–133.
12. Perski O, Blandford A, West R, et al. Conceptualising
engagement with digital behaviour change interventions: A
systematic review using principles from critical interpretive
synthesis. Transl Behav Med 2017; 7: 254–267.
13. Zeng Z, Wu T, Lv M, et al. Impact of mobile health and tele-
health technology on medication adherence of stroke patients:
a systematic review and meta-analysis of randomized con-
trolled trials. Int J Clin Pharm 2022; 44: 4–14.
14. Wildenbos GA, Jaspers MWM, Schijven MP, et al. Mobile
health for older adult patients: using an aging barriers frame-
work to classify usability problems. Int J Med Inf 2019; 124:
68–77.
15. Banskota S, Healy M and Goldberg EM. 15 Smartphone apps
for older adults to use while in isolation during the COVID-19
pandemic. Western J Emergency Med 2020; 21: 514.
16. Park JYE, Li J, Howren A, et al. Mobile phone apps targeting
medication adherence: quality assessment and content ana-
lysis of user reviews. JMIR Mhealth Uhealth 2019; 7: e11919.
17. Piran P, Thomas J, Kunnakkat S, et al. Medical mobile appli-
cations for stroke survivors and caregivers. J Stroke
Cerebrovasc 2019; 28: 104318.
18. Cao W, Kadir AA, Wang Y, et al. Description of apps target-
ing stroke patients: A review of apps store. 2023. https://doi.
org/10.1177/20552076231181473
19. Mummah SA, King AC, Gardner CD, et al. Iterative develop-
ment of Vegethon: A theory-based mobile app intervention to
increase vegetable consumption. Int J Behav Nutr Phys Act
2016; 13: 1–12.
20. Saei Ghare Naz M, Simbar M, Rashidi Fakari F, et al. Effects
of model-based interventions on breast cancer screening
behavior of women: A systematic review. Asian Pac J
Cancer Prev 2018; 19: 2031–2041.
21. Nasution A, Yusuf A, Keng SL, et al. Development of mobile
app for breast examination awareness using health belief
model: A qualitative study. Asian Pac J Cancer Prev 2021;
22: 3151–3163.
22. Yang Z, Wang Y and Sun C. Emerging information technol-
ogy acceptance model for the development of smart construc-
tion system. J Civ Eng Manage 2018; 24: 457–468.
23. Wang Y, Wang X, Wang X, et al. Translation and validation
of the Chinese version of the general medication adherence
scale (GMAS) in patients with chronic illness. Curr Med
Res 2021; 37: 829–837.
24. Yang L, Zheng L, Meiying H, et al. The reliability and valid-
ity of the Chinese version of beliefs about medical question-
naire among elderly patients with depressive disorder. Chin
J Nurs 2014; 49: 389–393.
25. Jiru L, Juan L and Fang T. Development and reliability and
validity test of the health literacy scale for stroke patients.
Chinese nursing research 2020; 34: 3220–3225. https://
doi:10.12102/j.issn.1009-6493.2020.18.007
26. Effectiveness of mobile application interventions for stroke
survivors: systematic review and meta-analysis.
27. Van De A and Delbecq AL. Nominal versus interacting group
processes for committee decision-making effectiveness. Acad
Manage J 1971; 14: 203–212.
28. Nelson V, Dubov A, Morton K, et al. Using nominal group tech-
nique among resident physicians to identify key attributes of a
burnout prevention program. Plos One 2022; 17: e0264921.
29. Rice DB, Cañedo-Ayala M, Turner KA, et al. Use of the nominal
group technique to identify stakeholder priorities and inform
survey development: An example with informal caregivers of
people with scleroderma. BMJ Open 2018; 8: 1–9.
30. McMillan SS, King M and Tully MP. How to use the nominal
group and Delphi techniques. Int J Clin Pharm 2016; 38:
655–662.
31. Potter M. The Nominal Group Technique: A useful consensus
methodology in physiotherapy research. 2004; 32.
32. Finch C and Disney W. The art of Walt Disney: from Mickey
Mouse to the Magic Kingdoms and beyond. 2011; 503. http://
www.worldcat.org/oclc/704121271
33. Liu S and Joines S. Developing a framework of guiding
interface design for older adults. Proc Hum Factors Ergon Soc
2012; 56(1): 1967–1971. https://doi.org/10.1177/107118131
2561291
34. Phiriyapokanon T. Is a big button interface enough for elderly
users? Towards user interface guidelines for elderly users. 2011.
https://www.europeana.eu/item/9200111/BibliographicResource_
1000086000732?utm_source=api&utm_medium=api&utm_
campaign=YuvuWBeCa
35. Roblyer MD and Edwards J. Integrating educational technol-
ogy into teaching. Upper Saddle River: Merrill, Prentice-Hall
International, 2000.
36. Nielsen J. Heuristic evaluation. In: Usability inspection
methods. New York: John Wiley& Sons, 1994, pp. 25–62.
37. Wilson K, Bell C, Wilson L, et al. Agile research to comple-
ment agile development: a proposal for an mHealth research
lifecycle. Npj Digit Med 2018; 1: 1–6.
38. Mohr DC, Schueller SM, Montague E, et al. The behavioral
intervention technology model: an integrated conceptual and
technological framework for eHealth and mHealth interven-
tions. J Med Internet Res 2014; 16: e3077.
39. Allesi SM and Trollip SR. Multimedia for learning: Methods
and development. Boston, MA: Allyn & Bacon, 2001. https://
csuglobal.blackboard.com/bbcswebdav/institution/FCC
Content/csfiles/home_dir/externalFiles_20130401041211/
library__xid-1005_5/Textbook Reserve__xid-13309_5/OTL__
xid-14610_5/OTL543__xid-17718_5/OTL543_Module2__xid-
14582_5-2.PDF.
40. Wang Y, Chai J and Lei T. Chinese UMUX: cross-cultural
adaptation for perceived usability measurement. Int J Hum
Comput Interact 2021; 37: 1538–1550.
41. Finstad K. The usability metric for user experience. Interact
Comput 2010; 22: 323–327.
42. Berkman MI and Karahoca D. Re-assessing the usability
metric for user experience (UMUX) Scale[J]. J Usability
Stud 2016; 11: 89–109.
14 DIGITAL HEALTH
43. Lewis JR, Utesch BS and Maher DE. Measuring perceived
usability: the SUS, UMUX-LITE, and AltUsability. Int J
Hum Comput Interact 2015; 31: 496–505.
44. Al-Tahat KS. An arabic adaptation of the usability metric for
user experience (UMUX). Int J Hum Comput Interact 2020;
36: 1050–1055.
45. Borsci S, Federici S, Bacci S, et al. Assessing user satisfaction
in the era of user experience: comparison of the SUS, UMUX,
and UMUX-LITE as a function of product experience. Int J
Hum Comput Interact 2015; 31: 484–495.
46. Zhao J, Li X, Liu X, et al. Changing the strategy and culture of
stroke awareness education in China: implementing Stroke
1-2-0. Stroke Vasc Neurol 2020; 5: 374–380.
47. Wildenbos GA, Peute L and Jaspers M. Aging barriers influen-
cing mobile health usability for older adults: A literature based
framework (MOLD-US). Int J Med Inf 2018; 114: 66–75.
48. Fang KY, Ginige JA, Basilakis J, et al. Smart homecare
research translation into broader practice: enablers, barriers
and directions. IEEE Access 2022; 10: 134726–134743.
49. Ramaswamy S, Gilles N, Gruessner AC, et al. User-centered
mobile applications for stroke survivors (MAPPS): A mixed-
methods study of patient preferences. Arch Phys Med Rehabil
2023; 104: 1573–1579.
50. Amin SU and Hossain MS. Edge intelligence and internet of
things in healthcare: A survey. IEEE Access 2021; 9: 45–59.
51. Gao C, Zhou L, Liu Z, et al. Mobile application for diabetes
self-management in China: do they fit for older adults? Int J
Med Inf 2017; 101: 68–74.
52. Walrave M, Waeterloos C and Ponnet K. Adoption of a contact
tracing app for containing COVID-19: A health belief model
approach. JMIR Public Health Surveill 2020; 6: e20572.
53. Mitzner TL, Rogers WA, Fisk AD, et al. Predicting older
adults’perceptions about a computer system designed for
seniors. Univers Access Inf Soc 2016; 15: 271–280.
54. Knox L, Gemine R, Rees S, et al. Using the technology
acceptance model to conceptualise experiences of the usabil-
ity and acceptability of a self-management app (COPD.Pal®)
for chronic obstructive pulmonary disease. Health Technol
2021; 11: 111–117.
55. Gallagher M, Hares T, Spencer J, et al. The nominal group
technique: A research tool for general practice? Fam Pract
1993; 10: 76–81.
56. Santo K, Richtering SS, Chalmers J, et al. Mobile phone apps
to improve medication adherence: A systematic stepwise
process to identify high-quality apps. JMIR Mhealth
Uhealth 2016; 4: e132. https://doi.org/10.2196/mhealth.6742
57. Haase J, Farris KB and Dorsch MP. Mobile applications to
improve medication adherence. Telemed E-Health 2017; 23:
75–79.
58. Ahmed I, Ahmad NS, Ali S, et al. Medication adherence apps:
review and content analysis. JMIR Mhealth Uhealth 2018; 6:
e62. https://doi.org/10.2196/mhealth.6432
59. Ridho A, Alfian SD, van Boven JFM, et al. Digital health tech-
nologies to improve medication adherence and treatment outcomes
in patients with tuberculosis: systematic review of randomized con-
trolled trials. JMedInternetRes2022; 24: e33062.
60. Tan EH, Wong ALA, Tan CC, et al. Improving medication
adherence with adjuvant aromatase inhibitor in women with
breast cancer: A randomised controlled trial to evaluate the
effect of short message service (SMS) reminder. The Breast
2020; 53: 77–84.
61. Fikarwin Z and Lubis R. Model of family health empower-
ment preventing stroke in Puskesmas Ulee Kareng Banda
Aceh. J Phys Conf Ser 2020; 1460: 012076.
62. Dharani G and Akiladevi D. A systemic review on the role
of clinical pharmacists in stroke patients. Int J Pharm Res
2021; 13. https://doi: 10.1088/1742-6596/1460/1/012076
364–368.
63. Sauro J. A practical guide to the system usability scale:
Background, benchmarks & best practices. In Measuring
Usability LLC, 2011. https://doi.org/10.1371/journal.pone.0257035
64. Macefield R. How to specify the participant group size for usabil-
ity studies: A practitioner’sguide.J Usability Stud 2009; 5: 34–
45. http://www.usabilityprofessionals.org/upa_publications/jus/
2009november/JUS_Macefield_Nov2009.pdf
65. Parati G, Torlasco C, Omboni S, et al. Smartphone applications
for hypertension management: A potential game-changer that
needs more control. Curr Hypertens Rep 2017; 19: 1–9.
Cao et al. 15
Content uploaded by Wang Yuhui
Author content
All content in this area was uploaded by Wang Yuhui on Mar 07, 2024
Content may be subject to copyright.
