Access to this full-text is provided by SAGE Publications Inc.
Content available from Digital Health
This content is subject to copyright.
Evaluating remote healthcare practices:
Experiences and recommendations of healthcare
professionals on smart applications
Fadime Baştürk
1
, Arif Osman Tokat
2
, Osman Öztürk
3
,Çiğdem Kader
4
and Levent Işikay
5
Abstract
Objective: With the digitalization of objects and spaces, healthcare services are being reshaped globally, creating many
potential applications. This study aimed to determine the application potential of remote healthcare services (RHS) in a hos-
pital by considering the experiences, interests, and suggestions of health professionals, and examples of useful applications
that can be used, developed, or invented for healthcare systems.
Methods: A semi-structured, face-to-face interview survey was conducted with 176 healthcare professionals working at
Bozok University.
Results: Branches with the highest practice experience were internal medicine, cardiac, pediatric, infection, and orthopedics.
Experienced participants rated the usability of “Consultation,”“Support,”and “Monitoring”applications higher than other
apps, and indicated that they would prefer to use them for themselves (η²≤0.12). Requirements adequacy was lower for
older adults, internal/surgical branches, and physicians than for other groups (η²≥0.05). Application categories showed
a significant relationship (0.4 ≤r≤0.8, p < 0.05). Several variables significantly explained the models (p < 0.001): applica-
tion-usability (64%), user-demand (64%), and requirements-adequacy (25%). Professionals’demand (r =0.83) was more
strongly correlated to usability than patient demand (r =0.63). Health professionals (N =105) from 17 branches provided
57 available, 51 developable, and 19 innovative application recommendations. These were coded according to application
type, critical features, presence, integration status, and usefulness.
Conclusion: RHS’application potential in a hospital was revealed considering demographic factors and application categor-
ies based on health professionals’experiences, practical interests, and suggestions, with a strong, comprehensive, and up-
to-date methodology. The findings have the potential for international application and can contribute to implementing useful
and developing original applications.
Keywords
Remote healthcare, medical informatics applications, digital technology, smart materials, healthcare management
Submission date: 4 June 2024; Acceptance date: 8 January 2025
1
Patient Rights Department, Health Practice and Research Center, Yozgat
Bozok University, Yozgat, Turkey
2
Department of Thoracic Surgery, Faculty of Medicine, Yozgat Bozok
University, Yozgat, Turkey
3
Child Health and Diseases, Faculty of Medicine, Yozgat Bozok University,
Yozgat, Turkey
4
Department of Internal Medical Sciences, Faculty of Medicine, Yozgat
Bozok University, Yozgat, Turkey
5
Department of Surgical Medical Sciences, Faculty of Medicine, Yozgat
Bozok University, Yozgat, Turkey
Corresponding author:
Fadime Baştürk, Patient Rights Department, Health Practice and Research
Centre, Yozgat Bozok University, Yozgat, Turkey.
Email: f.arslanbasturk@gmail.com
Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial
4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work
without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/
open-access-at-sage).
Original Research Article
DIGITAL HEALTH
Volume 11: 1–29
© The Author(s) 2025
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/20552076251315786
journals.sagepub.com/home/dhj
Background
Currently, smart technologies are developing at an unprece-
dented pace. Smart environments and objects are being
designed, and complex interventions in biological structures
are being performed remotely by leveraging the sensitive
sensors of smart devices.
1
The use of new technologies in
healthcare and medicine has created tremendous application
potential, especially given the digital transformations neces-
sitated by the COVID-19 pandemic.
2,3
For instance, applica-
tions that make impossible procedures possible, such as
remote robotic surgeries, have become widespread.
Moreover, important factors that advantage users are
finding solutions to the obstacles arising from spatial, tem-
poral, and individual limits, facilitating, accelerating, and
simplifying the work procedures by transferring some of
them to technology,
4
thus preventing human-caused
errors.
5
Therefore, the concept of digital healthcare plays
an important role in providing more up-to-date, efficient,
and quality healthcare services while benefiting individuals.
6
As digital health reforms progress globally, the situation
in hospitals—the cornerstones of the healthcare system—is
becoming particularly intriguing. In the developing version
of remote healthcare services (RHS), with the transforma-
tive effect of new health technologies, hospitals are
moving from being places of observation and intervention
to remote control centers. In this framework, institutions
should not only overcome their deficiencies but also
promptly take advantage of innovation opportunities.
Hospitals should structure their information systems, and
establish their application laboratories and markets.
In its traditional sense, RHS is generally used by private
healthcare providers, and is a format that includes only
patient and physician meetings. There is also conceptual
confusion among researchers and users about whether the
terms tele, e, mobile, online, virtual, smart, web-based
and digital health are within the scope of the RHS.
Therefore, traditional definitions must be updated within
the framework of technological development before design-
ing a study on the subject.
7
However, the transition to new practices depends on
established orders and standards.
8
For applications that
are in use and can be improved, studies have evaluated
certain types of RHS.
9,10
Furthermore, the applicability of
RHS has been investigated using a mixed methodology spe-
cific to certain health branches and generally for commonly
used applications.
11
Additionally, commonly used applica-
tions that healthcare professionals recommend to their
patients have been addressed in structured quantitative
studies. However, these designs provide a limited under-
standing of the basic and indispensable issues in developing
or adapting Internet-based RHS interventions.
12
Studies have also highlighted the potential for digital
health security and artificial intelligence (AI)-driven appli-
cations. For example, AI-driven applications in preventive
health can be considered an alternative treatment approach
for infectious diseases.
13
Alternately, secure, interactive
multimedia applications by enhancing remote media areas
with heterogeneous image sources; can also be used to
provide healthcare support in non-hospital settings.
14
The majority of future innovation recommendations are
based on literature reviews or qualitative studies with small
groups of participants.
7,15,16
Furthermore, e-health applica-
tions are often designed by researchers without consulting
the primary users. These studies do not consider the
needs of end users but require intense effort, resulting in
poor usability, frustration, and limited acceptance.
12,17
Therefore, more studies are required on both the multifa-
ceted evaluation of the usability of applications and devel-
oping new applications.
18–22
Physicians’adoption of
telemedicine can be explained by behavioral usage inten-
tion, actual usage, satisfaction, attitude, continuous usage
and recommendation intention variables.
23
Also a deeper
exploration of user needs can provide smarter healthcare
solutions that can better meet practical requirements.
24
Addressing these gaps, this study evaluates the applica-
tion potential of RHS in a hospital, considering application
type, healthcare branch, and professionals. It aims to answer
the question, “I
f experiences, practical interests, and recom-
mendations of health professionals improve RHS applic-
ability, what applications can be used, developed, or
invented considering these factors?”In particular, the fol-
lowing specific questions are answered:
Qualitative questions: 1. What are the types of RHS and
applicability categories? 2. Is there another app profes-
sionals would like to use in their work? 3. What applica-
tions are health professionals interested in or would like
to see developed?
Quantitative questions: 1. In which branches and to what
extent are the RHS types used? 2. Is there a relationship
between applicability categories and the usability of RHS
types? 3. Does RHS applicability differ among different
demographic groups? 4. How do usefulness, adequacy,
experience, demand, education, and age play a role in
RHS applicability categories and app-type usability?
Common inference questions: (1) What are the coded
recommendations based on application type and other
emerging themes? (2) What are the common inferences
from the quantitative and qualitative questions? (3) What
are the RHS application potentials for the hospital regarding
existing, developable, or innovative applications?
Overall, this study has important strengths as the use
of both qualitative and quantitative methods provides a
well-rounded understanding of the topic. For new
research topics, exploratory and explanatory data are
required. While qualitative data can provide sufficient
insight, quantitative data can provide easy, effective,
and powerful measurements. Description and generaliza-
tion become easier as the scope and diversity of partici-
pants increase.
2DIGITAL HEALTH
Moreover, this study addresses practical concerns and
suggestions from healthcare professionals, making the find-
ings highly relevant for improving RHS. Additionally,
including healthcare professionals from various branches
ensures a comprehensive view of the current state and
needs of RHS.
Methods
Research design
This study employed a cross-sectional, descriptive, and ana-
lytical design. The big picture of the study problem was
explained using quantitative data, important codes and
themes were explored using qualitative data. In addition
the quantitative data were complemented and explained
with qualitative data. A complex mixed-methods design
was used instead of a single method design.
25
Specifically, the study was conducted in three stages: an
exploratory sequential design, followed by an explanatory
sequential design, and finally, a concurrent design.
26
This
study utilized the six basic criteria determined by Hirose
and Creswell for mixed-method studies.
25
Scheme 1 pre-
sents the research design procedure and describes the data
collection, analysis methods, and associated outputs. The
research design and reporting were in accordance with the
standards specified in “Mixed Methods Reporting in
Rehabilitation & Health Sciences.”
27
Researcher Background and Contributions to the
Project:Data collection, analysis, and reporting were con-
ducted by researchers who have been working in manage-
ment and professional healthcare in various institutions
for a long time. Therefore, observations and expert evalua-
tions were included.
Environment and scope
The research was conducted in a university hospital in
Turkey between September 25, 2022, and March 25,
2023. Turkey’s health institutions have their own informa-
tion systems. Additionally, Turkey’s personal health record
application, “E-Nabız,”is used under the control of the
Ministry of Health.
Generally, health professionals with a medical background
work in at least one different health institution before being
appointed to an academic position at a university hospital.
Other employees may transfer from private hospitals or differ-
ent health institutions. Transfers from private hospitals are
common in the hospital where the study was conducted.
Permission and ethical approval to conduct the study
was received from the Hospital Management and
University Ethics Commission (September 21, 2022, No.:
36/18). The purpose of the research and confidentiality of
the data were explained to the participants, and informed
verbal and written consent was obtained.
Participants
The study population comprised 301 healthcare profes-
sionals actively working in 21 branches of a university hos-
pital clinic. To reach a statistically significant sample size,
the sample size was calculated as 170 people with a confi-
dence interval of 95% and a margin of error of 5%.
Interviews were conducted with 191 people, comprising
managers, assistant managers, supervisors, and unit
employees, who agreed to participate in the interview vol-
untarily with the guidance of the unit manager.
Subsequently, 15 participants who left 5% or more of the
survey items blank, double-ticked, or had inconsistencies
in their answers were excluded. Furthermore, 105 partici-
pants from 17 branches made RHS application recommen-
dations. At least one person from each branch participated
in the study. In qualitative research, a large amount of infor-
mation can be obtained with a small sample size in studies
conducted with participants who are experts with high
knowledge power.
28
Participants presented examples of
RHS relevant to their field and examples of other field
applications that make their field possible. This is signifi-
cant when considering the interdependence among
branches in providing health services.
Healthcare professionals have a very heterogeneous spe-
cialization structure. This study ensured participation from
every active branch in the hospital, especially by unit heads
or their assistants. Furthermore, non-physician healthcare
professionals can work in mixed units comprising more
than one branch. Therefore, participants who did not
work in a specific branch were included in the mixed
branch.
29,30
Subgroup comparisons were made if the
number of participants was at least 10 or if statistical signifi-
cance was achieved according to the analysis. Ultimately,
data obtained from 176 participants grouped into 15
branches were analyzed (Figure 1).
Data collection
Pilot study. In the first phase of this study, aimed at deter-
mining the RHS types and applicability categories, trial
interviews were conducted with participants representing
internal, surgical and mixed specialties. Based on this,
eight themes of RHS categories were determined, two
themes of “user demand,”and three themes of “adequacy
of requirements.”Subsequently, adjustments were made
to the interview protocol based on expert experience and
information provided by the participants. The semistruc-
tured interview form was revised, unnecessary expressions
were removed, and new ones were added or changed
31
(Appendix 1). Detailed information is given in Scheme 1.
Instrumentation
Remote Health Services and Application Categories. Many
current definitions of health have emerged with the
Baştürk et al. 3
combination of technological and health terms, such as tele,
electronic (e), mobile, online, virtual, smart, web-based,
and digital health/medicine. These refer to a broad or
narrow scope of services depending on the meaning of
the added word and technological features. For example,
e-health is the new name for health in the technology age
and is defined as the provision of health services electronic-
ally in a web-based environment.
32
New health technolo-
gies are changing and improving how health services and
many traditional practices are implemented. For instance,
AI, augmented reality, sensor sensing, blockchain,
Internet-of-Things (IoT), metaverse, and other technologies
are making it more realistic to consider health in new con-
texts and have also developed tools to increase physical
independence.
16,33
The traditional name for RHS is telehealth. Telehealth,
in addition to tele, includes medicine, consultancy, monitor-
ing, and several other aspects. Here, diverse definitions are
created by adding words. The most important differentiator
in telehealth is physical distance. Many applications, such
as monitoring with connectivity technologies, robotic
surgery, drug treatment, implants, diagnosis, and care, can
be performed over long distances.
34
Therefore, RHS can
be defined as remote delivery of healthcare services
through digital technologies such as wearable smart
devices, application software, information communication,
and connectivity technologies. Besides distance, RHS
covers other new definitions and key areas of healthcare
such as electronic or operational.
The application categories were determined according to
the general service areas of health services provided in the
hospital based on the literature and extant practices.
32–34
They were revised in the first and second phases, and at
the end of the study.
Applicability of Remote Health Services. In prior studies, to
evaluate the usability of an application, participants who had
experienced an application were usually asked about state-
ments that measure the usefulness, practicality, reliability,
user satisfaction, and other aspects of the application.
11
However, three main themes have been highlighted in the
research implications: “usability”
1,11,35–37
;“adequacy”
11
—
technical adequacy,
1,36–40
worker count adequacy,
38
worker
qualification adequacy,
28,35,40–42
reliability,
20,43
and financial
and legal adequacy; and “demand”
20,44,45
—profession,
patient,
35
supplier, and politician demand. The current study
investigated the usability of applications that employees not
only experience but are also interested in. Therefore, the
usability of each RHS type with specific characteristics was
investigated. To determine the application experience, partici-
pants were asked to indicate their previous use of the applica-
tion types via yes or no options. During the trial phase, the
reliability, financial adequacy, legal adequacy, and supplier
demand categories were eliminated. Healthcare professionals
perceive these issues as technical aspects under the control of
institutional owners, experts, or management. The applicabil-
ity categories included in the research questionnaire are
shown in Figure 2.
Implementation. The interview method was used to collect
data. During the interview, participants were asked to evalu-
ate the RHS categories specifically for their branches. The
responses were transcribed verbatim using a voice recording
or note-taking application with the participant’s permission.
The content was reviewed, coded, and anonymized.
Data analysis
Principal component analysis (PCA) was conducted to deter-
mine constructs in the applicability questionnaire: “app use-
fulness,”“user demand,”and “adequacy of requirements.”
IBM SPSS Program Statistics 29 (IMB Corp., NY, USA)
was used for the quantitative analysis. Compliance with a
normal distribution was examined using the Kolmogorov–
Figure 1. Participant flowchart.
4DIGITAL HEALTH
Smirnov test. Additionally, skewness and kurtosis values
were examined. If these values are between ±2, continuous
scores show normal distribution.
46
Descriptive statistics are
provided for the sociodemographic and applicability categor-
ies. A one-way analysis of variance (ANOVA) was used to
compare independent groups. Significant results in the
ANOVA test were examined using the multiple comparison
tests to determine which group the difference was in.
Pearson’s correlation (r) was used to measure the linear rela-
tionship between the applicability categories. Subsequently,
multivariate linear regression was performed to measure the
role of usefulness, adequacy, experience, demand, education,
and age in the applicability categories of RHS and app-type
usability. The basic assumptions required for statistical com-
parison methods were tested. The statistical significance level
was set at 5%. Content and thematic analyses were applied to
the sample application ideas in MAXQDA 2022. The data
were presented with weighted code models and crosstabs
using the frequency distribution of coded documents or sec-
tions. Key findings from the qualitative and quantitative ana-
lyses were combined into one table to illustrate the common
conclusions. Thus, stronger findings can be achieved by
adding the insights developed from integrating qualitative
and quantitative data.
25
Reliability and validity tests
Cronbach’s alpha and PCA were calculated for each con-
struct in the questionnaire.
Usefulness: Cronbach’s alpha: 0.925; PCA: eight com-
ponents (“physical examination,”“measurement,”“evalu-
ation,”“treatment/care,”“operation,”“monitoring,”and
“counseling”) explaining 65.9% of the variance.
User Demand: Cronbach’s alpha: 0.822; PCA: two com-
ponents (“own-demand”and “patient demand”) explaining
84.9% of the variance.
Adequacy of requirements: Cronbach’s alpha: 0.924;
PCA: three components (“technical adequacy,”“worker
count adequacy,”and “worker qualification adequacy”)
explaining 91.8% of the variance.
To ensure the reliability of the qualitative analysis,
data, analyst, and method triangulation was applied,
and application categories determined from the literature
were used.
47
Results
Evaluation of the applicability of remote
health applications
Participant profile. A total of 53% (94 people) and 47% (81
people) of the participants were in the 20–30 years and 30–
67 age groups, respectively. Furthermore, 39% (69 people)
had an undergraduate education, while 42.6% (75 people)
had a postgraduate education. Finally, 48.3% (85 people)
were nurses, 40.3% were physicians (71 people), and
11.4% (20 people) were other health workers (Table 1).
Regarding working experience in professional branches,
Figure 2. Applicability categories.
Baştürk et al. 5
95 (54%) participants had been working for 5 years or less,
61 (34.7%) for 5–15 years, and 20 (11.4%) for more than 15
years. The distribution of participants by branch was 10 at
the lowest and 15 at the highest.
Determining sample application usage experience according to
branch. A total of 175 (99.4%) participants reported
having used RHS applications before. Of these, 110
(62.5%) had used any RHS application category
through an informal or official digital application. The
branches with the highest practice experience were
internal medicine (13 people, 92.9%), cardiac (12
people, 80%), and pediatrics (8 people, 80%).
Participants mostly used consultation (cardiac: 11
people; surgery/child/infection: 9 people; total: 89
people, 51.4%), support (internal medicine: 9 people;
orthopedics: 5 people; total: 37 people, 21.6%), and mon-
itoring applications (internal medicine: 8 people; pediat-
rics: 6 people; orthopedics: 5 people; total: 33 people,
19.2%) (Table 2).
Pearson correlation analysis: identifying relationships affecting
applicability. Pearson’s correlation analysis was used to
determine the relationships between “adequacy”and
“demand”with the usability of examination, measurement,
evaluation, treatment/care, operation, monitoring, counsel-
ing, and support applications (Table 3).
Table 1: Participant Profile
Demographic Factor Count %
Branch Emergency 10 5.7%
Anesthesiology 12 6.8%
General Surgery 10 5.7%
Ophthalmology 12 6.8%
Gynecology/Obstetrics 13 7.4%
Urology 11 6.2%
Orthopedics 11 6.2%
İnfectious diseases 12 6.8%
Physiotheraphy 10 5.7%
Internal medicine 14 8.0%
Neurology 15 8.5%
Internal Other 11 6.2%
Pediatry-S 10 5.7%
Pulmonology-S 10 5.7%
Cardiology-S 15 8.5%
Total 176
Working Year in
Branch
Under 1 Year 21 11.9%
Between 1-5 Years 74 42.0%
Between 5-10 Years 38 21.6%
Between 10-15 Years 23 13.1%
Above 15 20 11.4%
Total 176
Age Under 25 26 14.9%
26-30 68 38.9%
31-35 35 20.0%
36-40 18 10.3%
Above 41 28 16.0%
Total 175
(continued)
Table 1: Continued.
Demographic Factor Count %
Education Primary education 0 0.0%
High school 20 11.4%
Associate Degree 12 6.8%
Licence 69 39.2%
Master's Degree 46 26.1%
Doctorate 29 16.5%
Total 176
Profession Physician 71 40.3%
Nurse 85 48.3%
Other 20 11.4%
Total 176
Note: Mixed (…S) Both internal and surgical branches.
6DIGITAL HEALTH
Table 2. Sample application usage experience according to branch.
Branch and application type
Application →
Physical Examination Measurement Evaluation Treatment / Care Operation Monitoring Counseling Support
Experienced
participant
Total
participants
Branch ↓NN%N%
Surgical Emergency 1 1 1 2 0 1 5 3 6 60.0 10 5.7
Anesthesiology 1 1 2 0 2 2 6 1 7 58.3 12 6.8
General Surgery 0 3 3 0 0 1 2 3 4 40.0 10 5.7
Ophthalmology 1 1 2 2 0 1 5 4 7 58.3 12 6.8
Gynecology/Obstetrics 1 0 1 0 0 1 2 1 6 46.2 13 7.4
Urology 0 1 0 0 0 1 7 1 6 54.5 11 6.3
Orthopedics 0 1 1 4 4 5 7 5 8 72.7 11 6.3
İnternal İnfectious diseases 1 1 2 1 0 1 9 2 9 75.0 12 6.8
Physiotherapy 0 0 0 1 0 0 7 1 7 70.0 10 5.7
Internal medicine 1 4 3 2 3 8 9 9 13 92.9 14 8.0
Neurology 1 1 0 3 1 2 8 2 10 66.7 15 8.5
Internal Other 0 1 1 1 0 1 1 1 3 27.3 11 6.3
Mixed Pediatry-S 1 4 4 4 2 6 9 2 8 80.0 10 5.7
Pulmonology-S 1 1 1 0 0 1 1 1 4 40.0 10 5.7
Cardiology-S 5 1 1 3 1 2 11 1 12 80.0 15 8.5
Total N 14 21 22 23 13 33 89 37 110 175 176 100
% 8.0 12.1 12.8 13.4 7.6 19.2 51.4 21.6 62.5 99.4 100
Note: Mixed (…S) Both internal and surgical branches.
Baştürk et al. 7
Table 3. Relationships between usability of application types, adequacy of requirements and user demand.
Physical
examination Measurement Evaluation
Treatment /
Care Operation Monitoring Counseling Support
Patient
demand
Own
demand
Adequacy of
requirements
Pearson
r
.424** .364** .283** .316** .329** .405** .358** .313** .407** .354**
Physical examination Pearson
r
1 .608** .517** .618** .586** .546** .560** .540** .444** .581**
Measurement Pearson
r
1 .764** .786** .676** .653** .527** .567** .495** .694**
Evaluation Pearson
r
1 .748** .649** .613** .464** .542** .504** .670**
Treatment / Care Pearson
r
1 .663** .609** .555** .550** .604** .731**
Operation Pearson
r
1 .704** .482** .560** .499** .680**
Monitoring Pearson
r
1 .659** .622** .549** .719**
Counseling Pearson
r
1 .639** .522** .656**
Support Pearson
r
1 .522** .689**
Patient demand Pearson
r
1 .698**
p< 0.01. Significance levels are two-way.
8DIGITAL HEALTH
“Care usability”and “own-demand”exhibited a highly
positive (r =0.731) and significant (p < 0.05) relationship,
with 53.4% of the variation within the “care usability”vari-
able being explained by the “own-demand”variable.
Participants reported wanting to use care applications that
they found useful in their work. In general, all variables
exhibited a moderate (r =0.40–0.59) to strong (r =0.60–
0.79) significant (p < 0.05) relationship. The relationships
between the “adequacy of requirements”variable, and
measurement, operation, examination, consultation,
support, and “own-demand”variables were weak (r =
0.20–0.39) but significant (p < 0.05).
One-way ANOVA: identifying demographic group differences
regarding RHS applicability. A one-way ANOVA was con-
ducted to determine demographic group differences
regarding RHS applicability. Significant differences were
found between the groups, especially regarding the
adequacy of requirements (Table 4).
The mean scores related to the adequacy of requirements
showed a significant difference by age group (F[4–170] =
3.33, p < 0.05). The multiple comparison test revealed that
the perceptions of the adequacy of requirements among
those in the under-25-years age group (M =2.64, SD =
0.513) were significantly (η²=0.07) higher than among
those in the 36–40-years (M =1.88, SD =0.786) and 41
+-years age groups (M =1.96, SD =0.820). The results
were similar for all three subcategories of the adequacy of
requirements.
Furthermore, comparing the applicability means of the
education groups revealed differences between the groups
in “count adequacy “(F[4–171] =2.509, p < 0.05) and
Table 4. Identifying Demographic Group Differences Regarding RHS Applicability
Groups 1. Group 2. Group 3. Group 4. Group 5. Group ANOVA
Age 25 –26–30 31–35 36–40 41 +df =4,170
Measures M SD M SD M SD M SD M SD F p η²
Technical 2,58 0,516 2,3 0,893 2,21 0,856 1,82 0,74 2,01 0,838 2,912 ,023 ,064
Count A. 2,66 0,547 2,31 0,859 2,33 0,93 1,92 0,865 1,97 0,916 3,060 ,018 ,067
Gualification 2,67 0,559 2,2 0,883 2,26 0,992 1,89 0,81 1,88 1,021 3,359 ,011 ,073
Adequacy 2,64 0,513 2,27 0,852 2,27 0,908 1,88 0,786 1,96 0,82 3,331 ,012 ,073
Education High Associate Licence Master Doctorate df =4,170
Measures M SD M SD M SD M SD M SD F p η²
Count A. 2,41 0,913 2,32 0,602 2,44 0,839 2,2 0,869 1,88 0,881 2,509 ,044 ,055
Care Use 2,65 1,018 3,33 0,811 2,75 1,108 3,32 1,192 3,18 1,148 2,781 ,028 ,061
Profession Physician Nurse Other df =2,173
Measures M SD M SD M SD F p η²
Technical 2,01 0,803 2,39 0,816 2,36 0,864 4,536 ,012 ,050
Count A. 1,97 0,828 2,49 0,809 2,46 0,937 8,020 <,001 ,085
Gualification 1,97 0,909 2,35 0,854 2,46 0,997 4,554 ,012 ,050
Adequacy 1,98 0,808 2,41 0,784 2,42 0,92 6,053 ,003 ,065
Support 3,53 1,164 3,4 0,97 2,76 1,108 4,162 ,017 ,046
Branch Surgical İnternal Mixed df =2,173
(continued)
Baştürk et al. 9
“care usability”(F[4–171] =2.781, p < 0.05). The percep-
tions of the “doctorate”group regarding “count adequacy”
(M =1.88, SD =0.881) were moderately (η²=0.055) lower
than those of the “high school”(M =2.41, SD =0.913) and
“license”groups (M =2.44, SD =0.839). Furthermore, the
perceptions of the “master”group regarding “care usabil-
ity”(M =3.32, SD =1.192) were significantly (η²=0.061)
higher than those of the “high school”(M =2.65, SD =
1.018) and “license”groups (M =2.75, SD =1.108).
Next, comparing the applicability means of the profes-
sion groups revealed significant differences in “adequacy”
(F[2–173] =6.053, p < 0.05) and “support usability”(F[2–
173] =4.162, p < 0.05). The perceptions of the “physician”
group regarding “adequacy”(M =1.98, SD =0.808) were
significantly (η²=0.065) lower than those of the “other”
(M =2.42, SD =0.920) and “nurse”groups (M =2.41,
SD =0.784). The results for the “adequacy”subcategories
were similar. However, the perceptions of the “other”
group regarding “support usability”(M =2.76, SD =
1.108) were moderately (η²=0.046) lower than those of
the “physician”(M =3.53, SD =1.164) and “nurse”
groups (M =3.40, SD =0.970).
Subsequently, comparing applicability means of experi-
ence groups revealed significant differences in “own-
demand”(F[2–173] =3.533, p < 0.05), “monitoring usabil-
ity”(F[2–173] =4.186, p < 0.05), “consultation usability”
(F[2–173] =11.412, p < 0.05), and “support usability”
(F[2–173] =5.508, p < 0.05). The perceptions of the “two
app+”group regarding “own-demand”(M =3.35, SD =
0.834) were moderately (η²=0.039) higher than those of
the “none”(M =2.92, SD =0.959) and “one app”groups
(M =3.03, SD =0.964). Moreover, the perception of “con-
sultation usability”of the “none”group (M =2.93, SD =
1.187) was significantly (η²=0.117) lower than those of
the “two app+”(M =3.78, SD =0.850) and “one app”
groups (M =3.55, SD =0.997). Furthermore, the percep-
tion of “support usability”of the “two app+”group (M =
3.75, SD =0.899) was significantly (η²=0.060) higher
than those of the “none”(M =3.13, SD =1.081) and “one
app”groups (M =3.28, SD =1.194). similarly, the “two
app+”group’s perceptions of “monitoring usability”(M
=3.47, SD =0.821) were moderately (η²=0.046) higher
than those of the “none”(M =2.96, SD =1.083) and “one
app”groups (M =3.08, SD =1.156).
Finally, comparing the applicability means of the branch
groups revealed a significant difference in “adequacy”
(F[
2–173
]=4.512, p < 0.05). the perceptions of the
“mixed”group regarding “Adequacy”(M =2.68, SD =
0.930) were moderately (η²=0.050) higher than those of
the “surgical”(M =2.13, SD =0.766) and “internal”
groups (M =2.21, SD =0.840). the results for the
“adequacy”subcategories were similar.
Table 4. Continued.
Groups 1. Group 2. Group 3. Group 4. Group 5. Group ANOVA
Age 25 –26–30 31–35 36–40 41 +df =4,170
Measures M SD M SD M SD F p η²
Technical 2,13 0,75 2,21 0,861 2,65 0,952 3,931 ,021 ,043
Count 2,18 0,788 2,22 0,895 2,74 0,937 4,475 ,013 ,049
Gualification 2,09 0,838 2,2 0,946 2,66 0,976 4,024 ,020 ,044
Adequacy 2,13 0,766 2,21 0,84 2,68 0,93 4,512 ,012 ,050
Experience None One App Two App+df =2,173
Measures M SD M SD M SD F p η²
Own D. 2,92 0,959 3,03 0,964 3,35 0,834 3,533 ,031 ,039
Monitoring_U 2,96 1,083 3,08 1,156 3,47 0,821 4,186 ,017 ,046
Consultation_U 2,93 1,187 3,55 ,997 3,78 ,850 11,412 <,001 ,117
Support_U 3,13 1,081 3,28 1,194 3,75 ,899 5,508 ,005 ,060
*P< 0,05, η²; Estimated eta-squared.
10 DIGITAL HEALTH
Multiple linear regression analysis: the role of usefulness,
adequacy, experience, demand, education, and age on RHS
applicability categories and app type usability. The role of
the applicability variables was evaluated together with the
impact of demographic changes. For each applicability cat-
egory, a model was created that included age, education,
and experience variables along with other applicability cat-
egories. The most appropriate model was determined by
removing variables that did not significantly contribute to
the model. All applicability variables together significantly
affected the model variables. Among the demographic vari-
ables, only education contributed significantly to the
adequacy model (Table 5).
Among the effects of the model variables on “useful-
ness,”the “demand”(β=0.749, p < 0.001) and “adequacy”
(β=0.106, p < 0.05) variables significantly explained
63.8% of the model (R
2
=0.638, F =152.478, p < 0.001).
Next, regarding the effects on “demand,”the “usefulness”
(β=0.751, p < 0.001) and “adequacy”(β=0.101, p <
0.05) variables significantly explained 63.7% of the
model (R
2
=0.637, F =151.814, p < 0.001). regarding the
effects on “adequacy,”the “demand”(β=0.244, p <
0.05), “usefulness”(β=0.248, p < 0.05), and “education”
(β=−0.236, p < 0.001) variables significantly explained
24.6% of the model (R
2
=0.246, F =18.722, p < 0.001).
While a one-unit increase in the “education”variable
Table 5. The Role of Usefulness, Adequacy, Experience, Demand, Education, and Age on RHS Applicability Categories and App Type
Usability
Models Variables B SE βt p Tolerance VIF
1. Usefulness (Constant) ,587 ,155 3,797 <,001
Demand ,769 ,052 ,749 14,916 <,001 ,830 1,205
Adequacy ,113 ,054 ,106 2,118 ,036 ,830 1,205
Summary R =,799 R
2
=,638 AR
2
=,634 F =152,478 df: 2,173 P<,001 DW: 2,027
2. Adequacy BSE βt p Tolerance VIF
(Constant) 1,552 ,275 5,638 <,001
Demand ,235 ,105 ,244 2,239 ,026 ,370 2,700
Usefulness ,232 ,102 ,248 2,280 ,024 ,371 2,695
Education −,168 ,048 −,236 −3,540 <,001 ,986 1,014
Summary R =496 R
2
=,246 AR
2
=,233 F =18,722 df: 3,172 P<,001 DW: 2,055
3. Demand BSE βt p Tolerance VIF
(Constant) ,435 ,153 2,832 ,005
Usefulness ,732 ,049 ,751 14,916 <,001 ,827 1,209
Adequacy ,104 ,052 ,101 1,998 ,047 ,827 1,209
Summary R =,798 R
2
=,637 AR
2
=,633 F =151,814 df: 2,173 P<,001 DW: 2,031
4. App Type Demand BSE βt p Tolerance VIF
(Constant) ,428 ,135 3,159 ,002
Care ,301 ,043 ,393 7,036 <,001 ,570 1,754
Monitoring ,193 ,052 ,233 3,721 <,001 ,454 2,202
(continued)
Baştürk et al. 11
reduced the “adequacy”variable by 0.24 units, a one-unit
increase in the “demand”variable increased the “adequacy”
variable by 0.24 units. Likewise, a one-unit increase in the
“usefulness”variable increased the “adequacy”variable by
0.25 units.
Finally, examining the effects of model variables on
“app type demand,”“care usability”(β=0.393, p <
0.001), “monitoring usability”(β=0.233, p < 0.001), “con-
sultation usability”(β=0.157, p < 0.05), and “support
usability”(β=0.215, p < 0.001) variables significantly
explained 70.2% of the model (R
2
=0.702, F =98.821, p
< 0.001). Furthermore, examining the effects on “app type
adequacy,”“examination usability”(β=0.289, p < 0.001)
and “monitoring usability”(β=0.247, p < 0.05) variables
significantly explained 22.3% of the model (R
2
=0.223,
F=24.851, p < 0.001).
Example application ideas
The responses received from 105 participants in 17
branches were categorized by content analysis using
MAXQDA 2022. Specifically, the application features,
RHS type, integration into the RHS system, and applicabil-
ity were examined. The number of coded sections (CS) was
calculated.
Features specified in applications. The features emphasized
as necessary or useful for the implementation of the appli-
cation were coded. The number of coded sections was cal-
culated. The main features required to use the application
were identified as data storage and transmission (CS =
70), followed by recognition-detection (CS =55), wearable
portable device (CS =55), multi-connection-integration
(CS =49), planning-setting (CS =47), warning-command
(CS =42), notification (CS =40), meeting (CS =31),
digital identity creation-verification (CS =27), display
(CS =25), practicability (CS =24), reporting (CS =23),
reminding (CS =19), digital assistant (CS =19), and loca-
tion determination (CS =18) (Figure 3). The application
suggestions are given in Appendix 2.
Application recommendations specific to health service type.
Sample application ideas were coded according to RHS
type. Although physical examination, evaluation, and mon-
itoring are determined to be different service types in the lit-
erature, because there is no detailed content about these
groups, the categories have been combined as diagnosis-
monitoring types. Application recommendations can be
categorized into many types of services by their content.
In this study, coding was performed to cover all of applica-
tion type; however, when selecting the examples given for
the service type, the main emphasis in the application pro-
posal was considered.
Diagnosis and monitoring applications. In this application
group, scanning, monitoring, and evaluation recommenda-
tions were made for the branch, disease, and health status.
The most frequently emphasized codes in this category
were follow-up and monitoring (CS =51), followed by
physical examination (CS =13), symptom scanning (CS =
10), emergency assessment (CS =7), mobility assessment
(CS =7), and branch-specific evaluation
7
(Figure 4).
Some application suggestions are given in Appendix 3.
Biometric measurement applications. In these applica-
tions, vital values (CS =27), blood sugar (CS =11), blood
pressure (CS =11), imaging (CS =11), mobility (CS =6),
and electrocardiography were the most frequently used
Table 5. Continued.
Models Variables B SE βt p Tolerance VIF
Consultation ,126 ,049 ,157 2,552 ,012 ,469 2,131
Support ,171 ,047 ,215 3,626 <,001 ,505 1,980
Summary R =,838 R
2
=,702 AR
2
=,695 F =98,821 df: 4,168 P<,001 DW: 1,971
5. App Type Adequacy BSE βt p Tolerance VIF
(Constant) 1,025 ,186 5,525 <,001
Examination ,220 ,061 ,289 3,619 <,001 ,702 1,424
Monitoring ,197 ,064 ,247 3,094 ,002 ,702 1,424
Summary R =,472 R
2
=,223 AR
2
=,214 F =24,851 df: 2,173 P<,001 DW: 1,954
F: ANOVA F value showing the significance of the model; DW: Autocorrelation between variables Durbin Watson; VIF: Variance Inflation Factor;
12 DIGITAL HEALTH
Figure 3. Application feature; code sub-code model.
Figure 4. Diagnosis and monitoring applications; code subcode model.
Baştürk et al. 13
codes (Figure 5). Some application suggestions are given in
Appendix 4.
Care and treatment applications. In these applications, the
medication (CS =12) and exercise (CS =11) codes were
most frequently emphasized (Figure 6). Thus, the recom-
mended practices should ensure the effectiveness, effi-
ciency, and controllability of treatment, especially for
patients who are sent home. Some application suggestions
are given in Appendix 5.
Operation applications. The most frequently used codes in
these applications were remote adjustment or use of devices/
equipment by an expert user (CS =12), emergency interven-
tion (CS =8), and drug administration (CS=7) (Figure 7).
Some application suggestions are given in Appendix 6.
Figure 5. Biometric measurement applications; hierarchical code sub-code model.
Figure 6. Care and treatment applications; code sub-code model.
14 DIGITAL HEALTH
Counseling and support applications. Among these appli-
cations, the most common code was counseling practice
between the patient and healthcare provider (CS =23), fol-
lowed by suggestions regarding remote health environ-
ments (CS =27) (Figure 8). Some application suggestions
are given in Appendix 7.
Integration status of the application into the hospital remote
health system. Recommended applications were classified
according to their development status, and their distribution
according to branch was provided by creating the
MAXQDA 2022 Cross Table (Table 6).
The majority of recommendations were regarding the
integration of existing applications into the system (N =
57, 44.88%). This was followed by recommendations for
developing and integrating existing applications (N =51,
40.16%). While internal branches mostly recommended
integrating existing applications by improving them
Figure 7. Operation applications; code sub-code model.
Figure 8. Counseling and support applications; code sub-code model.
Baştürk et al. 15
(internal: 24-surgical: 17), surgical branches mostly sug-
gested integrating new applications (surgery: 13, internal:
4). Some application suggestions are given in Appendix 8.
Sample applications according
to their application potential
The potential for using applications for each application
recommendation was graded according to expression
style, emotion analysis, and development status. The use
of positive expressions about the application, such as beau-
tiful, useful, good, and excellent, in the text content and
application suggestion made by more than one participant
were considered in the rating (Table 7).
The data in Table 7 were obtained using the analysis
made with the MAXQDA 2022 Interactive Quote Matrix.
Many RHS applications with high application potential
exist for all branches. The majority of the recommendations
related to integrating existing applications into the corpor-
ate system either directly or by improving them.
Qualitative-Quantitative integrated findings
on RHS applicability
Different data types and analyses yielded findings that
support, explain, or extend each other (Table 8).
“Experience”(N =110) and “quan-usefulness”(M =
3.10) categories had high values in the same application
types: “counseling”(N =89/M =3.40), “support”(N =37/
Table 6. Distribution of application development status based on branch.
Coded sections were counted once per document.
Table 7. Distribution of Application Potential and Integration Status Based on Main Branch
a
Main Branch
Participant Application Potential Application Integration Status
Total Very High High Middle Low Total Available New By İmproving
Mixed 16 12 11 6 1 30 12 3 15
İnternal 42 42 14 5 2 63 29 4 30
Surgical 47 33 15 9 3 60 25 14 21
Total 105 87 40 20 6 153 66 21 66
a
The number of applications was determined by the sum of the same applications suggested by different participants or different applications suggested by the
same participants.
16 DIGITAL HEALTH
Table 8. Qualitative-quantitative integrated findings on RHS applicability.
Analysis Data
Applicability
categories
Physical
examination Measurement Evaluation
Treatment /
Care Operation Monitoring Counseling Support
General or
total (T)
Descriptive,
Coding and
Themes
Guan Experience
1
N14 2122231333 89 37 110
Usefulness M 2.67 2.99 3.12 3.00 2.98 3.17 3.40 3.38 3.10
N 176 176 175 175 176 176 176 176 176
Gual Usefulness Coded
section
14 103 105 76 44 38 63 137
Suggestions by
branch
Coded
document
Mixed 4 11 11 848410 16
İnternal 5 17 31 17 12 21 15 25 42
Surgical 4 20 29 14 12 18 10 23 47
Total 13 48 71 39 28 47 29 58 105
Examples Figures
2
45467488
Appendices
3
45467488
Correlation
4
and
Rgression
5
Guan
Demand/
Usefulness
Correlate
Own d. .581** .694** .670** .731** .680** .719** .656** .689** .831**
Patient d. .444** .495** .504** .604** .499** .549** .522** .522** .631**
Demand .556** .644** .637** .724** .639** .688** .639** .657** .793**
Regression
6
App
Demand
β=.393 β=.233 β=.157 β=.215 R
2
=.702
Adequacy/
Usefulness
Correlate
Technical .420** .380** .315** .313** .304** .375** .331** .303** .413**
Count .435** .363** .273** .315** .334** .410** .369** .299** .413**
Qualification .367** .310** .233** .284** .311** .380** .332** .301** .369**
Adequacy .424** .365** .284** .317** .330** .405** .359** .314** .415**
(continued)
Baştürk et al. 17
M=3.38), and “monitoring”(N =33/ M =3.17).
Applications with which users are more experienced are
perceived as more usable. However, the applications that
received high values in the “qual-usefulness”category
were “support”(CS =137), “evaluation”(CS =105), and
“measurement”(CS =103). Considering the relationship
of “measurement usability”with “technical adequacy”
(r =0.380) and “own-demand”(r =0.694), the usability of
applications increases when “technical adequacy”and
“demand”are ensured. Considering the relationship
between “monitoring app usability,”and the categories
“adequacy”(r =0.405, β=.247), “demand”(r =0.688,
β=.233), and their subcategories, more “adequacy”
and “demands”are required to use the monitoring appli-
cation than other applications. Furthermore, the stron-
ger correlations of the categories “examination
usability”(r =0.424, β=.289) and “monitoring usabil-
ity”(r =0.405, β=.405) with “app adequacy,”com-
pared with the other categories, including the
subcategories, suggest that the design of these applica-
tions is not functional or they are not used effectively
due to insufficient training.
Furthermore, the “treatment/care usability”(r =0.724,
β=.393) application requires more “demand”than the
more experienced “counseling”(r =0.639, β=.157) and
“support”(r =0.657, β=.215) applications. The “adequacy”
(r =317) relationship of the “treatment/care usability”appli-
cation was lower than that of the other applications.
Similarly, more “demand”was required for “support usabil-
ity”(r =0.657, β=0.215) compared with “adequacy”
(r =0.314). Thus, these applications can be used without
needing too much “adequacy”under high “demand.”simi-
larly, more “adequacy”is needed for “physical examination
usability”(r =0.424, β=.289) compared with “demand”
(r =0.556).
The “operation usability”application was less related to
“adequacy”(r =330) and “demand”(r =639) than the
other applications. However, the values of “quan-
usefulness”(r =317), “qual-usefulness”(CS =44), and
“experience”(M =13) were low. Thus, the relationship
between the usability of “operation usability,”and the cat-
egories of “demand”and “adequacy”was weaker than that
with other applications. The same is true for “quan/
qual-usability”and “experience.”Professional statements
also supported this inference.
In the multiple regression analysis, the variables signifi-
cantly explained the models (p < 0.001): “app type demand”
model:Care β=0.39, monitoring β=0.23, consultation β=
0.16, and support β=0.22, (70%); “usefulness”model:
demand β=0.75 and adequacy β=0.11, (64%);
“demand”model: usefulness β=0.75 and adequacy β=
0.10, (64%); “adequacy”model: demand β=0.24, useful-
ness β=0.25, and education β=−0.24, (25%); and “app
type adequacy”model: examination β=0.29 and monitor-
ing β=0.25, (22%).
Table 8. Continued.
Analysis Data
Applicability
categories
Physical
examination Measurement Evaluation
Treatment /
Care Operation Monitoring Counseling Support
General or
total (T)
Regression
6
App
Adequacy
β=.289 β=.247 R2=.223
One-Way Anova
7
Demographic G/
Usefulness
Differences
Experience 3&1|2* 1&2|3* 3&1|2* p> 0.05
Education 4&1|3* p> 0.05
Profession 3&1|2* p> 0.05
*p< 0.05, **p< 0.01 level. η²: estimated eta-squared. Groups differences (GD: 1., 2., 3., 4. and 5.) were identified by “&;and, |;or”signs.
1See Table 1 for more details.
2
See Figures for more details.
3
See Appendices for more details.
4
See Table 3 for more details.
5
See Table 4 for more details.
6
See Table 5 for more details.
7
See Table 4 for more details.
18 DIGITAL HEALTH
Discussion
Overall, the results of this study reveal that the majority of
healthcare professionals used at least one application from
each type of RHS either informally or formally. The most
experienced branches are internal medicine, cardiac, pediat-
rics, infection, and orthopedics. Internal (surgical) branches
come to the fore in terms of application experience (appli-
cation recommendations) and current (new) applications.
Illustrating the constant pursuit of progress in healthcare,
the growth curve of robotic surgery points to innovation.
48
While application recommendations are mostly for
support, measurement, and evaluation applications, appli-
cations which employees have more experience of using,
such as “Consulting,”“Support,”and “Monitoring,”are
perceived to be more usable. Unlike the current study, a
study conducted with physiotherapists revealed that the
applications were mostly used in monitoring, treatment,
and evaluation.
11
In another study, doctors recommended
support applications such as receiving test results, schedul-
ing appointments, and medication reminders to their
patients; however, they were reluctant to recommend appli-
cations such as test result evaluation, video conferencing,
and remote monitoring of vital parameters.
42,49
I
n this study, professionals find applications that they
actually used to be useful. Furthermore, professionals want
to use more technical applications if the necessary conditions
are met. Similar results were reported by a study conducted in
Korea: Those with experience in distance therapies exhibited
a significantly higher willingness to continue participation
throughout the institutionalization process.
50
Early adopters
of telemedicine applications may influence others to try
them. A strong recommendation intention indicates a posi-
tive experience and successful adoption. Recommendation
intention emerges as an important outcome variable influ-
enced by a number of psychological and motivational
factors, such as “performance expectancy”and “effort
expectancy.”
23
Medical practitioners’awareness and posi-
tive attitudes play an important role in the effective function-
ing of RHS systems.
51
An interesting finding of this study is
that professionals’self-use demand exhibits a stronger asso-
ciation with application usability than patients’demand. The
relationship of education and experience with applicability,
which is frequently emphasized in the literature; must be
examined from different perspectives. As such, rigorous,
in-depth research should be conducted on professional regu-
lations for RHS content creators, RHS operators, RHS engi-
neers, RHS safety experts, virtual health assistants, hologram
doctors, and so on, as a professional function or as new
professions.
The applicability categories in this study exhibit moder-
ate to high correlations. All three applicability categories
significantly explain each other in the multiple regression
models. Educational status, a demographic factor, is also
included in the “Adequacy of Requirements”model. The
adequacy of requirements for applicability is lower for
older adults than for the youth, for internal/surgical
branches than for mixed branches, and for physicians than
for other professionals. Furthermore, those with higher edu-
cation perceive “staff number”as low score and “care
usability”as high score . Thus, technical and human
resources must be carefully managed for RHS applicability.
Insufficient integration resulting in unnecessary repetitions
increases workload and affects user demand.
38
Healthcare
managers should also consider demographic influences
when making assignments. For example, because they
interact with technology more, the younger generation
may be more aware of the benefits of technology or new
applications than the older generation.
User participation in health technology design is very
important for application usability. The development of
RHS systems requires a multidisciplinary approach with
patient and healthcare professional participation.
35
By con-
sidering user suggestions during the design process, user-
friendly applications can be developed as technical solu-
tions can be incorporated in advance.
52
For instance, a
study evaluating access to remote treatment revealed that,
while the majority of individuals have access to digital
devices, the system used for remote therapy does not
meet the minimum requirements.
53
For remote monitoring
application in chronic patients, important features such as
data storage and transmission, user interface, and alarm
have been successful. Furthermore, the system design
allows the incorporation of data sources other than
medical data, such as temperature, humidity, or pollution
level. These data can improve predictions and may be espe-
cially important for chronic patients with respiratory pro-
blems.
9
In another study, healthcare providers indicated
that adopting, scaling, and sustaining technology-enhanced
nutrition care models benefits patients, clinicians, and
healthcare overall.
54
Potential barriers to RHS adoption include resistance to
change, resistance to use, perceived risk, status quo bias,
work-related issues, quality of care concerns, and organiza-
tional challenges.
55
System complexity, lack of user train-
ing, lack of system integration, security and privacy,
inadequate technical support, and inflexibility of systems
in use have been cited by healthcare professionals.
56
Resistance from healthcare providers may arise from lack
of training, limited clinical knowledge, low levels of staff
participation, reduced productivity, lack of required rou-
tines, communication problems, unplanned and ineffective
implementation, limited resources leading to infrastructure
deficiencies, technology usability challenges, design and
software barriers, integration issues with other information
technology systems, fears related to system effectiveness or
performance risk, and doubts about the clinical and cost-
effectiveness of the system.
55
The metaverse could offers a tremendous opportunity for
RHS applicability. The metaverse comprises various virtual
Baştürk et al. 19
components that represent embodiments of their physical
counterparts, such as digital avatars and three-dimensional
virtual environments. Technologies that enable the meta-
verse include augmented reality, AI, distributed computing,
digital twins, and telecommunications.
15
These components
can enhance the quality of remote interaction.
Lack of demand for use of RHS may be due to privacy
and data security concerns.
15
To address security concerns;
Real-world evaluations such as “wireless hacking test”are
carried out in environments where IoT devices are used in
RHS applications; safety precautions must be taken.
57
To
practically ensure the scalability and security of RHS appli-
cations; security models suggested for smart application
design can be used.
58
Studies have suggested security solu-
tions such as rigorous authentication processes, data
encryption, resistance to attacks, and continuous monitor-
ing, emphasizing the need to support patient autonomy,
ensure confidentiality of data, and maintain equal access
to healthcare in the context of IoT communication security.
Measures against the ever-evolving threats in smart health-
care environments must be incorporated into application
design.
43
These privacy-enhancing strategies will align
RHS applications with global data protection standards in
ensuring the security of patient data.
Overall, combining current study findings with those
from the literature, the critical features needed in an RHS
application are illustrated in Figure 9. Although the features
vary depending on the type of service, most features are
indispensable in a digital application owing to the connec-
tion between the services. Another crucial inference is that
design requirements demonstrate the importance of the
digital health engineering field.
The strategic gaps in smart healthcare applications are
“legal regulations,”“economic factors,”“user behavior
habits,”“politics and culture,”and “environment and tech-
nology.”
24
Studies have emphasized the integration of pol-
itical, technological,
59
organizational,
60
and cultural
61
aspects, as well as the development of new applications
in these directions to increase the socioeconomic benefits
in RHS delivery. The current study’s proposed future
version of RHS includes features such as multi-faceted
translation, multiple and flexible integration capabilities,
personalization, alternative options, sensor detection, and
advanced interaction. These features can significantly
increase the applicability of the RHS from a socio-
demographic perspective.
In summary, the extant application features experienced
related to a RHS operate within a limited framework.
Furthermore, the system is not useful enough, applications
are not integrated and used officially, and application fea-
tures need to be significantly improved. Several applica-
tions with high application potential exist for every
branch and service type, and are even used in some institu-
tions. Application recommendations are mostly related to
integrating existing applications into the corporate system
either directly or through development. Nonetheless, there
Figure 9. Critical application feature.
20 DIGITAL HEALTH
are interesting suggestions for future innovation. To move
forward in this regard, the necessary conditions and user
demand must be met. Table 9 presents meta-inferences
and sample applications regarding application recommen-
dations based on RHS application types.
Physical examination
The application type wherein usability is most dependent on
adequacy is physical examination. However, compared with
other applications, a lower demand is required for its usabil-
ity. The use of physical examination was mostly recom-
mended in the outcome evaluation phase or for greenfield
patients (Appendix 3). Remote examination can be used
effectively in a system that enables support, measurement,
monitoring, and evaluation applications. For example, appli-
cations that recommend examination or treatment decisions
can be developed by adequately detecting the patient’s con-
dition. These decisions can be verified by remote examin-
ation. Furthermore, more effective results can be obtained
when using applications with advanced technology.
Studies have exhibited the potential of new technology to
remotely enable individuals to experience the sensations of
touch, sight, and sound, which are indispensable for physical
examination.
35
Another study comparing simultaneous
remote and in-person musculoskeletal examination using
augmented reality, tactile sensors, and three-dimensional
imaging found that the evaluations matched.
62
In another
study comparing remote and face-to-face application of
heart and lung auscultation, remote application was found
to be an acceptable alternative to face-to-face application.
63
Measurement
Compared with consulting practice, the usability of meas-
urement application depends more on technical adequacy
and health professionals’own demand. Concerns about
the reliability of applications may impact professionals’
demand for RHS use. A study on biometric measurement
and monitoring using a smartwatch found that patients
were satisfied with the application; however, technical pro-
blems reduced usability.
64
In the current study, suggestions were reported for mea-
surements such as sound, hearing, touch, vision, rhythm/
motion, pressure, and temperature using sensors. Other
studies have shown that remote measurements such as
height/weight,
65
auditory brainstem response testing,
66
heart rate,
67
or blood pressure,
68
among others, are feasible.
Evaluation
The evaluation application is the least related application to
“adequacy”and second least related application to
“demand.”However, it can be considered the third most
useful application. This study suggests that, medical
device data should be transferred to the RHS information
system for better evaluation. Here, health status-specific
screening and assessment applications can be used.
Additionally, applications that evaluate functions, such as
sleep, nutrition, movement, respiration, circulation, excre-
tion, and emotionality, can be integrated into the system.
Innovative suggestions for malignancy prediction using
noninvasive chips or digital twins have also been presented.
Some applications on evaluation have been reported in
the literature. For instance, patients at risk of mortality
can be identified by the remote assessment of heart failure
symptoms.
69
Another study found that “prehospital assess-
ment using commercial mobile phones with fifth-generation
wireless communication technology is feasible and reliable
during ambulance transport in urban areas.”
70
Additionally,
the telemedical approach can significantly reduce the wait
time for specialist evaluation in a real-world setting.
71
Treatment/care
Compared with other applications, less adequacy and more
demands are needed for usability of treatment/care applica-
tion. Improved versions of applications exist, such as medi-
cation use, mobilization, diabetes, circulation, nutrition,
wound care, excretion, stress, sleep, and respiratory man-
agement, which are recommended to be used within the
scope of remote care-treatment. Many applications are actu-
ally used unofficially. Health professionals believe that their
use will be beneficial. Moreover, those with higher educa-
tion perceived “Care Usability”as high.
A scoping review of distance physiotherapy found
remote therapy to be safe, feasible, and acceptable to
patients, and more cost-effective than face-to-face
therapy.
72
Another study found a positive usability evalu-
ation of a digital self-management app for depression.
73
Furthermore, the use of the remote health program was
associated with significant clinical improvement in
anxiety and depression.
74
Additionally, studies have con-
cluded that remote health intervention programs can be
applicable, safe, and effective in the rehabilitation process
of neurological diseases.
75
Operation
Operation is the fourth application that the participants per-
ceived as related to the “demand”and “adequacy”categories.
This application is the least experienced but the second most
usable application. In applications involving direct intervention,
security or uncertainty may affect applicability more than other
factors. For example, contemporary robotic surgery systems
face challenges such as security, privacy, reliability, latency,
and costly impacts of blockchain-based storage.
76
Developments such as augmented reality, AI, machine
learning, integration of imaging and visualization technolo-
gies, improved precision and dexterity of robotic arms,
Baştürk et al. 21
Table 9. Remote health application: examples of useful RHS recommendations—existing, developable, or innovative applications.
Application Definition Resource Available Must be developed Future innovations
Physical
Examination
Physical examination is usually performed remotely with the
patient and doctor meeting simultaneously. The ability to
perform fixed/moving images, tissue density/saturation,
voice and speech analyses, and tactile analyses via sensors
during the examination will increase the usability of these
applications.
7,15,16,35,57,58
1. Pre or post-examination
interview.
1. İnterview supported by symptom checker
app or wearable devices
1. Simultaneous analysis of image, tissue
density/fullness, sound, and speech
Using remote sensors.
2. Examination in an environment
supported by metaverse whith 3D
visualizations
Measurement It involves remote measurement of a person’s measurable
physical and behavioral values using a digital device and
application. Measurements such as image, vital, O2, EEG,
EMG, ECG, blood, urine, sugar, color, and mobility
measurements should be given as examples of these
applications.
9,35,76,81
1. Temperature, pulse, blood or
pressure measurements
1. Function measurements such as respiration
and excretion
2. Spermiogram
3. Tumor markers
1. Glasses measuring intraocular
pressure,
2. Delivering a digital tablet through
the urethra to diagnose kidney stones.
Evaluation It includes the remote identification and evaluation of the
individual’s health-specific conditions using tools such as
digital devices/applications, artificial intelligence,
algorithms, etc. Evaluation types such as disease risk,
complications, health status, screening, prediction,
prioritization/triage, and wound evaluation should be given
as examples of evaluation applications.
11,33,36,38,81
. 1. Health status-specific
screening, and evaluation
applications
2. Evaluation of sleep,
nutrition, movement,
respiration, circulation,
excretion or emotionality.
1. Transferring medical device data to the RHS
information system.
1. Malignancy prediction using a
noninvasive chip or digital twin.
2. Real-time hologram doctor review
of critical health results via an app that
continuously monitors and reports
biometric data.
Treatment / Care It includes remote management and implementation of care
and treatment services using digital devices and
applications. The prominent application features of these
services are disease-specific and personalized reminders,
alerts, task lists, adjustments made according to
measurement and/or monitoring results, treatment
compliance, progress, and side effect checks. Practices such
as nutrition, sleep, exercise, medication, wound care, and
stress management should be given as examples of these
applications.
11,12,15,38,83
. 1. Apps to manage personal
care functions such as diet,
sleep, exercise, stress
1. Medication use control and management.
2. Treatment applications for elderly and
chronic patients, powered by sensors, voice
digital assistants, user-friendly interfaces
and translation features.
1. Prescribing personalized treatments for
specific patient care such as drug use,
wound care, diabetic patient care
2. Applying treatment practices with
digital nurse support
Operation It involves carrying out a transaction in person remotely, using
a digital device and applications. Applications such as
robotics, surgery, interventional procedures, drug
administration, environment setting, and device setting
should be given as examples of these applications.
7,15,16,36,84
1. Adjustment of environment,
device or drug
1. Emergency intervention apps for elderly
falls, heart attacks, e.g.,
2. Robotic surgery
1. Operations platforms, powered by
Metaverse, for example, personalized
simulations on digital twin/avatar
2. Neuron implants that will enable
remote intervention and control
movement
Monitoring It involves remote monitoring of an individual’s health-specific
conditions using tools such as digital devices/applications,
artificial intelligence, algorithms, etc. For example, features
such as remote monitoring of health findings, warning
when there is a deviation in values, and remote patient
visits should be given.
9–11,15,38,74
. 1. Health status-specific
monitoring applications
1. Patient wristband application with biometric
measurement, tracking, and analysis
features.
1. Monitoring the sedated patient’s level of
consciousness.
2. Continuous monitoring of physical,
mental, and emotional health status
via a personalized chip.
3. Virtual visits
(continued)
22 DIGITAL HEALTH
Table 9. Continued.
Application Definition Resource Available Must be developed Future innovations
Counseling It is the simultaneous or asynchronous meeting with relevant
people via a digital application when needed regarding a
certain subject. There are ways of applying it between
individuals or groups, formally or informally, related to
health issues. It can be done between the same or different
levels of expertise based on patient, branch, profession, or
institution.
3,15,42,77,78
1. Interview between experts
with a formal app
1. Formal consultation app, involving experts or
institutions at various levels.
1. International consultant group
application consisting of different
interest groups.
2. Consultation platforms involving
virtual experts and expert’s avatar
Support It is the implementation of activities that assist in the provision
of health services using digital tools. Applications such as
recording, decision support, personal digital assistant,
information, training, and meeting applications should be
given as examples of support applications.
7,16,42,43,78,82,85
. 1. Patient admission, navigation,
decision support,
information, training, etc.
1. Audio, speech, sign, text, etc. transformation.
2. Voice reporting application
3. Current content production and/or
supervision by health professionals
1. Informational applications in digital
games.
2. Digital identity application that
includes personal health information,
with features such as identification,
verification, authorization, censorship,
etc.
3. Virtual health Assistant special for
professionals and patients
General RHS
system
RHS can be defined as remote delivery of healthcare services
through digital technologies such as wearable smart
devices, application software, information communication,
and connectivity technologies.
For RHS, an application platform is required that includes
smart systems, equipment, environment, connectivity,
digital users, data management, security, translation,
telecommunications, sensors and analytics.
33,35,40,41,44,57,66,69,77,81,85
1. Promotion, compliance
training, etc. Activities for
using the existing system.
Currently generally for
individual use
1. İntegrated data, devices, system,
organization and applications. 2. Mobile
phone as a medical device and mobile
version of apps.
Official use of institutional regional
integration
1. Application environments (cabin) at
specific locations, supported by the
virtual universe, connected to RHS
centers.
2. Critical features in RHS system and
applications design (see Figure 9)
Multiple and Global integrations.
Coded documents N 57 51 19
Baştürk et al. 23
robotic design allowing flexible surgery, tactile feedback,
and sensory enhancement, are increasing the potential for
robotic surgery application.
48
Despite reservations regarding using RHS in surgical
branches, important suggestions have been made for opera-
tive patient groups related to planning, operation simula-
tion, robotic surgery, three-dimensional imaging, or a
common platform, in addition to pre- or post-procedure
follow-ups. Considering these suggestions, a common plat-
form can be created, supported by AI and augmented and
virtual reality technologies, wherein surgeries performed
throughout Turkey can be monitored or relevant experts
can be involved when necessary. The technology to simul-
taneously connect multiple consoles to perform a single
procedure is especially beneficial for patients with
complex medical conditions that require a multi-specialty
approach. Experts from different disciplines can connect
from different locations and collaborate on the same
patient in real time.
77
The application system may have a
feature that allows operation planning and simulation on a
person-by-person basis. Such an application not only sup-
ports surgeons but also helps assistants obtain experience
from cases they have not encountered before. AI technol-
ogy can help create visual animations and aid surgeons in
understanding what they cannot see.
78
Additionally, deci-
sion support applications can be used during the operation,
providing both visual and informational support on a trans-
action basis.
Monitoring
In terms of usability, the second most dependent application
on “Adequacy”and “Demand”is “Monitoring.”This appli-
cation is also the third most experienced and most usable
application. A study on monitoring, diagnosis, and treat-
ment applications revealed that wireless technology is
mostly used in the monitoring area.
79
Based on the results
of the current study, the monitoring application must be
strengthened with supporting applications. For instance,
the “patient wristband application integrated with the
RHS system, with biometric measurement, tracking and
analysis features”suggested in this study may be an inter-
esting application.
Similar to the results of this study, other studies have
also recommended using health condition-specific applica-
tions. For instance, in a study conducted with heart failure
patients, the monitoring application was found to be
effective.
80
Similar results have been obtained for patients
with chronic
9,81
and psychotic issues,
10
Parkinson’s,
64
and so on.
Counseling
The usability of the consulting application is higher than
that of other applications when sufficient conditions and
patient demand are provided. Participants suggested that
the most used consultancy application should be done
through the formal application and one with more technical
features. Previous studies support this conclusion. For
example, remote consultation has been shown to be import-
ant and reduces the number of patient visits, especially
when isolating those with infectious diseases. However,
the current RHS system must be improved with support
applications.
40,82
Another study found that chatbots devel-
oped for medication counseling supported healthcare pro-
fessionals.
83
Currently, counseling is generally performed
via one-on-one meetings. In informal applications, it can
also be done through a portal with certain groups of consul-
tants or counselee. Consultation groups practice (consultant
groups) is being carried out informally among health pro-
fessionals usually using WhatsApp. Participants stated
that exchanging ideas on a subject is useful and frequently
used in groups comprising professionals or experts from
different branches. The application is especially effective
in using multiple minds and reaching a common conclusion
in evaluating critical situations. The official implementation
system should be developed considering these situations.
As an application feature, transaction-based communication
styles should be created.
Support
An application whose applicability is secondarily depend-
ent on patient demand is the support application. This
application is the second most experienced and most
usable application; compared with other applications,
less adequacy and more demands are needed for its
usability. Notably, most application recommendations
have been made for this application. This application
type is widely discussed, especially in digital health lit-
erature. Nonetheless, it has a wide application and devel-
opment potential. For instance, a study evaluating the
support practices used by the Turkish Ministry of
Health found that the most used E-Nabızservicewas
finding out the examination results.
84
Similar applications
were recommended to patients by doctors.
42
In this study,
application recommendations were made for decision
support, education, information, guidance, preventive
health, and patient acceptance. Thus, there is a wide-
ranging potential for information applications. Scholars
can examine digital content production on a selected
topic, such as exercise, patient information, and obtaining
consent. Content does exist on most subjects; however,
this the content should be produced by professionals, or
at least revised and updated. This also reveals “digital
health information”content production as a new profes-
sional function. Thus, virtual health assistant application
based on AI
85
can be more effective than existing
applications.
24 DIGITAL HEALTH
The most striking code in support applications is RHS
environments. In the literature, location designs such as
smart hospital, house, city, building, apartment, outdoor,
or mobile health-based design have been mentioned for
RHS.
33,79,86
Considering the recommendations made in
this study, RHS environments can be in certain locations
such as a health institution, home, or in the city/intercity.
Certain applications such as measurement, interview, mon-
itoring, treatment, or operation must be possible in the
environment. For this, application features must be
designed very well. Especially for treatment and operation
applications, adjustments such as device, medication, and
environment settings should be made remotely. The recom-
mendations in this study support and expand the literature
on future innovation.
7,15,16
Based on these recommendations, voice recognition can
be especially used in support applications for various tasks
and processes, such as note taking, reporting, transaction
recording, and commanding. This technology is used in
smart hospitals, including electronic medical document
transcription, pathological voice recognition, and medical
process optimization through human–medical equipment
interaction.
87
Furthermore, remote support and inspection
applications with sound, image, sign, and text features
can be designed with a feature that enables communication
functions, such as messaging when necessary, and performs
different converting different functions, such as converting
voice to text.
Integration
Integration should include user, data, device, application,
and system. The participants emphasized that, the applica-
tions can be integrated with E-Nabız or hospital information
systems, all necessary data can be transferred to the system,
and access to the system can be provided through a
common application. In this context, the management of
big data is crucial to ensure health data standardization,
quality, and integrity. To ensure this, a general health data-
base should be developed at the national level to which
other applications and standardized health institution data-
bases can be connected. Moreover, applications should be
accessible within the framework of the needs and authority
of the institution and application users, and digital health
identities should be capable of being integrated into differ-
ent health systems.
Furthermore, the integration of frequently used personal
devices, such as smart watches and mobile phones, into the
general system is among the notable suggestions. It is stated
by the participants that, the applications used must have a
mobile version and be installed on the mobile phones of
users such as patients or healthcare professionals.
However, paradoxically, mobile phones, one of the
wearable-portable devices that are indispensable for RHS
use, do not have a specific standard as an RHS device
and are often used informally. This issue must be addressed.
An in-depth examination of the integration of both existing
applications and digital technologies that will enable the use
of these applications into the RHS system is needed.
Limitations
This study has some limitations. First, it was conducted on a
sample from a university hospital within the Turkish health-
care system. Nonetheless, besides minor institutional differ-
ences, the results provide useful information suitable for a
regional or international audience as health services are uni-
versal, and especially standardized in hospital settings.
Additionally, the contributions of academic health profes-
sionals who have worked in various institutions are
valuable.
Second, this study considered only some feasibility
factors owing to the nature of the study. Other dimensions,
such as social, economic, cultural, politics, law, and secur-
ity, are important in terms of feasibility and each may
require detailed investigations. Some of these dimensions
were added in the initial versions of this study’s question-
naire; however, based on the expert evaluation, it was
decided that the dimensions would be a separate study
subject owing to their scope and sensitivity. For example,
finances are under the control of the institutional owner
and management, especially in hospitals. As such, this
factor should be examined more comprehensively by
healthcare service managers, providers, and financiers.
Furthermore, during the trial phase, healthcare profes-
sionals perceived these issues as technical aspects and
described them as subjects they had secondary knowledge
about. Similarly, the demand factor was limited to the
primary users of healthcare services. Politicians and sup-
plier companies have quite important roles or expectations
in the use of RHS. Finally, the security dimension should be
studied in more detail to reveal the different risks of each
type of application. An up-to-date measurement tool can
be developed based on the type of application by also bene-
fiting from the features outlined here regarding the func-
tionality of the applications.
Conclusion
This study provides strong inferences regarding the charac-
teristics of RHS applications. Thus, it contributes unique
insights regarding the potential for RHS implementation.
The results reveal that the adequacy of requirements,
patient–professional demand, and usability categories are
important for RHS applicability. Professionals find the
applications they have experienced useful and want to use
other applications as well if the necessary conditions are
met. Interestingly, patient–professional demand has a
greater impact on the usability of applications compared
with the adequacy of requirements. Additionally, the
Baştürk et al. 25
professional’s own-use demand for RHS usability is more
effective than the patient’s demand. Notably, our results
are obtained both from quantitative and qualitative data
which consider the needs and interests of professionals.
Consequently, our conclusions regarding the application
features are robust and valuable. Based on the results, we
suggest that, for a hospital of the future, RHS application
should have at least these critical features. For instance,
as defined in the specifications, virtual assistants and
avatars should be specific to patients and professionals.
Furthermore, we provide interesting examples of existing,
improvable, or innovative application suggestions that can
be integrated into the RHS system, such as meta-inferences
based on application type, which are derived from combin-
ing professional recommendations.
Moreover, education and experience significantly affect
applicability. This result, which is frequently emphasized in
other studies, draws attention to a missing issue in terms of
adaptation to the digital age: For RHS applicability, profes-
sions or professional functions should be updated according
to the newly emerging application potential. Furthermore,
differences between demographic groups are mostly
related to the adequacy of requirements. Hence, healthcare
managers should consider demographic factors, such as
age, education, profession and experience, when planning
resources and staff. Similarly, application developers
should consider demographic and applicability categories
along with the features that an RHS application should
have. For example, patient–professional application users
should have usage options that consider their age, educa-
tion, profession, health status, and so on. Given the substan-
tial demand and necessary conditions, tremendous potential
exists for developing and inventing usable applications.
List of abbreviations
AI artificial intelligence
ANOVA one-way analysis of variance
CS coded sections
IoT internet of things
PCA principal component analysis
RHS remote healthcare services.
Acknowledgements:We would like to thank the hospital’s head
nursing department for contributing to the implementation of the
surveys during the data collection process. We would also like
to thank the healthcare professionals who participated in the
study and shared their opinions, ideas, and experiences. We
would like to thank Editage (www.editage.com) for English
language editing. We would also like to thank the reviewers and
editorial team for their comments that contributed to the quality
and expressiveness of the study.
Contributorship: F.B.: conception or design of the work (65%),
data collection (65%), data analysis and interpretation (65%),
drafting the article (65%), and critical revision of the article
(65%). A.O.T.: conception or design of the work (20%), data
collection (20%), data analysis and interpretation (20%), drafting
the article (20%), and critical revision of the article (20%). O.Ö.:
conception or design of the work (5%), data collection (5%),
data analysis and interpretation (5%), drafting the article (5%),
and critical revision of the article (5%). Ç.K.: conception or
design of the work (5%), data collection (5%), data analysis and
interpretation (5%), drafting the article (5%), and critical
revision of the article (5%). L.I.: conception or design of the
work (5%), data collection (5%), data analysis and interpretation
(5%), drafting the article (5%), and critical revision of the article
(5%). Final approval of the version to be submitted: all named
authors have read and approved the final version of the
manuscript.
Availability of data and materials: The datasets generated and/
or analyzed during the current study are not publicly available due
to limitations of ethical approval involving the participants’data
and anonymity but are available from the corresponding author
on reasonable request.
Consent for publication: The purpose of the research and
confidentiality of the data obtained were explained to the
participants, and their informed verbal and written consent was
obtained.
Declaration of conflicting interests: The authors declared no
potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.
Ethics approval and consent to participate: The Yozgat Bozok
University Ethics Committee approved the research on 21
September 2022, with reference number 36/18. The purpose of
the research and confidentiality of the data obtained were
explained to the participants, and their informed verbal and
written consent was obtained.
Funding: The authors disclosed receipt of the following financial
support for the research, authorship, and/or publication of this
article: The study was supported by Yozgat Bozok University
Scientific Research Projects Commission under the code
THD-2023-1093.
ORCID iD: Fadime Bastürk https://orcid.org/0000-0003-2928-
824X
Supplemental material: Supplemental material for this article is
available online.
References
1. Mbunge E, Jiyane S and Muchemwa B. Towards emotive
sensory Web in virtual health care: Trends, technologies, chal-
lenges and ethical issues. Sens Int 2022; 3: 100134.
2. Hakim AA, Kellish AS, Atabek U, et al. Implications for the
use of telehealth in surgical patients during the Covid-19 pan-
demic. Am J Surg 2020 Jul; 220: 48–49.
26 DIGITAL HEALTH
3. Ohannessian R, Duong TA and Odone A. Global telemedi-
cine implementation and integration within health systems
to fight the Covid-19 pandemic: a call to action. JMIR
Public Health Surveill 2020; 6: e18810.
4. Asiri A, AlBishi S, AlMadani W, et al. The Use of
Telemedicine in Surgical Care: a Systematic Review. Acta
Inform Medica AIM J Soc Med Inform Bosnia Herzeg Cas
Drustva Za Med Inform BiH [Internet] 2018 Oct [cited
2022 Jun 21]; 26. Available from: https://pubmed.ncbi.nlm.
nih.gov/30515013/.
5. Di˙lbaz B, Kaplanog
lu M and Kaya D. Teletıp ve Telesag
lık:
Geçmis, Bugün ve Gelecek. Eurasian J Health Technol
Assess 2020 Nov 30; 4: 40–56.
6. Uysal B and Ulusi˙nan E. Güncel Dijital Sag
lik
Uygulamalarinin Incelenmesi. Selçuk Sag
lık Derg 2020 Apr
30; 1: 46–60.
7. Ag
aog
lu FO, Eki˙nci˙LO and Tosun N. Metaverse ve Sag
lik
Hi˙zmetleri˙Üzeri˙ne Bi˙r Deg
erlendi˙rme. Erzincan Binali
Ildırım Üniversitesi I
ktisadi Ve I
dari Bilim Fakültesi Derg
2022 Jun 30; 4: 95–102.
8. Musamih A, Yaqoob I, Salah K, et al. Metaverse in
Healthcare: Applications, Challenges, and Future Directions
[I
nternet]. [cited 2024 Sep 10]. Available from: https://
ieeexplore.ieee.org/abstract/document/9956737.
9. Morales-Botello ML, Gachet D, de Buenaga M, et al. Chronic
patient remote monitoring through the application of big data
and internet of things. Health Informatics J 2021 Jul 1; 27:
14604582211030956.
10. Zhang X, Lewis S, Carter LA, et al. Evaluating a smartphone-
based symptom self-monitoring app for psychosis in China
(YouXin): a non-randomised validity and feasibility study
with a mixed-methods design. Digit Health 2024 Jan 1; 10:
20552076231222097.
11. Aydin NS, Torlakcik C and Tonga E. Pos0073-Hpr applic-
ability of the digital physiotherapy interventions for rheumatic
and musculoskeletal conditions in Turkey: a mix method
study. Ann Rheum Dis 2023 Jun 1; 82: 247–248.
12. Semonella M, Marchesi G, Andersson G, et al. Usability
study of SOSteniamoci: an internet-based intervention plat-
form to support informal caregivers in Italy. Digit Health
2024 Jan 1; 10: 20552076231225082.
13. Molina-Carballo A, Palacios-López R, Jerez-Calero A, et al.
Protective effect of melatonin administration against
SARS-CoV-2 infection: a systematic review. Curr Issues
Mol Biol 2022 Jan; 44: 31–45.
14. Antón P, Maña A, Muñoz A, et al. An immersive view
approach by secure interactive multimedia proof-of-concept
implementation. Multimed Tools Appl 2015 Oct 1; 74:
8401–8420.
15. Musamih A, Yaqoob I, Salah K, et al. Metaverse in health-
care: applications, challenges, and future directions. IEEE
Consum Electron Mag 2023 Jul; 12: 33–46.
16. Söyler S and Averbek GS. Sag
lik Teknoloji˙leri˙Ve
Metaverse: potansi˙yel Uygulama Alanlari Ve Mevcut
Engeller. Int Anatolia Acad Online J Health Sci 2022 Sep
3; 8: 138–166.
17. Wetzel AJ, Koch R, Koch N, et al. ‘Better see a doctor?’Status
quo of symptom checker apps in Germany: a cross-sectional
survey with a mixed-methods design (CHECK.APP). Digit
Health 2024 Jan 1; 10: 20552076241231555.
18. Agnew JMR, Hanratty CE, McVeigh JG, et al. An investiga-
tion into the use of mHealth in musculoskeletal physiother-
apy: scoping review. JMIR Rehabil Assist Technol [Internet
2022; 9: e33609. Available from: https://rehab.jmir.org/
2022/1/e33609.
19. Andrei B, Biduski D, Pinto C, et al. Diabetes mellitus
m-Health applications: A systematic review of features and
fundamentals. Telemed E-Health [Internet] 2018; 24: 839–
852.
20. Dittrich F, Back DA, Harren AK, et al. Smartphone and app
usage in orthopedics and trauma surgery: survey study of phy-
sicians regarding acceptance, risks, and future prospects in
Germany. JMIR Form Res 2020 Nov 30; 4: e14787.
21. Eriksson L, Lindström B, Gard G, et al. Physiotherapy at a
distance: a controlled study of rehabilitation at home after a
shoulder joint operation. J Telemed Telecare 2009; 15: 215–
220.
22. Hamine S, Gerth-Guyette E, Faulx D, et al. Impact of Mhealth
chronic disease management on treatment adherence and
patient outcomes: a systematic review. J Med Internet Res
[I
nternet] 2015 Feb 24 [cited 2022 Jun 21]; 17, Available
from: https://pubmed.ncbi.nlm.nih.gov/25803266/?dopt=
Abstract.
23. Al-Emran M, Al-Qaysi N, Al-SharafiMA, et al. Factors
shaping physicians’adoption of telemedicine: a systematic
review, proposed framework, and future research agenda.
Int J Human–Computer Interact 2024; 1–20.
24. Yang Y and Lin GTR. Analyzing the shortcomings in smart
healthcare for remote home care—A case study of the
Taiwan market. Int J Environ Res Public Health 2024 Jul;
21: 838.
25. Hirose M and Creswell JW. Applying core quality criteria of
mixed methods research to an empirical study. J Mix Methods
Res 2023; 17: 12–28.
26. Creswell JW and Plano Clark VL. Designing and conducting
mixed methods research. 3rd ed. Los Angeles; London;
New Delhi; Singapore; Washington DC; Melbourne: Sage, 2018.
27. Tovin MM and Wormley ME. Systematic development of
standards for mixed methods reporting in rehabilitation
health sciences research. Phys Ther 2023 Nov 1; 103:
pzad084.
28. Rouleau G, Wu K, Parry M, et al. Providing compassionate
care in a virtual context: qualitative exploration of Canadian
primary care nurses’experiences. Digit Health 2024 Jan 1;
10: 20552076231224072.
29. Creswell JW. Research design: Qualitative, quantitative, and
mixed methods approaches. 4th ed. California, USA: Sage
publications, 2014.
30. Simsek A. Evren ve Örneklem. In: Sosyal Bilimlerde
Arastırma Yöntemleri. Eskisehir: Anadolu Üniversitesi,
2018, pp.108–133.
31. Kogan JR, Conforti LN, Bernabeo EC, et al. Faculty
staff perceptions of feedback to residents after direct
observation of clinical skills. Med Educ 2012 Feb 1; 46:
201–215.
32. Toygar SA. E-Sag
lık uygulamaları.Yasama Derg 2018 Jun 1;
37: 101–123.
33. Alshamrani. Iot and artificial intelligence implementations for
remote healthcare monitoring systems: a survey. J King Saud
Univ - Comput Inf Sci 2022 Sep 1; 34: 4687–4701.
Baştürk et al. 27
34. Sholla S, Naaz R and Chishti MA. Incorporating ethics in
internet of things (IoT) enabled connected smart healthcare.
In: 2017 IEEE/ACM international conference on connected
health: applications, systems and engineering technologies
(CHASE) [Internet]. Philadelphia, PA, USA: IEEE, 2017
[cited 2023 Sep 3], pp.262–263. Available from: http://
ieeexplore.ieee.org/document/8010648/.
35. Berlet M, Fuchtmann J, Krumpholz R, et al. Toward teleme-
dical diagnostics—clinical evaluation of a robotic examin-
ation system for emergency patients. Digit Health 2024 Jan
1; 10: 20552076231225084.
36. Sheng B, Wang Z, Qiao Y, et al. Detecting latent topics and
trends of digital twins in healthcare: a structural topic model-
based systematic review. Digit Health 2023 Jan 1; 9:
20552076231203672.
37. Vázquez A, Jenaro C, Flores N, et al. E-health interventions
for adult and aging population with intellectual disability: A
review. Front Psychol 2018; 9: 2323.
38. Dale CM, Ambreen M, Kang S, et al. Acceptability of the
long-term in-home ventilator engagement virtual intervention
for home mechanical ventilation patients during the
COVID-19 pandemic: a qualitative evaluation. Digit Health
2024 Jan 1; 10: 20552076241228417.
39. Giroux EE, Hagerty M, Shwed A, et al. It’s not one size fits
all: a case for how equity-based knowledge translation can
support rural and remote communities to optimize virtual
health care [I
nternet]. Vol. 22. 2022 [cited 2024 Jun 12].
Available from: https://www.rrh.org.au/journal/article/7252/.
40. Phillips D, Matheson L, Pain T, et al. Evaluation of an occu-
pational therapy led Paediatric Burns Telehealth Review
Clinic: exploring the experience of family/carers and clini-
cians [I
nternet]. Vol. 22. 2022 [cited 2024 Jun 12].
Available from: https://www.rrh.org.au/journal/article/6887/.
41. Blandford A, Wesson J, Amalberti R, et al. Opportunities and
challenges for telehealth within, and beyond, a pandemic.
Lancet Glob Health 2020 Nov 1; 8: e1364–5.
42. Burzyn
ska J, Bartosiewicz A and Januszewicz P. Dr. Google:
physicians—the web—patients triangle: digital skills and atti-
tudes towards e-health solutions among physicians in south
eastern Poland—A cross-sectional study in a Pre-COVID-19
era. Int J Environ Res Public Health 2023 Jan; 20: 978.
43. Jaime FJ, Muñoz A, Rodríguez-Gómez F, et al. Strengthening
privacy and data security in biomedical microelectromechani-
cal systems by IoT communication security and protection in
smart healthcare. Sensors 2023 Jan; 23: 8944.
44. Baigi SFM, Baigi SMM and Habibi MRM. Challenges and
opportunities of using telemedicine during Covid-19 epi-
demic: a systematic review. Front Health Inform 2022 Mar
5; 11: 109.
45. Reinecke F, Dittrich F, Dudda M, et al. Acceptance, barriers,
and future preferences of mobile health among patients
receiving trauma and orthopedic surgical care: paper-based
survey in a prospective multicenter study. JMIR MHealth
UHealth 2021 Apr 21; 9: e23784.
46. George D and Mallery P. IBM SPSS Statistics 26 Step by Step:
A Simple Guide and Reference. 16th ed. New York:
Routledge, 2019.
47. Baskale H. Nitel Arastırmalarda Geçerlik, Güvenirlik ve
Örneklem Büyüklüg
ünün Belirlenmesi. New York, USA:
Routledge, 2016.
48. Chatterjee S, Das S, Ganguly K, et al. Advancements in
robotic surgery: innovations, challenges and future prospects.
J Robot Surg 2024 Jan 17; 18: 28.
49. Wasi Abbas M, Nawaz Tahir H, Jaffar N, et al. Facilitators
and barriers in acceptance of telemedicine among healthcare
providers in Pakistan: a cross-sectional survey. J Med
Access 2024 Jan 1; 8: 27550834241266413.
50. Kim J, Kim S, Oh H, et al. Questionnaire survey on perception
and attitude toward of remote treatment by Korean medicine
doctors. J Korean Med 2024 Mar 1; 45: 99–112.
51. Allen MR, Webb S, Mandvi A, et al. Navigating the
doctor-patient-AI relationship - a mixed-methods study of
physician attitudes toward artificial intelligence in primary
care. BMC Prim Care 2024 Jan 27; 25: 42.
52. Erturkmen GBL, Juul NK, Redondo IE, et al. Design, imple-
mentation and usability analysis of patient empowerment in
ADLIFE project via patient reported outcome measures and
shared decision making. BMC Med Inform Decis Mak 2024
Jun 28; 24: 185.
53. Watson A, Mellotte H, Hardy A, et al. The digital divide:
factors impacting on uptake of remote therapy in a South
London psychological therapy service for people with psych-
osis. J Ment Health 2021; 24: 185.
54. Barnett A, Kelly JT, Wright C, et al. Technology-supported
models of nutrition care: perspectives of health service provi-
ders. Digit Health 2022 Jan 1; 8: 20552076221104670.
55. Talwar S, Dhir A, Islam N, et al. Resistance of multiple stake-
holders to e-health innovations: integration of fundamental
insights and guiding research paths. J Bus Res 2023 Nov 1;
166: 114135.
56. Tabaeeian RA, Hajrahimi B and Khoshfetrat A. A systematic
review of telemedicine systems use barriers: primary health
care providers’perspective. J Sci Technol Policy Manag
2022 Dec 22; 15: 610–635.
57. Muñoz A, Fernández-Gago C and López-Villa R. A test
environment for wireless hacking in domestic IoT scenarios.
Mob Netw Appl 2023 Aug 1; 28: 1255–1264.
58. Sánchez-Cid F, Maña A, Spanoudakis G, et al.
Representation of security and dependability solutions. In:
Kokolakis S, Gómez AM and Spanoudakis G (eds)
Security and dependability for ambient intelligence
[Internet]. Boston, MA: Springer US, 2009 [cited 2024
Dec 1], pp.69–95.
59. Eckersley L. Socioeconomic determinants of health: remote-
ness from care. Can J Cardiol 2024 Jun 1; 40: 1007–1015.
60. Barbalho RE, Schenkman S, Sousa A, et al. Innovative short-
cuts and initiatives in primary health care of rural/remote loca-
tions: a scoping review on how to overcome the COVID-19
pandemic. Rural Remote Health 2023 Dec; 23: 1–16.
61. Fitzpatrick KM, Ody M, Goveas D, et al. Understanding virtual
primary healthcare with indigenous populations: a rapid evi-
dence review. BMC Health Serv Res 2023 Mar 29; 23: 303.
62. Borresen A, Chakka K, Wu R, et al. Comparison of in-person
and synchronous remote musculoskeletal exam using aug-
mented reality and haptics: a pilot study. PM&R 2023 Jul 1;
15: 891–898.
63. Haskel O, Itelman E, Zilber E, et al. Remote auscultation of
heart and lungs as an acceptable alternative to legacy mea-
sures in quarantined COVID-19 patients—prospective evalu-
ation of 250 examinations. Sensors 2022 Jan; 22: 3165.
28 DIGITAL HEALTH
64. Maas BR, Speelberg DHB, de Vries GJ, et al. Patient experi-
ence and feasibility of a remote monitoring system in
Parkinson’s disease. Mov Disord Clin Pract [Internet]
2024; 11: 1223–1231. Available from: https://onlinelibrary.
wiley.com/doi/abs/10.1002/mdc3.14169.
65. Zhang E, Davis AM, Jimenez EY, et al. Validation of remote
anthropometric measurements in a rural randomized pediatric
clinical trial in primary care settings. Sci Rep 2024 Jan 3; 14: 411.
66. Sithi D, Govender SM and Ntuli TS. Evaluating the feasibility
of a tele-diagnostic auditory brainstem response service in a
rural context. S Afr J Commun Disord 2024 Jul 31; 71: 1020.
67. Xiao H, Liu T, Sun Y, et al. Remote photoplethysmography
for heart rate measurement: a review. Biomed Signal
Process Control 2024 Feb 1; 88: 105608.
68. Slapnic
ar G, Wang W and Luštrek M. Feasibility of remote blood
pressure estimation via narrow-band multi-wavelength pulse
transit time. ACM Trans Sen Netw 2024 May 11; 20: 77:1–77:21.
69. Wohlfahrt P, Jenc
a D, Melenovský V, et al. Remote heart failure
symptoms assessment after myocardial infarction identifies
patients at risk for death. J Am Heart Assoc 2024; 13: e032505.
70. Lee HW, Ko YC, Tang SC, et al. Prehospital neurologic
assessment using mobile phones: Comparison between neu-
rologists and emergency physicians. J Formos Med Assoc
2024; 13: e32505.
71. Barbieri VdO, Nakayama LF, Barbieri GA, et al. Transition
from an in-person to a telemedicine diabetic retinopathy screen-
ing program. Arq Bras Oftalmol 2024 Apr 19; 87: e2023.
72. Hawley-Hague H, Lasrado R, Martinez E, et al. A scoping
review of the feasibility, acceptability, and effects of physio-
therapy delivered remotely. Disabil Rehabil 2023 Nov 6; 45:
3961–3977.
73. Kandola A, Edwards K, Muller MA, et al. Digitally managing
depression: a fully remote randomised attention-placebo con-
trolled trial. Digit Health 2024 Jan 1; 10: 20552076
241260409.
74. Perlman A, Pickman Y, Dreyfuss M, et al. Digitally enabled
asynchronous remote medical management of anxiety and
depression: a cohort study. J Telemed Telecare 2024; 10:
1357633X241233788.
75. Despoti A, Megari K, Tsiakiri A, et al. Effectiveness of
remote neuropsychological interventions: a systematic
review. Appl Neuropsychol Adult 2024: 1–9.
76. Kumar N and Ali R. A smart contract-based robotic surgery
authentication system for healthcare using 6G-tactile internet.
Comput Netw 2024 Jan 1; 238: 110133.
77. Patel V, Marescaux J and Covas Moschovas M. The humani-
tarian impact of telesurgery and remote surgery in global
medicine. Eur Urol 2024 Aug 1; 86: 88–89.
78. Aktuel S. Sag
lık Aktüel. 2023 [cited 2023 Sep 7]. Robotik
Cerrahinin Önde Gelen I
simleri I
stanbul’da Bulustu.
Available from: https://www.saglikaktuel.com/haber/robotik-
cerrahinin-onde-gelen-isimleri-istanbulda-bulustu-92147.htm.
79. Akar T, Burmaog
lu S and Kidak LB. 5G Teknolojisinin
Sag
lık Alanındaki Uygulamaları.Eurasian J Health Technol
Assess 2023 Jun 30; 7: 1–22.
80. Brahmbhatt DH, Ross HJ, O ‘Sullivan M, et al. The effect of
using a remote patient management platform in optimizing
guideline-directed medical therapy in heart failure patients.
JACC Heart Fail 2024 Apr; 12: 678–690.
81. Dhamanti I, Nia IM, Nagappan K, et al. Smart home health-
care for chronic disease management: a scoping review.
Digit Health 2023 Jan 1; 9: 20552076231218144.
82. Vodic
ka S and Zelko E. Remote consultations in general prac-
tice –A systematic review. Slov J Public Health 2022 Dec 1;
61: 224–230.
83. Albogami Y, Alfakhri A, Alaqil A, et al. Safety and quality of
AI chatbots for drug-related inquiries: a real-world compari-
son with licensed pharmacists. Digit Health 2024 Jan 1; 10:
20552076241253523.
84. Gündog
du S and Erkek S. VatandasGözünden E-Sag
lık
Hizmetlerinin Deg
erlendirilmesi: farkındalık, Kullanımve
Memnuniyet Düzeyleri. Selçuk Üniversitesi Sos Bilim Mesl
Üksekokulu Derg 2022 Nov 30; 25: 646–667.
85. Samala AD and Rawas S. Generative AI as virtual healthcare
assistant for enhancing patient care quality. Int J Online
Biomed Eng 2024 May 1; 20: 174–187.
86. Mirjalali S, Peng S, Fang Z, et al. Wearable sensors for
remote health monitoring: potential applications for early
diagnosis of Covid-19. Adv Mater Technol 2022 Jan 1; 7:
2100545.
87. Zhang J, Wu J, Qiu Y, et al. Intelligent speech technologies
for transcription, disease diagnosis, and medical equipment
interactive control in smart hospitals: a review. Comput Biol
Med 2023 Feb 1; 153: 106517.
Baştürk et al. 29