ArticlePDF Available

Abstract and Figures

Digital health interventions (DHIs) enable improvements in health strategy and address health system challenges. The World Health Organization provides a formal classification for DHIs. However, safety claims, about such interventions, vary in quality and are often vague as to how they are communicated between technical, clinical experts and stakeholders. By combining the classifications with a method of safety analysis and justification, we postulate confidence in the safety of digital technology. Confidence is resulting from the application of the framework to the DHI, using defined health system challenges. The framework and derived safety justifications can be applied to any DHI. It can serve as guideline for health strategy, regulatory and standards based compliance.
Content may be subject to copyright.
A Method of Justifying Confidence in the
Safety of Digital Health Interventions
Stuart HARRISON a,1 George DESPOTOU a and Theodoros N. ARVANITIS a
a Institute of Digital Healthcare, WMG, University of Warwick, UK
Abstract. Digital health interventions (DHIs) enable improvements in health
strategy and address health system challenges. The World Health Organization
provides a formal classification for DHIs. However, safety claims, about such
interventions, vary in quality and are often vague as to how they are communicated
between technical, clinical experts and stakeholders. By combining the
classifications with a method of safety analysis and justification, we postulate
confidence in the safety of digital technology. Confidence is resulting from the
application of the framework to the DHI, using defined health system challenges.
The framework and derived safety justifications can be applied to any DHI. It can
serve as guideline for health strategy, regulatory and standards based compliance.
Keywords. Digital health, safety, justification, health system, confidence, hazard
analysis
1. Introduction
The lack of the adoption of the fundamental concepts of clinical risk management and
safety methods, within health informatics, demonstrates safety’s limited influence in the
development of digital health technologies. It has been shown that the foundations of
safety engineering concepts and methods can improve quality and safety [1]. The impact
of digital health interventions (DHIs) on the safety of patients, and potential harm
exercised by the unsafe actions of clinical users, is not documented openly. Evidence
suggests a lack of rigor within the industry, where strategies for innovation to improve
clinical outcomes and advance health using new technologies, overlook the principles of
patient safety [2,3]. As these strategies often, see rigor as a barrier not an enabler to
innovation. In contrast to the digital healthcare industry, traditional safety critical
engineering industries have the capability of in-depth analysis and assessment, while
they have been established over decades. Additionally, these, more open, safety cultures
bring together concepts of quality, benefits and safety objectives into a more rigorous,
systematic environment and innovation ready.
The World Health Organization (WHO) classification of DHIs [4], and their
relationship to Health System Challenges (HSC),provides us with an opportunity to
establish or affirm safety claims by the application of safety analysis methods. The HSC
is a health service problem (e.g. lack of access to information or data, poor patient
experience) and DHI is the class of technology intervention that aims to address the
problem. This can offer insight and affirm confidence that the DHIs are safe and fit for
1 Corresponding Author, S. Harrison, IDH, WMG, University of Warwick, CV4 7AL, UK;
E-mail: stuart.harrison@warwick.ac.uk.
The Importance of Health Informatics in Public Health during a Pandemic
J. Mantas et al. (Eds.)
© 2020 The authors and IOS Press.
This article is published online with Open Access by IOS Press and distributed under the terms
of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0).
doi:10.3233/SHTI200523
179
purpose, by applying safety methods. The WHO classification promotes an accessible
and bridging language between technical and clinical experts, aimed at simplifying
dialogue and aiding digital health implementation. The classification represents discrete
functionality of DHI, in order to achieve health sector objectives and meet the health
system challenge (HSC), aimed at commissioners of digital services. We can apply these
same classifications and challenges to identify hazards and construct a safety claim and
justification. The objective of this paper is to implement the framework for the synthesis
of safety justification for digitally enabled healthcare services. The ultimate aim is to
apply the framework to a DHI and generate an assurance case, thereby provide a
justification of safety and elicit confidence that the DHI is fit for purpose, not just to meet
the health system challenge. This, in turn, will bridge the understanding of the health
delivery organization and manufacturer clinical risk management processes, by way of
guidance for each DHI classification. This will guide and influence the right behaviors
of innovation within the boundary of good practice and safety methods.
2. Method
The aim is to identify hazards and construct a safety claim and justification. A safety
claim follows an approach to safety justification that is commonly used in safety
engineering industry. It is also used in traditional medical device safety assurance claims
and, through graphical notation, provides a more efficient way of demonstrating safety
between differing experts (technical and clinical). The method is explained below in
Figure 1, and can be completed retrospectively or, ideally, in line with the requirements
and definition phase of the planned DHI.
Figure 1 Method & sequence of DHI analysis
Select the DHI is a straightforward exercise, as the interventions are well defined and
utilize established taxonomies, from mobile and more traditional digital health solutions.
Assess credible failures is completed by examination of the use of the DHI and the
deviation from that use. The inclusion of health system challenges provides synergy
between intended operational use and the challenge faced in the health system. It is
purposeful for examining the relationship between HSC, Hazard, Effect and Contribution,
which is important when safety claims are made. Examine safety significance &
Identify safety controls is where the clinical risk management methods are used –
hazard analysis. An examination of DHI hazards is completed using likelihood and
consequence to derive a severity level. Safety controls are identified to enable mitigation
Clients
(Patients/Public/Caregivers)
Healthcare Providers
Health System or Resource
Managers
Data Services
Digital Health
Intervention
Information
Availability
Quality
Acceptability
Utilization
Efficiency
Cost
Accountability
Health System
Challenges 1. Establish scope of analysis
2. Explore Credible Faliures
3. Examine safety
significance
4. Identify safety controls
5. Implement safety controls
6. Verify safety controls
Safety Justification
Framework
Safety Argument
S. Harrison et al. / A Method of Justifying Confidence in the Safety of DHIs180
of hazards, and this is where HSC form an important link into the framework.
Implementation and verification, the final stage, evidences controls implemented and
aligned to hazards and shape the final safety justification.
3. Results
We have applied the framework to a DHI, a self-management mobile app & web based
portal for children / young people with Type 1 diabetes. The hazard assessment shows at
least two health system challenges forming part of the causes to safety hazards. The
framework has been applied to the DHI category of Client, Targeted Client
Communication and Transmit Targeted Health Information to Client(s) Based on Health
Status of Demographics. A Hazard Identification (HAZID) was undertaken to identify
hazards that could cause harm to a “User” Patient. The assessment includes health system
challenges in bold type as contributory causes (table 1).
Table 1 Hazard Analysis of an example Digital Health Intervention (mobile app for Diabetes Type 1)
HazID
Hazard
Clinical Safety Impact
Cause
Control
1
Mobile App
and/or linked
clinical website
unavailable
Inability to support
clinical services, stress
or anxiety to service
users, delayed action of
treatment plans.
Unsupported mobile device
configuration, Key
information is not available,
security issue, technical /
configuration error. Lack of
out of hours or system
outage messages.
Poor Patient Experience.
Lack of access to
information or data.
Care planning
and intervention
includes outage
continuity plans.
Alternative
services
information
available through
other sources.
Technical
assurance
coverage includes
mobile variants,
webpage content
and OS.
2
Clinical
information
presented is
incorrect
and/or
misinterpreted
Reliance on information
leads to inappropriate
action of treatment plan
or advice to manage
condition.
Lack of quality/reliable
data Insufficient
utilization of data and
information.
UX issues
with information presented.
Out of date clinical
guideline
s. Lack of or
inappropriate referrals.
Clinical care
plans and
workflows are
controlled by
policy and
governance.
Technical
assurance
includes UX &
accessibility.
Content change
processes and
training is
implemented
regularly.
3
Users rely on
digital health
intervention
solely for care
and advice and
exclude care
giver/clinical
support
Service users/patients,
care givers and health
care practitioners lose
confidence in the DHI.
Reduced benefit of
using the DHI. Patient
condition may be
uncontro
lled and
adversely impacted.
Lack of alignment with
local norms. Poor
adherence to guidelines.
Inadequate supportive
supervision.
Lack of
underst
anding of the
service by users.
Low technical awareness
within the cohort.
Demo version of
the DHI is
available
for
training. Human
factors / codesign
workshop
as
part
of the content and
workflow
management.
S. Harrison et al. / A Method of Justifying Confidence in the Safety of DHIs 181
Performance,
outcomes and
benefits
indicators.
4
Inappropriate /
incorrect
implementation
of DHI into
health system
Poor and/or
declining
quality of
clinical
information or
data
Potential delay in the
ongoing
care of a
patient, transfer or
communication of
critical information to
support the treatment of
the patient.
User(s) adopt the
application informally and
evolve its use into
“informal” clinical c
are
pathways.
Insufficient
health worker
competence. Low health
worker motivation. Poor
adherence to guidelines.
Inadequate workflow
management. Poor
planning and
coordination.
High risk patients
prioritized by
local health
worker
s.
Use of
recommended
gove
rnance,
planning and
clinical
engagement
agreements. Use
of feedback
mechanisms to
monitor
performance and
accountability.
4. Discussion and Conclusions
The exponential growth, diversity of DHIs and associated regulatory position are the
biggest challenges to the industry. Policy makers, manufacturers, health organizations
and digital technology users (healthcare professionals and patients) have different
understandings and objectives of DHIs. The benefit for the communication between
stakeholders, for safety claims aligned with the classifications of DHIs. The presented
framework and associated justifications contribute to application guidance and best
practice. The DHI classification scheme has been used to generate guidance on
effectiveness for DHIs [5]. The results of this work indicate that the domains of security,
safety and effectiveness can be correlated. The use of taxonomies, synonyms and
ontologies, with established graphical notation methods, allow us to automate, predefine
and guide through case studies. Further work is needed, in order to demonstrate this
method and build the guidance across the DHI classification scheme. The
implementation and verification of DHIs, justified this way, will provide a direct
correlation to the health system challenge.
References
[1] Habli I, White S, Sujan M, Harrison S, Ugarte M, What is the safety case for health IT? A study of
assurance practices in England. Saf Sci. (2018),110.
[2] Sujan M, Scott P, Cresswell K, Digital health and patient safety: Technology is not a magic wand, Health
Informatics J. 2019.
[3] Iakovleva T, Oftedal E, Bessant J, Responsible Innovation in Digital Health. Responsible Innov Digit
Heal. (2019), 1–8.
[4] WHO, Classification of Digital Health Interventions v 1.0 [Internet]. [cited 2019 Jun 4]. Available from:
http://who.int/reproductivehealth/topics/mhealth/en/.
[5] NICE. Evidence standards framework for digital health technologies. Nice [Internet]. 2019;(March):1–
35. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/our-
programmes/evidence-standards-framework/functional-classification-case-
studies.pdf%0Ahttps://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/evidence-
standards-framework/digital-evid
S. Harrison et al. / A Method of Justifying Confidence in the Safety of DHIs182
... (Fatehi et al., 2015;Host et al., 2018;Lucas et al., 2016;Sevean et al., 2009), yet the United Kingdom (UK) remained reluctant and slow on the implementation of either VC or TC. This reluctance appeared to be due to several reasons, including competence and motivation in the use of digital technology (Harrison et al., 2020), and the clinical suitability of different patient groups to this form of consultation (Greenhalgh et al. (2015). Imlach et al. study (Imlach et al., 2020) suggested that some patients are unsuitable to move to remote (video or telephone) consultation, with many clinicians unconvinced of the NHS′s ability in the UK to make this happen. ...
Article
Full-text available
Aims: To identify, evaluate and summarize evidence of patient and clinician experiences of being involved in video or telephone consultations as a replacement for in-person consultations. Design: Narrative synthesis. Data sources: Medline; EMBASE; EMCARE; CINAHL and BNI. Searching took place from January 2021 to April 2021. Papers included were published between 2013 and 2020. Review methods: Papers were appraised by two independent reviewers for methodological quality. Data extraction was conducted according to the standardized tool from Joanna Briggs Institute. Results: Seven qualitative studies were included, from five countries and from the perspective of patients, relatives, administrators, nurses, physiotherapists and physicians. We developed two main themes: Pragmatic Concerns and Therapeutic Concerns. Each theme contained two categories: Pragmatic Concerns: (a) the convenience of non-face to face consultations; (b) using technology and equipment in a consultation; Therapeutic Concerns (c) building therapeutic relationships; and (d) embracing benefits and addressing challenges. Conclusion: This narrative synthesis presents the existing evidence on clinician and patient experience of participating in non-face to face consultations. Experiences are varied but largely focus on communication and forming relationships, using the technology successfully and the ability for patients to self-manage with support from clinicians who are not in-person. More high-quality studies are required to explore the experiences of patients and clinicians accessing remote consultations as a result of global implementation post-SARS-CoV-2 pandemic to identify any learning and education opportunities. Impact: Health care staff can provide high-quality care through video or telephone appointments as well as face to face appointments. This review has, however, identified that the evidence is limited and weak in this area and recommends there is research further to inform practice and influence future care.
Article
Full-text available
The use of novel health information technology provides avenues for potentially significant patient benefit. However, it is also timely to take a step back and to consider whether the use of these technologies is safe-or more precisely what the current evidence for their safety is, and what kinds of evidence we should be looking for in order to create a convincing argument for patient safety. This special issue on patient safety includes eight papers that demonstrate an increasing focus on qualitative approaches and a growing recognition that the sociotechnical lens of examining health information technology-associated change is important. We encourage a balanced approach to technology adoption that embraces innovation, but nonetheless insists upon suitable concerns for safety and evaluation of outcomes.
Article
Objective Health IT (HIT) systems are increasingly becoming a core infrastructural technology in healthcare. However, failures of these systems, under certain conditions, can lead to patient harm and as such the safety case for HIT has to be explicitly made. This study focuses on safety assurance practices of HIT in England and investigates how clinicians and engineers currently analyse, control and justify HIT safety risks. Methods Three workshops were organised, involving 34 clinical and engineering stakeholders, and centred on predefined risk-based questions. This was followed by a detailed review of the Clinical Safety Case Reports for 20 different national and local systems. The data generated was analysed thematically, considering the clinical, engineering and organisational factors, and was used to examine the often implicit safety argument for HIT. Results Two areas of strength were identified: establishment of a systematic approach to risk management and close engagement by clinicians; and two areas for improvement: greater depth and clarity in hazard analysis practices and greater organisational support for assuring safety. Overall, the dynamic characteristics of healthcare combined with insufficient funding have made it challenging to generate and explain the safety evidence to the required level of detail and rigour. Conclusion Improvements in the form of practical HIT-specific safety guidelines and tools are needed. The lack of publicly available examples of credible HIT safety cases is a major deficit. The availability of these examples can help clarify the significance of the HIT risk analysis evidence and identify the necessary expertise and organisational commitments.
Responsible Innovation in Digital Health
  • T Iakovleva
  • E Oftedal
  • J Bessant
Iakovleva T, Oftedal E, Bessant J, Responsible Innovation in Digital Health. Responsible Innov Digit Heal. (2019), 1-8.