A Method of Justifying Confidence in the
Safety of Digital Health Interventions
Stuart HARRISON a,1 George DESPOTOU a and Theodoros N. ARVANITIS a
a Institute of Digital Healthcare, WMG, University of Warwick, UK
Abstract. Digital health interventions (DHIs) enable improvements in health
strategy and address health system challenges. The World Health Organization
provides a formal classification for DHIs. However, safety claims, about such
interventions, vary in quality and are often vague as to how they are communicated
between technical, clinical experts and stakeholders. By combining the
classifications with a method of safety analysis and justification, we postulate
confidence in the safety of digital technology. Confidence is resulting from the
application of the framework to the DHI, using defined health system challenges.
The framework and derived safety justifications can be applied to any DHI. It can
serve as guideline for health strategy, regulatory and standards based compliance.
Keywords. Digital health, safety, justification, health system, confidence, hazard
The lack of the adoption of the fundamental concepts of clinical risk management and
safety methods, within health informatics, demonstrates safety’s limited influence in the
development of digital health technologies. It has been shown that the foundations of
safety engineering concepts and methods can improve quality and safety . The impact
of digital health interventions (DHIs) on the safety of patients, and potential harm
exercised by the unsafe actions of clinical users, is not documented openly. Evidence
suggests a lack of rigor within the industry, where strategies for innovation to improve
clinical outcomes and advance health using new technologies, overlook the principles of
patient safety [2,3]. As these strategies often, see rigor as a barrier not an enabler to
innovation. In contrast to the digital healthcare industry, traditional safety critical
engineering industries have the capability of in-depth analysis and assessment, while
they have been established over decades. Additionally, these, more open, safety cultures
bring together concepts of quality, benefits and safety objectives into a more rigorous,
systematic environment and innovation ready.
The World Health Organization (WHO) classification of DHIs , and their
relationship to Health System Challenges (HSC),provides us with an opportunity to
establish or affirm safety claims by the application of safety analysis methods. The HSC
is a health service problem (e.g. lack of access to information or data, poor patient
experience) and DHI is the class of technology intervention that aims to address the
problem. This can offer insight and affirm confidence that the DHIs are safe and fit for
1 Corresponding Author, S. Harrison, IDH, WMG, University of Warwick, CV4 7AL, UK;
The Importance of Health Informatics in Public Health during a Pandemic
J. Mantas et al. (Eds.)
© 2020 The authors and IOS Press.
This article is published online with Open Access by IOS Press and distributed under the terms
of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0).
purpose, by applying safety methods. The WHO classification promotes an accessible
and bridging language between technical and clinical experts, aimed at simplifying
dialogue and aiding digital health implementation. The classification represents discrete
functionality of DHI, in order to achieve health sector objectives and meet the health
system challenge (HSC), aimed at commissioners of digital services. We can apply these
same classifications and challenges to identify hazards and construct a safety claim and
justification. The objective of this paper is to implement the framework for the synthesis
of safety justification for digitally enabled healthcare services. The ultimate aim is to
apply the framework to a DHI and generate an assurance case, thereby provide a
justification of safety and elicit confidence that the DHI is fit for purpose, not just to meet
the health system challenge. This, in turn, will bridge the understanding of the health
delivery organization and manufacturer clinical risk management processes, by way of
guidance for each DHI classification. This will guide and influence the right behaviors
of innovation within the boundary of good practice and safety methods.
The aim is to identify hazards and construct a safety claim and justification. A safety
claim follows an approach to safety justification that is commonly used in safety
engineering industry. It is also used in traditional medical device safety assurance claims
and, through graphical notation, provides a more efficient way of demonstrating safety
between differing experts (technical and clinical). The method is explained below in
Figure 1, and can be completed retrospectively or, ideally, in line with the requirements
and definition phase of the planned DHI.
Figure 1 Method & sequence of DHI analysis
Select the DHI is a straightforward exercise, as the interventions are well defined and
utilize established taxonomies, from mobile and more traditional digital health solutions.
Assess credible failures is completed by examination of the use of the DHI and the
deviation from that use. The inclusion of health system challenges provides synergy
between intended operational use and the challenge faced in the health system. It is
purposeful for examining the relationship between HSC, Hazard, Effect and Contribution,
which is important when safety claims are made. Examine safety significance &
Identify safety controls is where the clinical risk management methods are used –
hazard analysis. An examination of DHI hazards is completed using likelihood and
consequence to derive a severity level. Safety controls are identified to enable mitigation
Health System or Resource
Challenges • 1. Establish scope of analysis
2. Explore Credible Faliures
3. Examine safety
4. Identify safety controls
5. Implement safety controls
6. Verify safety controls
S. Harrison et al. / A Method of Justifying Conﬁdence in the Safety of DHIs180
of hazards, and this is where HSC form an important link into the framework.
Implementation and verification, the final stage, evidences controls implemented and
aligned to hazards and shape the final safety justification.
We have applied the framework to a DHI, a self-management mobile app & web based
portal for children / young people with Type 1 diabetes. The hazard assessment shows at
least two health system challenges forming part of the causes to safety hazards. The
framework has been applied to the DHI category of Client, Targeted Client
Communication and Transmit Targeted Health Information to Client(s) Based on Health
Status of Demographics. A Hazard Identification (HAZID) was undertaken to identify
hazards that could cause harm to a “User” Patient. The assessment includes health system
challenges in bold type as contributory causes (table 1).
Table 1 Hazard Analysis of an example Digital Health Intervention (mobile app for Diabetes Type 1)
Clinical Safety Impact
Inability to support
clinical services, stress
or anxiety to service
users, delayed action of
Unsupported mobile device
information is not available,
security issue, technical /
configuration error. Lack of
out of hours or system
Poor Patient Experience.
Lack of access to
information or data.
Reliance on information
leads to inappropriate
action of treatment plan
or advice to manage
Lack of quality/reliable
utilization of data and
with information presented.
Out of date clinical
s. Lack of or
includes UX &
Users rely on
solely for care
and advice and
care givers and health
care practitioners lose
confidence in the DHI.
Reduced benefit of
using the DHI. Patient
condition may be
Lack of alignment with
local norms. Poor
adherence to guidelines.
anding of the
service by users.
Low technical awareness
within the cohort.
Demo version of
the DHI is
factors / codesign
of the content and
S. Harrison et al. / A Method of Justifying Conﬁdence in the Safety of DHIs 181
of DHI into
Potential delay in the
care of a
patient, transfer or
critical information to
support the treatment of
User(s) adopt the
application informally and
evolve its use into
“informal” clinical c
competence. Low health
worker motivation. Poor
adherence to guidelines.
High risk patients
4. Discussion and Conclusions
The exponential growth, diversity of DHIs and associated regulatory position are the
biggest challenges to the industry. Policy makers, manufacturers, health organizations
and digital technology users (healthcare professionals and patients) have different
understandings and objectives of DHIs. The benefit for the communication between
stakeholders, for safety claims aligned with the classifications of DHIs. The presented
framework and associated justifications contribute to application guidance and best
practice. The DHI classification scheme has been used to generate guidance on
effectiveness for DHIs . The results of this work indicate that the domains of security,
safety and effectiveness can be correlated. The use of taxonomies, synonyms and
ontologies, with established graphical notation methods, allow us to automate, predefine
and guide through case studies. Further work is needed, in order to demonstrate this
method and build the guidance across the DHI classification scheme. The
implementation and verification of DHIs, justified this way, will provide a direct
correlation to the health system challenge.
 Habli I, White S, Sujan M, Harrison S, Ugarte M, What is the safety case for health IT? A study of
assurance practices in England. Saf Sci. (2018),110.
 Sujan M, Scott P, Cresswell K, Digital health and patient safety: Technology is not a magic wand, Health
Informatics J. 2019.
 Iakovleva T, Oftedal E, Bessant J, Responsible Innovation in Digital Health. Responsible Innov Digit
Heal. (2019), 1–8.
 WHO, Classification of Digital Health Interventions v 1.0 [Internet]. [cited 2019 Jun 4]. Available from:
 NICE. Evidence standards framework for digital health technologies. Nice [Internet]. 2019;(March):1–
35. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/our-
S. Harrison et al. / A Method of Justifying Conﬁdence in the Safety of DHIs182