PreprintPDF Available
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

Healthcare systems across the globe are struggling with increasing costs and worsening outcomes. This presents those responsible for overseeing healthcare with a challenge. Increasingly, policymakers, politicians, clinical entrepreneurs and computer and data scientists argue that a key part of the solution will be 'Artificial Intelligence' (AI)-particularly Machine Learning (ML). This argument stems not from the belief that all healthcare needs will soon be taken care of by "robot doctors." Instead, it is an argument that rests on the classic counterfactual definition of AI as an umbrella term for a range of techniques that can be used to make machines complete tasks in a way that would be considered intelligent were they to be completed by a human. Automation of this nature could offer great opportunities for the improvement of healthcare services and ultimately patients' health by significantly improving human clinical capabilities in diagnosis, drug discovery, epidemiology, personalised medicine, and operational efficiency. However, if these AI solutions are to be embedded in clinical practice, then at least three issues need to be considered: the technical possibilities and limitations; the ethical, regulatory and legal framework; and the governance framework. In this article, we report on the results of a systematic analysis designed to provide a clear overview of the second of these elements: the ethical, regulatory and legal framework. We find that ethical issues arise at six levels of abstraction (individual, interpersonal, group, institutional, sectoral, and societal) and can be categorised as epistemic, normative, or overarching. We conclude by stressing how important it is that the ethical challenges raised by implementing AI in healthcare settings are tackled proactively rather than reactively and map the key considerations for policymakers to each of the ethical concerns highlighted.
Content may be subject to copyright.
1
The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review
Jessica Morley1* **, Caio C. V. Machado1*, Christopher Burr1, Josh Cowls1,2, Indra Joshi4, Mariarosaria
Taddeo1,2,3, Luciano Floridi1,2,3
1 Oxford Internet Institute, University of Oxford, 1 St Giles’, Oxford, OX1 3JS, UK
2 Alan Turing Institute, British Library, 96 Euston Rd, London NW1 2DB, UK
3 Department of Computer Science, University of Oxford, 15 Parks Rd, Oxford, OX1 3QD, UK
* These authors contributed equally to the writing of this paper
** Email of correspondence author: Jessica.morley@kellogg.ox.ac.uk
Abstract
Healthcare systems across the globe are struggling with increasing costs and worsening outcomes. This
presents those responsible for overseeing healthcare with a challenge. Increasingly, policymakers,
politicians, clinical entrepreneurs and computer and data scientists argue that a key part of the solution
will be ‘Artificial Intelligence’ (AI) particularly Machine Learning (ML). This argument stems not
from the belief that all healthcare needs will soon be taken care of by “robot doctors.” Instead, it is an
argument that rests on the classic counterfactual definition of AI as an umbrella term for a range of
techniques that can be used to make machines complete tasks in a way that would be considered
intelligent were they to be completed by a human. Automation of this nature could offer great
opportunities for the improvement of healthcare services and ultimately patients’ health by
significantly improving human clinical capabilities in diagnosis, drug discovery, epidemiology,
personalised medicine, and operational efficiency. However, if these AI solutions are to be embedded
in clinical practice, then at least three issues need to be considered: the technical possibilities and
limitations; the ethical, regulatory and legal framework; and the governance framework. In this article,
we report on the results of a systematic analysis designed to provide a clear overview of the second of
these elements: the ethical, regulatory and legal framework. We find that ethical issues arise at six levels
of abstraction (individual, interpersonal, group, institutional, sectoral, and societal) and can be
categorised as epistemic, normative, or overarching. We conclude by stressing how important it is that
the ethical challenges raised by implementing AI in healthcare settings are tackled proactively rather than
reactively and map the key considerations for policymakers to each of the ethical concerns highlighted.
Keywords
Artificial Intelligence; Ethics; Healthcare; Health Policies; Machine Learning.
2
1. Introduction
Healthcare systems across the globe are struggling with increasing costs and worsening outcomes
(Topol, 2019). This presents those responsible for overseeing healthcare systems with a wicked
problem, meaning that the problem has multiple causes, is hard to understand and define, and hence
will have to be tackled from multiple different angles. Against this background, policymakers,
politicians, clinical entrepreneurs and computer and data scientists increasingly argue that a key part of
the solution will be ‘Artificial Intelligence’ (AI), particularly Machine Learning (Chin-Yee & Upshur,
2019). The argument stems not from the belief that all healthcare needs will soon be taken care of by
robot doctors (Chin-Yee & Upshur, 2019). Instead, the argument rests on the classic counterfactual
definition of AI as an umbrella term for a range of techniques (summarised in Figure 1 below) that
can be used to make machines complete tasks in a way that would be considered intelligent were they
to be completed by a human. For example
1
, as mapped by (Harerimana, Jang, Kim, & Park, 2018),
decision tree techniques can be used to diagnose breast cancer tumours (Kuo, Chang, Chen, & Lee,
2001); Support Vector Machine techniques can be used to classify genes (Brown et al., 2000) and
diagnose Diabetes Mellitus (Barakat, Bradley, & Barakat, 2010); ensemble learning methods can predict
outcomes for cancer patients (Kourou, Exarchos, Exarchos, Karamouzis, & Fotiadis, 2015); and
neural networks can be used to recognise human movement (Jiang & Yin, 2015). From this
perspective, AI represents a growing resource of interactive, autonomous, and often self-learning (in the
machine learning sense, see Figure 1) agency, that can be used on demand (Floridi, 2019), presenting
the opportunity for potentially transformative cooperation between machines and doctors (Bartoletti,
2019).
1
For a full overview of all supervised and unsupervised Machine Learning techniques and their applications in
healthcare, see Harerimana, Jang, Kim, & Park, 2018 and for a detailed look at the number of papers related to
AI techniques and their clinical applications see (Tran et al., 2019)
3
Figure 1. AI Knowledge Map (AIKM). Source: Corea (2019), reproduced with permission courtesy
of F. Corea.
If harnessed effectively, such AI-clinician coordination (AI-Health) could offer great opportunities for
the improvement of healthcare services and ultimately patients’ health (Taddeo & Floridi, 2018) by
significantly improving human clinical capabilities in diagnosis (Arieno, Chan, & Destounis, 2019; De
Fauw et al., 2018; Kunapuli et al., 2018), drug discovery (Álvarez-Machancoses & Fernández-Martínez,
2019; Fleming, 2018), epidemiology (Hay, George, Moyes, & Brownstein, 2013), personalised medicine
(Barton et al., 2019; Cowie, Calveley, Bowers, & Bowers, 2018; Dudley, Listgarten, Stegle, Brenner, &
Parts, 2015) or operational efficiency (H. Lu & Wang, 2019; Nelson, Herron, Rees, & Nachev, 2019).
However, as Ngiam & Khor (2019) stress, if these AI solutions are to be embedded in clinical practice,
then at least three issues need to be considered: the technical possibilities and limitations; the ethical,
regulatory and legal framework; and the governance framework.
The task of the following pages is to focus on the second of these elements the ethical,
regulatory and legal framework by stressing how important it is that the ethical challenges raised by
4
implementing AI in healthcare settings are tackled proactively (Char, Shah, & Magnus, 2018). If they are
not, there is a risk of incurring significant opportunity costs (Cookson, 2018) due to what Floridi terms
a ‘double bottleneck’ whereby a double bottleneck: ethical mistakes or misunderstandings may lead
to social rejection and/or distorted legislation and policies, which in turn may cripple the acceptance
and advancement of [the necessary] data science.
2
Although essential, encouraging this kind of
proactive ethical analysis is challenging because although bioethical principles for clinical research
and healthcare are well established, and issues related to privacy, effectiveness, accessibility and utility
are clear (Nebeker, Torous, & Bartlett Ellis, 2019) other issues are less obvious (Char et al., 2018).
For example, AI processes may lack transparency, making accountability problematic, or may be
biased, and leading to unfair behaviour or mistaken decisions (Mittelstadt, Allo, Taddeo, Wachter, &
Floridi, 2016). Identification of these less obvious concerns requires input from the medical sciences,
economics, computer sciences, social sciences, law, and policy-making. Yet, research in these areas is
currently happening in siloes, is overly focused on individual level impacts (Morley & Floridi, 2019b),
or does not consider the fact that the ethical concerns may vary depending on the stage of the
algorithm development pipeline (Morley, Floridi, Kinsey, & Elhalal, 2019). Taken together, these issues
are inhibiting the development of a coherent ethical framework.
Whilst AI-Health remains in the early stages of development and relatively far away from
having a major impact on frontline clinical care (Panch, Mattie, & Celi, 2019) there is still time to
develop this framework. However, this window of opportunity is closing fast, as the pace at which AI-
Health solutions are gaining approval for use in clinical care in the US is accelerating (Topol, 2019).
Both the Chinese (Zhang, Wang, Li, Zhao, & Zhan, 2018) and British governments (Department of
Health and Social Care, 2019) have made it very clear that they intend on investing heavily in the spread
and adoption of AI-Health technologies. It is for these reasons that the goal of this article is to offer
a cross-disciplinary systematic review mapping the potential ethical implications of the development
of AI-Health in order to support the development of better design practices, and transparent and
accountable deployment strategies. We will do this in terms of digital ethics. That is, we will focus on
the evaluation of moral problems related to data, algorithms and corresponding practices (Floridi &
Taddeo, 2016), with the hope of enabling governments and healthcare systems looking to adopt AI-
Health to be ethically mindful (Floridi, 2019a).
2
https://digitalethicslab.oii.ox.ac.uk/the-ethics-of-medical-data-analytics-opportunities-and-challenges/
5
2. Methodology in Brief
A detailed outline of the methodology used to conduct the review can be found in the appendix. For
now, suffice to say that a traditional thematic review methodology (following Abdul, Vermeulen,
Wang, Lim, & Kankanhalli, 2018) was used to find literature from across disciplinary boundaries that
highlighted ethical issues unique to the use of AI algorithms in healthcare. This means that the review
did not focus on issues such as lack of evidence, privacy and security (Vayena, Tobias, Afua, &
Allesandro, 2018), or definitions and secondary uses of healthcare data, as these are ethical issues for
digital health at large and not unique to AI. More detailed discussion of these issues is highlighted in
Table 1.
General Digital Health Ethical Concern
Example References
Data Sharing/Data Privacy
(Kalkman, Mostert, Gerlinger, Van Delden, & Van Thiel,
2019)(Olimid, Rogozea, & Olimid, 2018) (Parker, Halter,
Karliychuk, & Grundy, 2019)(Quinn, 2017) (Richardson, Milam,
& Chrysler, 2015) (Townend, 2018)
Secondary use of Healthcare Data
(Lee, 2017) (Nittas, Mütsch, Ehrler, & Puhan, 2018) (O’Doherty
et al., 2016)
Surveillance, Nudging and Paternalism
(Maher et al., 2019) (Marill, 2019)(Nebeker et al., 2017) (Morley &
Floridi, 2019d)(Burr, Mariarosaria, & Floridi, 2019)
Consent
(S. Millett & O’Leary, 2015) (T. Ploug & Holm, 2016) (Mann,
Savulescu, & Sahakian, 2016) (Balthazar, Harri, Prater, & Safdar,
2018)
Definition of Health Data
(Floridi et al., 2018) (Voigt, 2019) (Holzinger, Haibe-Kains, &
Jurisica, 2019) (Kleinpeter, 2017)
Ownership of Health Data
(Chiauzzi & Wicks, 2019) (Krutzinna, Taddeo, & Floridi, 2018)
(Shaw, Gross, & Erren, 2016) (Sterckx, Rakic, Cockbain, &
Borry, 2016) (Stephan Millett & O’Leary, 2015)
Embodied Intelligence/Robotics
(Fiske, Henningsen, & Buyx, 2019)
Digital Divide/eHealth Literacy
(Celi et al., 2016) (Kuek & Hakkennes, 2019)
Patient involvement
(Aitken et al., 2019) (Page, Manhas, & Muruve, 2016)
Patient Safety
(Barras, 2019)
Evidence of Efficacy
(Ferretti, Ronchi, & Vayena, 2019) (Henson, David, Albright, &
Torous, 2019) (Larsen et al., 2019)
Table 1: Example literature related to ethical concerns that are relevant for all digital health intervention, not unique to AI-Health and
therefore not included in this review
To ensure that the focus stayed on the unique ethical issues, the map, developed by (Mittelstadt et al.,
2016), of the epistemic, normative, and overarching ethical concerns related to algorithms was used as
a base. First, the selected literature was reviewed to identify healthcare examples of each of the
6
concerns highlighted in the original map, as shown in Table 2, and then reviewed more thoroughly to
identify how the ethical issues may vary depending on whether the analysis was being conducted at: (i)
individual, (ii) interpersonal, (iii) group (e.g. family or population), (iv) institutional, (v) sectoral, and/or
(v) societal levels of abstraction (LoA)
3
(Floridi, 2008). This helped the review avoid the narrow focus
on individual-level impacts highlighted in the introduction. This approach is not intended to imply that
there is no overlap between the levels.
Ethical Concern
Explanation
Medical Example
Epistemic concerns
Inconclusive Evidence
Algorithmic outcomes (e.g.
classification) are
probabilistic and not
infallible. They are rarely
sufficient to posit the
existence of a causal
relationship.
EKG readers in smartwatches may ‘diagnose’ a patient
as suffering from arrhythmia when it may be due to a
fault with the watch not being able to accurately read that
user’s heartbeat (for example due to the colour of their
skin) or the ‘norm’ is inappropriately calibrated for that
individual (Hailu, 2019)
Inscrutable Evidence
Recipients of an algorithmic
decision very rarely have full
oversight of the data used
to train or test an algorithm
or the data points used to
reach a specific decision.
A clinical decision support system deployed in a hospital
may make a treatment recommendation, but it may not
be clear on what basis it has made that ‘decision’ raising
the risk that it has used data that are inappropriate for
the individual in question or that there is a bug in the
system leading to issues with over or under prescribing
(Wachter, 2015).
Misguided Evidence
Algorithmic outcomes can
only be as reliable (but also
as neutral) as the data they
are based on.
Watson for Oncology is in widespread use in China for
‘diagnosis’ via image recognition but has primarily been
trained on a Western data set leading to issues with
concordance and poorer results for Chinese patients than
their Western counterparts (Liu et al., 2018).
Normative Concerns
Unfair outcomes
An action can be found to
having more of an impact
(positive or negative) on
one group of people
An algorithm ‘learns’ to prioritise patients it predicts to
have better outcomes for a particular disease. This turns
out to have a discriminatory effect on people within the
Black and minority ethnic communities (Garattini,
Raffle, Aisyah, Sartain, & Kozlakidis, 2019).
Transformative effects
Algorithmic activities, like
profiling, re-conceptualise
reality in unexpected ways.
An individual using personal health app has limited
oversight over what passive data it is collecting and how
that is being transformed into a recommendation to
improve, limiting their ability to challenge any
recommendations made and a loss of personal autonomy
and data privacy (Kleinpeter, 2017).
Overarching
Traceability
Harm caused by algorithmic
activity is hard to debug (to
detect the harm and find its
cause), and it is hard to
identify who should be held
responsible for the harm
caused.
If a decision made by clinical decision support software
leads to a negative outcome for the individual, it is unclear
who to assign the responsibility and or liability to and
therefore to prevent it from happening again (Racine,
Boehlen, & Sample, 2019)..
Table 2: A summary of the epistemic, normative and overarching ethical concerns related to algorithmic use in healthcare based on
Mittelstadt et al (2016) from (Jessica Morley & Floridi, In Press) .
3
A level of abstraction can be imagined as an interface that enables one to observe some aspects of a system
analysed, while making other aspects opaque or indeed invisible. For example, one may analyse a house at the
LoA of a buyer, of an architect, of a city planner, of a plumber, and so on. LoAs are common in computer
science, where systems are described at different LoAs (computational, hardware, user-centred etc.). Note that
LoAs can be combined in more complex sets, and can be, but are not necessarily hierarchical, with higher or
lower ‘resolution’ or granularity of information.
7
3. Thematic Analysis
What follows is a detailed discussion of the issues uncovered. A summary table (Table 3) is provided
at the end of the section.
3.1. Epistemic Concerns: Inconclusive, Inscrutable, and Misguided Evidence
Many factors are encouraging the development of AI-Health (Chin-Yee & Upshur, 2019). One of the
main driving forces is the belief that algorithms can make more objective, robust and evidence-based
clinical decisions (in terms of diagnosis, prognosis or treatment recommendations) than a human
health care provider (HCP) can (Kalmady et al., 2019). This is not an unfounded position. Machine
learning methods, especially ensemble and unsupervised methods (Harerimana et al., 2018), can take
into account a far greater range of evidence (data) than a Health Care Provider (HCP) when making a
clinical decision, including five of the seven dimensions of healthcare data provided by the US
Department of Health and Human services: (1) demographic and socioeconomic data; (2) symptom
and existing diagnosis data; (3) treatment data; (4) outcome data; and (5) other omic data (Holzinger
et al., 2019)
4
. If designed taking into account the multiple epistemic concerns, this ability enables
clinical algorithms to act as digital companions (Morley & Floridi, 2019d), reducing the information
asymmetry that exists between a HCP and the individual seeking care by making available information
accessible to both parties and helping ensure that the most informed decision possible is made by the
person who has the right to make it (Morley & Floridi, 2019a).
It is at least in part due to this ability to make evidence-based’ decisions that, as AI-health
research has shown, AI techniques can considerably augment or surpass human capabilities when it
comes to tasks including: (1) analysis of risk factors (De Langavant, Bayen, & Yaffe, 2018; Deng, Luo,
& Wang, 2018); (2) prediction of disease (Moscoso et al., 2019); (3) prediction of infection (Barton et
al., 2019)(López-Martínez, Núñez-Valdez, Lorduy Gomez, & García-Díaz, 2019); (4) population
health monitoring (F. S. Lu, Hattab, Clemente, Biggerstaff, & Santillana, 2019; Zacher & Czogiel,
2019); (5) prediction of adverse effects (Ding, Tang, & Guo, 2019; Mortazavi et al., 2017); (6)
prediction of outcome and/or likelihood of survival (Dong et al., 2019; Popkes et al., 2019; Topuz,
Zengul, Dag, Almehmi, & Yildirim, 2018); and (7) analysing electronic health records (Shickel, Tighe,
Bihorac, & Rashidi, 2018). These capabilities should not be underestimated, particularly as AI-Health
solutions can operate at scale, diagnosing or predicting outcomes for multiple people at once
4
The other two categories refer to data from the Healthcare system, such as expenditure and healtchcare
resources data.
8
something that an HCP could never do. Yet in many ways this almost unwavering faith in the truth-
telling power of AI-Health is flawed.
As has been highlighted multiple times in the wider ethical AI literature, the belief that
algorithms are more objective than humans is a ‘carefully crafted myth’ (Gillespie, Boczkowski, &
Foot, 2014), and just because an algorithm can recognise a pattern (for example) does not necessarily
make it meaningful (Floridi, 2014). In the context of healthcare, existing methods and studies
(potentially including those referenced) suffer from overfitting due to small numbers of samples,
meaning that the majority of results (e.g. patterns of disease risk factors, or presence of disease) are
inconclusive (Holzinger et al., 2019). This is a problem that is further magnified by the lack of
reproducibility, and external validity, of results. AI-Health solutions are often untranslatable between
different settings and rarely work in settings different to those in which the initial result was obtained
(Vollmer, Mateen, Bohner, Király, Ghani, et al., 2018), raising serious questions about the scientific
rigor of AI-Health and its safety (Vayena, Blasimme, & Cohen, 2018). Furthermore, the results can
often be heavily value-laden, based on the definition of ‘healthy’ by influential people or powerful
companies (McLaughlin, 2016). This raises a number of significant ethical concerns.
At the individual LoA there is considerable risk of misdiagnosis. This can happen in at least
two ways: either, as highlighted in table 2, by an individual using a wearable device that has a bug, or
is inappropriately calibrated for them could be ‘told’ that they are suffering from a health condition
when they are not (or vice versa) or, an HCP relying on clinical decision support software (CDSS)
(Ruckenstein & Schüll, 2017) could be given an inaccurate diagnosis or recommendation which they
do not question due to a tendency to uncritically accept the decisions of automated systems (Challen
et al., 2019). Moreover, this can have impacts in medical practice, causing overreliance on the machine
diagnostics and deskilling of practitioners (Cabitza, Rasoini, & Gensini, 2017). Not only is this a risk
for individuals, but it also reverses the advantage of AI-Health solutions being able to operate at scale
by introducing the group LoA ethical concern of misdiagnosis or missed diagnosis happening
repeatedly. Whilst an HCP might give one person the wrong diagnosis and then be corrected, a faulty
algorithm, based on the misguided, inscrutable or inconclusive evidence could give the same wrong
diagnosis to hundreds or thousands of people at a time (Topol, 2019). The scale of the problems is as
large as the scale of the solutions.
Building on this, there are also ethical implications at the interpersonal LoA. HCP-patient
relationships are primarily based on trust and empathy, and whilst AI-Health solutions can take over
tasks that are more routine and standardised, they cannot reproduce the emotional virtues of which
human HCPs are capable (Ngiam & Khor, 2019). Consequently, an over-reliance on the ‘quantitative’
9
and objective evidence that fuels clinical algorithms (Cabitza et al., 2017) could discredit other forms
of diagnosis (Rosenfeld et al., 2019) a prominent concern in the case of clinical psychiatry
(Burns, 2015) and lead to the de-humanisation or impersonalisation of care provision (Juengst,
McGowan, Fishman, & Settersten, 2016), from a service based on listening and theory to one based
purely on categorisation (an issue that could again lead to a group LoA harm of group-profiling and
associated discrimination by providers including insurers, see section 3.2.). Not only is this effectively
‘paternalism in disguise’ (Juengst et al., 2016) but it could also lead to poorer health outcomes due to
the lack of disconnect between pure medical evidence and actual behaviour change (Emanuel &
Wachter, 2019).
Finally, scaling up to the institutional, sectoral and societal LoAs, there is the concern that
public health decisions are increasingly made on predictive AI-Health algorithms, which too often rely
on the same flawed assumptions as outlined above. Regarding these assumptions, consider what is by
now a classic example: the Google Flu Trends monitoring of the influenza virus. The initial algorithm
distorted the spread of the virus in the US (Vayena, Salathé, Madoff, & Brownstein, 2015). If policy
decisions about where to deploy health resources are based on such poor-quality evidence, this could
result in the waste of public funds (e.g., promoting vaccination campaigns where they are not needed),
damage local economies (e.g., scaring away tourists from a region) which would result in a positive
feedback loop of less money available for public expenditure and lead to poorer quality public
healthcare provision and thus worse health outcomes for society at large. This worry is particularly
paramount when it is considered that the ultimate ambition of AI-Health is to create a learning
healthcare system where the ‘system’ is constantly learning from the data it receives on the
performance of its interventions (Faden et al., 2013). Furthermore, it is worth noting that, at this
juncture, the example offered above of Flu Trends does not represent the limits of Google’s interest
and that of its subsidiaries and its siblings under parent company Alphabet in public health. As we
discuss below, the engagement between Alphabet’s artificial intelligence subsidiary DeepMind and a
major UK hospital has attracted the attention of data protection regulators, the press, and academics
(Information Commissioner, 2018; Powles & Hodson, 2017). The challenge of ensuring that AI-
Health systems function accurately has in turn sparked debates about the appropriateness of sharing
data between public and private entities. In response to claims that patient data transferred from the
Royal Free Hospital to DeepMind was far in excess of the requirements of those publicly stated
needs” (Powles & Hodson, 2017), DeepMind representatives argued that “data processed in the
application have been defined by and are currently being used by clinicians for the direct monitoring
10
and care of AKI [acute kidney injury] patients” (King et al., 2018). Powles and Hodson responded in
turn that it is a “statement of fact that the data transferred is broader than the requirements of AKI”
(Powles & Hodson, 2018). As this series of claims and counter-claims demonstrates, the quality and
quantity of data required for a particular AI-Health application is likely to be a matter of dispute in the
context of the collection and sharing of patient data in training AI-Health.
Ultimately, data is necessary for medical practice and thus so are AI-Health solutions that can
take in greater volumes of data. But data collected and used in this way is insufficient to inform medical
practice; it must be transformed to be useful (Car, Sheikh, Wicks, & Williams, 2019) and if this
transformation process is flawed the results could be hugely damaging, resulting in either wasted funds
and poorer health provision, or undue sharing of patient data with private sector actors under the guise
of AI-Health.
3.2. Normative Concerns: Unfair Outcomes and Transformative Effects
As referenced in the introduction, healthcare systems across the globe are struggling with increasing
costs and decreasing outcomes (Topol, 2019) and their administrators increasingly believe that the
answer may well lie in making healthcare systems more informationally mature and able to capitalise
on the opportunities presented by AI-Health significantly to improve outcomes for patients, and to
reduce the burdens on the system (Cath, Wachter, Mittelstadt, Taddeo, & Floridi, 2017). Whilst it
would be ethically remiss to ignore these opportunities (Floridi, 2019a), it would be equally ethically
problematic to ignore the fact that these opportunities are not created by AI-Health technologies per
se but by their ability to re-ontologise (that is, fundamentally transform the intrinsic nature of) the ways in
which health care is delivered by coupling, re-coupling and de-coupling different parts of the system
(Floridi, 2017a). For example (Morley & Floridi, 2019b):
Coupling: patients and their data are so strictly and interchangeably linked that the
patients are their genetic profiles, latest blood results, personal information, allergies
etc. (Floridi, 2017a). What the legislation calls ‘data subjects” become “data patients”;
Re-Coupling: research and practice have been sharply divided since the publication
of the National Commission for the Protection of Human Subjects in the 1970s, but
in the digital scenario described above, they are re-joined as one and the same again
(Petrini, 2015) (Faden et al., 2013);
De-Coupling: presence of Health Care Provider (HCP) and location of Patient
become independent, for example because of the introduction of online consultations
(NHS England, 2019).
11
As a result of these transformations a number of ethical concerns arise.
Starting once again with the individual LoA: as more diagnostic and therapeutic interventions
become based on AI-Health solutions, individuals may be encouraged to share more and more
personal data about themselves (Racine et al., 2019) data that can then be used in opaque ways
(Sterckx et al., 2016). This means that the ability for individuals to be meaningfully involved in shared
decision making is considerably undermined (Vayena et al., 2018) As a result, the increasing use of
algorithmic decision-making in clinical settings can have negative implications for individual
autonomy, as for an individual to be able to exert agency over the AI-Health derived clinical decision,
they would need to have a good understanding of the underlying data, processes and technical
possibilities that were involved in it being reached (DuFault & Schouten, 2018) and be able to ensure
their own values are taken into consideration (McDougall, 2019). The vast majority of the population
do not have the level of eHealth literacy necessary for this (Kim & Xie, 2017), and those that do
(including HCPs) are prevented from gaining this understanding due to the black-box nature of AI-
Health algorithms (Watson et al., 2019). In extreme instances, this could undermine an individual’s
confidence in their ability to refuse treatment (Thomas Ploug & Holm, 2019). Such issues pose a
substantial threat to an individual’s integrity of self (the ability of an individual to understand the forces
acting on them) (Cheney-Lippold, 2017). Given that damage to a person’s psychological integrity can
be perceived as a ‘harm’, not accounting for this potentiality poses the risk of creating a system that
violates the first principle of medical ethics: primum non nocere (first, do no harm) (Andorno, 2004;
Morley & Floridi, 2019d).
It is not necessarily the case that harmful impacts will primarily be felt by the patients. At the
interpersonal LoA, HCPs may themselves feel increasingly left ‘out of the loop’ as decisions are made
by patients and their ‘clinical advice’ algorithm in a closed digital loop (Nag, Pandey, Oh, & Jain, 2017).
As a result, HCPs may too feel unable to exert their own agency over the decision-making capacity of
AI-Health solutions. Though the use of algorithmic decision-making makes diagnostics seem like a
straightforward activity of identifying symptoms and fitting them into textbook categories, medical
practice is much less clear-cut than it seems (Cabitza et al., 2017). Clinical practice involves a series of
evaluations, trial and error, and a dynamic interaction with the patient and the medical literature. As a
result, formal treatment protocols should be seen more as evaluative guidelines than well-defined,
isolated categories. AI-Health solutions may not be in accordance with current best practice, which is
12
necessary to handle the great degree of uncertainty
5
and can only be fully evaluated by physicians
(Cabitza et al., 2017). Therefore, AI-Health solutions need to allow HCPs to exert influence in the
decision-making process.
At the group LoA the concern is that AI-Health systems may well be able to cope better with
illnesses and injuries that have well-established and fairly set (and therefore automatable) treatment
protocols. These are more likely to exist for afflictions most commonly suffered by white men as there
is a greater volume of medical trials data for this group than there is for almost any other group.
Algorithms trained on such biased datasets could make considerably poorer predictions for, for
example, younger black women (Vayena et al., 2018). If HCPs are left out of the loop completely and
learning healthcare systems primarily rely on automated decisions, there is considerable potential to
exacerbate existing inequalities between the haves and the have-nots of the digital healthcare
ecosystem, i.e., those that generate enough data on themselves to ensure accurately trained algorithms
and those that do not (Topol, 2019).
To mitigate these and associated risks, institutions need to be asking the crucial question:
how much clinical decision-making should we be delegating to AI-Health solutions (Di Nucci, 2019)?
If it is known that algorithms which enable profiling (e.g. those that determine genetic risk profiles)
can ignore outliers and provide the basis for discrimination (Garattini et al., 2019), so deciding whether
healthcare also ought to be a means to promote social justice is crucial in order to establish what type
of data services will be embedded in the system (Voigt, 2019), what data should be collected, and
which values should be embedded in algorithmic decision-making services (McDougall, 2019). This
decision also determines what sort of population-level behavioural change the health system should
be able to aim for depending on cost management, data collection and fairness in data-driven systems
(Department of Health and Social Care, 2018.). If not carefully considered, this process of
transforming the provision of care risks over-fitting the system to a specific set of values that may not
represent those of society at large (McDougall, 2019).
Another, more subtle yet pervasive transformative effect arises at the sectoral level. Powles
& Hodson (2017) argue that one risk that may arise from collaboration between public and private
sector entities such as that between the Royal Free London hospital and DeepMind is that the positive
5
Here we are discussing fairly routine illnesses and injuries that have set treatment protocols that may need to
be flexibly interpreted on a case-by-case basis. We recognize that there are other instances, such as in the case
of rare diseases, where algorithmic systems might be better equipped to deal with diagnostic uncertainty (for
example in cases of rare disesaes) by being able to draw on a wider range of data points and information
sources than a human clinician could.
13
benefits of AI-Health “solutions” will be siloed within private entities. They note that in the Royal
Free case, DeepMind [was given] a lead advantage in developing new algorithmic tools on otherwise
privately-held, but publicly-generated datasets (Powles & Hodson, 2017, p. 362). This, they suggest,
may mean that the only feasible way that future advances may be developed is via DeepMind on
DeepMind’s terms”. This interpretation was contested by DeepMind, who called it “unevidenced and
untrue” and claimed that the Information Commissioner agreed with their stance in her 2018 ruling
(King et al., 2018). Whatever the circumstances of this particular case, the broader risk of privately
held AI-Health solutions trained on datasets that have been generated about the public by public
actors but then (lawfully) shared with private companies is worthy of caution going forward.
As may now be clear, these transformative effects also have significant ethical implications at
the societal LoA. Before institutions can establish where and how (and, from the sectoral perspective,
whether) AI-Health solutions can improve care, society itself must make difficult decisions about what
care is and what constitutes good care (Coeckelberg, 2014). To offer a simplistic example, does it mean
purely providing a technical diagnosis and an appropriate prescription or does it involve contemplating
a series of human necessities that revolve around well-being (Burr, Taddeo, & Floridi, 2019)? If it is
the former, then it is relatively easy to automate the role of non-surgical clinicians through AI (although
this does not imply that doctors should be substituted by AI systems). However, if is the latter, then
providing good health care means encompassing psychological wellbeing and other elements related
to quality of life, which would make human interaction an essential part of healthcare provision, as a
machine does not have the capability to make emotionally-driven decisions. Consequently, certain
decisions may completely exceed the machine’s capabilities and thus delegating these tasks to AI-
Health would be ethically concerning (Matthias, 2015).
Consider, for example, a situation where an AI-Health solution decides which patients are
sent to the Intensive Care Unit (ICU). Intensive care is a limited resource and only people who are at
risk of losing their lives or suffering grave harms are sent there. Triage decisions are currently made by
humans with the aim of maximising well-being for the greatest number of people. Doctors weigh
different factors when making this decision, including the likelihood of people surviving if they are
sent to the ICU. These situations often involve practitioners (implicitly) taking moral stances, by
prioritising individuals based on their age or health conditions. These cases are fundamentally oriented
by legal norms and medical deontology, yet personal expertise, experience and values also inevitably
play a role. Having the support of AI-Health in the ICU screening increases the number of agents and
complicates the norms involved in these decisions, since the doctor may follow his or her professional
guidelines, while the algorithm will be oriented by the values embedded in its code. Unless there is a
14
transparent process for society to be involved in the weighing of values embedded in these decision-
making tools (for instance, how is ‘fair’ provision of care defined?) (Cohen, Amarasingham, Shah, Xie,
& Lo, 2014), then the use of algorithms in such scenarios could result in the overfitting of the health
system to a specific set of values that are not representative of society at large.
In response to this risk, some attempts have already been made to involve the public at large
in decisions over the design and deployment of AI systems. In early 2019, the UK’s Information
Commissioner’s Office and the National Institute for Health Research staged a series of “citizens’
juries” to obtain the opinions of a representative cross-section of British society regarding the use of
AI in health (Information Commissioner, 2019). The “juries” were presented with four scenarios, two
relating to health using AI to diagnose strokes, and using it to find potential matches for a kidney
transplant and another two relating to criminal justice. Notably, the juries “strongly favoured
accuracy over explanation” in the two scenarios involving AI in health (National Institute for Health
Research, 2019). This is just one example of research which attempted to obtain public opinion data
regarding AI in health, and there are reasons to suppose that the apparent preference among
participants for accurate over explainable AI systems reflects the high-stakes and fast-moving scenarios
that were presented (as opposed to, say, the more routine illnesses and injuries we are focusing on
here). Nonetheless, it demonstrates the plausibility and preferability of involving the public in
designing AI-Health systems.
To conclude this sub-section, the notion that AI-Health technologies are ethically neutral is
unrealistic, and having them perform moral decision-making and enforcement may provoke immoral
and unfair results (Rajkomar, Hardt, Howell, Corrado, & Chin, 2018). The direct involvement of the
public in the design of AI-Health may help mitigate these risks. This should be borne in mind by all
those involved in the AI-driven transformation of healthcare systems.
3.3 Overarching Concerns: Traceability
The previous sub-section outlined how the increasing use of AI-Health is fundamentally transforming
the delivery of healthcare and the ethical implications of this process, particularly in terms of potentially
unfair outcomes. This transformation process means that healthcare systems now rely on a dynamic,
cyclical and intertwined series of interactions between human, artificial and hybrid agents (Vollmer,
Mateen, Bohner, Király, & Ghani, 2018)(Turilli & Floridi, 2009). This is making it increasingly
challenging identify interaction-emerging risks and allocate liability, raising ethical concerns with
regards to moral responsibility.
15
Moral responsibility involves both looking forward, where an individual, group or
organisation is perceived as being in charge of guaranteeing a desired outcome, and looking backwards
to appropriate blame and possibly redress, when a failure has occurred (Wardrope, 2015). In a well-
functioning healthcare system, this responsibility is distributed evenly and transparently across all
nodes so that the causal chain of a given outcome can be easily replicated in the case of a positive
outcome, or prevented from repeating in the case of a negative outcome (Floridi, 2013, 2016). In an
algorithmically-driven healthcare system, a single AI diagnostic tool might involve many people
organising, collecting and brokering data, and performing analyses on it, making this transparent
allocation of responsibility almost impossible. In essence, not only is the decision-making process of
a single algorithm a black-box, but the entire chain of actors that participate in the end product of AI-
Health solutions is extremely complex. This makes the entire AI-Health ecosystem inaccessible and
opaque, making responsibility and accountability difficult.
To unpack the ethical implications of this at-scale lack of traceability, let us take the example
of a digital heart-rate monitor that ‘intelligently’ processes biological and environmental data to signal
to its user their risk of developing a heart condition.
At the individual LoA this process relies on what can be termed the ‘digital medical gaze’
(Morley & Floridi, 2019d) and is based on this micro-cycle of self-reflection adapted from (Garcia,
Romero, Keyson, & Havinga, 2014):
1. Gaining Knowledge: Algorithm reads multi-omic dataset to determine risk of heart attack
and providers individual with a ‘heart health score’
2. Gaining Awareness: on the advice of the algorithm, individual starts monitoring their
activity level and becomes aware of how active they are
3. Self-reflection: as directed by the algorithm the individual reflects on how much high fat
food they are eating in a day and compares this to their optimal diet based on their
genomic profile and their level of activity
4. Action: individual takes the advice of the algorithm and takes specific actions to improve
their heart-health score e.g. starts regular exercise.
If this does not work, and the individual still ends up experiencing heart failure, this process of
algorithmic surveillance (Rich & Miah, 2014) risk creating an elaborate mechanism for victim-blaming
(Danis & Solomon, 2013; McLaughlin, 2016). The individual may be seen as being a ‘bad user’ for
failing to act upon the allegedly objective and evidence-based advice of the algorithm (see section 3.1),
and may therefore be framed as being morally responsible for their poor health and not deserving of
state-provided healthcare. Yet, due to the lack of traceability, there can be no certainty that the poor
16
outcome was due to the lack of action by the individual: it could be a faulty device, buggy code, or the
result of biased datasets (Topol, 2019). Moreover, even if a negative outcome were to result purely
from an individual disregarding the guidance, the adoption of digital infrastructure that enables failure
to be ascribed to a morally culpableindividual is itself a matter of ethical concern. These new insights
may enable lives to be saved and quality of life to be drastically improved, yet they also shift the ethical
burden of living wellsquarely onto newly accountable individuals. The ontological shift that this new
infrastructure permits from individuals-as-patients deserving quality healthcare, regardless of their
prior choices as fallible humans, to individuals-as-agents expected to take active steps to pre-empt
negative outcomes raises stark questions for bioethics, which has traditionally been seen as an
“ethics of the receiver” (Floridi 2008). Moreover, these technological changes might prompt a shift in
the ethical framework, burdening the individuals, while not providing de facto means of behavioural
change. Many concerns stem from socio-demographic issues which entail harmful habits, and cannot
oversimplified to a matter of delivering the adequate information to the patient (Owens & Cribb,
2019).
Due to issues of bias (discussed further in section 3.2), there is, further, a group LoA ethical
risk that some groups may come to be seen as being more morally irresponsible about their healthcare
than others. Heart-rate monitors, for example, are notoriously less accurate for those with darker skin
(Hailu, 2019), meaning that they could give considerably less accurate advice to people of colour than
to those with light skin. If this results in people of colour being less able to use AI-Health advice to
improve their heart-health, then these groups of people may be seen as morally reprehensible when it
comes to their health. Furthermore, the healthcare could then ‘learn’ to predict that people of colour
have worse heart-health, potentially resulting in these groups of individuals being discriminated against
by, for example, insurers (Martani, Shaw, & Elger, 2019).
At the interpersonal, institutional and sectoral LoAs, this moral responsibility translates
into liability. If for example, instead of the heart-health algorithm providing the advice back to the
individual, it provides the data to the individual’s HCP and the HCP provides advice that either fails
to prevent an adverse event or directly causes an adverse event, this could be the basis of a medical
malpractice suit (Price, 2018). In this scenario, it remains unclear where the liability will eventually sit
(Ngiam & Khor, 2019). Current law implies that the HCP would be at fault, and therefore liable, for
an adverse event as the algorithm in this scenario would be considered a diagnostic support tool just
like a blood test with no decision making capacity, so it is the HCP’s responsibility to act
appropriately based on the information provided (Price, Gerke, & Cohen, 2019; Schönberger, 2019;
Sullivan & Schweikart, 2019). However, the supply chain for any clinical algorithm is considerably
17
more complex and less transparent than that of a more traditional diagnostic tool meaning that many
are questioning whether this is actually how the law will be interpreted in the future. For example, does
the liability really sit with the HCP for not questioning the results of the algorithm, even if they were
not able to evaluate the quality of the diagnostic against other sources of information, including their
own personal knowledge of the patient due to the black-box nature of the algorithm itself? And what
about the role of the hospital or care facility: does it have a responsibility to put in place a policy
allowing HCPs to overrule algorithmic advice when this seems indicated? Similarly, what role do
commissioners or retailers of the device that contains the algorithm play? Do they not carry some
responsibility for not checking its accuracy, or do they assume that this responsibility sits with the
regulator (for example MHRA in the UK, the FDA in the US or the CFDA in China) who should,
therefore, carry the burden for not appropriately assessing the product before it was deployed in the
market? What if the problem is further back in the chain, stemming from inaccurate coding or poor-
quality training data? There is a clear lack of distributed responsibility (Floridi, 2013, 2016) a problem
that is exacerbated by a lack of transparency making it hard to hold individual parts of the chain
accountable for poor outcomes which poses a significant ethical risk.
In their overview of patient-safety issues with AI in healthcare He et al. (2019) state that those
working in the field are trying to establish a systems-wide approach that does not attribute blame to
individuals or individual companies, but conclude that where liability will ultimately rest remains to be
seen. This is problematic because, as Hoffman et al. (2019) stress, uptake of algorithmic-decision-
making tools by the clinical community is highly unlikely until this liability question is resolved (Vayena
et al., 2018), which could result in the overarching ethical concern raised in the introduction that of
a significant missed opportunity. Many, including (Holzinger et al., 2019) believe that explainability is
the answer to solving this problem and that, if HCPs can understand how a decision was reached, then
reflecting on the output of an algorithm is no different from any other diagnostic tool. Indeed
Schönberger (2019) argues that legally this is the case and that as long as it can be proven that the duty
of care was met, then harm caused to a patient by an erroneous prediction of an AI-Health system
would not yet constitute medical negligence but that it might in the near future constitute negligence to
not rely on the algorithmic output which brings us back to the issues outlined in section 3.1.
Overall, this lack of clarity will continue to persist for some time (Schönberger, 2019), making
it once again a social issue. Society will ultimately dictate what the socially acceptable and socially
preferable (Floridi & Taddeo, 2016) answers are to these pressing questions. The ethical issue is
whether all parts of society will have an equal say in this debate, as in the example of citizens’ juries
above, or whether it will be those individuals or groups with the loudest voices that get to set the rules.
18
As (Beer, 2017) attests, when thinking about the power of an algorithm, we need to think beyond the
impact and consequences of the code, to the powerful ways in which notions and ideas about the
algorithm circulate throughout the social world.
A. Individual
B. Interpersonal
C. Group
D. Institutional
E. Sectoral
F. Societal
1. Epistemic
concern
(inconclusive,
inscrutable and
misguided
evidence)
Misdiagnosis
or missed
diagnosis
Loss of trust in
HCP-Patient
relationships, de-
personalisation of
care
Misdiagnosis or
missed diagnosis
at scale some
groups more
affected than
others
Waste of funds
and resources not
directed to areas
of greater need
Excessively
broad data
sharing
between
public and
private
entities
Poorer public
healthcare
provision and
worsening
health
outcomes for
society
2. Normative
(transformative
effects and
unfair
outcomes)
Surveillance &
undermining
of autonomy
and integrity
of self
Deskilling of
HCPs, overreliance
on AI-tools, and
undermining of
consent practices
and redefining
roles in the
healthcare system
Profiling and
discrimination
against certain
groups seen as
being less
healthy or
higher risk
Transformation
of care pathways
& imposing of
specific values at
scale redefining
‘good care’
Siloing of
new AI tool
development
within
private
sector
Inequalities in
outcomes
3. Overarching
(traceability)
‘Bad Users’
could come to
be blamed for
their own ill-
health
See institutional
Specific groups
framed as being
more morally
irresponsible
with regards to
their health than
others
Lack of clarity
over liability with
regards to issues
with safety and
effectiveness
could halt
adoption or
result in certain
groups being
blamed more
often than others
See
institutional
Society must
decide through
regulation
preferable
answers to the
questions
regarding
liability and
risk allocation
in healthcare
provision.
However, all
groups in
society may
not be given
an equal say in
this process
Table 3: summary of the epistemic, normative and overarching ethical concerns associated with AI-Health at the five different LoAs as
identified by the literature review
4.The Need for an Ethically-Mindful and Proportionate Approach
The literature surveyed in this review clearly indicates the need for an agreed pro-ethical blueprint for
AI-Health that considers the epistemic, normative and traceability ethical concerns at the five different
LoAs. Protecting people from the harms of AI-Health goes beyond protecting data collection and
applying a valid model. Normative frameworks need to contemplate the complexities of the human
interactions where these technologies will be introduced and their emotional impacts (Luxton, 2014;
Riek, 2016).
Importantantly, an adequate normative framework should deal with the key question related
to liability allocation in cases of medical error. Much of the risk of handling data and algorithms stems
from professionals not adopting measures to protect privacy and support cybersecurity. The solution
of risk management will come not only from drawing the boundaries of responsibility, but also
19
promoting capacitation, understanding and interfaces for handling AI tools. For one, promoting
doctors’ and patients’ understanding and control over how AI-Health produces predictions or
recommendations that are used in treatment plans, and access to and protection of patient data (Ngiam
& Khor, 2019). Also, there needs to be control over how the interface and design of AI-Health
products influences HCP-patient-artificial-agent interactions (Cohen et al., 2014). Finally, a
certification for professionals seeking to use AI-Health tools is also necessary for the adequate
implementation and use of AI (Kluge, Lacroix, & Ruotsalainen, 2018).
To tackle these challenges, healthcare systems will need to update outdated governance
mechanisms (policies, standards and regulations). These can be replaced with both hard and soft
mechanisms, meaning what ought to be done and what may be done based on the existing moral
obligations (Floridi, 2018), that balance the need to protect individuals from harm, whilst still
supporting innovation that can deliver genuine system and patient benefit (Morley & Joshi, 2019). In
short, healthcare systems should not be overly cautious about the adoption of AI-Health solutions,
but should be mindful of the potential ethical impacts (Floridi, 2019a) so that proportionate
governance models can be developed (Sethi & Laurie, 2013) which can, in turn, ensure that those
responsible for ensuring that healthcare systems are held accountable for the delivery of high-quality
equitable and safe care, can continue to be so.
What these regulations, standards and policies should cover and how they should be
developed remain open questions (Floridi, 2017b) which will likely be ‘solved’ multiple times over by
different healthcare systems operating in different settings. However, in order to lend a more
systematic approach to addressing these outstanding questions, enabling greater coherence and speed
in addressing these challenges, in Table 4 below we have assembled a list of essential cross-cutting
considerations that emerge from our literature review. The table indicates from which aspect of our
systematic review (ethical concern x LoA, corresponding to a cell in Table 3) each consideration is
drawn.
20
Consideration
Key supporting literature
Relevant aspects (ascending LoA6)
The professional skills required of the
healthcare workforce, including information
governance skills
(Kluge et al., 2018)
Epistemic (A, B, C, F)
Normative (B, C, D, E)
Overarching (A, C)
Which tasks should be delegated to AI-Health
solutions
(Di Nucci, 2019)
Epistemic (A, B, C, D, F)
Normative (B, C, D, F)
Overarching (A, C, D)
What evidence is needed to ‘prove’ clinical
effectiveness of an AI-Health solution
(Greaves et al., 2018)
Epistemic (A, B, C, E, F)
Normative (E)
Overarching (A, C, D, F)
Mechanisms for reporting or seeking redress
for AI-Health associated harms
(Schönberger, 2019)
Epistemic (A, C, E, F)
Normative (A, C, E, F)
Overarching (A, C, D)
Mechanisms for the inclusion of all relevant
stakeholder views in the development of AI-
Health systems
(Aitken et al., 2019)
Epistemic (C, E, F)
Overarching (A, C, D, F)
Explainability of specific AI-Health decisions
(Watson et al., 2019)
Epistemic (A, C)
Normative (A, C, E)
Overarching (A, D)
Reliability, replicability and safety of AI-
Health solutions
(Challen et al., 2019)
Epistemic (A, C, F)
Normative (C, E, F)
Overarching (A, C, D)
Transparency over how algorithmic tools are
integrated into the healthcare workflow, how
it shapes decisions, and how it affects process
optimization within medical services
(Effy Vayena et al., 2015)
Epistemic (A, B, C, D, E, F)
Normative (A, B, D, F)
Overarching (A, D, F)
How traditional and non-traditional sources of
health data can be incorporated into AI-
Health decision making, how it can be
appropriately protected and how it can be
harmonised
(e.g. Maher et al., 2019; Ploug & Holm,
2016; Richardson, Milam, & Chrysler,
2015; Townend, 2018)
Epistemic (A, C, D, E, F)
Normative (A, C, D, E, F)
Overarching (A, C, D, E)
How bioethical concepts (beneficence, non-
maleficence, autonomy and justice
(Beauchamp & Childress, 2013) are challenged
by AI-Health
(Mittelstadt, 2019)
Epistemic (B, F)
Normative (A, C, D, F)
Overarching (A, F)
How concepts such as fairness, accountability
and transparency can be maintained at scale
(Morley & Floridi, 2019b) so that, for
example, the output of algorithmic diagnostics
does not result in economic benefits to
specific drug producers or technology
companies
(Rosenfeld et al., 2019)
Epistemic (C, D, E, F)
Normative (D, E, F)
Overarching (F)
Table 4: the 11 key considerations for policymakers that arose from the literature review.
Awareness of the need to consider these questions is increasing, and efforts are being made at both a
national and international level to adapt existing regulations so that they remain fit for purpose (The
6
Denoted by an increasing Level of Analysis: Individual (A), Interpersonal (B), Group (C), Institutional (D),
Sectoral (E) and Societal (F).
21
Lancet Digital Health, 2019). The American Food and Drug Administration (FDA) is now planning
on regulating Software as a Medical Devices (SaMD) (Food and Drug Administration (FDA), 2019)
and in both the EU and the UK Regulation 2017/745 on medical devices comes into effect in April
2020 and significantly increases the range of software and non-medical products that will need to be
classed (and assessed) as medical devices. Additionally, the UK has published its Code of Conduct for
data-driven health and care technologies, standards for evidence of clinical effectiveness for digital
health technologies (Greaves et al., 2018) a digital assessment questionnaire standards for apps and
is currently developing a ‘regulation as a service’ model to ensure that there are appropriate regulatory
checks at all stages of the AI development cycle (Morley & Joshi, 2019). The World Health
Organisation has a number of projects under way to develop guidance for member states (Aicardi et
al., 2016) (World Health Organisation, 2019). In China, several norms
7
provide specific and detailed
instructions to ensure health data security and confidentiality (Wang, 2019) to ensure that health and
medical big data sets can be used as a national resource to develop algorithms (Zhang et al., 2018) for
the improvement of public health (Li, Li, Jiang, & Lan, 2019).
These are all steps in the right direction, however, their development is progressing slowly
(which is why the relevant literature is unlikely to reflect all current developments) and almost all focus
solely on interventions positioning themselves as being health-related in the medical sense, not in the
wider, wellbeing sense, e.g., healthy exercise, diet, sleeping habits). They will not necessarily mitigate
risks are associated with the expanding wellness industry, which provides algorithmic tools that
potentially enable people to bypass formal and well-regulated healthcare systems entirely by accessing
technology directly, either by using a wearable device or consulting online databases (Burr et al., 2019).
Similarly, although some technical solutions have been put forward for mitigating issues with data bias
(Gebru et al., 2018; Holland, Hosny, Newman, Joseph, & Chmielinski, 2018) and data quality (Dai,
Yoshigoe, & Parsley, 2018) and ensuring social inclusion in decision making (Balthazar et al., 2018;
Friedman, Hendry, & Borning, 2017; Rahwan, 2018), these remain relatively untested. Unless a
competitive advantage of taking such pro-ethical steps becomes clear without these approaches being
made mandatory, it is unlikely that they will have a significant impact on the ethical impacts of AI-
Health in the near future. As a result, there is still little control over the procedures followed and quality
7
Article 6 of the Regulations on the Management of Medical Records of Medical Institutions, Article 8 of the
Management Regulations on Application of Electronic Medical Records, Article 6 of the Measures for the
Management of Health Information, the Cybersecurity Law of the People’s Republic of China, and the new
Personal Information Security Specification.
22
control mechanisms (Cohen et al., 2014) involved in the development, deployment and use of AI-
Health.
As these comparatively easier to tackle problems do not yet have adequate solutions, it is
unsurprising that the bigger issues regarding the protection of equality of care (Powell & Deetjen, 2019,
fair distribution of benefits (Balthazar et al., 2018) (Kohli & Geis, 2018) and the protection and
promotion of societal values (Mahomed, 2018) have barely even been considered. Given that
healthcare systems in many ways act as the core of modern societies this is concerning. If mistakes are
made too early in the adoption and implementation of AI in healthcare, the fall-out could be significant
enough to undermine public trust, resulting in significant opportunity costs, and potentially
encouraging individuals to seek their healthcare from outside of the formal systems where they may
be presented with even greater risks. A coherent approach is needed and urgently, hopefully this
systematic overview of the issues to be considered can help speed up its development.
5. Conclusion
This thematic literature review has sought to map out the ethical issues around the incorporation of
data-driven AI technologies into healthcare provision and public health systems. In order to make this
overview more useful, the relevant topics have been organised into themes and five different levels of
abstraction (LoAs) have been highlighted. The hope is that by encouraging a discussion of the ethical
implications of AI-Health at individual, interpersonal, group, institutional and societal LoAs,
policymakers and regulators will be able to segment a large and complex conversation into tractable
debates around specific issues, stakeholders, and solutions. This is important, as Topol (2019) states
‘there cannot be exceptionalism for AI in medicine,’ especially not when there is potentially so much
to gain (Miotto, Wang, Wang, Jiang, & Dudley, 2018).
With this in mind, the review has covered a wide range of topics while also venturing into the
specificity of certain fields. This approach has made it to develop a fuller and more nuanced
understanding of the ethical concerns related to the introduction of AI into healthcare systems than
has been previously seen in the literature. Inevitably, there are limitations to this approach, which are
specified in the appendix, detailing our methodology and pointing towards opportunities for further
research.
In this article, we hope to have provided a sufficiently comprehensive, detailed, and systematic
analysis of the current debates on ethical issues related to the introduction of AI into healthcare
systems. The aim is to help policymakers and legislators develop evidence-based and proportionate
23
policy and regulatory interventions. In particular, we hope to encourage the development of a system
of transparent and distributed responsibility, where all those involved in the clinical algorithm supply
chain can be held proportionately and appropriately accountable for the safety of the patient at the
end, not just the HCP. It is only by ensuring such a system is developed that policymakers and
legislators can be confident that the inherent risks we have described are appropriately mitigated (as
far as possible) and only once this is the case will the medical community at large feel willing and able
to adopt AI technologies.
24
Appendix Methodology
The data collection for this research was divided into three stages as outlined in the below schematic.
This process resulted in approximately 147 papers suitable for analysis and inclusion in the
initial review. Subsequent relevant papers that met the criteria were added at a later date during the
writing up of the results.
This literature review also included accessory readings and case studies that were
encountered during the research process. This includes bibliography obtained from the references of
the papers analysed, and case studies identified in the readings (e.g. the Deep Mind case study). It is
our belief that these exploratory readings enrich our systematic approach by developing on
interesting findings and topics identified throughout our investigation.
It is important to note that the selection of articles and policy documents was restricted to
those written in English. This means that some ethical issues will have been overlooked (e.g. those in
Spanish-speaking countries or in China). Second, academic literature, much like regulation, tends to
struggle to keep pace with technological development. This literature review did not seek to identify
25
ethical issues associated with specific use cases of AI first-hand, for example, by reviewing recently
published studies available on pre-print servers such as arXiv, but instead focused on providing an
overview of the ethical issues already identified. As a result, there may well be ethical concerns that
are associated with more emergent use cases of AI for healthcare that we have not identified as they
have not yet been discussed in formal peer-reviewed publications.
To overcome these limitations, further research could seek to expand the literature review
by including a wider range of search queries, and by taking a case-study approach to analysing the
ethical issues of specific practices and then aggregating these. This could be complemented by a
comprehensive review of the policies, standards and regulations in development in different
healthcare systems across the globe to assess the extent to which these are likely to be effective at
mitigating these ethical concerns.
Technology
Ethics/Concerns
Field
Algorithm*
Fairness
Health*
Artificial Intelligence / AI
Moral*
Machine Learning
Ethics
Governance
Table 5: Showing terms refined from Mittelstadt et al (2016) and selected to focus the literature
search on publications focusing specifically on the ethics of AI for health. It is important to note that
the search parameters were not exactly the same in all databases. Adaptation was necessary since not
all databases operated with the same syntax or accepted the same number of search queries. As a
result, the arrangement of Boolean operators and a search parameter were adapted to ensure that all
possible combinations were covered.
Database
Search Query
Results
Titles
Selected
Titles Downloaded
SCOPUS
ethic* AND algorithm* AND health*
596
39
19
(ethic* AND ( "Artificial
Intelligence" OR ai ) AND health* )
239
37
15
( moral* AND ( "Artificial
Intelligence" OR ai ) AND health* )
46
2
0
( fair* AND ( "Artificial
Intelligence" OR ai ) AND health* )
122
6
3
(moral* OR ethic*) AND "machine learning" AND
health*
91
14
9
( fair* ) AND "machine learning" AND health* )
70
5
3
Web of Science
((fair* OR moral* OR ethic*) AND ("machine learning"
OR "Artificial Intelligence" OR "AI" OR algorithm*)
AND health*)
668
45
26
Philpapers
'"machine learning" AND health*
3
1
1
Artificial Intelligence AND health* AND ethic*
1000+
-
-
algorithm* AND health* AND ethic*
5
0
0
ethics AND "artificial intelligence" AND health
3
2
2
AI or Artificial Intelligence or Fair AND ethic or moral or
health AND health8
9
0
0
Google Scholar
ethics algorithms health
15,400
18
18
ethics of machine learning in health
21,300
11
10
ETHICS & HEALTH
and at least one of:
algorithm OR machine learning OR artificial intelligence
OR AI
716,000
2
1
26
ETHICS & HEALTH
and at least one of:
algorithm OR AI
105,000
2
2
MORAL & HEALTH
and at least one of:
algorithm OR AI
26,900
2
1
FAIR & HEALTH
And at least one of:
algorithm OR AI
38,000
0
0
PubMed
ETHICS & ARTIFICIAL INTELLIGENCE OR
MACHINE LEARNING
34, 193
37
37
Table 6: Showing the final results from all searches. It is important to note that multiple search
queries were made to cover all the combinations and the numbers in the table thus represent the sum
of results, titles evaluated and downloaded (not all found files were accessible for download). It is
also important to note that only the first 500 most relevant results from Google Scholar were
reviewed and anything written before 2014 was excluded to make the number of results more
manageable.
Funding
Taddeo and Floridi’s work was partially supported by Privacy and Trust Stream Social lead of the
PETRAS Internet of Things research hub PETRAS is funded by the UK Engineering and Physical
Sciences Research Council (EPSRC), grant agreement no. EP/N023013/1. Caio’s, Taddeo’s and
Floridi’s work was also partially supported by a Microsoft grant and a Google grant.
27
References
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018). Trends and Trajectories for
Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. Proceedings of the 2018
CHI Conference on Human Factors in Computing Systems - CHI ’18, 118.
https://doi.org/10.1145/3173574.3174156
Aicardi, C., Del Savio, L., Dove, E. S., Lucivero, F., Tempini, N., & Prainsack, B. (2016). Emerging ethical
issues regarding digital health data. On the World Medical Association Draft Declaration on Ethical
Considerations Regarding Health Databases and Biobanks. Croatian Medical Journal, 57(2), 207213.
https://doi.org/10.3325/cmj.2016.57.207
Aitken, M., Tully, M. P., Porteous, C., Denegri, S., Cunningham-Burley, S., Banner, N., … Willison, D. J.
(2019). Consensus Statement on Public Involvement and Engagement with Data-Intensive Health
Research. International Journal of Population Data Science, 4(1). https://doi.org/10.23889/ijpds.v4i1.586
Álvarez-Machancoses, Ó., & Fernández-Martínez, J. L. (2019). Using artificial intelligence methods to speed up
drug discovery. Expert Opinion on Drug Discovery, 14(8), 769777.
https://doi.org/10.1080/17460441.2019.1621284
Andorno, R. (2004). The right not to know: An autonomy based approach. Journal of Medical Ethics, 30(5), 435
439. https://doi.org/10.1136/jme.2002.001578
Arieno, A., Chan, A., & Destounis, S. V. (2019). A review of the role of augmented intelligence in breast
imaging: From automated breast density assessment to risk stratification. American Journal of
Roentgenology, 212(2), 259270. https://doi.org/10.2214/AJR.18.20391
Balthazar, P., Harri, P., Prater, A., & Safdar, N. M. (2018). Protecting Your Patients’ Interests in the Era of Big
Data, Artificial Intelligence, and Predictive Analytics. Journal of the American College of Radiology, 15(3),
580586. https://doi.org/10.1016/j.jacr.2017.11.035
Barakat, N., Bradley, A. P., & Barakat, M. N. H. (2010). Intelligible Support Vector Machines for Diagnosis of
Diabetes Mellitus. IEEE Transactions on Information Technology in Biomedicine, 14(4), 11141120.
https://doi.org/10.1109/TITB.2009.2039485
Barras, C. (2019). Mental health apps lean on bots and unlicensed therapists. Nature Medicine.
https://doi.org/10.1038/d41591-019-00009-6
Bartoletti, I. (2019). AI in healthcare: Ethical and privacy challenges. Lecture Notes in Computer Science (Including
Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11526 LNAI, 710.
https://doi.org/10.1007/978-3-030-21642-9_2
Barton, C., Chettipally, U., Zhou, Y., Jiang, Z., Lynn-Palevsky, A., Le, S., … Das, R. (2019). Evaluation of a
machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs.
Computers in Biology and Medicine, 109, 7984. https://doi.org/10.1016/j.compbiomed.2019.04.027
Beauchamp, T. L., & Childress, J. F. (2013). Principles of biomedical ethics (7th ed). New York: Oxford University
Press.
Beer, D. (2017). The social power of algorithms. Information, Communication & Society, 20(1), 113.
https://doi.org/10.1080/1369118X.2016.1216147
Brown, M. P. S., Grundy, W. N., Lin, D., Cristianini, N., Sugnet, C. W., Furey, T. S., … Haussler, D. (2000).
Knowledge-based analysis of microarray gene expression data by using support vector machines.
Proceedings of the National Academy of Sciences, 97(1), 262267. https://doi.org/10.1073/pnas.97.1.262
Burns, T. (2015). Our necessary shadow: The nature and meaning of psychiatry. Place of publication not identified:
Pegasus Books.
Burr, C., Mariarosaria, T., & Floridi, L. (2019, February 1). The Ethics of Digital Well-Being: A Thematic
Review. Retrieved 19 April 2019, from SSRN website:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3338441&download=yes
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017). Unintended Consequences of Machine Learning in Medicine.
JAMA, 318(6), 517. https://doi.org/10.1001/jama.2017.7797
Car, J., Sheikh, A., Wicks, P., & Williams, M. S. (2019). Beyond the hype of big data and artificial intelligence:
Building foundations for knowledge and wisdom. BMC Medicine, 17(1).
https://doi.org/10.1186/s12916-019-1382-x
28
Cath, C., Wachter, S., Mittelstadt, B., Taddeo, M., & Floridi, L. (2017). Artificial Intelligence and the ‘Good
Society’: The US, EU, and UK approach. Science and Engineering Ethics.
https://doi.org/10.1007/s11948-017-9901-7
Celi, L. A., Davidzon, G., Johnson, A. E. W., Komorowski, M., Marshall, D. C., Nair, S. S., … Stone, D. J.
(2016). Bridging the health data divide. Journal of Medical Internet Research, 18(12).
https://doi.org/10.2196/jmir.6400
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial
intelligence, bias and clinical safety. BMJ Quality & Safety, 28(3), 231237.
https://doi.org/10.1136/bmjqs-2018-008370
Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing Machine Learning in Health CareAddressing
Ethical Challenges. The New England Journal of Medicine, 378(11), 981983.
https://doi.org/10.1056/NEJMp1714229
Cheney-Lippold, J. (2017). We are data: Algorithms and the making of our digital selves. New York: New York
University Press.
Chiauzzi, E., & Wicks, P. (2019). Digital Trespass: Ethical and Terms-of-Use Violations by Researchers
Accessing Data From an Online Patient Community. Journal of Medical Internet Research, 21(2), e11985.
https://doi.org/10.2196/11985
Chin-Yee, B., & Upshur, R. (2019). Three Problems with Big Data and Artificial Intelligence in Medicine.
Perspectives in Biology and Medicine, 62(2), 237256. https://doi.org/10.1353/pbm.2019.0012
Coeckelberg, M. (2014). Good Healthcare Is in the “How”: The Quality of Care, the Role of Machines, and the
Need for New Skills. In Machine medical ethics (Vol. 74). New York: Springer.
Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014). The Legal And Ethical Concerns That
Arise From Using Complex Predictive Analytics In Health Care. Health Affairs, 33(7), 11391147.
https://doi.org/10.1377/hlthaff.2014.0048
Cookson, C. (2018, September 6). Artificial intelligence faces public backlash, warns scientist. Financial Times.
Retrieved from https://www.ft.com/content/0b301152-b0f8-11e8-99ca-68cf89602132
Cowie, J., Calveley, E., Bowers, G., & Bowers, J. (2018). Evaluation of a digital consultation and self-care
advice tool in primary care: A multi-methods study. International Journal of Environmental Research and
Public Health, 15(5). https://doi.org/10.3390/ijerph15050896
Dai, W., Yoshigoe, K., & Parsley, W. (2018). Improving Data Quality through Deep Learning and Statistical
Models. ArXiv:1810.07132 [Cs], 558, 515522. https://doi.org/10.1007/978-3-319-54978-1_66
Danis, M., & Solomon, M. (2013). Providers, Payers, The Community, And Patients Are All Obliged To Get
Patient Activation And Engagement Ethically Right. Health Affairs, 32(2), 401407.
https://doi.org/10.1377/hlthaff.2012.1081
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., … Ronneberger, O.
(2018). Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine,
24(9), 13421350. https://doi.org/10.1038/s41591-018-0107-6
De Langavant, L. C., Bayen, E., & Yaffe, K. (2018). Unsupervised machine learning to identify high likelihood
of dementia in population-based surveys: Development and validation study. Journal of Medical Internet
Research, 20(7). https://doi.org/10.2196/10493
Deng, X., Luo, Y., & Wang, C. (2018). Analysis of Risk Factors for Cervical Cancer Based on Machine
Learning Methods. 2018 5th IEEE International Conference on Cloud Computing and Intelligence Systems
(CCIS), 631635. https://doi.org/10.1109/CCIS.2018.8691126
Department of Health and Social Care. (n.d.-a). Annual Report of the Chief Medical Office 2018: Health 2040Better
Health Within Reach. Retrieved from
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file
/767549/Annual_report_of_the_Chief_Medical_Officer_2018_-_health_2040_-
_better_health_within_reach.pdf
Department of Health and Social Care. (n.d.-b). Health Secretary announces £250 million investment in
artificial intelligence [Gov.uk]. Retrieved 8 August 2019, from
https://www.gov.uk/government/news/health-secretary-announces-250-million-investment-in-
artificial-intelligence
Di Nucci, E. (2019). Should we be afraid of medical AI? Journal of Medical Ethics.
https://doi.org/10.1136/medethics-2018-105281
29
Ding, Y., Tang, J., & Guo, F. (2019). Identification of drug-side effect association via multiple information
integration with centered kernel alignment. Neurocomputing, 325, 211224.
https://doi.org/10.1016/j.neucom.2018.10.028
Dong, R., Yang, X., Zhang, X., Gao, P., Ke, A., Sun, H., … Shi, G. (2019). Predicting overall survival of
patients with hepatocellular carcinoma using a threecategory method based on DNA methylation
and machine learning. Journal of Cellular and Molecular Medicine, 23(5), 33693374.
https://doi.org/10.1111/jcmm.14231
Dudley, J. T., Listgarten, J., Stegle, O., Brenner, S. E., & Parts, L. (2015). Personalized medicine: From genotypes,
molecular phenotypes and the quantified self, towards improved medicine. 342346.
DuFault, B. L., & Schouten, J. W. (2018). Self-quantification and the datapreneurial consumer identity.
Consumption Markets & Culture, 127. https://doi.org/10.1080/10253866.2018.1519489
Emanuel, E. J., & Wachter, R. M. (2019). Artificial Intelligence in Health Care: Will the Value Match the Hype?
JAMA, 321(23), 22812282. https://doi.org/10.1001/jama.2019.4914
Faden, R. R., Kass, N. E., Goodman, S. N., Pronovost, P., Tunis, S., & Beauchamp, T. L. (2013). An Ethics
Framework for a Learning Health Care System: A Departure from Traditional Research Ethics and Clinical
Ethics. Hastings Center Report, 43(s1), S16S27. https://doi.org/10.1002/hast.134
Ferretti, A., Ronchi, E., & Vayena, E. (2019). From principles to practice: Benchmarking government guidance
on health apps. The Lancet Digital Health, 1(2), e55e57. https://doi.org/10.1016/S2589-
7500(19)30027-5
Fiske, A., Henningsen, P., & Buyx, A. (2019). Your Robot Therapist Will See You Now: Ethical Implications
of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy. Journal of Medical
Internet Research, 21(5), e13216. https://doi.org/10.2196/13216
Fleming, N. (2018). How artificial intelligence is changing drug discovery. Nature, 557(7707), S55S57.
https://doi.org/10.1038/d41586-018-05267-x
Floridi, L., Luetge, C., Pagallo, U., Schafer, B., Valcke, P., Vayena, E., … Kalra, D. (2018). Key Ethical
Challenges in the European Medical Information Framework. Minds and Machines, 117.
https://doi.org/10.1007/s11023-018-9467-4
Floridi, Luciano. (2008). The Method of Levels of Abstraction. Minds and Machines, 18(3), 303329.
https://doi.org/10.1007/s11023-008-9113-7
Floridi, Luciano. (2013). Distributed Morality in an Information Society. Science and Engineering Ethics, 19(3),
727743. https://doi.org/10.1007/s11948-012-9413-4
Floridi, Luciano. (2014). The 4th revolution: How the infosphere is reshaping human reality. Oxford: Oxford Univ.
Press.
Floridi, Luciano. (2016). Faultless responsibility: On the nature and allocation of moral responsibility for
distributed moral actions. Philosophical Transactions of the Royal Society A: Mathematical, Physical and
Engineering Sciences, 374(2083), 20160112. https://doi.org/10.1098/rsta.2016.0112
Floridi, Luciano. (2017a). Digital’s Cleaving Power and Its Consequences. Philosophy & Technology, 30(2), 123
129. https://doi.org/10.1007/s13347-017-0259-1
Floridi, Luciano. (2017b). The Logic of Design as a Conceptual Logic of Information. Minds and Machines,
27(3), 495519. https://doi.org/10.1007/s11023-017-9438-1
Floridi, Luciano. (2018). Soft ethics, the governance of the digital and the General Data Protection Regulation.
Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 376(2133).
https://doi.org/10.1098/rsta.2018.0081
Floridi, Luciano. (2019a). AI opportunities for healthcare must not be wasted. Health Management, 19.
Floridi, Luciano. (2019b). What the Near Future of Artificial Intelligence Could Be. Philosophy & Technology,
32(1), 115. https://doi.org/10.1007/s13347-019-00345-y
Floridi, Luciano, & Taddeo, M. (2016). What is data ethics? Philosophical Transactions of the Royal Society A:
Mathematical, Physical and Engineering Sciences, 374(2083), 20160360.
https://doi.org/10.1098/rsta.2016.0360
Friedman, B., Hendry, D. G., & Borning, A. (2017). A Survey of Value Sensitive Design Methods. Foundations
and Trends® in HumanComputer Interaction, 11(2), 63125. https://doi.org/10.1561/1100000015
Garattini, C., Raffle, J., Aisyah, D. N., Sartain, F., & Kozlakidis, Z. (2019). Big Data Analytics, Infectious
Diseases and Associated Ethical Impacts. Philosophy & Technology, 32(1), 6985.
https://doi.org/10.1007/s13347-017-0278-y
30
Garcia, J., Romero, N., Keyson, D., & Havinga, P. (2014). Reflective healthcare systems: Mirco-cylce of self-
reflection to empower users. Interaction Design and Architecture(s), 23(1), 173190.
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumeé III, H., & Crawford, K.
(2018). Datasheets for Datasets. ArXiv:1803.09010 [Cs]. Retrieved from
http://arxiv.org/abs/1803.09010
Gillespie, T., Boczkowski, P. J., & Foot, K. A. (2014). Media technologies: Essays on communication, materiality, and
society. Cambridge, Massachusetts: The MIT Press.
Greaves, F., Joshi, I., Campbell, M., Roberts, S., Patel, N., & Powell, J. (2018). What is an appropriate level of
evidence for a digital health intervention? The Lancet, 392(10165), 26652667.
https://doi.org/10.1016/S0140-6736(18)33129-5
Hailu, R. (2019). Fitbits and other wearables may not accurately track heart rates in people of color. STAT.
Retrieved from https://www.statnews.com/2019/07/24/fitbit-accuracy-dark-skin/
Harerimana, G., Jang, B., Kim, J. W., & Park, H. K. (2018). Health Big Data Analytics: A Technology Survey.
IEEE Access, 6, 6566165678. https://doi.org/10.1109/ACCESS.2018.2878254
Hay, S. I., George, D. B., Moyes, C. L., & Brownstein, J. S. (2013). Big data opportunities for global infectious disease
surveillance.
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019). The practical implementation of artificial
intelligence technologies in medicine. Nature Medicine, 25(1), 3036. https://doi.org/10.1038/s41591-
018-0307-0
Henson, P., David, G., Albright, K., & Torous, J. (2019). Deriving a practical framework for the evaluation of
health apps. The Lancet Digital Health, 1(2), e52e54. https://doi.org/10.1016/S2589-7500(19)30013-5
Hoffman, L., Benedetto, E., Huang, H., Grossman, E., Kaluma, D., Mann, Z., & Torous, J. (2019).
Augmenting Mental Health in Primary Care: A 1-Year Study of Deploying Smartphone Apps in a
Multi-site Primary Care/Behavioral Health Integration Program. Frontiers in Psychiatry, 10, 94.
https://doi.org/10.3389/fpsyt.2019.00094
Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2018). The Dataset Nutrition Label: A
Framework To Drive Higher Data Quality Standards. ArXiv:1805.03677 [Cs]. Retrieved from
http://arxiv.org/abs/1805.03677
Holzinger, A., Haibe-Kains, B., & Jurisica, I. (2019). Why imaging data alone is not enough: AI-based
integration of imaging, omics, and clinical data. European Journal of Nuclear Medicine and Molecular
Imaging. https://doi.org/10.1007/s00259-019-04382-9
Information Commissioner. (2018, June 6). Royal FreeGoogle DeepMind trial failed to comply with data
protection law. Retrieved 15 September 2019, from https://ico.org.uk/about-the-ico/news-and-
events/news-and-blogs/2017/07/royal-free-google-deepmind-trial-failed-to-comply-with-data-
protection-law/
Information Commissioner. (2019, June 3). Project ExplAIn interim report. Retrieved 15 September 2019,
from https://ico.org.uk/about-the-ico/research-and-reports/project-explain-interim-report/
Jiang, W., & Yin, Z. (2015). Human Activity Recognition Using Wearable Sensors by Deep Convolutional
Neural Networks. Proceedings of the 23rd ACM International Conference on Multimedia - MM ’15, 1307
1310. https://doi.org/10.1145/2733373.2806333
Juengst, E., McGowan, M. L., Fishman, J. R., & Settersten, R. A. (2016). From “Personalized” to “Precision”
Medicine: The Ethical and Social Implications of Rhetorical Reform in Genomic Medicine. Hastings
Center Report, 46(5), 2133. https://doi.org/10.1002/hast.614
Kalkman, S., Mostert, M., Gerlinger, C., Van Delden, J. J. M., & Van Thiel, G. J. M. W. (2019). Responsible
data sharing in international health research: A systematic review of principles and norms. BMC
Medical Ethics, 20(1). https://doi.org/10.1186/s12910-019-0359-9
Kalmady, S. V., Greiner, R., Agrawal, R., Shivakumar, V., Narayanaswamy, J. C., Brown, M. R. G., …
Venkatasubramanian, G. (2019). Towards artificial intelligence in mental health by improving
schizophrenia prediction with multiple brain parcellation ensemble-learning. Npj Schizophrenia, 5(1), 2.
https://doi.org/10.1038/s41537-018-0070-8
Kim, H., & Xie, B. (2017). Health literacy in the eHealth era: A systematic review of the literature. Patient
Education and Counseling, 100(6), 10731082.
31
King, D., Karthikesalingam, A., Hughes, C., Montgomery, H., Raine, R., Rees, G., & On behalf of the
DeepMind Health Team. (2018). Letter in response to Google DeepMind and healthcare in an age of
algorithms. Health and Technology, 8(1), 1113. https://doi.org/10.1007/s12553-018-0228-4
Kleinpeter, E. (2017). Four Ethical Issues of “E-Health”. IRBM, 38(5), 245249.
https://doi.org/10.1016/j.irbm.2017.07.006
Kluge, E.-H., Lacroix, P., & Ruotsalainen, P. (2018). Ethics Certification of Health Information Professionals.
Yearbook of Medical Informatics, 27(01), 037040. https://doi.org/10.1055/s-0038-1641196
Kohli, M., & Geis, R. (2018). Ethics, Artificial Intelligence, and Radiology. Journal of the American College of
Radiology, 15(9), 13171319. https://doi.org/10.1016/j.jacr.2018.05.020
Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015). Machine learning
applications in cancer prognosis and prediction. Computational and Structural Biotechnology Journal, 13, 8
17. https://doi.org/10.1016/j.csbj.2014.11.005
Krutzinna, J., Taddeo, M., & Floridi, L. (2018). Enabling Posthumous Medical Data Donation: An Appeal for
the Ethical Utilisation of Personal Health Data. Science and Engineering Ethics.
https://doi.org/10.1007/s11948-018-0067-8
Kuek, A., & Hakkennes, S. (2019). Healthcare staff digital literacy levels and their attitudes towards
information systems. Health Informatics Journal, 146045821983961.
https://doi.org/10.1177/1460458219839613
Kunapuli, G., Varghese, B. A., Ganapathy, P., Desai, B., Cen, S., Aron, M., … Duddalwar, V. (2018). A
Decision-Support Tool for Renal Mass Classification. Journal of Digital Imaging, 31(6), 929939.
https://doi.org/10.1007/s10278-018-0100-0
Kuo, W.-J., Chang, R.-F., Chen, D.-R., & Lee, C. C. (2001). Data mining with decision trees for diagnosis of
breast tumor in medical ultrasonic images. Breast Cancer Research and Treatment, 66(1), 5157.
https://doi.org/10.1023/A:1010676701382
Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B. (2019). Using science to sell
apps: Evaluation of mental health app store quality claims. Npj Digital Medicine, 2(1), 18.
https://doi.org/10.1038/s41746-019-0093-1
Lee, L. M. (2017). Ethics and subsequent use of electronic health record data. Journal of Biomedical Informatics, 71,
143146. https://doi.org/10.1016/j.jbi.2017.05.022
Li, B., Li, J., Jiang, Y., & Lan, X. (2019). Experience and reflection from China’s Xiangya medical big data
project. Journal of Biomedical Informatics, 93. https://doi.org/10.1016/j.jbi.2019.103149
Liu, C., Liu, X., Wu, F., Xie, M., Feng, Y., & Hu, C. (2018). Using Artificial Intelligence (Watson for Oncology)
for Treatment Recommendations Amongst Chinese Patients with Lung Cancer: Feasibility Study.
Journal of Medical Internet Research, 20(9), e11087. https://doi.org/10.2196/11087
López-Martínez, F., Núñez-Valdez, E. R., Lorduy Gomez, J., & García-Díaz, V. (2019). A neural network
approach to predict early neonatal sepsis. Computers & Electrical Engineering, 76, 379388.
https://doi.org/10.1016/j.compeleceng.2019.04.015
Lu, F. S., Hattab, M. W., Clemente, C. L., Biggerstaff, M., & Santillana, M. (2019). Improved state-level
influenza nowcasting in the United States leveraging Internet-based data and network approaches.
Nature Communications, 10(1). https://doi.org/10.1038/s41467-018-08082-0
Lu, H., & Wang, M. (2019). RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement
Pathway Optimization. ArXiv:1906.01407 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1906.01407
Luxton, D. D. (2014). Artificial intelligence in psychological practice: Current and future applications and
implications. Professional Psychology: Research and Practice, 45(5), 332339.
https://doi.org/10.1037/a0034559
Maher, N. A., Senders, J. T., Hulsbergen, A. F. C., Lamba, N., Parker, M., Onnela, J.-P., … Broekman, M. L.
D. (2019). Passive data collection and use in healthcare: A systematic review of ethical issues.
International Journal of Medical Informatics, 129, 242247. https://doi.org/10.1016/j.ijmedinf.2019.06.015
Mahomed, S. (2018). Healthcare, artificial intelligence and the Fourth Industrial Revolution: Ethical, social and
legal considerations. South African Journal of Bioethics and Law, 11(2), 93.
https://doi.org/10.7196/SAJBL.2018.v11i2.00664
Mann, S. P., Savulescu, J., & Sahakian, B. J. (2016). Facilitating the ethical use of health data for the benefit of
society: Electronic health records, consent and the duty of easy rescue. Philosophical Transactions of the
32
Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083).
https://doi.org/10.1098/rsta.2016.0130
Marill, M. (2019, May 30). Why Tracking Your Symptoms Can Make You Feel Worse. Wired. Retrieved from
https://www.wired.com/story/why-tracking-your-symptoms-can-make-you-feel-
worse/?&utm_campaign=the_download.unpaid.engagement&utm_source=hs_email&utm_medium
=email&utm_content=73227565&_hsenc=p2ANqtz-
8UIkyITAj5PaGHIR7rQySwYgVDMJAdHABei6CktNIvvIaayTN3lsHJHhDMSWxcl0rGKAf23ckp
D3uUAEZow-HQyxjsPJVKPfptCzqqpl2XART5QOA&_hsmi=73227565
Martani, A., Shaw, D., & Elger, B. S. (2019). Stay fit or get bitEthical issues in sharing health data with
insurers’ apps. Swiss Medical Weekly, 149, w20089. https://doi.org/10.4414/smw.2019.20089
Matthias, A. (2015). Robot Lies in Health Care: When Is Deception Morally Permissible? Kennedy Institute of
Ethics Journal, 25(2), 169162. https://doi.org/10.1353/ken.2015.0007
McDougall, R. J. (2019). Computer knows best? The need for value-flexibility in medical AI. Journal of Medical
Ethics, 45(3), 156160. https://doi.org/10.1136/medethics-2018-105118
McLaughlin, K. (2016). Empowerment: A critique. Retrieved from
http://public.eblib.com/choice/publicfullrecord.aspx?p=4332655
Millett, S., & O’Leary, P. (2015). Revisiting consent for health information databanks. Research Ethics, 11(3),
151163. https://doi.org/10.1177/1747016115587964
Millett, Stephan, & O’Leary, P. (2015). Revisiting consent for health information databanks. Research Ethics,
11(3), 151163. https://doi.org/10.1177/1747016115587964
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018). Deep learning for healthcare: Review,
opportunities and challenges. Briefings in Bioinformatics, 19(6), 12361246.
https://doi.org/10.1093/bib/bbx044
Mittelstadt, B. (2019). The Ethics of Biomedical ‘Big Data’ Analytics. Philosophy & Technology, 32(1), 1721.
https://doi.org/10.1007/s13347-019-00344-z
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping
the debate. Big Data & Society, 3(2), 205395171667967. https://doi.org/10.1177/2053951716679679
Morley, J, & Joshi, I. (2019). Developing effective policy to support Artificial Intelligence in Health and Care.
Eurohealth, 25(2). https://doi.org/Forthcoming
Morley, Jessica, & Floridi, L. (In Press). The need for an ethically mindful approach to AI for healthcare. The
Lancet.
Morley, Jessica, & Floridi, L. (2019a). Enabling digital health companionship is better than empowerment. The
Lancet Digital Health, S2589750019300792. https://doi.org/10.1016/S2589-7500(19)30079-2
Morley, Jessica, & Floridi, L. (2019b). How to design a governable digital health ecosystem. Health Policy and
Technology. https://doi.org/10.13140/rg.2.2.28320.74244/1
Morley, Jessica, & Floridi, L. (2019c). The Limits of Empowerment: How to Reframe the Role of mHealth
Tools in the Healthcare Ecosystem. Science and Engineering Ethics. https://doi.org/10.1007/s11948-
019-00115-1
Morley, Jessica, Floridi, L., Kinsey, L., & Elhalal, A. (2019). From What to How. An Overview of AI Ethics
Tools, Methods and Research to Translate Principles into Practices. Science and Engineering Ethics.
Retrieved from http://arxiv.org/abs/1905.06876 (Pre-Print)
Mortazavi, B. J., Desai, N., Zhang, J., Coppi, A., Warner, F., Krumholz, H. M., & Negahban, S. (2017).
Prediction of Adverse Events in Patients Undergoing Major Cardiovascular Procedures. IEEE Journal
of Biomedical and Health Informatics, 21(6), 17191729. https://doi.org/10.1109/JBHI.2017.2675340
Moscoso, A., Silva-Rodríguez, J., Aldrey, J. M., Cortés, J., Fernández-Ferreiro, A., Gómez-Lado, N., … Aguiar,
P. (2019). Prediction of Alzheimer’s disease dementia with MRI beyond the short-term: Implications
for the design of predictive models. NeuroImage: Clinical, 23, 101837.
https://doi.org/10.1016/j.nicl.2019.101837
Nag, N., Pandey, V., Oh, H., & Jain, R. (2017). Cybernetic Health. ArXiv:1705.08514 [Cs]. Retrieved from
http://arxiv.org/abs/1705.08514
National Institute for Health Research. (2019, June 14). Involving the public in complex questions around
artificial intelligence research. Retrieved 15 September 2019, from
https://www.nihr.ac.uk/blog/involving-the-public-in-complex-questions-around-artificial-
intelligence-research/12236
33
Nebeker, C., Harlow, J., Espinoza Giacinto, R., Orozco-Linares, R., Bloss, C. S., & Weibel, N. (2017). Ethical
and regulatory challenges of research using pervasive sensing and other emerging technologies: IRB
perspectives. AJOB Empirical Bioethics, 8(4), 266276.
https://doi.org/10.1080/23294515.2017.1403980
Nebeker, C., Torous, J., & Bartlett Ellis, R. J. (2019). Building the case for actionable ethics in digital health
research supported by artificial intelligence. BMC Medicine, 17(1). https://doi.org/10.1186/s12916-
019-1377-7
Nelson, A., Herron, D., Rees, G., & Nachev, P. (2019). Predicting scheduled hospital attendance with artificial
intelligence. Npj Digital Medicine, 2(1), 26. https://doi.org/10.1038/s41746-019-0103-3
Ngiam, K. Y., & Khor, I. W. (2019). Big data and machine learning algorithms for health-care delivery. The
Lancet Oncology, 20(5), e262e273. https://doi.org/10.1016/S1470-2045(19)30149-4
NHS England. (2019). The NHS Long Term Plan. Retrieved from NHS website:
https://www.longtermplan.nhs.uk/wp-content/uploads/2019/01/nhs-long-term-plan.pdf
Nittas, V., Mütsch, M., Ehrler, F., & Puhan, M. A. (2018). Electronic patient-generated health data to facilitate
prevention and health promotion: A scoping review protocol. BMJ Open, 8(8).
https://doi.org/10.1136/bmjopen-2017-021245
O’Doherty, K. C., Christofides, E., Yen, J., Bentzen, H. B., Burke, W., Hallowell, N., … Willison, D. J. (2016).
If you build it, they will come: Unintended future uses of organised health data collections Donna
Dickenson, Sandra Soo-Jin Lee, and Michael Morrison. BMC Medical Ethics, 17(1).
https://doi.org/10.1186/s12910-016-0137-x
Olimid, A. P., Rogozea, L. M., & Olimid, D.-A. (2018). Ethical approach to the genetic, biometric and health
data protection and processing in the new EU general data protection regulation (2018). Romanian
Journal of Morphology and Embryology, 59(2), 631636. Retrieved from
https://www.scopus.com/inward/record.uri?eid=2-s2.0-
85052735984&partnerID=40&md5=6b8d7af00f77e58b4439983e09148535
Owens, J., & Cribb, A. (2019). ‘My Fitbit Thinks I Can Do Better!’ Do Health Promoting Wearable
Technologies Support Personal Autonomy? Philosophy & Technology, 32(1), 2338.
https://doi.org/10.1007/s13347-017-0266-2
Page, S. A., Manhas, K. P., & Muruve, D. A. (2016). A survey of patient perspectives on the research use of
health information and biospecimens. BMC Medical Ethics, 17(1). https://doi.org/10.1186/s12910-
016-0130-4
Panch, T., Mattie, H., & Celi, L. A. (2019). The “inconvenient truth” about AI in healthcare. Npj Digital
Medicine, 2(1), 77. https://doi.org/10.1038/s41746-019-0155-4
Parker, L., Halter, V., Karliychuk, T., & Grundy, Q. (2019). How private is your mental health app data? An
empirical study of mental health app privacy policies and practices. International Journal of Law and
Psychiatry, 64, 198204. https://doi.org/10.1016/j.ijlp.2019.04.002
Petrini, C. (2015). On the ‘pendulum’ of bioethics. Clinica Terapeutica, 166(2), 8284.
https://doi.org/10.7417/CT.2015.1821
Ploug, T., & Holm, S. (2016). Meta consentA flexible solution to the problem of secondary use of health
data. Bioethics, 30(9), 721732. https://doi.org/10.1111/bioe.12286
Ploug, Thomas, & Holm, S. (2019). The right to refuse diagnostics and treatment planning by artificial
intelligence. Medicine, Health Care, and Philosophy. https://doi.org/10.1007/s11019-019-09912-8
Popkes, A.-L., Overweg, H., Ercole, A., Li, Y., Hernández-Lobato, J. M., Zaykov, Y., & Zhang, C. (2019).
Interpretable Outcome Prediction with Sparse Bayesian Neural Networks in Intensive Care.
ArXiv:1905.02599 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1905.02599
Powles, J., & Hodson, H. (2017). Google DeepMind and healthcare in an age of algorithms. Health and
Technology, 117. https://doi.org/10.1007/s12553-017-0179-1
Powles, J., & Hodson, H. (2018). Response to DeepMind. Health and Technology, 8(1), 1529.
https://doi.org/10.1007/s12553-018-0226-6
Price, W. N. (2018). Medical Malpractice and Black-Box Medicine. In I. G. Cohen, H. F. Lynch, E. Vayena, &
U. Gasser (Eds.), Big Data, Health Law, and Bioethics (1st ed., pp. 295306).
https://doi.org/10.1017/9781108147972.027
Price, W. N., Gerke, S., & Cohen, I. G. (2019). Potential Liability for Physicians Using Artificial Intelligence.
JAMA. https://doi.org/10.1001/jama.2019.15064
34
Quinn, P. (2017). The Anonymisation of Research DataA Pyric Victory for Privacy that Should Not Be
Pushed Too Hard by the EU Data Protection Framework? European Journal of Health Law, 24(4), 347
367. https://doi.org/10.1163/15718093-12341416
Racine, E., Boehlen, W., & Sample, M. (2019). Healthcare uses of artificial intelligence: Challenges and
opportunities for growth. Healthcare Management Forum. https://doi.org/10.1177/0840470419843831
Rahwan, I. (2018). Society-in-the-Loop: Programming the Algorithmic Social Contract. Ethics and Information
Technology, 20(1), 514. https://doi.org/10.1007/s10676-017-9430-8
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018). Ensuring Fairness in Machine
Learning to Advance Health Equity. Annals of Internal Medicine, 169(12), 866.
https://doi.org/10.7326/M18-1990
Rich, E., & Miah, A. (2014). Understanding Digital Health as Public Pedagogy: A Critical Framework. Societies,
4(2), 296315. https://doi.org/10.3390/soc4020296
Richardson, V., Milam, S., & Chrysler, D. (2015). Is Sharing De-identified Data Legal? The State of Public
Health Confidentiality Laws and Their Interplay with Statistical Disclosure Limitation Techniques.
Journal of Law, Medicine and Ethics, 43(s1), 8386. https://doi.org/10.1111/jlme.12224
Riek, L. D. (2016). Robotics Technology in Mental Health Care. In Artificial Intelligence in Behavioral and Mental
Health Care (pp. 185203). https://doi.org/10.1016/B978-0-12-420248-1.00008-8
Rosenfeld, A., Benrimoh, D., Armstrong, C., Mirchi, N., Langlois-Therrien, T., Rollins, C., … Yaniv-
Rosenfeld, A. (2019). Big Data Analytics and AI in Mental Healthcare. ArXiv:1903.12071 [Cs].
Retrieved from http://arxiv.org/abs/1903.12071
Ruckenstein, M., & Schüll, N. D. (2017). The Datafication of Health. Annual Review of Anthropology, 46(1), 261
278. https://doi.org/10.1146/annurev-anthro-102116-041244
Schönberger, D. (2019). Artificial intelligence in healthcare: A critical analysis of the legal and ethical
implications. International Journal of Law and Information Technology, 27(2), 171203.
https://doi.org/10.1093/ijlit/eaz004
Sethi, N., & Laurie, G. T. (2013). Delivering proportionate governance in the era of eHealth: Making linkage
and privacy work together. Medical Law International, 13(23), 168204.
https://doi.org/10.1177/0968533213508974
Shaw, D. M., Gross, J. V., & Erren, T. C. (2016). Data donation after death: A proposal to prevent the waste of
medical research data. EMBO Reports, 17(1), 1417. https://doi.org/10.15252/embr.201541802
Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018). Deep EHR: A Survey of Recent Advances in Deep
Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE Journal of Biomedical and
Health Informatics, 22(5), 15891604. https://doi.org/10.1109/JBHI.2017.2767063
Sterckx, S., Rakic, V., Cockbain, J., & Borry, P. (2016). “You hoped we would sleep walk into accepting the
collection of our data”: Controversies surrounding the UK care.data scheme and their wider
relevance for biomedical research. Medicine, Health Care and Philosophy, 19(2), 177190.
https://doi.org/10.1007/s11019-015-9661-6
Sullivan, H. R., & Schweikart, S. J. (2019). Are current tort liability doctrines adequate for addressing injury
caused by AI? AMA Journal of Ethics, 21(2), 160166. https://doi.org/10.1001/amajethics.2019.160
Taddeo, M., & Floridi, L. (2018). How AI can be a force for good: Science, 361(6404), 751752.
https://doi.org/10.1126/science.aat5991
Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature
Medicine, 25(1), 4456. https://doi.org/10.1038/s41591-018-0300-7
Topuz, K., Zengul, F. D., Dag, A., Almehmi, A., & Yildirim, M. B. (2018). Predicting graft survival among
kidney transplant recipients: A Bayesian decision support model. Decision Support Systems, 106, 97109.
https://doi.org/10.1016/j.dss.2017.12.004
Townend, D. (2018). Conclusion: Harmonisation in genomic and health data sharing for research: An
impossible dream? Human Genetics, 137(8), 657664. https://doi.org/10.1007/s00439-018-1924-x
Tran, B. X., Vu, G. T., Ha, G. H., Vuong, Q.-H., Ho, M.-T., Vuong, T.-T., … Ho, R. C. M. (2019). Global
Evolution of Research in Artificial Intelligence in Health and Medicine: A Bibliometric Study. Journal
of Clinical Medicine, 8(3). https://doi.org/10.3390/jcm8030360
Turilli, M., & Floridi, L. (2009). The ethics of information transparency. Ethics and Information Technology, 11(2),
105112. https://doi.org/10.1007/s10676-009-9187-9
35
Vayena, E., Tobias, H., Afua, A., & Allesandro, B. (2018). Digital health: Meeting the ethical and policy
challenges. Swiss Medical Weekly, 148(34). https://doi.org/10.4414/smw.2018.14571
Vayena, Effy, Blasimme, A., & Cohen, I. G. (2018). Machine learning in medicine: Addressing ethical
challenges. PLoS Medicine, 15(11), e1002689. https://doi.org/10.1371/journal.pmed.1002689
Vayena, Effy, Salathé, M., Madoff, L. C., & Brownstein, J. S. (2015). Ethical Challenges of Big Data in Public
Health. PLOS Computational Biology, 11(2), e1003904. https://doi.org/10.1371/journal.pcbi.1003904
Voigt, K. (2019). Social Justice, Equality and Primary Care: (How) Can ‘Big Data’ Help? Philosophy & Technology,
32(1), 5768. https://doi.org/10.1007/s13347-017-0270-6
Vollmer, S., Mateen, B. A., Bohner, G., Király, F. J., & Ghani, R. (2018). Machine learning and AI research for
Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness. 25.
Vollmer, S., Mateen, B. A., Bohner, G., Király, F. J., Ghani, R., Jonsson, P., … Hemingway, H. (2018).
Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency,
Replicability, Ethics and Effectiveness. ArXiv:1812.10404 [Cs, Stat]. Retrieved from
http://arxiv.org/abs/1812.10404
Wachter, R. M. (2015). The digital doctor: Hope, hype, and harm at the dawn of medicine’s computer age. New York:
McGraw-Hill Education.
Wang, Z. (2019). Data integration of electronic medical record under administrative decentralization of medical
insurance and healthcare in China: A case study. Israel Journal of Health Policy Research, 8(1).
https://doi.org/10.1186/s13584-019-0293-9
Wardrope, A. (2015). Relational Autonomy and the Ethics of Health Promotion. Public Health Ethics, 8(1), 50
62. https://doi.org/10.1093/phe/phu025
Watson, D. S., Krutzinna, J., Bruce, I. N., Griffiths, C. E., McInnes, I. B., Barnes, M. R., & Floridi, L. (2019).
Clinical applications of machine learning algorithms: Beyond the black box. BMJ, 364, l886.
https://doi.org/10.1136/bmj.l886
World Health Organisation. (n.d.). Big data and artificial intelligence. Retrieved 29 June 2019, from
https://www.who.int/ethics/topics/big-data-artificial-intelligence/en/
Zacher, B., & Czogiel, I. (2019). Supervised learning improves disease outbreak detection. ArXiv:1902.10061
[Cs, Stat]. Retrieved from http://arxiv.org/abs/1902.10061
Zhang, L., Wang, H., Li, Q., Zhao, M.-H., & Zhan, Q.-M. (2018). Big data and medical research in China. BMJ,
j5910. https://doi.org/10.1136/bmj.j5910
... To the end of managing this challenge of controllability and the epistemic uncertainties around "black box" decision-making, recently issued ethical guidelines and codices give prominence to the issue of transparency [45] and thus, in technical, legal, economic, political, and social terms, to the call for the development of explainable AI (XAI). XAI appears to have evolved into a new moral principle for the development and design of responsible AI [46,47]. Applying this concept to the idea of the "digital tumor board", it seems clear that explainability is a central aspect of a clinician's assessment and is particularly key to the justification of a decision to the patient affected. ...
Article
Full-text available
Good decision-making is a complex endeavor, and particularly so in a health context. The possibilities for day-to-day clinical practice opened up by AI-driven clinical decision support systems (AI-CDSS) give rise to fundamental questions around responsibility. In causal, moral and legal terms the application of AI-CDSS is challenging existing attributions of responsibility. In this context, responsibility gaps are often identified as main problem. Mapping out the changing dynamics and levels of attributing responsibility, we argue in this article that the application of AI-CDSS causes diffusions of responsibility with respect to a causal, moral, and legal dimension. Responsibility diffusion describes the situation where multiple options and several agents can be considered for attributing responsibility. Using the example of an AI-driven ‘digital tumor board’, we illustrate how clinical decision-making is changed and diffusions of responsibility take place. Not denying or attempting to bridge responsibility gaps, we argue that dynamics and ambivalences are inherent in responsibility, which is based on normative considerations such as avoiding experiences of disregard and vulnerability of human life, which are inherently accompanied by a moment of uncertainty, and is characterized by revision openness. Against this background and to avoid responsibility gaps, the article concludes with suggestions for managing responsibility diffusions in clinical decision-making with AI-CDSS.
... One potential limitation to this study is that given the field of AI is evolving at an unprecedented rate [1], there is a possibility that new records in the academic and grey literatures will have been published after the conclusion of our search, and prior to publication. Some recent examples of related articles have very much been in line with our findings, drawing light to many of the pertinent ethical issues of AI in healthcare discussed in the literature reviewed [18,[127][128][129][130][131][132]. Few, however, appear to have discussed the ethical application of AI in LMICs [117,133] or public health [117,130], so despite any new literature that may have arisen, there is still further work to be done in these areas. ...
Article
Full-text available
Background Artificial intelligence (AI) has been described as the “fourth industrial revolution” with transformative and global implications, including in healthcare, public health, and global health. AI approaches hold promise for improving health systems worldwide, as well as individual and population health outcomes. While AI may have potential for advancing health equity within and between countries, we must consider the ethical implications of its deployment in order to mitigate its potential harms, particularly for the most vulnerable. This scoping review addresses the following question: What ethical issues have been identified in relation to AI in the field of health, including from a global health perspective? Methods Eight electronic databases were searched for peer reviewed and grey literature published before April 2018 using the concepts of health, ethics, and AI, and their related terms. Records were independently screened by two reviewers and were included if they reported on AI in relation to health and ethics and were written in the English language. Data was charted on a piloted data charting form, and a descriptive and thematic analysis was performed. Results Upon reviewing 12,722 articles, 103 met the predetermined inclusion criteria. The literature was primarily focused on the ethics of AI in health care, particularly on carer robots, diagnostics, and precision medicine, but was largely silent on ethics of AI in public and population health. The literature highlighted a number of common ethical concerns related to privacy, trust, accountability and responsibility, and bias. Largely missing from the literature was the ethics of AI in global health, particularly in the context of low- and middle-income countries (LMICs). Conclusions The ethical issues surrounding AI in the field of health are both vast and complex. While AI holds the potential to improve health and health systems, our analysis suggests that its introduction should be approached with cautious optimism. The dearth of literature on the ethics of AI within LMICs, as well as in public health, also points to a critical need for further research into the ethical implications of AI within both global and public health, to ensure that its development and implementation is ethical for everyone, everywhere.
... One potential limitation to this study is that given the eld of AI is evolving at an unprecedented rate (1), there is a possibility that new records in the academic and grey literatures will have been published after the conclusion of our search, and prior to publication. Some recent examples of related articles have very much been in line with our ndings, drawing light to many of the pertinent ethical issues of AI in healthcare discussed in the literature reviewed (18,(127)(128)(129)(130)(131)(132). Few, however, appear to have discussed the ethical application of AI in LMICs (117,133) or public health (117,130), so despite any new literature that may have arisen, there is still further work to be done in this area. ...
Preprint
Full-text available
Background: Artificial intelligence (AI) has been described as the “fourth industrial revolution” with transformative and global implications, including in healthcare, public health, and global health. AI approaches hold promise for improving health systems worldwide, as well as individual and population health outcomes. While AI may have potential for advancing health equity within and between countries, we must consider the ethical implications of its deployment in order to mitigate its potential harms, particularly for the most vulnerable. This scoping review addresses the following question: What ethical issues have been identified in relation to AI in the field of health, including from a global health perspective? Methods: Eight electronic databases were searched for peer reviewed and grey literature published before April 2018 using the concepts of health, ethics, and AI, and their related terms. Records were independently screened by two reviewers and were included if they reported on AI in relation to health and ethics and were written in the English language. Data was charted on a piloted data charting form, and a descriptive and thematic analysis was performed. Results: Upon reviewing 12,722 articles, 103 met the predetermined inclusion criteria. The literature was primarily focused on the ethics of AI in health care, particularly on carer robots, diagnostics, and precision medicine, but was largely silent on ethics of AI in public and population health. The literature highlighted a number of common ethical concerns related to privacy, trust, accountability and responsibility, and bias. Largely missing from the literature was the ethics of AI in global health, particularly in the context of low- and middle-income countries (LMICs). Conclusions: The ethical issues surrounding AI in the field of health are both vast and complex. While AI holds the potential to improve health and health systems, our analysis suggests that its introduction should be approached with cautious optimism. The dearth of literature on the ethics of AI within LMICs, as well as in public health, also points to a critical need for further research into the ethical implications of AI within both global and public health, to ensure that its development and implementation is ethical for everyone, everywhere.
... One potential limitation to this study is that given the eld of AI is evolving at an unprecedented rate (1), there is a possibility that new records in the academic and grey literatures will have been published after the conclusion of our search, and prior to publication. Some recent examples of related articles have very much been in line with our ndings, drawing light to many of the pertinent ethical issues of AI in healthcare discussed in the literature reviewed (18,(124)(125)(126)(127)(128)(129). ...
Preprint
Full-text available
Background: Artificial intelligence (AI) has been described as the “fourth industrial revolution” with transformative and global implications, including in healthcare, public health, and global health. AI approaches hold promise for improving health systems worldwide, as well as individual and population health outcomes. While AI may have potential for advancing health equity within and between countries, we must consider the ethical implications of its deployment in order to mitigate its potential harms, particularly for the most vulnerable. This scoping review addresses the following question: What ethical issues have been identified in relation to AI in the field of health, including from a global health perspective? Methods: Eight electronic databases were searched for peer reviewed and grey literature published before April 2018 using the concepts of health, ethics, and AI, and their related terms. Records were independently screened by two reviewers and were included if they reported on AI in relation to health and ethics and were written in the English language. Data was charted on a piloted data charting form, and a descriptive and thematic analysis was performed. Results: Upon reviewing 12,722 articles, 103 met the predetermined inclusion criteria. The literature was primarily focused on the ethics of AI in health care, particularly on carer robots, diagnostics, and precision medicine, but was largely silent on ethics of AI in public and population health. The literature highlighted a number of common ethical concerns related to privacy, trust, accountability, and bias. Largely missing from the literature was the ethics of AI in global health, particularly in the context of low- and middle-income countries (LMICs). Conclusions: The ethical issues surrounding AI in the field of health are both vast and complex. While AI holds the potential to improve health and health systems, our analysis suggests that its introduction should be approached with cautious optimism. The dearth of literature on the ethics of AI within LMICs, as well as in public health, also points to a critical need for further research into the ethical implications of AI within both global and public health, to ensure that its development and implementation is ethical for everyone, everywhere.
Article
Full-text available
As the UK sits in painful deadlock over Brexit, it is important to remember that governments are regularly faced with crises, and their responses can create enduring benefit for future generations. Back in 1858, for example, the UK parliament was dealing with another messy crisis: “the great stink.” In a world before sanitation, the river Thames had become an open latrine, and as summer blossomed parliament was engulfed in a pestilential stench. £2.5 million (about £300 million in today’s money) was hastily approved to build a network of sewers throughout the capital.1 This particular model of sanitation, developed by Bazalgette, was adopted by other cities around the world and the rest, as they say, is history. It is now unthinkable that a developed nation would not have sanitation infrastructure. However, back in 1858 the debate was whether sanitation infrastructure was worthy of investment and whether it was a public or private good. A similar debate has been simmering for some time regarding health data infrastructure, defined as the hardware and software to securely aggregate, store, process and transmit healthcare data. Is data infrastructure necessary for healthcare organizations and if so, is it the responsibility of individual healthcare organizations, of local health systems, or is it a public good? [First paragraph]
Article
Full-text available
The increased availability of data has enabled the development of Artificially Intelligent Systems (AIS) for health, but implementing these systems and capitalising on the associated opportunities is not straightforward. To mitigate these risks, outdated governance mechanisms need to be updated and key questions answered. To achieve this, whilst still supporting innovation, a new joint organisation for digital, data and technology in the English NHS (NHSX) is developing a 'principled proportionate governance' model that involves focusing on proactively and objectively evaluating current AIS technology and regularly involving all those who rely on and serve the health and care system.
Article
Full-text available
In an analysis of artificially intelligent systems for medical diagnostics and treatment planning we argue that patients should be able to exercise a right to withdraw from AI diagnostics and treatment planning for reasons related to (1) the physician’s role in the patients’ formation of and acting on personal preferences and values, (2) the bias and opacity problem of AI systems, and (3) rational concerns about the future societal effects of introducing AI systems in the health care sector.
Article
Full-text available
The digital revolution is disrupting the ways in which health research is conducted, and subsequently, changing healthcare. Direct-to-consumer wellness products and mobile apps, pervasive sensor technologies and access to social network data offer exciting opportunities for researchers to passively observe and/or track patients 'in the wild' and 24/7. The volume of granular personal health data gathered using these technologies is unprecedented, and is increasingly leveraged to inform personalized health promotion and disease treatment interventions. The use of artificial intelligence in the health sector is also increasing. Although rich with potential, the digital health ecosystem presents new ethical challenges for those making decisions about the selection, testing, implementation and evaluation of technologies for use in healthcare. As the 'Wild West' of digital health research unfolds, it is important to recognize who is involved, and identify how each party can and should take responsibility to advance the ethical practices of this work. While not a comprehensive review, we describe the landscape, identify gaps to be addressed, and offer recommendations as to how stakeholders can and should take responsibility to advance socially responsible digital health research.
Article
Full-text available
Big data, coupled with the use of advanced analytical approaches, such as artificial intelligence (AI), have the potential to improve medical outcomes and population health. Data that are routinely generated from, for example, electronic medical records and smart devices have become progressively easier and cheaper to collect, process, and analyze. In recent decades, this has prompted a substantial increase in biomedical research efforts outside traditional clinical trial settings. Despite the apparent enthusiasm of researchers, funders, and the media, evidence is scarce for successful implementation of products, algorithms, and services arising that make a real difference to clinical care. This article collection provides concrete examples of how “big data” can be used to advance healthcare and discusses some of the limitations and challenges encountered with this type of research. It primarily focuses on real-world data, such as electronic medical records and genomic medicine, considers new developments in AI and digital health, and discusses ethical considerations and issues related to data sharing. Overall, we remain positive that big data studies and associated new technologies will continue to guide novel, exciting research that will ultimately improve healthcare and medicine—but we are also realistic that concerns remain about privacy, equity, security, and benefit to all.
The concept of distributed moral responsibility (DMR) has a long history. When it is understood as being entirely reducible to the sum of (some) human, individual and already morally loaded actions, then the allocation of DMR, and hence of praise and reward or blame and punishment, may be pragmatically difficult, but not conceptually problematic. However, in distributed environments, it is increasingly possible that a network of agents, some human, some artificial (e.g. a program) and some hybrid (e.g. a group of people working as a team thanks to a software platform), may cause distributed moral actions (DMAs). These are morally good or evil (i.e. morally loaded) actions caused by local interactions that are in themselves neither good nor evil (morally neutral). In this article, I analyse DMRs that are due to DMAs, and argue in favour of the allocation, by default and overridably, of full moral responsibility (faultless responsibility) to all the nodes/agents in the network causally relevant for bringing about the DMA in question, independently of intentionality. The mechanism proposed is inspired by, and adapts, three concepts: back propagation from network theory, strict liability from jurisprudence and common knowledge from epistemic logic.
Article
Artificial intelligence (AI) is perceived as the most transformative technology of the 21st century. Healthcare has been identified as an early candidate to be revolutionized by AI technologies. Various clinical and patient-facing applications have already reached healthcare practice with the potential to ease the pressure on healthcare staff, bring down costs and ultimately improve the lives of patients. However, various concerns have been raised as regards the unique properties and risks inherent to AI technologies. This article aims at providing an early stage contribution with a holistic view on the ‘decision-making’ capacities of AI technologies. The possible ethical and legal ramifications will be discussed against the backdrop of the existing frameworks. I will conclude that the present structures are largely fit to deal with the challenges AI technologies are posing. In some areas, sector-specific revisions of the law may be advisable, particularly concerning non-discrimination and product liability.
Article
Health-care systems worldwide face increasing demand, a rise in chronic disease, and resource constraints. At the same time, the use of digital health technologies in all care settings has led to an expansion of data. For this reason, policy makers, politicians, clinical entrepreneurs, and computer and data scientists argue that a key part of health-care solutions will be artificial Intelligence (AI), particularly machine learning AI forms a key part of the National Health Service (NHS) Long-Term Plan (2019) in England, the US National Institutes of Health Strategic Plan for Data Science (2018), and China’s Healthy China 2030 strategy (2016). The willingness to embrace the potential future of medical care, expressed in these national strategies, is a positive development. Health-care providers should, however, be mindful of the risks that arise from AI’s ability to change the intrinsic nature of how health care is delivered. This paper outlines and discusses these potential risks.
Article
Artificial intelligence (AI) is quickly making inroads into medical practice, especially in forms that rely on machine learning, with a mix of hope and hype.¹ Multiple AI-based products have now been approved or cleared by the US Food and Drug Administration (FDA), and health systems and hospitals are increasingly deploying AI-based systems.² For example, medical AI can support clinical decisions, such as recommending drugs or dosages or interpreting radiological images.² One key difference from most traditional clinical decision support software is that some medical AI may communicate results or recommendations to the care team without being able to communicate the underlying reasons for those results.