ArticlePDF Available

An exploration of expectations and perceptions of practicing physicians on the implementation of computerized clinical decision support systems using a Qsort approach

Authors:

Abstract and Figures

Background There is increasing interest in incorporating clinical decision support (CDS) into electronic healthcare records (EHR). Successful implementation of CDS systems depends on acceptance of them by healthcare workers. We used a mix of quantitative and qualitative methods starting from Qsort methodology to explore expectations and perceptions of practicing physicians on the use of CDS incorporated in EHR. Methods The study was performed in a large tertiary care academic hospital. We used a mixed approach with a Q-sort based classification of pre-defined reactions to clinical case vignettes combined with a thinking-aloud approach, taking into account COREQ recommendations The open source software of Ken-Q Analysis version 1.0.6. was used for the quantitative analysis, using principal components and a Varimax rotation. For the qualitative analysis, a thematic analysis based on the four main themes was performed based on the audiotapes and field notes. Results Thirty physicians were interviewed (7 in training, 8 junior staff and 15 senior staff; 16 females). Nearly all respondents were strongly averse towards interruptive messages, especially when these also were obstructive. Obstructive interruption was considered to be acceptable only when it increases safety, is adjustable to user expertise level and/or allows deviations when the end-user explains why a deviation is desirable in the case at issue. Transparency was deemed an essential feature, which seems to boil down to providing sufficient clarification on the factors underlying the recommendations of the CDS, so that these can be compared against the physicians’ existing knowledge, beliefs and convictions. Conclusion Avoidance of disruptive workflows and transparency of the underlying decision processes are important points to consider when developing CDS systems incorporated in EHR.
This content is subject to copyright. Terms and conditions apply.
VanBiesenetal.
BMC Medical Informatics and Decision Making (2022) 22:185
https://doi.org/10.1186/s12911-022-01933-3
RESEARCH
An exploration ofexpectations
andperceptions ofpracticing physicians
ontheimplementation ofcomputerized
clinical decision support systems using aQsort
approach
Wim Van Biesen1,2*, Daan Van Cauwenberge2,3, Johan Decruyenaere2,4, Tamara Leune1,2 and Sigrid Sterckx2,3
Abstract
Background: There is increasing interest in incorporating clinical decision support (CDS) into electronic healthcare
records (EHR). Successful implementation of CDS systems depends on acceptance of them by healthcare workers.
We used a mix of quantitative and qualitative methods starting from Qsort methodology to explore expectations and
perceptions of practicing physicians on the use of CDS incorporated in EHR.
Methods: The study was performed in a large tertiary care academic hospital. We used a mixed approach with
a Q-sort based classification of pre-defined reactions to clinical case vignettes combined with a thinking-aloud
approach, taking into account COREQ recommendations The open source software of Ken-Q Analysis version 1.0.6.
was used for the quantitative analysis, using principal components and a Varimax rotation. For the qualitative analysis,
a thematic analysis based on the four main themes was performed based on the audiotapes and field notes.
Results: Thirty physicians were interviewed (7 in training, 8 junior staff and 15 senior staff; 16 females). Nearly all
respondents were strongly averse towards interruptive messages, especially when these also were obstructive.
Obstructive interruption was considered to be acceptable only when it increases safety, is adjustable to user expertise
level and/or allows deviations when the end-user explains why a deviation is desirable in the case at issue. Transpar-
ency was deemed an essential feature, which seems to boil down to providing sufficient clarification on the factors
underlying the recommendations of the CDS, so that these can be compared against the physicians’ existing knowl-
edge, beliefs and convictions.
Conclusion: Avoidance of disruptive workflows and transparency of the underlying decision processes are important
points to consider when developing CDS systems incorporated in EHR.
Keywords: Big data, Clinical decision support, Medicine, Artificial intelligence
© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco
mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Introduction
Diagnosing and managing patients’ conditions are key
factors of patient care [1]. Interest in application of
Computerized clinical decision support systems (CDSS)
fueled by artificial intelligence (AI) is progressively
gaining momentum. Whilst external validation is still
Open Access
*Correspondence: wim.vanbiesen@ugent.be
1 Department of Nephrology, Ghent University Hospital, Corneel
Heymanslaan 10, 0K12IA, Ghent, Belgium
Full list of author information is available at the end of the article
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 2 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
lacking [25], expectations of how implementation of
AI-based CDSS can improve health care are high [6]. In
literature, CDSS is mostly defined as a computer-based
system intended to support clinical decision making
in everyday patient care by presenting to the health-
care worker an integrated summary of clinically rel-
evant patient information. e emergence of automated
CDSS is facilitated by the introduction of electronic
health records (EHRs) and Computerized Provider
Order Entry systems (CPOE). However, the introduc-
tion of EHR/CPOE has also been linked to increased
incidence of physician burnout and decreased patient
satisfaction. ese observations have been attributed to
the enhanced additional administrative burden induced
by EHR/CPOE (“death by a thousand clicks” [7]), and
the feeling that computers obstruct true involvement
between physician and patient [8] whilst they do not
improve clinically relevant outcomes [9]. As a conse-
quence, some perceive the implementation of CDSS
into everyday care a major step forward [6], whereas
others perceive it rather as a threat [10].
e success of the implementation and performance
of CDSS depends on technical characteristics of the
software, the clinical aspects of the task at hand and
the expertise of the physician with the CDSS. Next to
these, a substantial human factor remains, and accept-
ance of the CDSS is essential. Many interruptive medi-
cation alerts are e.g. simply ignored by the operator
[1113]. In addition, the problem of alert fatigue is a
well-established downside of interruptive messaging
in CDSS [5, 14]. Different aspects of successful imple-
mentation of CDS devices have been explored, however
mostly in narrow contexts for well-defined and deline-
ated clinical problems [4, 5]. Little evidence is available
on which factors should be taken into account to maxi-
mize uptake by clinicians when incorporating CDSS in
to general EHRs/CPOEs [15, 16]. Most EHRs/CPOEs
available on the market today are designed from an
administrative and informatics perspective [8]. ey
rarely consider the specific requirements of clinical
tasks [17]. Most systems do not take into account local
conditions and culture, and most offer general solu-
tions for general problems, rather than specific solu-
tions for the actual problems the clinicians and their
patients are facing [18, 19]. As a consequence, they pro-
duce unrealistic, inapt or plainly unsuitable advice for
the local setting. erefore, there is a huge gap between
what health care workers have to put in to the system to
make it work, mainly administrative information, and
what they get out in terms of improved care for their
patients.
We hypothesized that this friction causes frus-
tration and decreases the probability of successful
implementation of CDSS in an EHR/CPOE. On the other
hand, overconfidence in the computerized CDSS might
also occur even when the advice is misleading [20, 21].
erefore, we designed a study with a mix of a quan-
titative and a qualitative method to thematically explore
the reactions and underlying reasoning of physicians
when confronted with vignettes in which hypothetical
CDSS incorporated in an EHR were presented.
Methods
e study was performed in an academic hospital in a
transition to selecting, customizing and implementing
a new EHR/CPOE system. e current EHR/CPOE has
been in use for more than 10years, but does not incor-
porate possibilities for CDSS. All participants were thus
familiar with the concept of EHR/CPOE, however, their
exposure to CDSS incorporated in EHR/CPOE was low.
We used an approach with a Q-sort based [22] classifi-
cation of pre-defined reactions to clinical case vignettes
(Additional file 1: appendix 1), in combination with a
thinking-aloud approach in which reasoning and atti-
tudes of the participant during the classification task
were solicited. All sessions were done by the same inter-
viewer (WVB) with expertise in the field of computerized
CDSS. All sessions were audiotaped and typed out ver-
batim afterwards. During the whole process, we adhered
strictly to COREQ recommendations [23].
Based on literature and consensus by the research
team, a concourse of statements was created. ese state-
ments described potential actions of a (hypothetical)
CDSS in four well defined clinical settings (Additional
file1: appendix 1). We opted for a structured approach
based on a prior hypotheses that acceptance of clini-
cal decision support by clinicians is influenced 1/by the
transparency of the CDSS; 2/ the degree of certainty
regarding the advice provided by the CDSS; 3/ interrup-
tion of workflow with or without obstructivity; 4/ the
type of problem at hand. None of the vignettes explicitly
mentioned whether the CDSS relied on a rule based sys-
tem or a self-learning system.
We developed thirty statements [24], distributed over
four clinical case vignettes, varied in four different attrib-
utes [25]:
1. Transparency of the support how well is the reasoning
of the device explained to the user?
2. Ways of expressing (un)certainty of the system on the
correctness of its advice absolute truth and certainty
are rare in medical conditions. erefore, it is essen-
tial that a CDSS can express this uncertainty in its
advice. Different ways and gradations of expressing
this uncertainty were introduced in the vignettes.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 3 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
3. Interruption of workflow with or without obstruction
CDSS can produce advice on request, but also in an
unsolicited (automated) fashion while working with
the system. is can interrupt the workflow, mean-
ing the user is distracted from her activity and needs
to perform an unplanned action. is interrup-
tion can be non-obstructive (workflow can go on as
planned e.g. by pressing an “ignore” button), or be
obstructive (planned workflow can only go on after
the unplanned action has actually been performed or
planned workflow needs to be aborted).
4. Type of problem CDSS can be used for easy or more
complex tasks.
In a pilot, statements were evaluated for clarity and
lack of ambiguity (Additional file2).
In the first two vignettes, the focus was on decision
support for medication orders, covering formulary, drug-
drug interaction and drug-condition alerts [26]. In the
third vignette, a diagnostic problem was raised, assessing
automated CDSS for order entry as well as for presenting
and interpreting results of the ordered diagnostic tests.
In the last vignette, a more advanced CDSS for handling
a complex clinical problem was presented (Additional
file3).
We intended to interview 30 physicians, with purposive
sampling to achieve a mix of gender and level of expertise
(trainee, junior staff, senior staff) [22, 25].
First, a list of 30 physicians consenting to take part was
created on a first come first served basis to avoid selec-
tion bias. Next, a random allocation of order for inter-
viewing was created. Respondents were interviewed in a
silent room, and all sources of distraction were avoided as
much as possible. All interviews took place in the period
July–August 2021 (Additional file4).
Statistics andanalysis
For the quantitative analysis of the Q-sort, we used the
open source software of Ken-Q Analysis version 1.0.6
(https:// shawn banas ick. github. io/ ken-q- analy sis) [27].
For each of the four vignettes, principal components
were extracted, and a Varimax rotation on the factors
with an Eigenvalue > 1.5 was performed subsequently as
it was deemed there was no theoretical justification for a
judgmental rotation, and we intended to reveal the range
of viewpoints favored by our participants [28].
For the qualitative analysis, a thematic extraction
was performed using NVIVO software based on the
audiotapes of the thinking-aloud during the interviews.
emes and concepts were grouped and re-grouped until
all concepts were placed under a non-overlapping header
conform the hypothesized four attributes. is was first
done in two groups (WVB and TL in group one and SS
and DVC in group two) separately, and was then triangu-
lated with the entire research team in two discussion ses-
sions. After this, SS and DVC checked all the audiotaped
interviews again to screen for fit of our thematic analysis
with what was actually conveyed during the interviews,
and to detect any missing viewpoints. e results and
interpretation of the thematic analysis were discussed at
length with a small group of interested peers at two dif-
ferent occasions. ese readback groups did not include
all the original respondents, and also contained partici-
pants who did not participate in the interviews (Addi-
tional file5).
Results
As planned, 30 physicians were interviewed, 7 in training
(IT), 8 junior staff (JS) and 15 senior staff (SS); 16 females.
Q‑sort: quantitative results
Vignette 1 (Additional file 1: appendix 1): Unrotated fac-
tor analysis yielded four factors, explaining 91% of vari-
ance (66%, 13%, 7% and 5% resp.). Factor one loaded
on to fifteen respondents (6 male, 10SS, 2JS and 3IT).
Obstructive interruption was scored as a strongly nega-
tive element by all participants, and was the major deter-
minant of this factor. Suggesting an alternative solution
by the CDSS consistently mitigated this negative attitude.
Factor two loaded on six respondents (3 male, 2IT, 2JS,
1SS). Also in this factor, obstructive interruption ranked
high. is was mitigated by transparency and pragmatic
suggestions by the CDSS, making it different from factor
1. Factor 3 loaded on five respondents (2 female, 3SS, 2JS)
and was also mainly driven by interruption, that could
however be overruled. Factor 4 loaded only to one person
(male trainee) although it explained 7% of total variance.
With a negative attitude towards obstructivity, it also
highlighted a preference for absolute advice rather than
(soft) suggestions. One statement (1A, Additional file1:
appendix 1) consistently ranked lowest (consensus state-
ment, rank -2, Z-score variance 0.0007) in all 4 factors.
is statement was obstructive, and it did not provide
transparency. Another obstructive statement without
transparency nor certainty on the advice (1H) was not
scored positively in any of the factors.
Vignette 2 Unrotated factor analysis yielded four fac-
tors explaining 92% of variance (43%, 24%, 14% and 13%
resp). Factor 1 loaded on to seven respondents (4 male,
2SS, 4JS and 1IT). Obstructive interruption was scored
strongly negative by all participants. However, when this
obstruction served to increase safety (statements 2B and
2F), it was scored very positively. Factor 2 loaded on only
three respondents (2 male, 1IT, 2SS). Within this factor,
obstructive interruption ranked high, and this was not
overruled by safety concerns. Automated presentation
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 4 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
of results was positively appreciated, especially when
presented on a smartphone. Factor 3 loaded only on one
respondent, despite explaining 18% of variance (1 female,
SS). It was not driven by obstructive interruption, regard-
less of safety concerns. Suggestions made by the CDSS
were appreciated, unless presented on a smartphone.
Factor 4 loaded to five respondents (3 male, 3SS, 1JS
and 1IT). Besides a negative attitude towards obstruc-
tive interruption, automated presentation of results was
highly appreciated when presented as a pop-up in the
EHR, but not on a smartphone. Within this vignette,
one statement (2A) ranked lowest in 3 out of 4 fac-
tors (rank -2, Z score variance 0.31). is statement was
highly obstructive, and did not provide any transparency.
Another statement (2H) was not scored negatively by any
of the factors. is statement was helpful, transparent
and enhanced safety.
Vignette 3 Unrotated factor analysis yielded four fac-
tors explaining 97% of variance (58%, 17%, 12% and 10%
resp.). Factor 1 loaded on to eight respondents (5 male,
4SS, 3JS and 1IT). Obstructive interruption was scored
strongly negative by all participants. Adding transpar-
ency to help understand the advice was appreciated by
all respondents. Factor 2 loaded on eight respondents (4
male, 6IT, 2SS), and was thus highly driven by physicians
in training, all rating obstructive interruption highly neg-
ative. Respondents all highly appreciated CDSS sugges-
tions when presented in a non-obstructive manner.
Factor 3 loaded only on two respondents, but explained
12% of variance (1 female, 2SS). It strongly preferred
safety over obstructivity (statements 3B, 3C, and 3D).
Factor 4 even loaded only to one respondent (female, SS),
but explained 15% of variance. It expressed a negative
attitude towards obstructive interruption. Statements 3E
and 3F did not get any negative ranking in any of the 4
factors. ese statements were not obstructive, provided
transparency and some degree of AI like advice. State-
ments 3A and 3D with an explicit obstructive character
did not get any positive ranking in any of the factors.
Vignette 4. Unrotated factor analysis yielded four fac-
tors explaining 87% of variance (37%, 23%, 15% and 12%
resp). Factor 1 loaded on to eight respondents (5 male,
4SS, 3JS and 1IT). Within this factor, respondents pre-
ferred a CDSS providing recommendations suggesting
some form of underlying reasoning by the system (AI-
like decision support) over a CDSS that simply worked
as a dictionary or encyclopedia (plain thesaurus func-
tions). Transparency provided as information on why a
suggestion was given by the CDSS was appreciated by all
respondents loading on this factor. However, respond-
ents favored statements in which the final decision was
left in their hands. Factor 2 loaded on six respondents (5
male, 1IT, 3SS, 1JS), and was thus highly driven by male
respondents. Personal contribution to the handling of the
case was considered very important.
Factor 3 loaded to six respondents (3 female, 1SS, 3JS,
2IT). ese respondents attributed less relevance to the
AI-like support. Factor 4 loaded only to three respond-
ents (2 female, 1SS, 2IT) and differed from factor 1 as
it appreciated statement 4A, which describes a simple
automated literature search. Statement 4G, providing a
diagnostic suggestion and management proposal while
still allowing a contribution by the physician, ranked pos-
itive in all factors.
Qsort: thematic analysis ofthinking aloud duringsurveys
Figure1 shows the subthemes related to the four themes
that were hypothesized in the Qsort, and which emerged
during the thinking aloud as derived from the field notes.
We identified four subthemes within the theme of
transparency:
(1) Possibility for discussion and argumentation regard-
ing the advice provided by the CDSS. Several
respondents indicated that it is impossible to reason
or argue with a CDSS, whereas with other experts,
peers or colleagues this interactive pro-con argumen-
tation and discussion is an essential part of how the
advice provided by the clinician comes about.
“the advice (of a CDSS) is absolute, and you can-
not argue or ask for additional explanation as you
would do with a colleague if you do not agree or
have doubts” (R3)
“calling a colleague for advice is far more easy and
effective” (R14)
“ [With people] you are able to ask why they sug-
gested something, they are able to give arguments
for their position and I am able to react to those
arguments. But here [with the AI] it ends with a
suggestion. ” (R1).
(2) Feeling in control. is was mainly associated with
a good explanation of the underlying reasoning of
the CDSS, or when different alternative options are
offered and the physician has to make a choice.
“I like it that the system provides additional expla-
nation on why it made this suggestion” (R9)
“Transparancy is good to create trust, and it pro-
vides control over the final decision” (R4)
“I feel more confident if I see that the system follows
the same reasoning as I did” (R16)
“this is good as alternative solutions are offered, but
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 5 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
Fig. 1 Thematic analysis. Inner side of the wheel: main themes (orange). More outward themes can be considered as subthemes of the more
innerbound themes. Green color indicates a positively appreciated aspect, whereas red indicates a negatively appreciated theme
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 6 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
I can make the final decision” (R11)
“e more information they get [from the AI], the
more willing the physician will be to follow [the AI’s
suggestion].” (R15)
(3) Trustworthiness and accuracy of the advice. Here
some respondents indicated that there is a need for
peer review of CDSS before it becomes implemented
in the clinical workspace.
“Not only the system, but also medicine [as a field
of study] has to have a certain level of accuracy [in
order for these AI to function properly]. ” (R1)
“the quality and performance should be tested in a
randomized trial” (R13)
“these (CDSS) should be peer reviewed, how else
would I know their performance? ” (R12)
Other respondents suggested that trustworthiness
can grow as the physician uses the system increasingly
frequently and experiences the quality of the advice
provided.
“if we start using them (CDSS), our confidence will
grow as we will better understand what triggers the
system and what makes it go astray” (R3)
(4) Potential educational role of the CDSS. It was sug-
gested by several respondents that the advice as
provided by the CDSS could be used to train more
junior staff by flagging up alternatives.
“popping up potential alternatives
with their probability can be very instructive”
For the theme of expressing (un)certainty of the sys-
tem regarding the correctness of its advice, we identified
three subthemes:
(1) Dependence on the quality of the data feeding the
system:
“even the best system (CDSS) is only
as good as the quality of the data it is using, and we
all know how many mistakes there are in medical
records” (R4).
“e quality of the data the system is using will
determine the quality of the system” (R2).
“yes, that system (a CDSS on infection control) is
very helpful, but it is time consuming to get the data
correctly in the system. ” (R17).
(2) Uncertainty and probabilistic nature of medicine
as a specialty. According to this line of thought, the
use of a CDSS is a potential pitfall, in view of the
fact that medicine is too complex to be captured by
a machine.
“In medicine it is always important
to doubt. … Our domain [medicine] is very hard to
automate, because it is difficult to reduce it to well-
defined patterns. With us there are way too many
dimensions to take into account.” (R23).
It could however also be seen as a benefit, as a
CDSS can be ideally suited to calculate and present
probabilities.
(3) e notion that trustworthiness and accuracy should
be higher for CDSS than for human decision makers.
For the theme of work flow interruption, we identi-
fied four subthemes:
e potential to increase safety was considered a
benefit,
“[e CDSS obstructing certain actions] could be
a good thing when you are dealing with cowboys,
people who think they know it best. [Obstructing
them via the CDSS] could be a good thing to pro-
tect people against themselves. ” (R20).
e other three subthemes, by contrast, were related
to potential drawbacks of the CDSS: creation of tun-
nel vision, loss of human control, alert fatigue, and
the added administrative burden (so-called ‘death by a
thousand clicks’) [7].
“Do not bother me with unimportant news” (R5).
“I do not want 199 notifications” (R1).
Respondents seemed to make a difference in their
appreciation of and trust in a CDSS according to the
task being considered as administrative vs medical.
CDSS were deemed to increase efficiency and safety
of more simple, routine administrative tasks, whereas
CDSS were considered unsuitable for medical tasks.
“[Unlike with medical decisions] I do trust the AI
when it takes administrative decisions. ose do
not look difficult to me. ” (R14).
[Administrative tasks] are trivial. ey are very
easy and should, obviously, be integrated [in the
system]. (R6).
However, the added burden on clinicians to “feed the
system” with data was also mentioned as a potential
powerful distraction from their clinical tasks.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 7 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
Discussion
In summary, the current study suggests that avoidance of
disruptive workflows and transparency of the underlying
decision processes are important considerations when
developing clinical decision support systems (CDSS)
incorporated in Electronic Health Record (EHR) and
Computerized Provider Order Entry systems (CPOE) sys-
tems. We conclude that not accounting for these aspects
reduces the likelihood physicians will experience the
CDSS as a useful tool, induces frustration and decreases
job satisfaction. e study also found that obstructive
interference is only acceptable when it increases safety, it
is adjustable to user expertise level and/or allows devia-
tions when explained by the end-user on why a deviation
is desirable in this case. e latter can also contribute to
education or improve the system’s decision making pro-
cess. To the respondents in this study, transparency and
explainability seem to boil down to providing sufficient
clarification on the underlying determining factors and
flowpaths within the CDSS, so that these can be cross-
checked to their own knowledge, beliefs and convictions.
All current integrated CDSS can, next to offering an
advice on demand of the user, also produce interruptive
messages, i.e. messages that pop up automatically and
interrupt the workflow planned by the user [26]. is can
be intended to either improve safety, indicate logistical/
practical issues or to guide the user to a change in behav-
ior towards the solution presumed to be most optimal by
the CDSS [5]. Such messages can be non-obstructive if
the user can bypass it so she can continue the planned
workflow, or obstructive when the message cannot be
bypassed, and the user needs to change her originally
planned workflow. Nearly all respondents in our Qsort
study were strongly averse towards such interruptive
messages, especially when also obstructive. Interruptive
messaging has best been studied in drug-prescription
applications [26]. Over half of the studies report a ben-
eficial effect on prescriber behavior, but the impact on
patient related outcomes is less clear [26]. Of note, such
alerts have a rather high override rate [29, 30]. Obstruc-
tive alerts may also have unintended negative effects,
such as a delay in prescribing off-label medication [31].
Our study indicates that interruptive messages are
accepted by physicians if they add to patient safety, e.g.
by blocking undesirable actions. Some indicated that
obstructive messages can be allowed provided they allow
overruling if the user has sufficient expertise (seniority)
or when an explanation on why the decision was over-
ruled is provided. Several studies suggest that an option
to adapt alerts to local circumstances, expertise and
workflows has a substantial positive impact on perceived
user friendliness and task behavior [32, 33]. Such an
approach allows to valorize the clinical expertise of the
healthcare worker and restores their feeling of control. It
also avoids the frustration of being forced to make deci-
sions perceived to be inappropriate, for example in excep-
tional cases. is is an important point also for liability,
as lay people may negatively judge physicians when they
do not follow AI advice, except in non-standard settings
[34]. In addition, some of our respondents suggested that
explanation by experts on why they overruled the sugges-
tions of the CDSS provide opportunities to improve the
system.
e problem of alert fatigue is a well-established down-
side of interruptive messaging in CDSS [5, 14]. Never-
theless, our respondents only hinted at it indirectly as
a theoretical problem. is can be due to the fact the
current EHR system in our hospital does not already
have a CDSS, so users were not yet confronted in their
daily practice with this aspect. Some suggested to pri-
oritize which messages should pop-up or not to avoid
alert fatigue. Such tier systems reduce alert fatigue, and
result in better uptake of the provided guidance [35]. It
adds another argument for substantial local adaptability
of EHR systems [4]. Automation complacency, operators
not conducting sufficient (manual) checks themselves
due to excessive thrust in the automated system, are the
opposite of alert fatigue [21]. Respondents indicated peo-
ple are most sensible to miss incorrect diagnoses of the
CDSS if it is reliable most of the time, when the case and/
or the advice appear as “near normal” and when they are
overloaded with alternative tasks or tired. In relation to
this, there was concern amongst the respondents that
implementation of CDSS might lead to deskilling, or to
delegation of tasks to staff with lower education, result-
ing in dangerous situations when the CDSS fails or when
human interference is needed, such as exceptional cases.
Different facets of tunnel vision as another type of
CDSS rigidity were brought up. First, once a suggestion is
provided by the CDSS, this cannot simply be ignored. e
physician needs to consider the suggestion, and reflect
on it, at least internally. is might lead to unnecessary
additional tests and tasks being done to be on the safe
side. Second, there is little possibility to introduce a scale
of gray in CDSS systems. ird, respondents identified
the risk of anchoring bias [36, 37], i.e. looking for facts
that confirm the suggestion of the CDSS whilst ignoring
facts that don’t fit. Of note, physicians often fail to ignore
inaccurate advice, regardless of it coming from another
human or from an AI [38]. is effect is mitigated by the
expertise level of the user [20], which would again sup-
port approaches in which suggestions of the CDSS can be
overruled by more senior staff.
Some respondents indicated the CDSS might come up
with useful suggestions they would not have considered
themselves, thereby breaking their own internal mental
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 8 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
tunnel vision [39]. Using a CDSS might also diminish
confirmation bias, the process by which physicians
believe data that support their own beliefs and dismiss
those that refute their own intuition [39].
Our data also suggest that the type of task the CDSS
takes on makes an impact on the acceptance of the gener-
ated advice. A CDSS is substantially better accepted for
tasks considered to be administrative or easy, whereas
there is much more reluctance to use CDSS for tasks
considered to be more intellectual or complex. Adminis-
trative tasks can be thought of as CPOEs, or electronic
medication prescription. Some authors distinguish sys-
tems only used for these tasks from true CDSS, whereas
others don’t [5]. It might be that CDSS support in admin-
istrative tasks is more transparent as these are often rule
based, whereas CDSS systems used for clinical/intellec-
tual tasks tend be more based on machine learning and
are thus less transparent. In general, implementation
of CDSS for administrative or simple tasks is consid-
ered to be promising to increase efficiency [40]. Some
raise concern this increased efficiency will be used to
increase workload or reduce contact time with patients
[10]. Others mention that the gain in efficiency is mostly
for administrative and logistical departments, whereas
the actual data input that enables this efficiency gain is
an extra burden for the clinicians [4, 5]. It would also be
interesting to know whether appreciation of CDSS is dif-
ferent for recommendations concerning primarily diag-
nostic tasks vs intervention and management oriented
tasks. Unfortunately, the design of the current vignettes
does not allow for that question to be answered, as this
aspect was not elicited by any of the cases, hence insuf-
ficient data are available to draw any firm conclusions on
this intriguing question.
Nearly all respondents highlighted that uncertainty is
an essential aspect of clinical practice [41]. Accordingly,
they expect the CDSS to provide a representation of how
certain it is the provided advice is correct. ere appears
to be no clear preference for verbal vs numeric indica-
tion of that certainty amongst our respondents. Some
argue that a verbal indication (might, likely, probably) is
vague and not very useful. Most agreed that providing
no grading of certainty automatically suggested that the
CDSS was absolutely sure about the advice. ere was
also a general conviction amongst our respondents that
the standards for certainty of advice should be higher for
CDSS than for human providers. is request can either
be explained by the absence of opportunity to discuss the
provided advice, or by a lack of transparency.
Transparency and explainability are key issues in
CDSS [42, 43]. ere is however a lot of debate on what
exactly is understood by these constructs, and what is
essential to achieve them [44]. For our respondents,
understanding the determining factors or the flow of
arguments of the CDSS was considered crucial to cre-
ate confidence in the advice. Trustworthiness was con-
sidered to be higher if the accuracy of the CDSS was
validated in a clinical trial.
Providing a thesaurus with exact meaning of the ter-
minology used by the CDSS was also considered nec-
essary to enhance semantic transparency [45]. Using
different concepts for the same disease, or different def-
initions for the same concept results in confusion e.g.
in what exactly is predicted by prediction models [46].
Remarkably, merely providing an explanation was suf-
ficient for most respondents, irrespective of the epis-
temic correctness of such explanation, as it provided
a means of control. Others stressed that such explana-
tions could be educative as they allowed “learning by
example”.
In conclusion, acceptance of a CDSS incorporated in
EHR/CPOE is improved by ensuring a swift integration
into the workflow, with a system of options tailored to
the expertise of the users to overrule the system. Trans-
parency on the underlying processes is essential to gain
trust. is perceived trust can be further enhanced by
providing proof of the accuracy of the CDSS, either by
personal experience or in randomized trials.
Supplementary Information
The online version contains supplementary material available at https:// doi.
org/ 10. 1186/ s12911- 022- 01933-3.
Additional le1. Qsort vignettes.
Additional le2. Factor Scores with Corresponding Ranks.
Additional le3. Factor Matrix with Defining Sorts Flagged.
Additional le4. Factor Q-sort Values for Statements sorted by Consen-
sus vs. Disagreement.
Additional le5. Factor Q-sort Values for Statements sorted by Consen-
sus vs. Disagreement.
Acknowledgements
Not applicable
Author contributions
WVB has conceived the concept of the study, helped design the vignettes,
performed all the Qsorts, did the first thematic analysis together with TL, took
part in the triangulation sessions and has written the draft version of the
paper. He also compiled the final version based on comments and sugges-
tions and the discussion round. DVC did the first thematic analysis together
with SS, took part in the triangulation sessions, and re-analysed the thematic
analysis in the second round. He also reviewed and commented on the draft
paper. JDC contributed to the conceptualization of the project, helped design
the vignettes by contributing expertise in CDSS, took part in the triangulation
sessions and reviewed and commented on the draft version of the paper. TL
did the first thematic analysis together with WVB, took part in the triangula-
tion sessions, conceptualized the figure, and reviewed and commented on
the draft paper. She also organized the logistics of the Qsort. SS contributed
to the concept of the study, helped in the design of the vignettes. She did
the first thematic analysis with DVC, and re-analysed the thematic analysis in
the second round. She took part in the triangulation sessions. She provided
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 9 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
several rounds of comments and suggestions on the draft versions of the
paper.All authors read and approved the final manuscript
Funding
This work is part of a larger project funded by Fonds Wetenschappelijk Onder-
zoek (FWO): FWO.OPR.2019.0045.01.
Availability of data and materials
All data are supplied as supplementary materials. The precise content of the
vignettes is available as appendix 1: Qsort vignettes; The data and analysis of
the Qsort are available as dataset 1–4: Qsort analysis.
Declarations
Ethics approval and consent to participate
This study was approved by the Ethical Committee of the University Hospital
Gent (number BC 08294). All participants provided written informed consent.
All research and work was performed in accordance with the relevant guide-
lines and regulations. We adhered to the Consolidated criteria for reporting
qualitative research (COREQ) regulations throughout the process.
Consent for publication
Not applicable.
Competing interests
None of the authors has competing interests with regard to the content of
this publication. the Authors therefore declare no competing financial or non-
financial interests.
Author details
1 Department of Nephrology, Ghent University Hospital, Corneel Heymanslaan
10, 0K12IA, Ghent, Belgium. 2 Consortium for Justifiable Digital Healthcare,
Ghent University Hospital, Ghent, Belgium. 3 Department of Philosophy
and Moral Sciences, Bioethics Institute Ghent, Ghent University, Ghent,
Belgium. 4 Department of Intensive Care Medicine, Ghent University Hospital,
Ghent, Belgium.
Received: 30 January 2022 Accepted: 7 July 2022
References
1. Armstrong KA, Metlay JP. Annals clinical decision making: communicating
risk and engaging patients in shared decision making. Ann Intern Med.
2020;172:688–92. https:// doi. org/ 10. 7326/ M19- 3495.
2. van de Sande D, van Genderen ME, Huiskens J, Gommers D, van Bom-
mel J. Moving from bytes to bedside: a systematic review on the use
of artificial intelligence in the intensive care unit. Intensive Care Med.
2021;47:750–60. https:// doi. org/ 10. 1007/ s00134- 021- 06446-7.
3. Beam AL, Manrai AK, Ghassemi M. Challenges to the reproducibility
of machine learning models in health care. JAMA J Am Med Assoc.
2020;323:305–6. https:// doi. org/ 10. 1001/ jama. 2019. 20866.
4. Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physi-
cian medication order entry in hospitalized patients–a systematic review.
Int J Med Inform. 2008;77:365–76. https:// doi. org/ 10. 1016/j. ijmed inf. 2007.
10. 001.
5. Black AD, et al. The impact of eHealth on the quality and safety of health
care: a systematic overview. PLoS Med. 2011;8:e1000387. https:// doi. org/
10. 1371/ journ al. pmed. 10003 87.
6. Topol EJ. High-performance medicine: the convergence of human and
artificial intelligence. Nat Med. 2019;25:44–56. https:// doi. org/ 10. 1038/
s41591- 018- 0300-7.
7. Schulte F, Fry, E. Death by a thousand clicks: where electronic health
records went wrong. Fortune, March 18, 2019.
8. Loper PL Jr. The electronic health record and acquired physician autism.
JAMA Pediatr. 2018;172:1009. https:// doi. org/ 10. 1001/ jamap ediat rics.
2018. 2080.
9. Brocklehurst P, et al. Computerised interpretation of the fetal heart rate
during labour: a randomised controlled trial (INFANT). Health Technol
Assess. 2018;22:1–186. https:// doi. org/ 10. 3310/ hta22 090.
10. Toll E. A piece of my mind. The cost of technology. JAMA J Am Med
Assoc. 2012;307:2497–8. https:// doi. org/ 10. 1001/ jama. 2012. 4946.
11. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts
in computerized physician order entry. J Am Med Inform Assoc.
2006;13:138–47. https:// doi. org/ 10. 1197/ jamia. M1809.
12. Wong A, et al. Prospective evaluation of medication-related clinical
decision support over-rides in the intensive care unit. BMJ Qual Saf.
2018;27:718–24. https:// doi. org/ 10. 1136/ bmjqs- 2017- 007531.
13. Poly TN, Islam MM, Yang HC, Li YJ. Appropriateness of overridden alerts in
computerized physician order entry: systematic review. JMIR Med Inform.
2020;8:e15653. https:// doi. org/ 10. 2196/ 15653.
14. Ancker JS, et al. Effects of workload, work complexity, and repeated alerts
on alert fatigue in a clinical decision support system. BMC Med Inform
Decis Mak. 2017;17:36. https:// doi. org/ 10. 1186/ s12911- 017- 0430-8.
15. Nanji KC, et al. Medication-related clinical decision support alert overrides
in inpatients. J Am Med Inform Assoc. 2018;25:476–81. https:// doi. org/ 10.
1093/ jamia/ ocx115.
16. Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may
be reduced via interaction design and clinical role tailoring: a systematic
review. J Am Med Inform Assoc. 2019;26:1141–9. https:// doi. org/ 10. 1093/
jamia/ ocz095.
17. Taichman DB, Williams SV, Laine C. Clinical decision making: nurturing
our core skills. Ann Intern Med. 2020;172:627. https:// doi. org/ 10. 7326/
M20- 1423.
18. Trinkley KE, et al. Applying clinical decision support design best practices
with the practical robust implementation and sustainability model
versus reliance on commercially available clinical decision support tools:
randomized controlled trial. JMIR Med Inform. 2021;9:e24359. https:// doi.
org/ 10. 2196/ 24359.
19. Gold R, et al. “I didn’t know you could do that”: a pilot assessment of
EHR optimization training. ACI Open. 2021;5:e27–35. https:// doi. org/ 10.
1055/s- 0041- 17310 05.
20. Gaube S, et al. Do as AI say: susceptibility in deployment of clini-
cal decision-aids. NPJ Digit Med. 2021;4:31. https:// doi. org/ 10. 1038/
s41746- 021- 00385-9.
21. Parasuraman R, Manzey DH. Complacency and bias in human use of
automation: an attentional integration. Hum Factors. 2010;52:381–410.
https:// doi. org/ 10. 1177/ 00187 20810 376055.
22. Akhtar-Danesh N, Baumann A, Cordingley L. Q-methodology in nursing
research: a promising method for the study of subjectivity. West J Nurs
Res. 2008;30:759–73. https:// doi. org/ 10. 1177/ 01939 45907 312979.
23. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative
research (COREQ): a 32-item checklist for interviews and focus groups.
Int J Qual Health Care J Int Soc Qual Health Care ISQua. 2007;19:349–57.
https:// doi. org/ 10. 1093/ intqhc/ mzm042.
24. Dennis KE. Q methodology: relevance and application to nursing
research. ANS Adv Nurs Sci. 1986;8:6–17. https:// doi. org/ 10. 1097/ 00012
272- 19860 4000- 00003.
25. Bachmann LM, et al. Vignette studies of medical choice and judgement
to study caregivers’ medical decision behaviour: systematic review. BMC
Med Res Methodol. 2008;8:50. https:// doi. org/ 10. 1186/ 1471- 2288-8- 50.
26. Page N, Baysari MT, Westbrook JI. A systematic review of the effectiveness
of interruptive medication prescribing alerts in hospital CPOE systems to
change prescriber behavior and improve patient safety. Int J Med Inform.
2017;105:22–30. https:// doi. org/ 10. 1016/j. ijmed inf. 2017. 05. 011.
27. Banasick S. Ken-Q Analysis (Version 1.0.6) [Software]. (2019). Available
from https:// shawn banas ick. github. io/ ken-q- analy sis/. doi:10.5281/
zenodo.1300201
28. Watts SSP. Doing Q methodology: theory method and interpretation.
Thousand Oaks: Sage Publications; 2012.
29. Beeler PE, Orav EJ, Seger DL, Dykes PC, Bates DW. Provider variation in
responses to warnings: do the same providers run stop signs repeatedly?
J Am Med Inform Assoc. 2016;23:e93-98. https:// doi. org/ 10. 1093/ jamia/
ocv117.
30. Lin CP, et al. Evaluating clinical decision support systems: monitor-
ing CPOE order check override rates in the Department of veterans
affairs’ computerized patient record system. J Am Med Inform Assoc.
2008;15:620–6. https:// doi. org/ 10. 1197/ jamia. M2453.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 10 of 10
VanBiesenetal. BMC Medical Informatics and Decision Making (2022) 22:185
fast, convenient online submission
thorough peer review by experienced researchers in your field
rapid publication on acceptance
support for research data, including large and complex data types
gold Open Access which fosters wider collaboration and increased citations
maximum visibility for your research: over 100M website views per year
At BMC, research is always in progress.
Learn more biomedcentral.com/submissions
Ready to submit your research
Ready to submit your research
? Choose BMC and benefit from:
? Choose BMC and benefit from:
31. Strom BL, et al. Unintended effects of a computerized physician order
entry nearly hard-stop alert to prevent a drug interaction: a randomized
controlled trial. Arch Intern Med. 2010;170:1578–83. https:// doi. org/ 10.
1001/ archi ntern med. 2010. 324.
32. Payne TH, et al. Recommendations to improve the usability of drug-drug
interaction clinical decision support alerts. J Am Med Inform Assoc.
2015;22:1243–50. https:// doi. org/ 10. 1093/ jamia/ ocv011.
33. Horsky J, et al. Interface design principles for usable decision support: a
targeted review of best practices for clinical prescribing interventions. J
Biomed Inform. 2012;45:1202–16. https:// doi. org/ 10. 1016/j. jbi. 2012. 09.
002.
34. Tobia K, Nielsen A, Stremitzer A. When does physician use of AI increase
liability? J Nucl Med. 2021;62:17–21. https:// doi. org/ 10. 2967/ jnumed. 120.
256032.
35. Paterno MD, et al. Tiering drug-drug interaction alerts by severity
increases compliance rates. J Am Med Inform Assoc. 2009;16:40–6.
https:// doi. org/ 10. 1197/ jamia. M2808.
36. Lee CS, Nagy PG, Weaver SJ, Newman-Toker DE. Cognitive and system
factors contributing to diagnostic errors in radiology. AJR Am J Roent-
genol. 2013;201:611–7. https:// doi. org/ 10. 2214/ AJR. 12. 10375.
37. Kahneman D, Slovic P, Tversky A. Judgment under uncertainty: heuristics
and biases. Cambridge: Cambridge University Press; 1982.
38. Tsai TL, Fridsma DB, Gatti G. Computer decision support as a source of
interpretation error: the case of electrocardiograms. J Am Med Inform
Assoc. 2003;10:478–83. https:// doi. org/ 10. 1197/ jamia. M1279.
39. Restrepo D, Armstrong KA, Metlay JP. Avoiding cognitive errors in clinical
decision making. Ann Intern Med. 2020;173:679. https:// doi. org/ 10. 7326/
L20- 1060.
40. Topol E. Preparing the healthcare workforce to deliver the digital future.
https:// topol. hee. nhs. uk/ wp- conte nt/ uploa ds/ HEE- Topol- Review- 2019.
pdf. (2019)
41. Simpkin AL, Schwartzstein RM. Tolerating uncertainty - the next medical
revolution? N Engl J Med. 2016;375:1713–5. https:// doi. org/ 10. 1056/
NEJMp 16064 02.
42. Rudin C. Stop explaining black box machine learning models for high
stake decisions and use interpretable models instead. Nat Mach Intell.
2019;1:206–215.
43. Langer M, Oster D, Speith T, Hermanns H, Kästner H, Schmidt E, Sesing A,
Baum K. What do we want from explainabe artificial intelligence: a stake-
holder perspective on XAI and a conceptual model guiding interdiscipli-
nary XAI research. Artif Intell. 2021;296:1-58.
44. London AJ. Artificial intelligence and black-box medical decisions: accu-
racy versus explainability. Hastings Cent Rep. 2019;49:15–21. https:// doi.
org/ 10. 1002/ hast. 973.
45. Pierce R, Sterckx S, Van Biesen W. A riddle, wrapped in a mystery, inside
an enigma: how semantic black boxes and opaque artificial intelligence
confuse medical decision-making. Bioethics. 2021. https:// doi. org/ 10.
1111/ bioe. 12924.
46. Van Acker P, et al. Risk prediction models for acute kidney injury in adults:
an overview of systematic reviews. PLoS One. 2021;16:e0248899. https://
doi. org/ 10. 1371/ journ al. pone. 02488 99.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub-
lished maps and institutional affiliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The use of artificial intelligence (AI) in healthcare comes with opportunities but also numerous challenges. A specific challenge that remains underexplored is the lack of clear and distinct definitions of the concepts used in and/or produced by these algorithms, and how their real world meaning is translated into machine language and vice versa, how their output is understood by the end user. This “semantic” black box adds to the “mathematical” black box present in many AI systems in which the underlying “reasoning” process is often opaque. In this way, whereas it is often claimed that the use of AI in medical applications will deliver “objective” information, the true relevance or meaning to the end‐user is frequently obscured. This is highly problematic as AI devices are used not only for diagnostic and decision support by healthcare professionals, but also can be used to deliver information to patients, for example to create visual aids for use in shared decision‐making. This paper provides an examination of the range and extent of this problem and its implications, on the basis of cases from the field of intensive care nephrology. We explore how the problematic terminology used in human communication about the detection, diagnosis, treatment, and prognosis of concepts of intensive care nephrology becomes a much more complicated affair when deployed in the form of algorithmic automation, with implications extending throughout clinical care, affecting norms and practices long considered fundamental to good clinical care.
Article
Full-text available
Background Informatics tools within electronic health records (EHRs)—for example, data rosters and clinical reminders—can help disseminate care guidelines into clinical practice. Such tools' adoption varies widely, however, possibly because many primary care providers receive minimal training in even basic EHR functions. Objectives This mixed-methods evaluation of a pilot training program sought to identify factors to consider when providing EHR use optimization training in community health centers (CHCs) as a step toward supporting CHC providers' adoption of EHR tools. Methods In spring 2018, we offered 10 CHCs a 2-day, 16-hour training in EHR use optimization, provided by clinician trainers, and customized to each CHC's needs. We surveyed trainees pre- and immediately post-training and again 3 months later. We conducted post-training interviews with selected clinic staff, and conducted a focus group with the trainers, to assess satisfaction with the training, and perceptions of how it impacted subsequent EHR use. Results Six CHCs accepted and received the training; 122 clinic staff members registered to attend, and most who completed the post-training survey reported high satisfaction. Three months post-training, 80% of survey respondents said the training had changed their daily EHR use somewhat or significantly. Conclusion Factors to consider when planning EHR use optimization training in CHCs include: CHCs may face barriers to taking part in such training; it may be necessary to customize training to a given clinic's needs and to different trainees' clinic roles; identifying trainees' skill level a priori would help but is challenging; in-person training may be preferable; and inclusion of a practice coach may be helpful. Additional research is needed to identify how to provide such training most effectively.
Article
Full-text available
Abstract Purpose: Due to the increasing demand for intensive care unit (ICU) treatment, and to improve quality and efficiency of care, there is a need for adequate and efficient clinical decision-making. The advancement of artificial intelligence (AI) technologies has resulted in the development of prediction models, which might aid clinical decision-making. This systematic review seeks to give a contemporary overview of the current maturity of AI in the ICU, the research methods behind these studies, and the risk of bias in these studies. Methods: A systematic search was conducted in Embase, Medline, Web of Science Core Collection and Cochrane Central Register of Controlled Trials databases to identify eligible studies. Studies using AI to analyze ICU data were considered eligible. Specifically, the study design, study aim, dataset size, level of validation, level of readiness, and the outcomes of clinical trials were extracted. Risk of bias in individual studies was evaluated by the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Results: Out of 6455 studies identified through literature search, 494 were included. The most common study design was retrospective [476 studies (96.4% of all studies)] followed by prospective observational [8 (1.6%)] and clinical [10 (2%)] trials. 378 (80.9%) retrospective studies were classified as high risk of bias. No studies were identified that reported on the outcome evaluation of an AI model integrated in routine clinical practice. Conclusion: The vast majority of developed ICU-AI models remain within the testing and prototyping environment; only a handful were actually evaluated in clinical practice. A uniform and structured approach can support the development, safe delivery, and implementation of AI to determine clinical benefit in the ICU. Keywords: Artificial intelligence, Machine learning, Intensive care unit, Clinical trials
Article
Full-text available
Background The incidence of Acute Kidney Injury (AKI) and its human and economic cost is increasing steadily. One way to reduce the burden associated with AKI is to prevent the event altogether. An important step in prevention lies in AKI risk prediction. Due to the increasing number of available risk prediction models (RPMs) clinicians need to be able to rely on systematic reviews (SRs) to provide an objective assessment on which RPM can be used in a specific setting. Our aim was to assess the quality of SRs of RPMs in AKI. Methods The protocol for this overview was registered in PROSPERO. MEDLINE and Embase were searched for SRs of RPMs of AKI in any setting from 2003 till August 2020. We used the ROBIS tool to assess the methodological quality of the retrieved SRs. Results Eight SRs were retrieved. All studies were assessed as being at high risk for bias using the ROBIS tool. Eight reviews had a high risk of bias in study eligibility criteria (domain 1), five for study identification and selection (domain 2), seven for data collection and appraisal (domain 3) and seven for synthesis and findings (domain 4). Five reviews were scored at high risk of bias across all four domains. Risk of bias assessment with a formal risk of bias tool was only performed in five reviews. Primary studies were heterogeneous and used a wide range of AKI definitions. Only 19 unique RPM were externally validated, of which 11 had only 1 external validation report. Conclusion The methodological quality of SRs of RPMs of AKI is inconsistent. Most SRs lack a formal risk of bias assessment. SRs ought to adhere to certain standard quality criteria so that clinicians can rely on them to select a RPM for use in an individual patient. Trial registration PROSPERO registration number is CRD 42020204236 , available at https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=204236 .
Article
Full-text available
Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.
Article
Full-text available
Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these stakeholders' desiderata) in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainability of artificial systems and reviews their desiderata. We provide a model that explicitly spells out the main concepts and relations necessary to consider and investigate when evaluating, adjusting, choosing, and developing explainability approaches that aim to satisfy stakeholders' desiderata. This model can serve researchers from the variety of different disciplines involved in XAI as a common ground. It emphasizes where there is interdisciplinary potential in the evaluation and the development of explainability approaches.
Article
Full-text available
Rationale: An increasing number of automated and artificially intelligent (AI) systems make medical treatment recommendations, including "personalized" recommendations, which can deviate from standard care. Legal scholars argue that following such nonstandard treatment recommendations will increase liability in medical malpractice, undermining the use of potentially beneficial medical AI. However, such liability depends in part on lay judgments by jurors: When physicians use AI systems, in which circumstances would jurors hold physicians liable? Methods: To determine potential jurors' judgments of liability, we conducted an online experimental study of a nationally representative sample of 2,000 U.S. adults. Each participant read one of four scenarios in which an AI system provides a treatment recommendation to a physician. The scenarios varied the AI recommendation (standard or nonstandard care) and the physician's decision (to accept or reject that recommendation). Subsequently, the physician's decision caused a harm. Participants then assessed the physician's liability. Results: Our results indicate that physicians who receive advice from an AI system to provide standard care can reduce the risk of liability by accepting, rather than rejecting, that advice, all else equal. However, when an AI system recommends nonstandard care, there is no similar shielding effect of rejecting that advice and so providing standard care. Conclusion: The tort law system is unlikely to undermine the use of AI precision medicine tools and may even encourage the use of these tools.
Article
Background Limited consideration of clinical decision support (CDS) design best practices, such as a user-centered design, is often cited as a key barrier to CDS adoption and effectiveness. The application of CDS best practices is resource intensive; thus, institutions often rely on commercially available CDS tools that are created to meet the generalized needs of many institutions and are not user centered. Beyond resource availability, insufficient guidance on how to address key aspects of implementation, such as contextual factors, may also limit the application of CDS best practices. An implementation science (IS) framework could provide needed guidance and increase the reproducibility of CDS implementations. Objective This study aims to compare the effectiveness of an enhanced CDS tool informed by CDS best practices and an IS framework with a generic, commercially available CDS tool. Methods We conducted an explanatory sequential mixed methods study. An IS-enhanced and commercial CDS alert were compared in a cluster randomized trial across 28 primary care clinics. Both alerts aimed to improve beta-blocker prescribing for heart failure. The enhanced alert was informed by CDS best practices and the Practical, Robust, Implementation, and Sustainability Model (PRISM) IS framework, whereas the commercial alert followed vendor-supplied specifications. Following PRISM, the enhanced alert was informed by iterative, multilevel stakeholder input and the dynamic interactions of the internal and external environment. Outcomes aligned with PRISM’s evaluation measures, including patient reach, clinician adoption, and changes in prescribing behavior. Clinicians exposed to each alert were interviewed to identify design features that might influence adoption. The interviews were analyzed using a thematic approach. Results Between March 15 and August 23, 2019, the enhanced alert fired for 61 patients (106 alerts, 87 clinicians) and the commercial alert fired for 26 patients (59 alerts, 31 clinicians). The adoption and effectiveness of the enhanced alert were significantly higher than those of the commercial alert (62% vs 29% alerts adopted, P<.001; 14% vs 0% changed prescribing, P=.006). Of the 21 clinicians interviewed, most stated that they preferred the enhanced alert. Conclusions The results of this study suggest that applying CDS best practices with an IS framework to create CDS tools improves implementation success compared with a commercially available tool. Trial Registration ClinicalTrials.gov NCT04028557; http://clinicaltrials.gov/ct2/show/NCT04028557