Categorizing the unintended sociotechnical consequences of computerized provider order entry.

Categ
of com
Joan S. A
James D
a Oregon He
b Northwest
c Providence
a r t i c
Article histor
Received 7 M
Accepted 11
Keywords:
Hospital inf
Online syste
Medical reco
Computeriz
Computeriz
1. Int
Use of com
widespread
the pioneer
lessons ge
issues invo
component
nical issues
tems imple
tended con
∗ Correspon
University, 3
E-mail a
1386-5056/$
doi:10.1016/i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 0 6 ) xxx–xxx
journa l homepage: www. int l .e lsev ierhea l th .com/ journa ls / i jmi
orizing the unintended sociotechnical consequences
puterized provider order entry
sha,∗, Dean F. Sittigb, Richard H. Dykstrab, Kenneth Guapponea,
. Carpenter c, Veena Seshadria
alth & Science University, Portland, OR, United states
Permanente, P.C., Portland, OR, United states
Health System, Portland, OR, United states
l e i n f o
y:
arch 2006
May 2006
a b s t r a c t
Objective: To describe the kinds of unintended consequences related to the implementation
of computerized provider order entry (CPOE) in the outpatient setting.
Design: Ethnographic and interview data were collected by an interdisciplinary team over a
7 month period at four clinics.
Measurements: Instances of unintended consequences were categorized using an expanded
Diffusion of Innovations theory framework.ormation systems
ms
rds systems
ed
ed physician order entry
Results: The framework was clarified
able unintended consequences, and t
many consequences that are not clear
depending on one’s perspective. The
concerns and issues related to alerts
reimplementations.
Conclusion: Consequences of impleme
plex. The expanded Diffusion of Innov
such consequences.
roduction
puterized provider order entry (CPOE) is not yet
[1], but valuable lessons can be learned from
ing organizations that have adopted CPOE. These
nerally involve sociotechnical issues, defined as
lving the interplay of organizational and technical
s of a system. The more we know about sociotech-
, the better prepared we can be for the clinical sys-
mentation process. For example, there are unin-
sequences resulting from CPOE implementation
ding author at: Department of Medical Informatics & Clinical Epidemio
181 SW Sam Jackson Park Rd., Portland, OR 97201-3098, United States.
ddress: ash@ohsu.edu (J.S. Ash).
[2], and kno
help avoid
CPOE, in
a provider
enter medi
for an inte
given by a
used in th
tion, but to
decision su
the receivin
– see front matter © 2006 Elsevier Ireland Ltd. All rights reserved.
j.ijmedinf.2006.05.017and expanded. There are both desirable and undesir-
hey can be either direct or indirect, but there are also
ly either desirable or undesirable or may even be both,
undesirable consequences include error and security
, workflow, ergonomics, interpersonal relations, and
nting and reimplementing clinical systems are com-
ations theory framework is a useful tool for analyzing
© 2006 Elsevier Ireland Ltd. All rights reserved.
logy, School of Medicine, Oregon Health and Science
Tel.: +1 503 494 4540; fax: +1 503 494 4551.
wledge about these consequences can potentially
them in the future.
the narrow sense, is defined as a process in which
who has ordering authority uses a computer to
cal orders directly. The process eliminates the need
rmediary to respond to written or verbal orders
provider. An expanded definition of CPOE was
is study, to include not only this narrow defini-
also encompass accompanying processes such as
pport, documentation, and order delivery, both to
g department and the patient. CPOE is receiving
IJB-2262; No. of Pages 7

2 a t i
growing wo
that it incr
[3–8].
Diffusio
work for t
Rogers as “
cated throu
bers of a so
idea, pract
ual, a grou
ory has be
great many
of innovati
been show
the diffusio
(is it better
consistent
plexity (is
you experi
ible to oth
sion is com
can be do
mass comm
the third e
vidual leve
adopters, e
laggards. A
the rate of a
fourth mai
of individu
diffuses [9]
The bas
communic
ied and va
edition of
researchers
ory and ext
technology
toward the
the late aut
changing t
such as re
of spacial d
[9, p. xiv].
Fig. 1 – Dif
loop.
1 de
e ad
dopt
Rog
d asp
s a c
irect
ende
cipa
uen
dies
focu
sear
suc
vely
ory)
chno
aim
uen
on s
e nu
he us
estig
tive
. We
an be
coni n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m
rldwide interest because there is some evidence
eases medical safety by reducing medical errors
n of Innovations (DOI) theory provided the frame-
his study. Diffusion has been defined by Everett
the process by which an innovation is communi-
gh certain channels over time among the mem-
cial system” and an innovation is defined as “an
ice, or objective perceived as new by an individ-
p, or an organization [9, p. 5].” The basic the-
en tested and validated in numerous studies. A
researchers have focused on the characteristics
ons. DOI theory outlines five attributes which have
n in many studies to be important in assessing
n potential of an innovation: relative advantage
than the idea it supercedes?); compatibility (is it
with existing values and needs of users?); com-
it hard to understand and use?); trialability (can
ment with it?); and observability (are results vis-
ers?). Another factor that is important in diffu-
munication, the process of sharing ideas, which
ne through a variety of channels ranging from
unications to face-to-face interactions. Time is
lement of the diffusion process, and at the indi-
l people can be categorized as innovators, early
arly majority adopters, late majority adopters, or
t the diffusion process level, time is a measure of
Fig.
with th
been a
ing to
studie
outline
able; d
“unint
unanti
conseq
Stu
ning to
[10]. Re
even if
effecti
tradict
was te
The
conseq
diffusi
sure th
that “t
for inv
qualita
theory
they c
subtledoption or spread of use through a population. The
n element of diffusion is the social system, a set
als or organizations through which the innovation
.
ic DOI theory elements of innovation attributes,
ation, time, and the social system have been stud-
lidated in over 5000 publications since the first
Rogers’ book was published [9, p. xviii]. Diffusion
have added to our knowledge of the basic the-
ended and enriched it over the years. Information
diffusion research was of special interest to Rogers
end of his career. In the last edition of the book,
hor stressed that information technology “may be
he diffusion process in certain fundamental ways
moving, or at least greatly diminishing, the role
istance in who talks to whom about a new idea”
fusion of Innovations Model with consequences
not know o
raphers ca
subjects. W
gatheringd
patient clin
sequences
and then w
scheme.
2. Me
2.1. Pri
Prior resea
described h
successfull
location. In
tors, inform
user staff
in a wide v
including p
gency depa
surgical un
theory app
scriptswas
then met to
from each n
ing analysi
reported el
analysis of
wide varie
menting CPc s x x x ( 2 0 0 6 ) xxx–xxx
picts the authors’ interpretation of the DOI model,
dition of “consequences.” Once an innovation has
ed, there are inevitable consequences, but, accord-
ers, the consequences of adoption are the least
ect of the innovation diffusion process. DOI theory
lassification of consequences: desirable or undesir-
or indirect; anticipated or unanticipated. The term
d” connotes consequences that are primarily both
ted and undesirable. Rogers has described types of
ces in words, but not graphically.
in the information technology literature are begin-
s on post-adoption behaviors such as continuance
chers have also recognized that complex systems,
cessfully adopted by some definitions, may not be
used and that “unanticipated (and sometimes con-
changes may result from an implementation that
logically labeled as successful.” [11, p. 1].
of this study is to describe kinds of unintended
ces related to CPOE in the outpatient setting. Many
tudies have been quantitative because they mea-
mbers of adopters over time, but Rogers, stating
ual survey research methods may be inappropriate
ating consequences,” [9, p. 470] has recommended
methods for investigating this aspect of diffusion
have selected several qualitative methods because
particularly useful for identifying both overt and
sequences, and many that users of a system do
ccur. By watching clinicians in the field, ethnog-
n discover issues previously unrecognized by the
e used observation and interview techniques for
ata over a 7-monthperiodof studyat four large out-
ics. We developed a categorization scheme for con-
based on Rogers’ Diffusion of Innovations model
e analyzed examples from our research using this
thods
or data collection
rch at four sites formed a backdrop for the study
ere. All of the organizations studied are using CPOE
y, but they vary according to type and geographic
terviews at each site were held with administra-
ation technology related staff, and clinical end
at all levels. Ethnographic observation was done
ariety of settings within the clinics and hospitals,
hysician offices, exam rooms, pharmacies, emer-
rtments, intensive care units, and medical and
its. Analysis was done iteratively using a grounded
roach: as each set of field notes or interview tran-
completed, individual researchers coded themand
agree on final coding, patterns, and themes. Data
ew site were blended with prior results in an ongo-
s cycle [12]. Results of these analyses have been
sewhere [13]. One theme that emerged during the
data from the fourth site was that there exists a
ty of unintended consequences related to imple-
OE.

a t i c
2.2. Cli
The organi
to as Clini
Data were
the same m
tronic heal
least 1997.
multidiscip
vation and
tion was do
and clerica
west data
for implem
notes was u
it included
theme of u
the semi-s
tions desig
Clinics Nor
vation peri
in April of 2
Researcher
29h and di
clinicians a
human sub
both Orego
referred to
2.3. Da
Field notes
investigato
duced from
tionists. W
total numb
As indicate
identify em
the informa
egories and
mined a pr
guide code
new data s
stant comp
both to val
[12]. Qualit
Internation
the researc
themes. Th
terns and t
met 32 tim
lyze the Cli
coded ever
these discu
one was Co
Ournex
grouped in
severity, gr
and freque
them in a m
ispla
es an
’ Dif
d to
ons
abou
ttern
.
Re
a d
sequ
l mo
uild a
Th
ons
see F
hod
an
ent
educ
sequi n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m
nics Northwest data collection
zation under study here, which will be referred
cs Northwest, uses outpatient CPOE exclusively.
collected at four large outpatient clinics under
anagement using the same vendor-supplied elec-
th record. All clinics have used the system since at
At these outpatient clinics, as at previous sites, a
linary team of qualitative researchers used obser-
interview techniques to gather data. Observa-
ne in physician offices, exam rooms, pharmacies,
l staff areas. The chief focus of the Clinics North-
collection effort was identifying success factors
enting CPOE. A semi-structured template for field-
sed by the five researchers doing observation and
themes developed from prior work, including the
nintended consequences. The interview guide for
tructured oral history interviews included ques-
ned to elicit comments about consequences. At
thwest, investigators were able to extend the obser-
od over 7 months: the first observations were done
003 and the last interview in October of that year.
s shadowed clinicians in four clinics for a total of
d 15h of interviewing. They shadowed 13 different
nd interviewed 12 individuals. The study received
jects approval for this component of the study from
n Health & Science University and the organization
work d
by Mil
Rogers
referre
CPOE C
heard
fied pa
groups
3.
Prior to
for con
archica
help b
3.1.
CPOE C
Please
thosew
so with
the int
error r
Con
as Clinics Northwest.
ta analysis
were transcribed from handwritten notes by the
rs themselves. Transcripts of interviews were pro-
audiotapes by experienced oral history transcrip-
hen put into a format suitable for analysis, the
er of single spaced pages was approximately 350.
d earlier, a grounded theory approach was used to
ergent themes. Grounded means that the words of
nt are used as a starting point for developing cat-
themes [12]. Rather than starting with a predeter-
iori list of code words, the informants’ own words
development. Applying the grounded approach as
ets are added shifts the analysis mode to “con-
arison” in which newly acquired data are used
idate existing themes and to uncover new themes
ative data analysis software (QSR N6 v 6.0, QSR
al Pty. Ltd., Melbourne, Australia, 2002) assisted
hers in reviewing and indexing these patterns and
e researchers worked together to name the pat-
hemes, thus creating a taxonomy. A team of five
es between April and the end of December to ana-
nics Northwest data. Each team member read and
y transcript and every set of field notes prior to
ssions. The result was a list of 20 themes, of which
nsequences.
t aimwas to analyze the instances ofConsequences
this theme. Consequences varied greatly in level of
anularity, direction (positive and negative), source,
ncy of occurrence. In a first attempt at organizing
eaningful way, we developed a hierarchical net-
tem as a re
from intent
earlier, acc
of consequ
indirect; an
During t
defined typ
stand them
the effort.
sequences
likelihood i
are akin to
processes y
comes that
at least one
Unantic
come as ha
them. Unan
that most
quences, so
gram in Fig
diagram fro
Decision
because th
quences. A
They also
fewer medi
illegible ord
quences su
order, a tra
physicians.
leave the o
more times x x x ( 2 0 0 6 ) xxx–xxx 3
y using a thematic conceptual model as described
d Huberman [14, pp. 131–132], which builds on
fusion of Innovations model. This model will be
as the Thematic Hierarchical Network Model for
equences. Investigators mapped instances seen or
t in the field to categories in the model, identi-
s within each theme, and further analyzed these
sults
etailed analysis, the team used the DOI framework
ences to develop a graphical depiction of a hier-
del, and then used this model during analysis to
n understanding of CPOE consequences.
e Thematic Hierarchical Network Model for
equences
ig. 2. According to Diffusion of Innovations theory,
ecide to implement an innovation such asCPOEdo
intent, with a definite reason in mind. With CPOE,
may be workflow efficiency, cost containment, or
tion, for example.
ences are changes to an individual or social sys-sult of adopting the innovation. They are different
s—they result from acting on the intent. As noted
ording to Rogers, there are three classifications
ences: desirable versus undesirable; direct versus
d anticipated versus unanticipated [3].
he data analysis process, the researchers further
es of consequences in an effort to better under-
. Desirable consequences are actually the goals of
On the other hand, anticipated undesirable con-
are tradeoffs because you knowingly accept their
n exchange for a greater good. Direct consequences
process measures in that they are measurable
ouwish to improve. Indirect consequences are out-
are generally less measurable and often they are
step removed from the direct consequences.
ipated consequences, if they are actually desirable,
ppy surprises, so we use the term serendipity for
ticipated undesirable consequences are the kind
of us mean when we discuss unintended conse-
we give them that term. Please refer to the dia-
. 2 for examples. This discussion will follow the
m left to right.
makers introduce innovations such as CPOE
ey anticipate certain desirable and direct conse-
n example would be elimination of illegible orders.
anticipate some indirect consequences such as
cation errors (partly resulting from elimination of
ers). Tradeoffs include direct undesirable conse-
ch as it taking the physician longer to write an
deoff better tolerated by decision makers than by
An indirect outcome might be that physicians may
rganization because the implemented CPOE takes
than they can tolerate.

4 a t i
Conside
of the diagr
example of
remember
deliberately
lary medica
formulary c
that by aler
learn and re
time. An un
decline in
errors caus
For exam
wrong patie
ple of an in
cians ignor
An impo
consequen
the left sid
If they are
offs. Once r
acknowled
that in exc
CPOE, phys
the system
the physic
mation is r
making vis
3.2. Fur
Once the i
chical Netwi n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r mFig. 2 – Thematic Hierarchical Network Model of
ring unanticipated consequences on the right side
am, some related to CPOE are indeed desirable. An
a direct one would be when a physician cannot
the recommended drug, but, knowing there is one,
enters the name of an appropriate but nonformu-
tion so the system will alert him to the appropriate
hoice. An indirect serendipitous outcome might be
ting him this way, the system helps the clinician to
member the right drug so that he orders it the next
desirable direct consequence of CPOE might be a
physician satisfaction. Another would be medical
ed by the information system.
ple, a physician might order something for the
nt because of poor screen design. Finally, an exam-
direct unintended consequence would be physi-
ing alerts because there are too many.
rtant goal of this work is that once unanticipated
ces can be predicted, they move to the known side,
e of the chart, because they are now anticipated.
undesirable but anticipated, they become trade-
ecognized as such, they can be managed or at least
ged. An example of managing a tradeoff might be
hange for the extra time it takes physicians to use
ician schedules are reduced until they acclimate to
. An example of acknowledgement might be that
ians understand and appreciate that more infor-
eadily available to them at the point of care, thus
its more effective.
ther analysis of unanticipated consequences
nvestigators had developed the Thematic Hierar-
ork Model, they analyzed each unintended conse-
quence exa
understand
consequen
given belo
unless it is
have added
quote can b
3.2.1. De
Many of the
relate to ex
with the p
the room. F
ing observa
screen to t
fying any q
dose writte
that a pati
front of you
more savvy
patient he
with the co
ing and com
go beyond
3.2.2. Un
These are
often and o
understood
offs, or ove
security co
• Error conc s x x x ( 2 0 0 6 ) xxx–xxxConsequences of CPOE.
mple from the data, attempting to categorize and
each consequence. They further categorized the
ces by theme within the network model. Quotes
w are representative examples from interviews
stated that they are from field notes. The authors
explanations in brackets so that the context of the
e understood.
sirable unintended consequences
desirable intended and unintended consequences
am room computing. The benefits are associated
hysician–patient interchange with a computer in
rom field notes done during exam room comput-
tion: “he then turns the patient instructions on the
he patient and they go over it line by line, clari-
uestions. Patient finds an error in the Premarin
n on the screen and it’s changed.” A clinician noted
ent will often ask “do you have my record up in
?” This clinician concluded: “I think they’re being
.” In another exam room observation “with the first
uses it in the exam room only to have her help him
ntent of a letter to her landlord.” Cooperative chart-
position seem to be serendipitous activities that
the original purposes of CPOE implementation.
desirable direct consequences
the unintended consequences mentioned most
f most interest because it is important that they be
so that they can be prevented, considered trade-
rcome. They are roughly divided into errors and
nsequences.
cerns

a t i c
Concerns
ity of err
entry err
these pro
usually r
notes fro
single m
liquid wh
drop dow
strength
Effexor (a
a day) wh
pharmac
regular b
by close
watched
vation of
and I wan
check, yo
know peo
from fiel
not on th
recogniz
said ‘see
ality, the
spelled c
• Potential
A numb
security.
nursing a
plained a
ing area
one. How
an impo
observat
‘secure’ a
tunately,
leaving t
3.2.3. Un
Issues relat
mentioned
were indire
• Issues w
There we
alerts or
when we
step. It’s
got some
this site
• Workflow
There w
workflow
be a lot
mation
patient i
“It pushe
speed of
doctors s
use
mem
e up
socia
f ho
r an
inter
kflow
s the
just
oma
nom
nom
d: “it
pain.
ficia
pos
ent i
Tw
rable
e of
in Fi
t alw
it di
sion
d byi n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m
were expressed in interviews about the possibil-
ors being caused by the EMR. Observers saw order
ors being corrected, so to the best of our knowledge
blems did not cause patient harm. The pharmacy
eceived and corrected erroneous orders. From field
m observing in the pharmacy: “on one occasion, a
orphine order was entered by the physician as oral
en in fact what was desired was injectable. . . the
n for order route, the first route selectable for this
was oral liquid.” In another instance “an order for
n antidepressant) was entered as QID (four times
en what was intended was QD (once a day).” The
y both detected and corrected these problems on a
asis. Such juxtaposition errors [2], errors caused
proximity on the screen, were also made as we
clinicians enter orders. From fieldnotes of obser-
a physician: “she says ‘oops, I just ordered it for X
ted it for Y.”’ One user said “you just have to double
u know. . . that’s what you’re supposed to do but I
ple get fast and that’s when errors are made.” Also
d notes: “she tried to show me that epigastric was
e coded list, but misspelled epigastric and did not
e the misspelling and then with some indignation
!’ when the system returned no matches.” In actu-
systemwouldhave accepted the term if it had been
orrectly, but she had no faith in the system.
security concerns
they
staff
mak
the
ing o
ente
An
wor
time
able
or w
• Ergo
Ergo
note
der
bene
best
prev
3.2.4.
undesi
Becaus
model
are no
found
impres
changeer of instances reported by observers concerned
From fieldnotes: “the terminals in the common
rea were always in a logged-in state. . . users com-
bout time to secure and then re-logon.” The nurs-
is quite secure, so the problem here is not a serious
ever, securing the terminal in the exam room is
rtant issue. From another set of field notes from
ions in an exam room: “he then put his cursor on
nd turned away as he clicked the mouse. Unfor-
the cursor moved and missed the ‘secure’ button,
he terminal open as he left the room.”
desirable indirect consequences
ed to alerts, workflow, and ergonomics were often
or observed to have negative consequences that
ct.
ith alerts
re numerous complaints about getting too many
alerts at an inappropriate time: “Now we get alerts
go to charting, which in my workflow is the last
after the patient’s gone. Now I get warned they’ve
drug interaction. Great!” In general, however, at
we heard far fewer complaints than elsewhere.
issues
ere a number of indirect consequences affecting
. For example, from fieldnotes: “There seems to
of duplication of work here, writing all the infor-
on paper, putting it into systems, printing the
nstructions.” One physician said quite eloquently
s the throttle of professional life. . . it drives the
your existence forward.” A nurse was sad that the
tarted “having to work through their lunch hour. . .
also by indi
hard it is to
glad the co
get how to
way, but on
an undesir
great tolera
quence, a c
everybody c
or a compl
desirable ou
visibility ab
one nurse n
better you
consequen
an importa
• Workflow
There w
viewed in
tive. One
nesses [i
not some
know mo
tence.” Fr
concerne
up. . . the
In some
a hum to
• Interpers
There w
quences
One perss x x x ( 2 0 0 6 ) xxx–xxx 5
d to actually take a break and socialize.” A support
ber stated that extra effort had to go into trying to
for that loss of social opportunity: “facilitation of
l conversations was every bit as facile as the train-
w to actually learn how to type and how to actually
order.”
esting impact of exam room computing on the
of computer support staff was also noted: “some-
re’s a patient in the room and I don’t feel comfort-
going in there [to fix a computer] and this poor man
n was in there.”
ic issues
ics were mentioned several times. One person
’s not ergonomically friendly. I have chronic shoul-
” Someone else said ergonomics training would be
l: “when you’re doing so much so fast. . . what’s the
ition? For people coming into it, [training] might
njuring from overuse.”
o sided consequences: both desirable and
unintended consequences
the complexity of consequences and because the
g. 2 forces a strict categorization of instances that
ays clearly either desirable or undesirable, we
fficult to classify many of the consequences. The
that the consequence was desirable or undesirable
participant group, i.e. MD, RN, etc. and sometimes
vidual within a group. Several examples show how
categorize consequences: one user stated, “I am
mputer goes down sometimes, otherwise I will for-
use paper.” This user was describing it in a positive
e might also say that the computer going down is
able consequence that this person is viewing with
nce. In another example of a two sided conse-
linician noted “It was in some ways an equalizer. . .
ould see a clinician’s notes. . . a one word sentence
ete evaluation.” While that person viewed it as a
tcome, some clinicians may not regard this kind of
out the quality of their notes as positively. Finally,
oted, “the more you know about the computer the
are at your job.” While she meant it as a desirable
ce, others might argue that it should not be such
nt and visible factor in doing their job well.
issues
ere several workflow consequences that could be
a plus or minus sense depending on your perspec-
clinician noted: “you can see where your weak-
n workflow] are.” Someone else said “workflow is
thing you learn about in medical school and you
st of it was sort of a level of unconscious incompe-
om field notes: “When the printer hums either the
d person or one of the medical assistants picks it
y have included the printer in their work routine.”
ways, however, it seems undesirable to depend on
alert someone to a new order.
onal issues
ere also some interpersonal unintended conse-
that are simultaneously desirable and undesirable.
on described the advantage of having a certain