Evaluation of Outpatient Computerized Physician Medication
Order Entry Systems: A Systematic Review
SAEID ESLAMI, PHARMD, AMEEN ABU-HANNA, PHD, NICOLETTE F. DE KEIZER, PHD
A b s t r a c t
setting on: safety; cost and efficiency; adherence to guideline; alerts; time; and satisfaction, usage, and usability.
Thirty articles with original data (randomized clinical trial, non-randomized clinical trial, or observational study
designs) met the inclusion criteria. Only four studies assessed the effect of CPOE on safety. The effect was not
significant on the number of adverse drug events. Only one study showed a significant reduction of the number
of medication errors. Three studies showed significant reductions in medication costs; five other studies could not
support this. Most studies on adherence to guidelines showed a significant positive effect. The relatively small
number of evaluation studies published to date do not provide adequate evidence that CPOE systems enhance
safety and reduce cost in the outpatient settings. There is however evidence for (a) increasing adherence to
guidelines, (b) increasing total prescribing time, and (c) high frequency of ignored alerts.
? J Am Med Inform Assoc. 2007;14:400–406. DOI 10.1197/jamia.M2238.
This paper provides a systematic literature review of CPOE evaluation studies in the outpatient
A 1999 Institute of Medicine report estimated that about
80,000 people are hospitalized and 7,000 die annually in the
United States due to medication errors in the inpatient
setting.1Of these errors, 32–69% are definitely or possibly
preventable. From an economic point of view, hospital costs
of preventable adverse drug events were estimated at $2
billion per year. Similar reports in other countries show that
medication errors indeed have important impact on mortal-
ity, morbidity, and cost of care.2,3Medication errors have
been defined as any error in the medication process, which
comprises taking history, ordering, pharmacy management,
administration management, and surveillance. A medica-
tion error may or may not result in patient harm. Adverse
drug events (ADEs) are usually considered to include both
medication errors that result in harm (preventable ADEs)
and adverse drug reactions (ADRs), which are considered
Although failure to monitor patients5and work pressure6
have been reported as possible causes of medical errors,
adverse outcomes are more commonly associated with the
problems during the medication process.7The ordering step
is crucial in the process: it is the point at which the
physician’s thoughts are transformed to decisions which
trigger a series of actions, ultimately resulting in the patient
receiving the medication.7
The Institute of Medicine and other important stakeholders
have identified Computerized Physician Order Entry
(CPOE) or Electronic Prescription (EP) as a key to reduce
medication errors and improve safety.1,8In this article, the
term CPOE refers specifically to medication ordering.
In an outpatient setting, patient information is often
scattered among various paper and electronic information
systems, rendering the identification of adverse drug
events and other outcomes very difficult.9This compli-
cates the evaluation of CPOE systems in the outpatient
setting. Not surprisingly, work on evaluation of CPOE
systems has focused on the inpatient setting where CPOE
systems have been shown to decrease medication errors,
especially when the CPOE systems include decision sup-
port.10–16However, most prescriptions occur in outpa-
tient settings and, due to less control in this environment,
more errors might be expected to occur with outpatient
prescriptions. A meta-analysis suggested that in 1994,
more than one million outpatients in the United States
experienced an ADE that required admission to the hos-
pital.17Of these ADEs 106,000 were fatal, placing them
between the fourth and sixth leading causes of death,
although these projections may have been somewhat
The main objective of this systematic review was to identify
and summarize published studies of outpatient CPOE sys-
tems that evaluated one of six aspects: safety; cost and
efficiency; adherence to guidelines; alerts; time; and satisfac-
tion, usage, and usability.
For the purposes of this review, an outpatient CPOE system
is a computer-based system that allows clinicians to enter
medication orders directly for outpatients or primary care
patients. In this context, a decision support system (DSS) is
Affiliation of the authors: Department of Medical Informatics,
Academic Medical Center, Universiteit van Amsterdam, Amster-
dam, The Netherlands.
Correspondence and reprints: Saeid Eslami, Academic Medical
Center, Universiteit van Amsterdam, Department of Medical Infor-
matics, J1b-124, Meibergdreef 15, 1105 AZ Amsterdam, The Neth-
erlands; e-mail: ?email@example.com?.
Received for review: 8/02/2006; accepted for publication: 4/02/
ESLAMI et al., Outpatient CPOE; A Systematic Review
any system designed to directly aid a health professional in
decision-making during medication ordering.
We searched for relevant English language articles based on
keywords in title, abstract, and MeSH terms, using Ovid
MEDLINE®(1950 to March 31, 2006) Ovid MEDLINE In-
Process®, and EMBASE®(1980 to March 31, 2006). The final
literature search was performed on March 31, 2006.
Figure 1 shows the search strategy used to identify the
relevant articles. In the first part (A), we applied keywords
without quotes and MeSH terms pertaining to electronic
prescription. These terms cover old and new ways to refer to
CPOE systems. In the second part (B), we searched for
medication related terms to identify studies that address
prescribing. In the third part (C), we searched for terms
related to outpatient care. The results of these three parts
were combined using the Boolean operator “and.” Searching
was supplemented by scanning bibliographies from identi-
Two reviewers individually examined all titles and ab-
stracts. Discrepancies among the two reviewers were re-
solved by consensus involving a third reviewer. Articles
were selected if they reported original data from a study in
an outpatient setting and if one of their main objectives
concerned evaluation (a) of a CPOE system for medication
ordering and/or (b) of a DSS used during the medication
ordering. Study designs include clinical trials as well as
observational studies such as questionnaire surveys on
satisfaction, and simulation studies. All studies reporting on
alerts, reminders, and DSS which are not part of the CPOE
system and which are not triggered during medication order
entry were excluded. Opinion papers, letters, and evaluation
studies of vaccination ordering systems and stand-alone
programs for ordering specific drugs such as Warfarin or
Digoxin were excluded.
From the selected papers, the same two reviewers extracted
data on the demographics (such as number of physicians,
number of patients, and duration of study), study design,
outcome measures, and results. Discrepancies between these
two reviewers were again resolved by consensus involving
the third reviewer.
We listed all reported outcome measures and then catego-
rized them into outcome groups. The measured effects were
classified as being one of: statistically significant positive
effects; demonstrated positive effects (when the authors
report a positive effect but without reporting statistical
significance); a mix of statistically significant and demon-
strated positive effects; no effect (when reported as such by
the authors, with or without statistical arguments); statisti-
cally negative effects; and, finally, a mix of positive, absence
of, and negative effects. To get insight into the heteroge-
neous nature of these evaluation studies we classified them
according to the hierarchy of study designs developed by
the University of California San Francisco Stanford Evi-
dence-Based Practice Center and implemented by Kaushal
et al.16in their review (Table 1).
Searching the online databases resulted in 1,032 articles
from Ovid MEDLINE®, Ovid MEDLINE In-Process®, and
EMBASE®after removing duplicates. Initial screening of
titles and abstracts rendered 52 articles eligible for further
full text review. Five additional articles were identified by
reviewing bibliographies, yielding a total of 57 articles for
full-text review. Based on the full-text review, 19 studies
were excluded because they turned out not to address a
specific CPOE system or an outpatient setting. One study
was excluded because the effects could not be attributed
to the CPOE system alone. Two studies were excluded
because they only predicted, instead of measuring, the
effects. Finally, five studies were excluded because eval-
uation was not a main objective of the study, leaving 30
articles for detailed analyses. Two of the 30 papers
reviewed were published in MEDLINE-indexed confer-
ence proceedings. We contacted the authors of these
studies, and, to our knowledge, they were not published
in a journal before this review was completed. The 30
studies are listed in Table 2 (available as a JAMIA on-line
data supplement at www.jamia.org).
For all these studies, we categorized the main outcome mea-
sures in six main groups: medication safety; cost and (organi-
zational) efficiency; adherence to guidelines; alerts and appro-
priateness of alerts; time; and satisfaction, usage and usability.
Most papers evaluated multiple outcome measures.
Table 3 shows a summary of all measured effects of the thirty
CPOE evaluation articles. Controlled study designs were also
described by the effects of CPOE. Articles may have two or
more listings in the table because they can describe studies
with multiple outcome measures and study designs.
Below we describe the effects per outcome group for all
Only four studies assessed the effect of CPOE, all with a
DSS, on safety.19–22One retrospective observational study
showed that there were no ADEs found in a set of randomly
selected cases in which the physician accepted the alert on
drug allergy or on high severity drug interaction (n ? 67).19
However, among the randomly selected cases in which
alerts were ignored there were 3 ADEs found (n ? 122, p ?
0.55). Since the number of cases (n ? 189) was limited, these
results did not amount to a significant difference. Another
prospective cohort study could not show a statistically
significant difference in number of ADEs and preventable-
ADEs between computerized and manual prescription sys-
tems.20An RCT showed that there was no significant
difference in the actual number of clinically relevant drug
interactions between a control group and the intervention
group which received alerts on interactions. However, usage
of the system in this study was optional and very low (2.8%
of medication orders were prescribed by CPOE).21
A recent non-RCT study showed that the provider did not
complete the medication order of 18 high-volume and
high-risk medications when an alert for an abnormal rule-
associated laboratory result was displayed (p ? 0.03). This
study did not show a statistically significant reduction in
percentage of definite or probable ADEs (p ? 0.23).22
Cost and (Organizational) Efficiency
Twelve studies evaluated the impact of CPOE on the phy-
sician’s office expenses, medication costs and (organiza-
Journal of the American Medical Informatics AssociationVolume 14Number 4 July / August 2007
Two studies, both non-RCT, showed that there was no
significant effect on medication cost when using a price
comparison module23or prescription cost information24
but another study with the same study design showed
that a CPOE system could reduce the cost of medication
by suggesting equally effective but cheaper drugs.25One
recent cohort study, resulting in two articles, showed that
clinicians who used electronic prescribing, with or with-
out receiving a list of prewritten prescriptions and patient
specific diagnostic information, had significantly lower
prescription costs than those in the control group.26,27
Rotman et al. performed an RCT and evaluated the cost
effect of a CPOE system including an interaction alert
module.21There was neither an effect on the number of
clinically relevant interactions nor a significant effect on
the cost of prescribing drugs. Two other RCTs showed
that a CPOE system with a DSS did not affect the number
of emergency department visits significantly.28,29At the
same time, one of these two studies showed that there was
no significant effect on the cost28while the other showed
that the cost significantly increased.29One study, a non-
RCT, evaluated the effect of a CPOE system on the
physician’s office expenses and showed that these had
increased.30Three studies, one observational, one non-
RCT, and one RCT, evaluated the effect of electronic
prescription on physician office resources and showed a
reductionin pharmacist interventions
tions.30–32One RCT showed there was no statistically
significant effect on consultation rate.33
Adherence to Guidelines
Eleven studies evaluated the impact of CPOE with a DSS on
the adherence to a guideline or another standard.25,28–30,32–38
Among these, four studies showed that there was a signifi-
cant positive effect on adherence;25,35,37,38two studies
showed a positive effect without reporting on statistical
F i g u r e 1.
Keywords and MeSH terms used in the search strategy (words in bold are MeSH terms) and the search flow.
ESLAMI et al., Outpatient CPOE; A Systematic Review
significance;34,36and five studies28–30,32,33did not find a
significant difference between the control and the interven-
tion group. Among the studies with a positive effect on
adherence to guidelines, one used an RCT design,37three
used a non-RCT design25,35,38and two were observational
studies34,36whereas in the studies without any effect three
were RCTs,28,29,33one was a non-RCT30and one was obser-
Table 1 y Hierarchy of Study Designs
LevelStudy design Description
I Randomized Controlled Trial (RCT) A study in which people are allocated at random (by chance alone) to receive one of several
clinical interventions. One of these interventions is the standard of comparison or control.
The investigator controls the exposure to the intervention.
A study in which people are allocated to receive one of several clinical interventions. One of
these interventions is the standard of comparison or control. The investigator controls the
exposure to the intervention but allocation of people is not based on chance. It includes
interrupted time series and before-after studies.
A study in which individuals are observed or certain outcomes are measured without a
specific attempt to affect the outcome (the investigator does not control the exposure to
the intervention, e.g., the use of a CPOE). The intent is to observe how exposure to risk
factors (implemented CPOE) influences the outcome of interest. Includes cross-sectional
studies to estimate the prevalence of the outcome of interest or the prevalence of
exposure to intervention or both; cohort (longitudinal) studies with control in which
individuals who are exposed to the intervention are followed for a defined length of time
and the effects of the intervention on the exposed group is compared to a group that was
not exposed; and case control studies in which a comparison of exposure to the CPOE in
a group of individuals with the outcome of interest (cases) is compared to those without
the outcome of interest (controls).
Includes cohort studies without controls or case series.
IINon-Randomized Controlled Trial
III Observational study with controls
IV Observational study without controls
Table 3 y Frequency of All Selected Articles According to Outcome Categories and Study Design
Study Designs with Control
ODWC ? Observation design without control; OWC ? observational with controls.
Controlled Studies are also described by the effects of CPOE.
Journal of the American Medical Informatics AssociationVolume 14Number 4July / August 2007
Alerts and Appropriateness of Alerts
Six observational studies19,39–43assessed the impact of
CPOE on the produced, accepted and ignored alerts from
two points of view: system weakness and user response.
Fernando et al. performed an observational study and
showed important weaknesses in generating alerts in four
commonly used commercial systems in Britain’s GP offices.39
None of them was able to generate all 18 predefined
established alerts for contraindicated drugs and hazardous
The remaining five studies addressed user response to
alerts. Four studies showed that most of the alerts (from 55%
to 91.2%) were ignored by the physicians.19,40–42Two stud-
ies showed that “clinically irrelevance” was the main re-
ported reason for overriding alerts.41,43
Three studies31,44,45one RCT and two non-RCTs, showed
that the total time for direct and indirect patient care
increased due to the introduction of the CPOE system.
Another observational study showed that physicians did not
believe that electronic prescription was more time consum-
ing than hand-written prescription.46
Satisfaction, Usability, and Usage
Fifteen studies evaluated the impact of a CPOE system on
user satisfaction and system usage and usability. Among
them, five observational studies25,30,36,43,44showed that after
the introduction of the CPOE system, the majority of users
were satisfied with the system and they believed that the
system is usable. Although the environment, questionnaire,
and target groups were different in these studies, the ma-
jority of users believed that CPOE improved drug manage-
ment and quality of care. Three others,21,23,35one RCT and
two non-RCT, showed that user satisfaction and usability
decreased. Two other RCTs showed that patient satisfac-
tion did not change significantly after introducing CPOE
with decision support.28,29There was a wide variability in
the degree of CPOE usage. Four studies showed that of
all prescriptions, 2.8% to 90%, were entered electronic-
ally21,32,46,47and another study showed that the levels of
system usage were low.33However, another study showed
there was continued improvement in system usage and
usability.47Finally, a simulation study showed the relation-
ship between usage of two decision support models and the
complexity of the cases.48During prescribing, physicians
were more willing to use on-demand decision support as the
clinical situation became more complex while for simple
cases the reminder-based support was appropriate.
Discussion and Recommendations
We have identified and described the results of 30 papers on
evaluation of CPOE systems in outpatients. The number of
such evaluation studies has clearly increased since 2002
(only 10 articles out of 30 before 2002). We used extensive
search criteria in order to capture the different ways a CPOE
system is referred to in the published literature. However,
the following are two limitations of our search. First, because
we only addressed studies in which evaluation formed a main
objective, we could have missed some studies with a limited
evaluation focus. Second, we may have missed some studies
that have targeted outpatient CPOE systems in specific special-
ties such as oncology and pediatrics.
To our knowledge this is the first review exclusively dedi-
cated to the evaluation of CPOE systems in outpatient
settings. Existing reviews of CPOE system evaluation stud-
ies focused on inpatients,13,16,49where advantages of these
systems have been reported. Recently, Chaudhry et al.
reviewed the impact of health information technology on
quality, efficiency, and cost of medical care in inpatient and
outpatient settings50but only two papers of our review
appeared there. Another recent review article focused on
overriding drug safety alerts in CPOE, which forms only one
aspect of CPOE system evaluation.51In contrast, we provide
a comprehensive characterization of outpatient studies in-
cluding description of the study design (with level of
evidence), methods, materials, and results.
In spite of the efforts made to enhance safety by introducing
CPOE, only four studies evaluated safety in an outpatient
setting. Three of them did not show significant reduction in
the number of ADEs. Moreover, only one study showed a
significant reduction in the number of errors. Recently some
observational studies in inpatients described how new med-
ication errors were associated with the use of the CPOE
system itself.52–54No such studies were found in the outpa-
tient setting, but one should be aware of the possible
existence of such associations in outpatients as well.
A plausible explanation for the low number of ADE-related
evaluation studies is the difficulty of obtaining quantitative
data on ADEs and the high cost associated with chart
reviewing in the face of incomplete [and scattered] patient
data.55The availability of an electronic patient record or a
CPOE is only part of the solution because there should also
be an infrastructure interconnecting information residing in
laboratory and radiology information systems with informa-
tion residing in the outpatient clinic or GP offices. However,
we believe that by focusing on specific patient groups, high
risk drugs, simple errors and typical ADEs, one can evaluate
the effect of CPOE systems on ADEs with a reasonable level
of effort. Another approach would be to identify the few
currently existing health care settings with comprehensive
paperless systems in place in order to perform evaluation
studies with errors and ADEs as outcomes.
Studies on alerts show that alerts were largely ignored by
physicians. This does not necessarily mean that safety is
compromised; alerts should not be used as proxies for the
number of errors or ADEs. Many alerts are not applicable to
the patient at hand or they are not clinically important. A
valid, although not a new, insight is that the provision of
non-patient specific advice is a considerable weakness of
CPOE systems, which may lead to low user response and
inattention. Five out of the six studies on alerts used the
observational study design which has a lower evidence
level. Future qualitative and quantitative studies are neces-
sary to show the reasons for overriding alerts and whether
redesigning the system is effective in reducing unnecessary
alerts and clinician overrides.
Another stated potential benefit of CPOE systems is the
reduction of medication cost. There is some indication,
although only from two studies with non-RCT design, that
advice on equally effective but cheaper drugs and evidence-
ESLAMI et al., Outpatient CPOE; A Systematic Review
based messages are more effective at reducing costs than
simply displaying a list of drugs with their prices. It is likely
that this effect will become more pronounced if the sugges-
tions become patient specific and also target specific expen-
sive groups of medications such as antidepressants and
Niinimaki and Forsstrom56described recommendations for
standardization and evaluation of CPOE systems in outpa-
tients. They suggested evaluating technical as well as med-
ical facets of these systems. None of the selected papers in
our study evaluated technical facets such as data security
and reliability of data transfer solutions, client interface,
technical functionality, the checking mechanism for danger-
ous drug dosage, etc. Technical facets such as the user
interface are important as they influence the way users
perceive and interact with the system. Such aspects deserve
more research attention in the future.
In the selected articles various study designs were used to
evaluate CPOE systems in outpatients. The results obtained
by non-randomized studies were more likely to report
statistically significant improvement in the outcome mea-
sures than RCTs. This is possibly an indication that non-
randomized studies might be biased.
In spite of the cited merits of enhancing safety and reducing
costs, published evaluation studies do not provide adequate
evidence that CPOE systems provide these benefits in out-
patient settings. A possible explanation is the small number
of such studies conducted to date and the relatively weak
study designs used. In contrast, there is more evidence on
the ability of CPOE systems to increase adherence to guide-
lines in outpatient settings. We hence hypothesize that cost
reduction can be achieved when guidelines are specifically
geared towards this goal and that safety can be improved
when guidelines are made more patient-specific. A second
conclusion of our study is that there is much to be gained in
insight when more direct outcome measures on safety are
included. However, this is hard to achieve due to the nature
of the scattered patient information and the non-controllable
environment in outpatients. A third related conclusion is
that although more studies with a randomized controlled
design are welcome to demonstrate and confirm the effects
of CPOE on the medication process outcomes in outpatients,
there are serious difficulties in conducting them in outpa-
tient settings. Hence we believe that there is also room for
new and nontraditional methodologies and case studies for
addressing the impact of information technology interven-
tions in dynamic environments such as health care.
In sum, we found that CPOE systems seem to support
adherence to guidelines which have the potential to influ-
ence costs and safety. To date, there is, however, little
evidence about improving safety as measured by medical
errors and ADEs. Focusing on outpatient subgroups and
specific drugs merits more attention in the future. The field
would also benefit from efforts to standardize evaluation
studies, such as the standard proposed in Bell et al.57which
aims at facilitating comparisons among studies. Finally,
standards for CPOE system requirements and functionality,
such as those pertaining to providing alerts, merit more
attention as they could facilitate the design and implemen-
tation of such systems in the future.
1. Kohn LT, Corrigan JM, Donaldson MS. To Err is Human:
Building a Safer Health System. Washington, DC: National
Academy Press; 1999.
2. Vincent C, Neale G, Woloshynowych M. Adverse events in
British hospitals: preliminary retrospective record review. Brit
Med J 2001;322:517–9.
3. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Hamil-
ton JD. The quality in Australian health care study. Med J Aust
4. Franklin BD, Vincent C, Schachter M, Barber N. The incidence of
prescribing errors in hospital inpatients: an overview of the
research methods. Drug Safety 2005;28(10):891–900.
5. Jordan S, Tunnicliffe C, Sykes A. Minimizing side-effects: the
clinical impact of nurse-administered “side effect” checklist.
J Adv Nurs 2002;37(2):155–65.
6. Griffith R, Griffiths H, Jordan S. Administration of medicine.
Part 1: The law and nursing. Nurs Stand 2003;18(2):47–53.
7. Kilbridge P, Classen D. A process model of inpatient medication
management and information technology interventions to im-
prove patient safety. VHA’s research series; 2001.
8. Milstein A, Galvin RS, Delbanco SF, Salber P, Buck CR Jr.
Improving the safety of health care: the Leapfrog initiative. Eff
Clin Pract 2000;3(6):313–6.
9. Gandhi TK, Burstin HR, Cook EF, et al. Drug complications in
outpatients. J Gen Intern Med 2000;15:149–54.
10. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD,
Bates DW. Effects of computerized physician order entry on
prescribing practices. Arch Intern Med 2000;160(18):2741–7.
11. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized
physician order entry and a team intervention on prevention of
serious medication errors. JAMA 1998;280(15):1311–6.
12. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N,
et al. The impact of computerized physician order entry on
medication error prevention. J Am Med Inform Assoc 1999;6(4):
13. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technol-
ogies on medication errors and adverse drug events. Am J
Health-Syst Pharm 2003;60:1447–58.
14. Mekhjian HS, Kumar RR, Kuehn L, et al. Immediate benefits
realized following implementation of physician order entry at
an academic medical center. J Am Med Inform Assoc 2002;9:
15. Galanter L, Didomenico RJ, Polikaitis A. A trial of automated
decision support alerts for contraindicate medication using
computerized physician. J Am Med Inform Assoc 2005;12:
16. Kaushal R, Shojania KG, Bates DW. Effects of computerized
physician order entry and clinical decision support systems on
medication safety. Arch Intern Med 2003;163:1409–16.
17. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug
reaction in hospitalized patients: a meta-analysis of prospective
studies. JAMA 1998;279:1200–5.
18. Bates DW. Drug and adverse drug reactions. How worried
should we be? JAMA 1998;279:1216–7.
19. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB,
Phillips RS. Physicians’ decisions to override computerized
drug alerts in primary care. Arch Intern Med 2003;163:2625–31.
20. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in
ambulatory care. N Eng J Med 2003;348(16):1556–64.
21. Rotman BL, Sullivan AN, McDonald TW, et al. A randomized
controlled trial of a computer-based physician workstation in an
outpatient setting: implementation barriers to outcome evalua-
tion. J Am Med Inform Assoc 1996;3:340–8.
Journal of the American Medical Informatics Association Volume 14 Number 4 July / August 2007
22. Steele AW, Eisert S, Witter J, et al. The effect of automated alert Download full-text
on provider ordering behavior in an outpatient setting. PloS
23. Vedsted P, Nielsen JN, Olesen F. Does a computerized price
comparison module reduce prescribing costs in general practice.
Fam Pract 1997;14(3):199–203.
24. Ornstein SM, MacFarlane LL, Jenkins RG, Pan Q, Wager KA.
Medication cost information in a computer-based patient record
system. Impact on prescribing in a family medicine clinical
practice. Arch Fam Med. 1999;8:118–21.
25. Walton RT, Gierl C, Yudkin P, Mistry H, Vessey MP, Fox J.
Evaluation of computer support for prescribing (CAPSULE)
using simulated cases. Brit Med J 1997;315:791–5.
26. McMullin ST, Lonergan TP, Rynearson CS, Doerr TD, Veregge
PA, Scanlan ES. Impact of an evidence-based computerized
decision support system on primary care prescription costs.
Ann Fam Med 2004;2:494–8.
27. McMullin ST, Lonergan TP, Rynearson CS. Twelve-month drug
cost savings related to use of an electronic prescribing system
with integrated decision support in primary care. J Manag Care
28. Murray MD, Harris LE, Overhage JM, et al. Failure of comput-
erized treatment suggestions to improve health outcomes of
outpatients with uncomplicated hypertension: results of a ran-
domized controlled trial. Pharmacotherapy 2004;24(3):324–37.
29. Tierney WM, Overhage JM, Murray MD, et al. Can computer-
generated evidence-based care suggestions enhance evidence-
based management of asthma and chronic obstructive pulmo-
nary disease? A randomized controlled trial. Health Serv Res
30. Wogen SE, Fulop G, Heller J. Electronic prescribing: improving
the efficiency of the prescription process and promoting plan
adherence. Drug Benefit Trends 2003;(September):35–40.
31. Beer J, Dobish R, Chambers C. Physician order entry: a mixed
blessing to pharmacy? J Oncol Pharm Practice 2002;8:119–26.
32. Ross SM, Papshev D, Murphy EL, Sternberg DJ, Taylor J, Barg R.
Effects of electronic prescribing on formulary compliance and
generic drug utilization in the ambulatory care setting: a retro-
spective analysis of administrative claims data. J Manag Care
33. Eccles M, McColl E, Steen N, et al. Effect of computerised
evidence based guidelines on management of asthma and
angina in adults in primary care: cluster randomised controlled
trial. Brit Med J 2002;325:941–48.
34. Chin HL, Wallace P. Embedding guidelines into direct physi-
cian order entry: simple methods, powerful results. Proc AMIA
35. Siegel C, Alexander MJ, Dlugacz YD, Fischer S. Evaluation of
computerized drug review system: impact, attitudes, and inter-
actions. Comp Biomed Research 1984;17:419–35.
36. Rivkin S. Opportunities and challenges of electronic physician
prescribing technology. Med Interface 1997;10(8):77–8, 83.
37. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara
FP, Davis RL. A randomized controlled trial of point of care
evidence to improve the antibiotic prescribing practices for otitis
media in children. Pediatrics 2001;107(2):15–9.
38. Bernstein SL, Whitaker D, Winograd J, Brennan JA. An elec-
tronic chart prompt to decrease proprietary antibiotic prescrip-
tion to self-pay patient. Acad Emerg Med 2005;12(3):225–31.
39. Fernando B, Savelyich BS, Avery AJ, et al. Prescribing safety
features of general practice computer systems: evaluation using
simulated test cases. Brit Med J 2004;328:1171–2.
40. Taylor L, Tamblyn R. Reasons for physician non-adherence to
electronic drug alerts. Proc MedInfo. 2004; :1101–5.
41. Magnus D, Rodgers S, Avery AJ. GPs’ views on computerized
drug interaction alerts: questionnaire survey. J Clin Pharm
42. Shah NR, Seger AC, Seger DL, et al. Improving acceptance of
computerized prescribing alerts in ambulatory care. J Am Med
Inform Assoc 2006;13:5–11.
43. Glassman PA, Simon B, Belperio P, Lanto A. Improving recog-
nition of drug interactions, benefits and barriers to using
automated drug alerts. Med Care. 2002;40(12):1161–71.
44. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled
trial of direct physician order entry: effects on physicians’ time
utilization in ambulatory primary care internal medicine prac-
tices. J Am Med Inform Assoc 2001;8(4):361–71.
45. Murray MD, Loos B, Tu W, Eckert GJ, Zhou XH, Tierney WM.
Effects of computer-based prescribing on pharmacist work
patterns. J Am Med Inform Assoc 1998;5:546–53.
46. Schectman JM, Schorling JB, Nadkarni MM, Voss JD. Determi-
nants of physician use of an ambulatory prescription expert
system. Int J Med Inform 2005;74:711–17.
47. Tamblyn R, Huang A, Kawasumi Y, et al. The development and
evaluation of an integrated electronic prescribing and drug
management system for primary care. J Am Med Inform Assoc
48. Seroussi B, Bouaud J. Reminder-based or on demand decision
support systems: a preliminary study in primary care with the
management of hypertension. Stud Health Technol Inform
49. Kuperman GJ, Gibson RF. Computer physician order entry:
benefits, costs, and issues. Ann Intern Med 2003;139:31–9.
50. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of
health information technology on quality, efficiency, and costs
of medical care. Ann Intern Med 2006;144(10):742–52.
51. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug
safety alerts in computerized physician order entry. J Am Med
Inform Assoc 2006;13:138–47.
52. Koppel R, Metlay JP, Cohen A, et al. Role of computerized
physician order entry systems in facilitating medication errors.
53. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a
medication dosing error related to CPOE. J Am Med Inform
54. Eslami S, Abu-Hanna A, de Keizer NF, de Jonge E. Error
associated with applying decision support by suggesting
default doses for aminoglycosides. Drug Safety 2006;29(9):
55. O’Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan
TA. Physician reporting compared with medical-record review
to identify adverse medical events. Ann Intern Med 1993;119:
56. Niinimaki J, Forsstrom J. Approaches for certification of elec-
tronic prescription software. Int J Med Inform 1997;47:175–82.
57. Bell DS, Cretin S, Marken RS, Landman AB. A conceptual
framework for evaluating outpatient electronic prescribing sys-
tems based on their functional capabilities. J Am Med Inform
ESLAMI et al., Outpatient CPOE; A Systematic Review