Return on investment for a computerized physician order entry system.
ABSTRACT Although computerized physician order entry (CPOE) may decrease errors and improve quality, hospital adoption has been slow. The high costs and limited data on financial benefits of CPOE systems are a major barrier to adoption. The authors assessed the costs and financial benefits of the CPOE system at Brigham and Women's Hospital over ten years.
Cost and benefit estimates of a hospital CPOE system at Brigham and Women's Hospital (BWH), a 720-adult bed, tertiary care, academic hospital in Boston.
Institutional experts provided data about the costs of the CPOE system. Benefits were determined from published studies of the BWH CPOE system, interviews with hospital experts, and relevant internal documents. Net overall savings to the institution and operating budget savings were determined. All data are presented as value figures represented in 2002 dollars.
Between 1993 and 2002, the BWH spent $11.8 million to develop, implement, and operate CPOE. Over ten years, the system saved BWH $28.5 million for cumulative net savings of $16.7 million and net operating budget savings of $9.5 million given the institutional 80% prospective reimbursement rate. The CPOE system elements that resulted in the greatest cumulative savings were renal dosing guidance, nursing time utilization, specific drug guidance, and adverse drug event prevention. The CPOE system at BWH has resulted in substantial savings, including operating budget savings, to the institution over ten years.
Other hospitals may be able to save money and improve patient safety by investing in CPOE systems.
- SourceAvailable from: Ali Amanati[Show abstract] [Hide abstract]
ABSTRACT: Medication dosing errors are frequent in neonatal wards. In an Iranian neonatal ward, a 7.5 months study was designed in three periods to compare the effect of Computerized Physician Order Entry (CPOE) without and with decision support functionalities in reducing non-intercepted medication dosing errors in antibiotics and anticonvulsants. Before intervention (Period 1), error rate was 53%, which did not significantly change after the implementation of CPOE without decision support (Period 2). However, errors were significantly reduced to 34% after that the decision support was added to the CPOE (Period 3; P<0.001). Dose errors were more often intercepted than frequency errors. Over-dose was the most frequent type of medication errors and curtailed-interval was the least. Transcription errors did not reduce after the CPOE implementation. Physicians ignored alerts when they could not understand why they appeared. A suggestion is to add explanations about these reasons to increase physicians' compliance with the system's recommendations.Journal of Medical Systems 07/2009; · 1.37 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Many healthcare providers are adopting clinical decision support (CDS) systems to improve patient safety and meet meaningful use requirements. Computerized alerts that prompt clinicians about drug-allergy, drug-drug, and drug-disease warnings or provide dosing guidance are most commonly implemented. Alert overrides, which occur when clinicians do not follow the guidance presented by the alert, can hinder improved patient outcomes.Ochsner Journal 01/2014; 14(2):195-202.
- [Show abstract] [Hide abstract]
ABSTRACT: There is a need to identify effective interventions to minimize the threat posed by medication administration errors (MAEs). Our objective was to review and critically appraise interventions designed to reduce MAEs in the hospital setting. Ten electronic databases were searched between 1985 and November 2013. Randomized controlled trials (RCTs) and controlled trials (CTs) reporting rates of MAEs or related adverse drug events between an intervention group and a comparator group were included. Data from each study were independently extracted and assessed for potential risk of bias by two authors. Risk ratios (RRs, with 95 % confidence intervals [CIs]) were used to examine the effect of an intervention. Six RCTs and seven CTs were included. Types of interventions clustered around four main themes: medication use technology (n = 4); nurse education and training (n = 3); changing practice in anesthesia (n = 2); and ward system changes (n = 4). Reductions in MAE rates were reported by five studies; these included automated drug dispensing (RR 0.72, 95 % CI 0.53-1.00), computerized physician order entry (RR 0.51, 95 % 0.40-0.66), barcode-assisted medication administration with electronic administration records (RR 0.71, 95 % CI 0.53-0.95), nursing education/training using simulation (RR 0.17, 95 % CI 0.08-0.38), and clinical pharmacist-led training (RR 0.76, 95 % CI 0.67-0.87). Increased or equivocal outcome rates were found for the remaining studies. Weaknesses in the internal or external validity were apparent for most included studies. Theses and conference proceedings were excluded and data produced outside commercial publishing were not searched. There is emerging evidence of the impact of specific interventions to reduce MAEs in hospitals, which warrant further investigation using rigorous and standardized study designs. Theory-driven efforts to understand the underlying causes of MAEs may lead to more effective interventions in the future.Drug Safety 04/2014; · 2.62 Impact Factor
Technology Evaluation n
Return on Investment for a Computerized Physician
Order Entry System
RAINU KAUSHAL, MD, MPH, ASHISH K. JHA, MD, CALVIN FRANZ, PHD, JOHN GLASER, PHD,
KANAKA D. SHETTY, MD, TONUSHREE JAGGI, BA, BLACKFORD MIDDLETON, MD, MBA, MSC,
GILAD J. KUPERMAN, MD, PHD, RAMIN KHORASANI, MD, MPH, MILENKO TANASIJEVIC, MD, MBA,
DAVID W. BATES, MD, MSC, Brigham and Women’s Hospital CPOE Working Group
A b s t r a c t
quality, hospital adoption has been slow. The high costs and limited data on financial benefits of CPOE systems are a
major barrier to adoption. The authors assessed the costs and financial benefits of the CPOE system at Brigham and
Women’s Hospital over ten years.
Objective: Although computerized physician order entry (CPOE) may decrease errors and improve
Design: Cost and benefit estimates of a hospital CPOE system at Brigham and Women’s Hospital (BWH), a 720-adult
bed, tertiary care, academic hospital in Boston.
Measurements: Institutional experts provided data about the costs of the CPOE system. Benefits were determined from
published studies of the BWH CPOE system, interviews with hospital experts, and relevant internal documents. Net
overall savings to the institution and operating budget savings were determined. All data are presented as value figures
represented in 2002 dollars.
Results: Between 1993 and 2002, the BWH spent $11.8 million to develop, implement, and operate CPOE. Over ten
years, the system saved BWH $28.5 million for cumulative net savings of $16.7 million and net operating budget
savings of $9.5 million given the institutional 80% prospective reimbursement rate. The CPOE system elements that
resulted in the greatest cumulative savings were renal dosing guidance, nursing time utilization, specific drug
guidance, and adverse drug event prevention. The CPOE system at BWH has resulted in substantial savings, including
operating budget savings, to the institution over ten years.
Conclusion: Other hospitals may be able to save money and improve patient safety by investing in CPOE systems.
j J Am Med Inform Assoc. 2006;13:261–266. DOI 10.1197/jamia.M1984.
Between 44,000 to 98,000 Americans die each year due to
medical errors, and about 1 million people are injured.1
Although there is controversy regarding the accuracy of the
mortality estimates, general agreement exists that iatrogenic
injuries are common, costly, and often preventable.2–8
Medications represent the single most common cause of iatro-
genic injuries, accounting for nearly 20% of all such events.9
Reducing medical errors requires a multifaceted approach.
Computerized physician order entry (CPOE) with clinical
decision support systems (CDSS) is a promising intervention
the serious medication error rate by 55%.11Other studies have
also demonstrated reductions in medication errors.12–14Most
CPOE systems allow physicians to enter medication orders
as well as laboratory, admission, radiology, and transfusion
orders electronically. When combined with clinical decision
support, CPOE improves medication safety, clinical labora-
tory and radiology testing, medication costs, adoption of crit-
ical pathways, and the efficiency of hospital workflow.13–17
Despite growing evidence and mandates to implement
CPOE, adoption has been slow, with only an estimated
15%of hospitalshaving even
CPOE.18,19High costs are one important barrier to faster
adoption.20Hospitals must make a large up-front capital in-
vestment without clear data on return on investment or con-
fidence in physician use of implemented systems. These
costs include the time required of hospital staff, particularly
physicians, for training and use of CPOE systems. Further,
Affiliations of the authors: Brigham and Women’s Hospital and Har-
vard Medical School, Boston, MA (RK, AKJ, TJ, BM, RKh, MT, DWB);
Department of Public Health, Weill Medical College of Cornell Uni-
versity, New York, NY (RK, GJK); Department of Health Policy and
Management, Harvard School of Public Health, Boston, MA (AKJ);
Eastern Research Group, Inc., Lexington, MA (CF); Department of
Medicine, Columbia University Medical Center, New York, NY
(KDS); Information Systems, Partners Healthcare System, Boston,
MA (JG, BM, DWB).
Most of the individual studies that were aggregated in this report
were supported by the Agency for Healthcare Research and Quality,
Members of the Brigham and Women’s Hospital CPOE Working
Group include Melissa Christino, William Churchill, Petr Jarolim,
Joshua Peterson, Eric Poon, Jeffrey Rothschild, Cynthia Spurr, Carl
Stapinski, Jonathan M. Teich, Debra Thomas, and Samuel Wang.
Correspondence and reprints: Rainu Kaushal, MD, MPH, Division of
General Medicine and Primary Care, Department of Medicine, Brig-
ham and Women’s Hospital, 1620 Tremont Street, Boston, MA
02120-1613; e-mail: <firstname.lastname@example.org>.
Received for review: 09/26/05; accepted for publication: 02/01/06.
Journal of the American Medical Informatics Association Volume 13 Number 3May / Jun 2006
while hospitals must shoulder the costs of CPOE, benefits
accrue to multiple parties including payers, employers, and
Given the tension between the clinical benefits of CPOE and
the high up-front costs, hospitals deciding whether to imple-
ment CPOE need to better understand how and when finan-
cial benefits of such systems accrue. Brigham and Women’s
Hospital (BWH), a 720-adult bed, academic, tertiary care
medical center in Boston, MA, implemented a home-grown
CPOE system in 1993. We assessed the costs and benefits as-
sociated with the implementation of the BWH CPOE system
over ten years.
BWH internally developed and implemented a CPOE system
in 1993 with CDSS introduced incrementally over the ensuing
years as the system was enhanced. We began by calculating
the costs and all financial benefits of the BWH CPOE system.
Since many CPOE savings are not demonstrable in an organi-
zation’s operating budget, we then calculated the benefits
that are actually reflected in the organization’s operating
budget such as decreased drug costs.
For BWH, based on internal documents and interviews with
the developers of the CPOE system, we determined the cap-
ital costs and assigned 60% of the costs to 1992, 20% to
1993, and 20% to 1994. We began to count operational costs
starting on January 1, 1993, and running through December
31, 2002. We included costs such as hardware including work-
stations and printers, software, network, leadership, and
training costs. However, we did not include costs such as
those for a pharmacy system, medication administration sys-
tem, or clinical data repository.
To estimate the benefits from CPOE, we identified each inter-
vention and calculated the associated cost savings. We
obtained actual BWH benefit data from relevant published
literature, institutional key informants, and internal docu-
ments. We depict the method and amount of cost savings
for the most financially profitable clinical decision elements
in Table 1. We reference relevant literature in Table 1 as well,
although much internal data were also used.
The most rigorously studied aspect of the BWH CPOE system
is CDSS to reduce adverse drug events.3,15,22,23Consequently,
we calculated many benefits based on the number of averted
adverse drug events (ADEs). For example, the BWH system
has drug–drug interaction alerts that check and warn for haz-
ardous interactions between drugs.3We determined cost sav-
ings from these alerts by multiplying the number of averted
ADEs by the average cost of an ADE ($4,685 in 1997
We estimated other benefits based on decreased drug costs
generally through decreased use. For example, a clinical deci-
sion support tool to decrease the frequency of ceftriaxone
from twice a day to once a day was introduced in 1999, result-
ing in 80% of orders being switched to daily dosing.25We cal-
culated cost savings by multiplying the number of saved
doses by the cost of each dose. Another important interven-
tion leading to decreased drug costs was early conversion
of intravenous to oral medications for patients who were
already taking either other oral medications or an oral
diet.26In this case, we estimated the amount of savings by
subtracting the costs of the oral medications from the costs
of the intravenous medications.
Some interventions saved money by decreasing laboratory
test usage such as an alert that warned physicians when re-
dundant laboratory tests were ordered.27For laboratory sav-
ings, we only had access to laboratory charge savings rather
than cost savings due to the internal structuring and record-
ing of hospital data. Although the ratio of charges to costs
varied for individual laboratory tests, we used a 0.2 charge-
to-cost ratio (i.e., a charge of $1 is associated with a cost of
$0.20) on the advice of institutional laboratory experts.
Other interventions addressed radiology test overuse and
misuse. For example, one intervention improved appropriate
ordering of radiology tests.28We calculated the savings from
this intervention by subtracting the costs of performed tests
from the costs of canceled inappropriate tests.
Improved workflow and efficiency led to additional cost sav-
ings. For example, improved nursing time utilization resulted
in staff and resource savings. Similarly there were savings in
physician time utilization.
For some interventions, particularly those that did not show
up in the operating budget, we were unable to calculate ben-
efits, and while benefits likely exist for many, we excluded
them from the analysis. These interventions include ad-
vanced chemotherapy decision support that prevented in-
correct dose calculations, lack of rescue medications, and
inappropriate administration of medications to the wrong pa-
tient. We did not include guided dose algorithms that prevent
either over- or underdose of medications requiring calcula-
tions such as heparin and digoxin. Nordid we include several
radiological interventions such as online scheduling, patient
preparatory instructions, and allergy tracking. We also did
not include pathway order sets, automated cosign and docu-
mentation requirements, or automated sign outs. Finally,
some interventions such as transfusion guidance were imple-
mented too recently to include in these analyses.
Live Date and End Date
CPOE CDSS interventions at BWH were phased in during the
period of analysis. To account for the costs and benefits of
each intervention, we used a number of conventions. When
an intervention became active (live date) in the middle of a
month, we assumed benefits would not start to accrue until
the first day of the following month. When we only had
data on the year in which an intervention went live, we
assumed that benefits would start to accrue at the midpoint
of the year (July 1). Many interventions were initially intro-
duced and then significantly enhanced. In these cases, we
reported multiple start dates in Table 1.
Operating Budget Analysis
We also performed an analysis to assess benefits that would
be included in the hospital’s operating budget. Many savings
from CPOE are not realized in the operating budget. We
analyzed each clinical decision support element to determine
whether the resulting savings were observable in the operat-
ing budget. For example, cost savings from lower drug costs
directly improve the hospital’s operating income. Savings
attributable to workflow improvements, on the other hand,
may not directly reduce operating costs since improvement
KAUSHAL ET AL., ROI of CPOE
in efficiency does not necessarily translate into full-time
Discounting, Annualization, Constant Dollars
We discounted all costs and benefits at a 7% annual percent-
age rate in accordance with the recommendations of the U.S.
Office of Management and Budget for economic analyses
performed for the federal government.29This represents a
societal discount rate rather than a hospital-specific rate. We
discounted all costs and benefits on a monthly basis with
costs discounted using a ‘‘beginning of period’’ convention,
In addition to discounted values, we calculated annualized
values. Annualization converts the entire stream of dis-
counted costs and benefits into a series of equal annual pay-
ments analogous to mortgage payments on a house.
All current dollar values for costs and benefits were con-
verted to a constant dollar basis to adjust for inflation. We
used the Bureau of Labor Statistics’ Producer Price Index
time series for General Medical and Surgical Hospitals to
deflate values to a constant 2002 base year.
Prospective reimbursement rates affect the amount of hospi-
tal savings from CPOE. If a patient’s care is not prospectively
reimbursed, then savings do not necessarily accrue to the
hospital from an avoided ADE or an unnecessary test since
the hospital may be reimbursed by the insurance company
regardless of whether the utilization was avoidable. For this
period of time at BWH, the average prospective reimburse-
ment rate was 80%.
The Brigham and Women’s Hospital Experience
All results are present value figures reported in constant 2002
dollars. In 1992, BWH spent approximately $3.7 million in
capital costs and between $600,000 to $1.1 million perensuing
year from 1993 to 2002 in operational costs for total costs of
Table 1 j Cumulative Benefits for CDSS Elements at Brigham and Women’s Hospital
CDSS ElementMethod of Cost Savings
Renal dosing guidance30
Decreased ADEs: decreased length of stay, decreased ADEs, and increased
appropriate prescriptions; 16,470 interventions per year
Improved work flow and efficiency: streamlined work flow for nurses
particularly by decreasing time to generate a medication
Decreased drug costs: decreased use or frequency resulting in decreased
doses. For example, 975 interventions per year suggest decreasing
frequency of ondansetron use from 4 to 3 times per day, resulting in an
overall decrease in frequency from 3.92 to 3.15 doses per day; 5,536
vancomycin interventions per year
Nurse time utilization
Specific or expensive drug
Adverse drug event prevention3,15,22–24
Decreased ADEs: decreased ADEs through drug dose, route, frequency,
allergy, drug interaction and laboratory warnings
Decreased laboratory tests: decreased ordering of laboratory tests. Charges
are displayed 10,608 times per year resulting in 4.5% fewer ordered tests.
Redundant laboratory warnings are issued 2,817 times per year resulting
in cancellation of 69% of suggested tests
Decreased ADEs: decreased time to treat ADEs through improved
communication; 6,720 alerts are generated each year regarding critical
Decreased drug costs: decreased use of intravenous medications by
a computerized report that identifies patients on expensive intravenous
medications who are taking either oral medications or food; 15,695 alerts
are generated per year
Decreased ADEs: decreased ADEs through early physician notification of
potential ADEs; generally 230 interventions per year
Improved work flow and efficiency: improved information access for
patients at time of discharge; decreases staff time otherwise needed to
generate a medication list
Improved work flow and efficiency: streamlined workflow for physicians
(e.g., reduced time finding chart or reduced re-work with pharmacists)
Decreased radiological utilization: decreased unnecessary testing and
improved documentation; an abdominal (KUB) radiograph assistant
generates 2,488 interventions per year to reduce overuse of KUBs radiographs
Decreased ADEs: decreased ADEs by recommending drug dose reduction
in geriatric patients
Decreased laboratory tests: approximately 120 rheumatologic test
recommendations per year result in fewer tests
Laboratory charge display and
redundant laboratory warnings27,32
Panic laboratory alerting33
Intravenous to oral guidance26
ADE monitor 5/001.0
Automated medication summary
at hospital discharge
Physician time utilization7/930.6
rule-out, and assistant28
Elderly dosing guidance12/970.1
Specific drug level guidance
CDSS 5 clinical decision support system; ADE 5 adverse drug event; KUB 5 kidney, ureter, and bladder.
This table depicts the cumulative benefits (in 2002 millions of dollars) from 1992 to 2002 for each element of CDSS at Brigham and Women’s
Hospital given an 80% prospective reimbursement rate.
Journal of the American Medical Informatics AssociationVolume 13Number 3 May / Jun 2006
$11.8 million to develop, implement, and operate CPOE
(Fig. 1). During these 11 years, the CPOE system saved a total
of $28.5 million given the 80% prospective reimbursement
rate at BWH. This resulted in a net benefit of $16.7 million
($2.2 million annualized). The operating budget benefits to-
taled $21.3 million for a net cumulative present value of
$9.5 million ($1.3 million annualized). As the project length
increased, benefits increased since most of the costs were in-
curred early in the implementation. It took over five years
for BWH to realize a net benefit and over seven years for
BWH to realize an operating budget benefit from CPOE.
Cumulative Computerized Physician Order Entry
Clinical Decision Support System Benefits for
Brigham and Women’s Hospital
The largest cumulative savings were from renal dosing guid-
ance ($6.3 million), improved nursing time utilization ($6.0
million), specific or expensive drug guidance (human growth
hormone, vancomycin, ceftriaxone, ondansetron, and hista-
mine receptor antagonists) ($4.9 million), ADE prevention
($3.7 million), laboratory charge display and redundant labo-
ratory warnings ($1.9 million), and panic laboratory alerting
($1.8 million)3,15,22–25,27,30–33(Table 1). Because many of the
clinical decision support elements were not implemented un-
til the latter part of the evaluated time period, the savings
were much greater during the latter years.
Renal dosing guidance saved the institution the most money.
In this intervention, the system recommends drug-dosing ad-
justments based onthepatient’srenalfunction. In a large study
of this intervention, the number of appropriate orders in-
creased from30% to 51% (p , 0.001) andmean adjusted length
of stay decreased from 4.5 days in the control group to 4.3 days
in the intervention group (p 5 0.009).30The annual cost sav-
ings weremost heavily driven by the decreased length of stay.
The BWH CPOE system improved the efficiency of nursing
time utilization. The largest efficiency was attributed to the
automation of the medication administration record eliminat-
ing the need for a handwritten medication administration
record. Other cost savings were due to decreased rework of
problematic orders to avoid medication errors or ADEs since
the CPOE system was performing many of these checks at the
time of order creation.
Specific or expensive drug guidance saved the institution
$4.9 million during the ten-year study period.15,25,31An inter-
vention that suggested appropriate indications for initiation
and continuation of vancomycin through explicit indications
and prompts to discontinue the drug after three days
resulted in 32% fewer vancomycin orders. The net effect
was $20,042 in annual cost savings ($80,166 in vancomycin
savings and $60,124 in replacement drug costs).31Prior to an-
other intervention, 80% of ceftriaxone orders were written
for twice-daily dosing. Providers were prompted to dose
daily instead with an 80% switch to once-daily dosing and
a cost savings of $175,094 annually.25An intervention to
reduce the use of human growth hormone by suggesting
appropriate indications resulted in $320,000 in annual cost
savings. Another intervention increased the number of on-
dansetron orders with a three-times daily frequency instead
of four times daily, from 5.9% to 93.5% of orders. This med-
ication was ordered an average of 975 times each year. After
adjusting for the average duration of each order, the annual
savings for this intervention was demonstrated to be
$249,218 in 2002 dollars given an 80% prospective reim-
bursement rate.15A final intervention recommended nizati-
dine and ranitidine as histamine-2 receptor antagonist of
choice in compliance with the BWH formulary at the time.
In a time-series analysis, nizatidine and ranitidine use in-
creased from 11.7% and 0%, respectively, to 97.9% and
A number of interventions aimed at improving patient safety
were demonstrated to decrease preventable ADEs from 4.69
to 3.88 per 1,000 patient days in one study.11Annual cost sav-
ings associated with these interventions were determined by
multiplying the reduction in ADEs by the cost of ADEs. These
interventions include, among others, default drug dose, fre-
quencies, and routes as well as drug–drug and drug–allergy
Another important intervention of the BWH CPOE system
allows critical laboratory results to be automatically detected
and the responsibleprovider paged. This system reduces time
to appropriate treatment thereby averting ADEs.33
Three important interventions were introduced too recently
to be included in the analyses; however, some data are avail-
able on cost savings associated with them. A pilot study has
indicated that a transfusion guidance system saved the hospi-
tal $1.3 million in 2002 dollars. An intervention that suggests
appropriate ordering of Clostridium difficile cytotoxin assays
is estimated to save $12,000 per year, while an intervention
to improve ordering of digoxin levels is estimated to save
$74,000 per year. These three interventions were not included
in the study.
Annual Computerized Physician Order Entry
Clinical Decision Support System Benefits for
Brigham and Women’s Hospital
Since the cumulative benefits are dependent on the length of
intervention, we also summarized the annual savings in 2002
dollars for each CDSS intervention (Table 2). Again renal dos-
ing guidance led the interventions (2.24 million), followed by
F i g u r e 1 .
ized physician order entry (CPOE) at Brigham and Women’s
Hospital (BWH) from 1992 to 2002 given an 80% prospective
reimbursement rate. Six years after the implementation of
CPOE, in 1998, BWH began to make a net profit from the
CPOE system. This net profit continues to grow steeply. In
1999, 7.5 years after the implementation of CPOE, a financial
benefit accrued in the operating budget.
The net cumulative present value of computer-
KAUSHAL ET AL., ROI of CPOE
ADE prevention, nurse time utilization, and specific orexpen-
sive drug guidance.
A CPOE system at a large academic hospital that was imple-
mented about 10 years ago saved the hospital about $2.2 mil-
lion annually with current savings of $16.7 million per year.
The operating budget savings were $9.5 million ($1.3 million
annualized). It took over five years for the BWH system to be-
gin accruing a netbenefit and over seven years tobegin accru-
ing an operating budget benefit.
The level and type of decision support weredirectlyrelated to
the amount of savings the hospital achieved. Renal drug dos-
ing, ADE prevention, and expensive or specific drug guid-
ance were the most financially beneficial interventions. It is
important to note that the majority of savings accrued from
a relatively small number of interventions. These results sug-
gest that hospitals should consider focusing on these CDSS
interventions to increase the chance of financial profitability
from their CPOE systems. In addition, hospitals should pay
careful attention to the method of workflow integration to
save nursing and physician time. Expensive drug guidance
is an increasingly important type of decision support given
the rapid emergence of new drugs. In this model, we included
cost containment of human growth hormone and ondanse-
tron but not other expensive medications.
Of note, we included only BWH CDSS elements for which
there were good estimates of cost savings in the model. We
were unable to include several BWH CDSS elements in the
study due to either their timing of implementation or the
lack ofreliablecost savings data. Even though many interven-
tions have been implemented at BWH, this CPOE system
lacks numerous other highly effective interventions such as
LDS Hospital’s antibiotic assistant.34,35Further studies of
the potential benefits of specific elements of CDSS are neces-
sary for hospitals to accurately understand the value of
In performing these analyses, we assumed that costs and ben-
efits would be equally affected by inflation over time. If the
price of medical services is growing more rapidly than gen-
eral inflation, as is likely, then the discount rate is actually
declining over time in real terms. If the discount is overesti-
mated in this way, the model would tend to underestimate
benefits rather than costs as we front-loaded costs and
back-loaded benefits in our analysis.
To achieve the types of benefits modeled based on BWH data,
a hospital must have nearly 100% physician use, well-
designed CDSS elements, and effective interfaces among
CPOE, pharmacy, laboratory, and medication administration
record systems. Some hospitals have overcome large financial
barriers to implement CPOE, only to fail to achieve wide-
spread use due to physician resistance.36,37In addition, the
automated knowledge necessary for CDSS elements must
be represented in ways that allow it to be readily inter-
changed between different computer systems. Benefits may
vary substantially among different vendor applications based
on factors like these. The benefits of increased workflow effi-
ciency are perhaps the most difficult to achieve as they
require all these factors along with a quick system, although
they are very important given the national shortages of
nurses and pharmacists.
This study has several limitations. Ideally, all benefit data
would have been collected prospectively rather than retro-
spectively. Active data collection would have decreased the
number of estimates by institutional experts. In addition,
we did not include less direct benefits of CPOE such as
averted malpractice litigation from fewer ADEs.38
excluded several decision support elements of the BWH sys-
tem for which we could not calculate benefits. Our model is
purely cost avoidance and does not directly address increased
revenue. CPOE systems often result in improved billing, but
these savings were not incorporated in our benefit estimates.
Nor did we include increased efficiencies for personnel such
as pharmacists since reliable institutional estimates were not
Of note, we did not include all the costs of knowledge engi-
neering by clinicians and engineers to create and encode clin-
ical information with CDSSs. Information technology staff
time is included, but not the time devoted by clinicians and
researchers in developing clinical rules. These hidden costs
were piecemeal over many years and probably represented
a relatively small part of the entire costs. Finally, it is essential
to note that the vast majority of implemented CPOE systems
are vendor based rather than home grown such as the BWH
system. Clearly the benefits of CDSS from vendor systems
may be different than those from the BWH system limiting
In conclusion, the BWH saved significant money by imple-
menting the CPOE system. Other hospitals may realize even
greaterbenefits, particularly if they havehigh levels ofclinical
decision support and rates of prospective reimbursement.
chase a CPOE system, the financial benefits may help justify
the expense. Furthermore, hospitals may not require long-
term ongoing financial support, although it will be important
a critical part of the health care mission and the implementa-
tion of CPOE can make health care safer and save money.
Table 2 j Annual Benefits for CDSS Elements at
Brigham and Women’s Hospital
CDSS Element Total Benefits
Renal dosing guidance
Nurse time utilization
Specific or expensive drug guidance (human growth
hormone, vancomycin, ceftriaxone, ondansetron,
histamine-2 receptor blockers)
Intravenous to oral guidance
Laboratory charge display and redundant laboratory
Panic laboratory alerting
Radiology indications, rule-out, and assistant
Automated medication summary at hospital discharge
Physician time utilization
Elderly dosing guidance
Specific drug level guidance (antiepileptics,
This table depicts the annual benefits (in 2002 millions of dollars) for
each element of CDSS at Brigham and Women’s Hospital given an
80% prospective reimbursement rate.
Journal of the American Medical Informatics AssociationVolume 13Number 3 May / Jun 2006
1. Institute of Medicine. To err is human. Building a safer health
system. Washington, DC: National Academy Press; 1999.
2. Barker KN, Mikeal RL, Pearson RE, Illig NA, Morse ML. Medi-
cation errors in nursing homes and small hospitals. Am J Hosp
3. Bates DW, Cullen DJ, Laird N, Petersen LA, Small S, Servi D,
et al. Incidence of adverse drug events and potential adverse
drug events: implications for prevention. ADE Prevention Study
Group. JAMA. 1995;274:29–34.
4. Dean BS, Allan EL, Barber ND, Barker KN. Comparison of med-
ication errors in an American and a British hospital. Am J Health
Syst Pharm. 1995;52:2543–9.
5. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse
drug events in hospitalized patients. Excess length of stay, extra
costs, and attributable mortality. JAMA. 1997;277:301–6.
6. Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A,
Leape LL. Preventable adverse drug events in hospitalized pa-
tients: a comparative study of intensive care and general care
units. Crit Care Med. 1997;25:1289–97.
7. Lesar TS, Lomaestro BM, Pohl H. Medication-prescribing errors
in a teaching hospital. A 9-year experience. Arch Intern Med.
8. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD,
Federico F, et al. Medication errors and adverse drug events in
pediatric inpatients. JAMA. 2001;285:2114–20.
9. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Law-
thers AG, et al. Incidence of adverse events and negligence in
hospitalized patients: results of the Harvard Medical Practice
Study I. N Engl J Med. 1991;324:370–6.
10. Kaushal R, Shojania KG, Bates DW. Effects of computerized phy-
sician order entry and clinical decision support systems on med-
ication safety: a systematic review. Arch Intern Med. 2003;163:
11. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM,
et al. Effect of computerized physician order entry and a team
intervention on prevention of serious medication errors.
12. Bates DW, Teich JM, Lee J, et al. The impact of computerized
physician order entry on medication error prevention. J Am
Med Inform Assoc. 1999;6:313–21.
13. Teich JM, Glaser JP, Beckley RF, et al. The Brigham integrated
computing system (BICS): advanced clinical systems in an
academic hospital environment. Int J Med Inform. 1999;54:
14. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A random-
ized trial of ‘‘corollary orders’’ to prevent errors of omission.
J Am Med Inform Assoc. 1997;4:364–75.
15. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates
DW. Effects of computerized physician order entry on prescrib-
ing practices. Arch Intern Med. 2000;160:2741–7.
16. Taylor R, Manzo J, Sinnett M. Quantifying value for physician
order-entry systems: a balance of cost and quality. Healthc
Financ Manage. 2002;56:44–8.
17. Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB,
McDonald CJ. A computerized reminder system to increase
the use of preventive care for hospitalized patients. N Engl J
18. Doolan DF, Bates DW. Computerized physician order entry sys-
tems in hospitals: mandates and incentives. Health Aff (Mill-
19. Ash JS, Gorman PN, Seshadri V, Hersh WR. Computerized phy-
sician order entry in U.S. hospitals: results of a 2002 survey. J Am
Med Inform Assoc. 2004;11:95–9.
20. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kau-
shal R. Overcoming barriers to adopting and implementing
computerized physician order entry systems in U.S. hospitals.
Health Aff (Millwood). 2004;23:184–90.
21. Kaushal R, Landrigan C, Bates DW. Improving Patient Safety in
Massachusetts. In: Moore RT (ed). Massachusetts Memos to the
Governor. Bloomington, Indiana; 2003, pp. 98–113.
22. Abookire SA, Teich JM, Sandige H, et al. Improving allergy alert-
ing in a computerized physician order entry system. Proc AMIA
23. Teich JM, Schmiz JL, O’Connell EM, Fanikos J, Marks PW, Shul-
man LN. An information system to improve the safety and effi-
ciency of chemotherapy ordering. Proc AMIA Annu Fall Symp.
24. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug
events in hospitalized patients. Adverse Drug Events Prevention
Study Group. 1997;277:307–11.
25. Bates DW. Using information systems to improve practice.
Schweiz Med Wochenschr. 1999;129:1913–9.
26. Teich JM, Petronzio AM, Gerner JR, Seger DL, Shek C, Fanikos J.
An information system to promote intravenous-to-oral medica-
tion conversion. Proc AMIA Symp. 1999:415–9.
27. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial
of a computer-based intervention to reduce utilization of redun-
dant laboratory tests. Am J Med. 1999;106:144–50.
28. Harpole LH, Khorasani R, Fiskio J, Kuperman GJ, Bates DW.
Automated evidence-based critiquing of orders for abdominal
radiographs: impact on utilization and appropriateness. J Am
Med Inform Assoc. 1997;4:511–21.
29. Guidelines and discount rates for benefit-cost analysis of federal
programs. October 29, 1992; http://www.whitehouse.gov/omb/
circulars/a094/a094.html#8. Accessed: March 20, 2006.
30. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication
dosing for inpatients with renal insufficiency. JAMA. 2001;286:
31. Shojania KG, Yokoe D, Platt R, Fiskio J, Ma’luf N, Bates DW. Re-
ducing vancomycin use utilizing a computer guideline: results of
a randomized controlled trial. J Am Med Inform Assoc. 1998;5:
32. Bates DW, Kuperman GJ, Jha A, et al. Does the computerized
display of charges affect inpatient ancillary test utilization?
Arch Intern Med. 1997;157:2501–8.
33. Kuperman GJ, Teich JM, Tanasijevic MJ, et al. Improving re-
sponse to critical laboratory results with automation: results of
a randomized controlled trial. J Am Med Inform Assoc. 1999;6:
34. Evans RS, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP.
Improving empiric antibiotic selection using computer decision
support. Arch Intern Med. 1994;154:878–84.
35. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted
management program for antibiotics and other antiinfective
agents. N Engl J Med. 1998;338:232–8.
36. Langberg M. Challenges to implementing CPOE: a case study of
a work in progree at Cedars-Sinai. Mod Physician. 2003;7:21–2.
37. Cedars-Sinai suspends CPOE use. http://www.ihealthbeat.org/
Accessed: March 20, 2006.
38. Rothschild JM, Federico FA, Gandhi TK, Kaushal R, Williams
DH, Bates DW. Analysis of medication-related malpractice
claims: causes, preventability, and costs. Arch Intern Med. Nov
39. Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman
GJ, Bates DW. A computer-based intervention for improving the
appropriateness of antiepileptic drug level monitoring. Am J
Clin Pathol. 2003;119:432–8.
40. Solomon DH, Shmerling RH, Schur PH, Lew R, Fiskio J, Bates
DW. A computer based intervention to reduce unnecessary sero-
logic testing. J Rheumatol. 1999;26:2578–84.
KAUSHAL ET AL., ROI of CPOE