Effect of Alerts for Drug Dosage Adjustment in Inpatients with Renal Insufficiency
Medication errors constitute a major problem in all hospitals. Between 20% and 46% of prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry decreases the proportion of inappropriate prescriptions based on the renal function of inpatients. Six alternating 2-month control and intervention periods were conducted between August 2006 and August 2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one of the 24 targeted drugs that required adjustment according to estimated glomerular filtration rate (eGFR). The main outcome measure was the proportion of inappropriate first prescriptions, according to recommendation. A total of 1,122 alerts were triggered. The rate of inappropriate first prescriptions did not differ significantly between intervention and control periods (19.9% vs. 21.3%; p=0.63). The effect of intervention differed significantly between residents and senior physicians (p=0.03). Residents tended to make fewer errors in intervention versus control periods (Odds ratio 0.69; 95% confidence interval 0.41 to 1.15), whereas senior physicians tended to make more inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% confidence interval 0.91 to 3.89). Alert activation was not followed by a significant decrease in inappropriate prescriptions in our study. Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.
Effect of Alerts for Drug Dosage Adjustment in Inpatients with
ELODIE SELLIER, MD, ISABELLE COLOMBET, MD, MPH, PHD, BRIGITTE SABATIER,PHARMD, PHD,
AELLE BRETON,PHARMD, JULIE NIES, MS, ERIC ZAPLETAL,PHDJEAN-BENOIT ARLET, MD,
OMINIQUE SOMME, MD, PIERRE DURIEUX, MD, MPH
Objectives: Medication errors constitute a major problem in all hospitals. Between 20% and 46% of
prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine
whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry
decreases the proportion of inappropriate prescriptions based on the renal function of inpatients.
Design: Six alternating 2-month control and intervention periods were conducted between August 2006 and August
2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were
included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one
of the 24 targeted drugs that required adjustment according to estimated glomerular ﬁltration rate (eGFR).
Measurements: The main outcome measure was the proportion of inappropriate ﬁrst prescriptions, according to
Results: A total of 1,122 alerts were triggered. The rate of inappropriate ﬁrst prescriptions did not differ signiﬁcantly
between intervention and control periods (19.9% vs. 21.3%; p ⫽ 0.63). The effect of intervention differed signiﬁcantly
between residents and senior physicians (p ⫽ 0.03). Residents tended to make fewer errors in intervention versus
control periods (Odds ratio 0.69; 95% conﬁdence interval 0.41 to 1.15), whereas senior physicians tended to make more
inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% conﬁdence interval 0.91 to 3.89).
Conclusion: Alert activation was not followed by a signiﬁcant decrease in inappropriate prescriptions in our study.
Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to
studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.
䡲 J Am Med Inform Assoc. 2009;16:203–210. DOI 10.1197/jamia.M2805.
Adverse events constitute a major problem in all hospitals, and
the most common are attributed to drugs. In the Harvard Medical
Study Practice II,
adverse drug events (ADEs) were estimated to
account for 19% of iatrogenic injuries. Adverse drug events are
associated with increases in the duration of hospital stay, addi-
tional costs, and mortality.
Medication errors, which account for
a large proportion (28%) of ADEs, are inherently preventable.
More than half of these errors occur at the ordering stage.
speciﬁcally, dosages are inappropriate for 20% to 46% of prescrip-
tions requiring dosage adjustments based on renal function.
Computerized physician order entry (CPOE)
review of all prescriptions by pharmacists
have been put
forward for reducing prescription errors. However, a recent
study performed in our hospital showed that pharmacy
validation produced only a moderate short-term impact on
decreasing potential prescription errors.
Only 26% of alerts
targeted to the prescribers resulted in a modiﬁcation of the
prescription. It was suggested that development of prescrip-
tion aids for drug-dose adjustment might prevent some
errors before the intervention of pharmacists, allowing them
to concentrate on the most relevant interventions.
Indeed, computerized clinical decision support systems
(CDSS) that automatically prompt users are often associated
with increased practitioner performance. Garg et al.
showed that 19 of 29 drug-dosing or prescribing systems
improved practitioner performance. However, in a recent
systematic review, Chaudhry et al.,
evaluating efﬁcacy of
Afﬁliations of the authors: Assistance Publique des Hôpitaux de
Paris-Georges Pompidou European Hospital, Medical Informatics
and Public Health Department (ES, IC, EZ, PD), Paris, France; Paris
Descartes University (IC, PD), Paris, France; Inserm (IC, JN, PD)
Paris, France; Pharmacy Department (BS, GB), Department of Inter-
nal Medicine (J-BA), Department of Geriatrics (DS), Georges Pom-
pidou European Hospital, Paris, France; MEDASYS (JN), Gif sur
Yvette Cedex, France.
Dr. Sellier was supported by a grant from the LEEM/Fédération
Hospitalière de France. J. Nies was a PhD student funded by a grant
from ANRT (CIFRE convention number: 498/2004).
The authors thank Jérôme Rosser, MD (nephrologist), Bruno Cassard
and Alexandre Stoehr (pharmacists) for developing dosage adjustment
tables, and Florence Gillaizeau (statistician) for comments during
Correspondence: Pierre Durieux, MD, MPH, Département
d’Informatique Hospitalière (DIH)-Evaluation et Gestion des Con-
naissances, Hôpital Européen Georges Pompidou, 20-40 Rue Leb-
lanc, 75908 Paris Cedex 15, France; e-mail: ⬍pierre.durieux@egp.
Received for review: 03/31/08; accepted for publication: 12/05/08
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009
health information technology in improving quality, pointed
out that of the 257 studies analyzed, 25% were concentrated
in four benchmark institutions. Thus, whether the results
reported would be similar for all institutions is questionable
and additional studies in other settings are needed.
We developed and implemented a system for drug dosage
adjustment integrated to the CPOE system in the two general
medical departments of our institution. The alert guided
physicians to adjust drug doses for patients suffering from
impaired renal function at the time of prescription. The main
objective of this study was to evaluate the effect of this alert
on the rate of inappropriate prescriptions. The second ob-
jective was methodological, i.e., to check whether an evalu-
ation based on routine data automatically retrieved from the
electronic health record (EHR) was feasible and reliable. We
suggested that the complete automation and computeriza-
tion of both the intervention and its evaluation would be a
condition for the long-term success of such a system.
Study Site and Setting
Our study was conducted in two medical departments of the
Georges Pompidou European Hospital, an 827-acute-bed
teaching hospital of the Assistance Publique-Hôpitaux de
Paris (metropolitan area public hospital network in Paris).
The hospital information system integrates an EHR (DxCare
MEDASYS) with CPOE. The EHR currently collates orders
and results of clinical tests and imaging procedures. The
DxCare EHR is at the center of care delivery. It is integrated
with other applications to allow the circulation of informa-
tion among wards, laboratories, and the pharmacy.
for laboratory tests are transmitted to the Laboratory Infor-
mation Management System, which returns the results to
the EHR. Drug prescriptions are transmitted to the Phedra
program, which is used by the pharmacy to manage pre-
All prescriptions are reviewed for interactive validation by
ward pharmacists. Four pharmacists (two senior pharma-
cists and two residents), assisted by two part-time pharmacy
students, perform these validations. The results of the phar-
macy validation can be accessed by the prescribers and/or
the nurses (depending on the pharmacist’s choice) via a
symbol inserted in front of a given prescription order line:
accepted (if the pharmacist agrees with the prescription),
refused (if the pharmacist has identiﬁed a potentially severe
prescribing error), or availability problem (if the drug
should be changed due to a problem of availability). The
physician may click on the symbol to view the pharmacist’s
comment, but is not obliged to take that comment into
This study was conducted between August 2006 and August
2007, and was based on an alternating time-series design,
with three 2-month control periods and three 2-month
intervention periods. All patients hospitalized in the Internal
Medicine Department or in the Geriatric Department and
prescribed one or more medications targeted by the alert
system were included in this study; this also included all
physicians working in these departments.
An expert panel, including nephrologists and pharmacists,
determined which renally cleared and/or nephrotoxic med-
ications were the most ordered in our institution. Using the
Summary of Product Characteristics and expert opinion, the
panel determined the dosage adjustments according to
the estimated glomerular ﬁltration rate (eGFR) for 24 drugs
(Table 1). The eGFR was estimated using the revised-4
component of the Modiﬁcation of Diet in Renal Disease
(MDRD) study equation,
except that we did not take into
account the race component because it is not validated in the
Required dosage adjustments were
based on level of kidney function impairment, which was
divided into three categories: mild (eGFR 60 to 79 mL/min/
), moderate (15 to 59 mL/min/1.73 m
) and ad-
vanced (⬍15 mL/min/1.73 m
). The expert panel deter-
mined the dose range and frequency for each of the
medications in each of these categories. During intervention
periods, the alert system was developed to compute these
recommendations and to automatically display the corre-
sponding alert if a targeted drug was selected for prescrip-
tion. Throughout DxCare, the alert system collected the data
transmitted from the Laboratory Information Management
System and required for computer-assisted decision making:
result of the last eGFR and its date. If the last eGFR was ⬍60
Table 1 y List of the Drugs Included in the Study
Drug Class Molecules Recommended Adjustments*
Antibiotics Vancomycin LD⫹ SC
Teicoplanin, Gentamicin, Amikacin, Tobramycin, LD⫹ I⫹SC
Ciproﬂoxacin, Levoﬂoxacin, Norﬂoxacin, LD⫹ I⫹SC
Erythromycin, Fosfomycin D
ACEI Ramipril, Perindropril, Captopril, Lisinopril D
B-adrenergic blocking agents Acebutolol, Bisoprolol, Sotalol D
Atenolol D⫹ 1
Antigout drugs Colchicine, Allopurinol D
Digoxin Digoxin, hemigoxin LD⫹ D⫹1
Antidiabetes drugs Metformin† D
ACEI ⫽ angiotensin converting enzyme inhibitor.
*An alert was triggered if the estimated glomerular ﬁltration rate (eGFR) of the patient was ⬍60 mL/min/m
: the recommendations could relate to
the loading dose (LD), the maintenance dose (D), the interval (I), and the need to further adjustment according to the serum concentration (SC).
†The metformin alert was triggered if eGFR ⬍ 80 mL/min/m
and medication was contraindicated for eGFR ⬍ 30 mL/min/m
204 Sellier et al., No Impact of an Alert for Drug Dosage
(or ⬍80 mL/min/1.73 m
for the met-
formin) and was measured ⬍5 days before the prescription
was written, a reminder displayed the last result of eGFR
and its date, the dosage adjustment table, and the reference
on the prescription screen (dose adjust alert, Figure 1). The
reminder appeared just after drug selection and before
dosage selection. If no eGFR was stored or if it was per-
formed 5 days or more before the prescription, the pre-
scriber was alerted to measure the GFR before prescribing
the drug (need for eGFR alert). The physician was not
required to acknowledge or justify why he chose to ignore
the electronic recommendation.
During control periods, no alert was displayed between
drug selection and drug completion. In control periods, as
well as in intervention periods, all prescriptions for drugs
targeted in the study were validated by the ward pharma-
cist, with physicians being able to access his recommenda-
Adjudication of Orders and Study Outcomes
Dose Adjust Alerts
A log was kept each time a reminder was displayed (in
intervention periods) or should have been displayed (in
control periods) to support the adjudication of dose adjust
alerts. The reviewer pharmacist (G.B.) judged each order
independently of the ward pharmacist by examining the
dose and frequency in combination with the recommenda-
tion that was displayed. There were three possible results:
appropriate dose if the dose and interval did not exceed the
recommendations displayed or if the drug was cancelled
before completion after being recommended (metformin
was indeed contraindicated when eGFR was ⬍30 mL/min/
), inappropriate dose if the dose or interval exceeded the
recommendations, and not applicable if the patient’s weight
was needed to calculate the adequate dose (for some antibi-
otics) and was unavailable in the database. The reviewer
pharmacist took no account of patient information, except
weight if necessary. To ensure consistency of the reviewer
pharmacist’s adjudication, a subset of 204 logs chosen at
random among the 700 ﬁrst logs was blindly rated by a
physician (E.S.). The physician was unaware of the pharma-
cist’s adjudication and the study period. Cases of discor-
dance were analyzed and consensus ratings were derived by
the two raters. The inter-rater reliability was reported using
the concordance rate and Kappa statistics.
Need for eGFR Alerts
To support the adjudication of the “need for eGFR” alerts, a
log was kept for all instances in which a targeted drug was
ordered without an eGFR measurement taken within the
previous 5 days. In this case, a log was considered inappro-
priate if it was followed by the completion of the prescrip-
tion and appropriate if the prescription was cancelled.
Our primary outcome assessed the proportion of inappro-
priate prescriptions among the ﬁrst prescriptions that re-
quired a dosage adjustment. A prescription was considered
a ﬁrst prescription if it was the ﬁrst order of a given
medication during a patient’s stay in the department, as
opposed to renewals or updates for the same medication
during his stay. The ﬁrst prescription could not be inﬂu-
enced by the ward pharmacist’s recommendations, in con-
trast to subsequent prescriptions. Thus, by including only
the ﬁrst prescription, we were able to estimate the effect of
alerts prior to input by ward pharmacists.
We also investigated secondary outcomes. First, we assessed
the proportion of inappropriate prescriptions among all
prescriptions that required a dosage adjustment (excluding
prescriptions adjusted on plasmatic concentration). In this
way, we evaluated the effect of alerts in addition to recom-
mendations of the ward pharmacist. We then evaluated the
Figure 1. Example of a dose-
adjusting alert. Translation: “Your
patient had an estimated glomeru-
lar ﬁltration rate of 10 mL/min/
on February 16, 2007, 14:04.
You can follow recommendations
above to adjust your order of cipro-
ﬂoxacin. Table, from top to bottom:
eGFR ⬎ 59 mL/min/1.73 m
age of 500 to 1,500 mg/day;
adjusted dosage/ordinary dos-
age ratio of 100%; eGFR of 15 to
59 mL/min/1.73 m
dosage/ordinary dosage ratio of
50% to 75%; eGFR ⬍ 15 mL/
; adjusted dosage/
ordinary dosage ratio of 50%.”
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009 205
proportion of prescriptions for which no eGFR was avail-
able, which were followed by the cancellation of the order
(rather than its completion).
In parallel, we also conducted a more automated process for
data extraction and validation. First, we directly selected
eligible prescriptions from the DxCare database, without
using recorded logs. Second, we ignored all prescriptions
that did not result in the drug being administered to the
patient (this method allowed us to provide the proportion of
drug dosage errors affecting patients). Third, we performed
an electronic validation using the recommendations set forth
by the expert panel. We ﬁnally assessed the proportion of
inappropriate ﬁrst prescriptions and the overall proportion
of inappropriate prescriptions among drugs effectively ad-
ministered to the patient.
Comparison of characteristics of patients and physicians
during intervention and control periods were tested using
the Chi square test and the Wilcoxon rank sum test when
appropriate. The nominal signiﬁcance level for the outcomes
was 0.05 (two-sided formulation). We assessed the effective-
ness of intervention on the rate of inappropriate prescrip-
tions using a logistic regression model. We estimated the
odds ratio (OR) for a prescription to be classiﬁed as inap-
propriate and its associated 95% conﬁdence interval (CI) in
intervention periods compared with control periods. Pre-
scriptions were not assumed to be independent due to the
hierarchical structure of data. Indeed, prescriptions (level 1)
were nested within patient and/or within physician clusters
(level 2). Patient and physician clusters were crossed at level
2: several physicians could have ordered a prescription for
one patient and a physician could have ordered prescrip-
tions for several patients. The departments (level 3) were
accounted for by including them as a dummy variable in the
model (i.e., ﬁxed effects), because only two departments
were involved. For each outcome, we determined the patient
effect on how appropriate the prescriptions were by testing
the signiﬁcance of variance between patients. We repeated
the procedure for the physician effect. The patient effect was
sometimes signiﬁcant; however, this was never the case for
the physician effect. Thus, if appropriate, we used a mixed
logistic model accounting for patients as a random effect. In
this case, explanatory variables (drug category, patient gen-
der, patient age, eGFR, prescriber gender, physician’s level
and department) were tested as ﬁxed effects. Parameters of
the model were estimated by a full maximum likelihood
method with adaptative quadrature.
In other cases, if the
patient effect was not signiﬁcant, we used an ordinary
multivariable logistic model.
We repeated analyses using the data retrieved from DxCare
and the automatic validation to assess how robust the results
were and to determine any meaningful effects of the selected
outcome and validation procedure.
All analyses were performed using STATA V9.0 (StataCorp,
College Station, Texas).
A total of 603 patients were included in the study, 321 in
control periods and 282 in intervention periods. Among
them, 294 (47.7 %) were ordered two or more targeted drugs.
A total of 38 physicians were involved (23 residents and 15
senior physicians), of which 29 made prescriptions in both
intervention and control periods, 3 only in intervention
periods, and 6 only in control periods. Senior physicians
were deﬁned as all qualiﬁed physicians, including academic
physicians (professors of internal medicine and assistant
professors) and full-time non-academic physicians. No sig-
niﬁcant differences were observed for patients’ and physi-
cians’ characteristics between the 2 groups (Tables 2 and 3).
During the study period, 1,122 logs were analyzed (Table 4).
A total of 955 logs corresponded to prescriptions for targeted
drugs that were ordered for patients with renal insufﬁciency
and that subsequently needed a dosage adjustment. Among
them, 707 were considered as ﬁrst prescriptions. The num-
ber and distribution by drug category are presented in Table
4. The remaining 167 logs corresponded to orders of targeted
drugs without a recent eGFR measurement.
The concordance rate was estimated at 90% (95% CI 0.86 to
0.94) and kappa at 0.72 (95% CI 0.61 to 0.83). Thirteen
instances of discordance were due to errors by the second
rater (E.S.). Nine were induced by the revalidation process,
which precludes the blinded physician from obtaining the
relevant data from the EHR required for validation. Seven
discordant validations were due to errors by the reviewer
pharmacist. The three drugs for which the pharmacist made
an error were systematically veriﬁed throughout the study
and corrected if necessary.
Rate of Inappropriate First Prescriptions
Seven of the 707 ﬁrst prescriptions evaluated by the re-
viewer pharmacist were rated as not applicable (0.99%) and
were dropped for subsequent analyses. During intervention
periods, 19.9% of orders (59 of 297) were inappropriate
versus 21.3% (86 of 403) in control periods (unadjusted odds
0.91, 95% CI 0.63 to 1.32, p ⫽ 0.63). Results according to
period are shown in Figure 2.
The rate of inappropriate prescriptions did not differ signif-
icantly between residents (62 of 288, 21.5%) and senior
physicians (24 of 115, 20.9%) in control periods (p ⫽ 0.88).
However, senior physicians made more errors (24 of 82,
29.3%) than residents (35 of 215, 16.3%) in intervention
periods (p ⫽ 0.01).
Table 2 y Comparison of Characteristics for
Inpatients Enrolled in Control and Intervention
(n ⫽ 321)
(n ⫽ 282) p Value
Age (yr), median (IQR) 82 (74–89) 84 (76–89) 0.12
Women, No. (%) 173 (53.9) 159 (56.4) 0.54
eGFR (mL/min/1.73 m
44 (34–52) 44 (34–52) 0.92
No. prescriptions per
patient, median (IQR)
1 (1–2) 1 (1–2) 0.83
eGFR ⫽ estimated glomerular ﬁltration rate; IQR ⫽ interquartile
*Data were missing for 72 patients (alerts with no GFR or result
dating from more than 5 days).
206 Sellier et al., No Impact of an Alert for Drug Dosage
In the multivariable analysis (Table 5, Model 1), the interac-
tion between intervention and prescriber level was signiﬁ-
cant, resulting in a different odds ratio for inappropriate
prescriptions between intervention and control periods for
residents (0.69, 95% CI 0.41 to 1.15) and for senior physicians
(1.88, 95% CI 0.91 to 3.89). The odds ratio of inappropriate
prescriptions for senior physicians compared with residents
during intervention periods was 2.35 (95% CI 1.53 to 4.60,
p ⫽ 0.03).
Drug category was strongly associated with whether a
prescription was inappropriate or not. Angiotensin-convert-
ing enzyme inhibitors were less prone to errors (6.7%).
Antibiotics accounted for most inappropriate prescriptions
(54.5%), as they were ordered often (26% of ﬁrst prescrip-
tions) and they were classiﬁed as inappropriate in 43.4% of
cases. B-adrenergic blocking agents, antigout drugs, and
digoxin were inappropriate in 11.9%, 15.5%, and 17.3% of
cases, respectively. Eleven of the 13 ﬁrst prescriptions for
metformin (84.6%) were inappropriate. Senior physicians
and residents did not order different drug classes (p ⫽ 0.36).
The effect of the intervention was not associated with the
drug class (p ⫽ 0.44).
There were no differences in the intervention effect between
the two departments and patient’s gender. Finally, an in-
crease in the age of the patient was associated with a
decrease in inappropriate prescriptions.
Figure 2. Rate of inappropriate ﬁrst prescriptions by
study periods. A. Rates of inappropriate ﬁrst prescriptions
for all physicians. B. Rates of inappropriate ﬁrst prescrip-
tions for residents. C. Rates of inappropriate ﬁrst prescrip-
tions for senior physicians.
Table 3 y Comparison of Characteristics for
Physicians Enrolled in Control and Intervention
(n ⫽ 35)*
(n ⫽ 32)* p Value
Women, No. (%) 20 (57) 18 (56) 0.94
Physician’s level, No. (%) 0.96
Residents 21 (60) 19 (59)
Senior physicians 14 (40) 13 (41)
No. prescriptions per
physician, median (IQR)
14 (7–23) 12 (2–22) 0.37
Internal medicine 21 (60) 21 (66)
Geriatric 14 (40) 11 (34)
IQR ⫽ interquartile range.
*Twenty-nine physicians made prescriptions in both intervention
and control periods, three only in intervention periods, and six only
in control periods.
Table 4 y Type of Recommendation and Drug
Category of Prescriptions Included in Control and
Type of Recommendation
Periods p Value*
Total 587 535
Need for measuring eGFR 64 103
Need for dosage adjustment 523 432
First prescriptions 406 301
Following prescriptions 117 131
Drug category†‡ 0.05
ACEI 136 (26.0) 147 (34.0)
B-adrenergic blocking agent 136 (26.0) 117 (27.1)
Antibiotic 141 (27.0) 87 (20.1)
Antigout drug 63 (12.0) 41 (9.5)
Digoxin 36 (6.9) 30 (7.0)
Antidiabetes drug 11 (2.1) 10 (2.3)
ACEI ⫽ angiotensin-converting enzyme inhibitor; eGFR ⫽ esti-
mated glomerular ﬁltration rate.
*Chi-square test comparing drug categories in intervention and
†Drug category (medications included): ACEI (ramipril, perindro-
pril, captopril, lisinopril); B-adrenergic blocking agents (acebutolol,
atenolol, bisoprolol, sotalol); antibiotics (ciproﬂoxacin, erythromycin,
gentamicin, vancomycin, levoﬂoxacin, sulfamethoxazole-timethropim,
amikacin, teicoplanin, tobramycin, norﬂoxacin, and fosfomycin); anti-
gout drug (colchicine, allopurinol); digoxin; antidiabetes drug (met-
‡Values are expressed as number (percentage of total prescriptions
needing dosage adjustment).
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009
Rate of Overall Inappropriate Prescriptions
Eight of the 955 prescriptions were rated as not applicable,
and ﬁve others were adjusted on plasmatic concentration.
These 13 prescriptions (1.4%) were subsequently excluded
for analyses. During control periods, 20.4% of orders were
inappropriate (106 of 520), whereas 18.5% (78 of 422) were
inappropriate in intervention periods (unadjusted OR 0.81;
95% CI 0.51 to 1.28, p ⫽ 0.37). In multivariable analysis
(Table 5, Model 2), the results were similar as previously
outlined, except for the interaction between the physician’s
level and intervention, which was not signiﬁcant.
Models 3 and 4 in Table 5 present the results obtained from
the analyses of data retrieved exclusively from the CPOE
and whose validation was automated. Only prescriptions
that were actually administered to patients were included.
The results were similar to those obtained previously and, as
for models with the validation carried out by the pharmacist
(Models 1 and 2), the patient effect was signiﬁcant only if all
prescriptions, not only the ﬁrst one, were included for
Rate of Prescriptions Cancelled if No eGFR Was
In control periods, 20 of 64 logs were followed by cancella-
tion of the prescription (31.3%), in contrast to 36 of 103 logs
(35.0%) in the intervention periods. The unadjusted odds of
having an appropriate order was 1.18 (95% CI 0.61 to 2.30,
p ⫽ 0.62) in intervention periods compared with control
Considerable attention has been drawn to electronic health
record systems as a means of reducing medication errors.
We implemented alerts for drug dosing adjustment into the
daily routine use of a CPOE, through a design for interven-
tion shown to be effective in previous studies.
ever, the intervention was not associated with a signiﬁcant
decrease in the number of inappropriate prescriptions for
inpatients with renal insufﬁciency. The design we chose was
the best alternative to a randomized controlled trial due to
the particular features of our study. Indeed, it was not
possible to randomize physicians to intervention or control
groups due to a contamination effect. Also, it was not
appropriate to randomize units, because there were only
two units with a different recruitment of patients, which
would have led to a baseline imbalance between the two
groups. This alternative design enabled us to avoid selection
bias. Furthermore, we made substantial efforts to control for
possible confounding factors, and our analyses accounted
for the clustering of the data (multilevel models).
Moreover, we conﬁrmed our results by various methods of
data retrieval and evaluation. We were able to retrieve all of
the relevant data from the hospital information system and
conduct an electronic validation of the prescriptions. The
overall results were similar for whichever method we used.
Although concordance and discordance cases for various
methods were not subtly evaluated, our results suggest that
it is possible to obtain a long-term indicator of physician
performance or that of the intervention effect without addi-
tional resources (i.e., log recording, manual validation).
Consequently, despite the negative results of our study, it is
conceivable that in the future the effect of these alerts may be
evaluated in other departments of our hospital.
In our study, the rate of inappropriate prescriptions during
control periods was 21.3%. This good performance may
represent a ceiling effect, which may have limited the impact
Table 5 y Adjusted Odds Ratio of Inappropriate Prescriptions According to the Method for Validation
(Pharmacist or Electronic) and to the Type of Prescriptions Included (First or All)
Electronic Validation First
Given to Patients
Electronic Validation All
Given to Patients
OR (CI 95%) N ⫽ 700 N ⫽ 942 N ⫽ 673 N ⫽ 843
Residents 0.69 (0.41–1.15) 0.70 (0.36–1.36) 0.70 (0.42–1.18) 0.75 (0.44–1.30)
Senior physicians 1.88 (0.91–3.89) 1.72 (0.92–3.23) 1.66 (0.82–3.36) 1.79 (0.85–3.75)
Patient’s age (yr) 0.98 (0.97–1.00)† 0.97 (0.95–0.99)† 0.98 (0.96–0.99)‡ 0.97 (0.95–0.99)‡
Drug category § § § §
ACEI 1.0 1.0 1.0 1.0
B-adrenergic blocking agent 2.0 (1.0–4.2) 3.4 (1.5–7.5) 3.8 (1.8–7.9) 5.3 (2.5–11.1)
Antibiotic 10.7 (5.5–20.7) 27.2 (10.7–69.3) 16.1 (7.8–33.2) 23.6 (10.3–54.2)
Antigout drug 2.5 (1.1–6.1) 3.3 (1.2–8.6) 3.2 (1.3–7.8) 3.2 (1.3–7.6)
Digoxin 3.2 (1.3–8.3) 7.3 (2.3–23.6) 4.4 (1.6–11.7) 6.4 (2.3–17.7)
Antidiabetes drug 78.0 (15.2–400.0) 527 (56–4785) — —
Model 1 and Model 3: ORs were estimated in an ordinary multivariable logistic regression model (no patient effect). Model 2 and Model 4: ORs
were estimated in mixed multivariable logistic regression model (signiﬁcant patient effect).
ACEI ⫽ angiotensin converting enzyme inhibitor; CI ⫽ conﬁdence interval; OR ⫽ odds ratio.
*Due to the signiﬁcance of interaction term Intervention
Physician’s level in Model 1, the odds ratio for the term Intervention was different
for residents and senior physicians. The odds ratio of inappropriate prescriptions for senior physicians in intervention versus control periods
was obtained by multiplying the odds ratio associated with the variable Intervention and the interaction term Intervention
†p ⬍ 0.05.
‡p ⬍ 0.01.
§p ⬍ 0.001.
208 Sellier et al., No Impact of an Alert for Drug Dosage
of our intervention. Some factors may contribute to the
quality of prescriptions in our hospital: these include access,
through the CPOE, to the Vidal electronic database and the
role of the ward pharmacists. Moreover, the purpose of our
alerts was to avoid excessive doses regardless of the renal
function only, and the error rates could have been greater if
we had taken into consideration the indication of each order
(reducing consequently the recommended dosing range for
We developed our alert to incorporate evidence-based fea-
associated with the success of decision support: we
used computers to deliver support, offering speciﬁc recom-
mendations and not just an assessment. Our application
generated alerts automatically as part of the clinical work-
ﬂow, at the time and the place that the decision was made.
Indeed, if a system providing recommendations is depen-
dent on the initiative of the physicians for use, they seldom
make the required effort.
However, the manner chosen to
provide overdosing alerts in our study may partly explain
the negative results. Our alert was possibly not directive
enough for the physician. Indeed, in the study by Chertow et
the investigators implemented and evaluated a system
for adjusting drug doses for inpatients with renal insufﬁ-
ciency using a similar design to that reported here. During
intervention periods, the adjusted dose list, default dose
amount, and default interval were displayed to the prescrib-
ers. The intervention resulted in a signiﬁcant decrease in
inappropriate dosages, from 67% to 54%. Similarly, Oppen-
heim et al.
implemented an alert displaying the correct
dosage of a drug only when a wrong dosage was selected by
the physician. Half of the orders generating alerts were
adjusted in response to alerts. In these two previous studies,
the exact dosage was suggested to the prescriber for each
prescription, whereas we provided an adjustment table as
rationale for advice. Also, the patients included in our study
were older than those in other studies that were investigated
with similar interventions.
As in a previous study,
we observed a signiﬁcant differ-
ence of the intervention effect between residents and senior
physicians: residents tended to improve their performance
on receipt of alerts, whereas senior physicians tended to
make more errors during similar periods. Although the
difference of inappropriate prescriptions was not signiﬁ-
cantly higher in intervention than in control periods among
senior physicians, it is worth trying to explain this unantic-
ipated effect. First, in French academic institutions, residents
write most prescriptions for hospitalized patients. Senior
physicians are likely to write prescriptions when doing the
weekly or bi-weekly review of all patients in the unit; this
review is carried out more frequently for more complicated
patients, which residents are unable to cope with, especially
young residents. Even if potentially there are situations in
which residents and senior physicians wrote different pre-
scriptions, and our multivariate analysis were unable to
adjust for these situations, it is unlikely that they occurred
differently in intervention and control periods. Second,
senior physicians may have judged that the beneﬁt of a
chosen drug-dosing regimen outweighed the risk of an
excessive dose, or they might have disagreed with the expert
panel’s recommendations. It is also likely that senior physi-
cians preferred disregarding the alerts in favor of their own
established practice, whereas younger physicians were
keener to improve and were more receptive to new infor-
mation. Although these factors may explain the absence of
an effect of intervention among senior physicians, they fail
to explain an increase in errors. Third, we cannot exclude
that the senior physicians may have misinterpreted informa-
tion relating to the dose, which may have been unclear.
Indeed, unclear dose-related information within the context
of alerts and an absence of time could lead to the misinter-
pretation of the alerts.
Finally, the intervention phases may
have induced a false sense of security among senior physi-
cians, because they knew these alerts were part of a research
Our study was subject to several limitations. First, the
intervention was conducted in only one hospital with its
own CPOE and health information system. Second, only two
departments were included, and thus a small number of
physicians. However, they were the most relevant depart-
ments in relation to this study, because they gathered the
oldest patients, who consequently are the most likely to
suffer from renal insufﬁciency. Third, our study may have
lacked power for detecting a difference between the two
groups. However, it is obvious that the alerts had no
meaningful clinical impact. Fourth, eGFR was calculated
thanks to the Modiﬁcation of Diet in Renal Disease
(MDRD) formula, which is imprecise for elderly patients,
but is more accurate than the Cockroft-Gault equation.
There is also little evidence on drug dose recommenda-
tions in elderly patients, because the randomized con-
trolled trials often exclude elderly patients. Finally, the
MDRD formula does not accurately reﬂect renal function
in unsteady-state conditions (such as acute increase or
decrease of creatinine), so the alert may have delivered
inappropriate recommendations in these cases. However,
patients with these unstable conditions are more likely to
be hospitalized in intensive care or in a renal unit in our
institution. If this was not the case, the ward pharmacist
could have alerted the clinician.
We assessed the effect of the intervention as a result of
process measures rather than patient outcomes measures
because process measures are the most suitable tool for
judging quality of care.
Our results contrast with those of other recent reviews,
which conclude the effectiveness of CDSS for improving
physician performance. However, these reviews can be
biased by the fact that negative studies are less published
than positive ones.
Moreover, Chaudhry et al.
caution in interpreting the results of identiﬁed studies of
CDSS because of ﬂaws in their design and analysis. Further-
more, positive studies have been carried out for the most
part by the developers of these systems. Finally, the results
of these reviews contradict other studies, which identify
high rates of alerts being overridden by physicians.
implementation of effective CDSS remains a challenging
and is not yet the miracle drug for improving the
performance of physicians. It is still necessary to evaluate
their impact if newly implemented in other settings and to
monitor for unanticipated effects and their underlying
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009 209
1. Leape LL, Brennan TA, Laird N, et al. The nature of adverse
events in hospitalised patients. Results of the Harvard Medical
Practice Study II. N Engl J Med 1991;324:377– 84.
2. Bates DW, Teich JM, Lee J, et al. The impact of computerized
physician order entry on medication error prevention. J Am
Med Inform Assoc 1999;6:313–21.
3. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse
drug events and potential adverse drug events. Implications
for prevention. ADE Prevention Study Group. JAMA 1995;
4. Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin
GA. The epidemiology of prescribing errors. Arch Intern Med
5. Salomon L, Deray G, Jaudon MC, et al. Medication misuse in
hospitalized patient with renal impairment. Int J Qual Health
Care 2003;15:235– 40.
6. A van Dijk E, Drabbe NRG, Kruijtbosch M, De Smet P AGM.
Drug dosage adjustments according to renal function at hospital
discharge. Ann Pharmacother 2006;40:1254 –60.
7. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication
dosing for inpatients with renal insufﬁciency. JAMA 2001;286:
8. Kaushal R, Shojania KG, Bates DW. Effect of computerized
physician order entry and clinical decision in support systems
on medication safety: a systematic review. Arch Intern Med
9. Leape LL, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ,
Erickson JI. Pharmacist participation on physician rounds and
adverse drug events in the intensive care unit. JAMA 1999;282:
10. Estellat C, Colombet I, Vautier S, Huault-Quentel J, Durieux P,
Sabatier B. Impact of pharmacy validation in a computerized
physician order entry context. Int J Qual Health Care 2007;19:
11. Garg AX, Adhikari NK, McDonald H, et al. Effects of comput-
erized clinical decision support systems on practitioner perfor-
mance and patient outcomes. A systematic review. JAMA
12. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of
health information technology on quality, efﬁciency and costs of
medical care. Ann Intern Med 2006;144:E12–E22.
13. Degoulet P, Marin L, Lavril M, et al. The HEGP component-
based clinical information system. Int J Med Inform 2003;69:
14. Levey AS, Greene T, Kusek JW, Beck GJ. A simpliﬁed equation
to predict glomerular ﬁltration rate from serum creatinine. J Am
Soc Nephrol 2000;11:155A.
15. Froissart M, Rossert J, Jacquot C, Paillard M, Houillier P.
Predictive performance of the modiﬁcation of diet in renal
disease and Cockroft-Gault equations for estimating renal func-
tion. J Am Soc Nephrol 2000;16:763–73.
16. Rabe-Hesketh S, Skrondal A, Pickles A. GLLAMM manual. UC
Berkeley Division of Biostatistics Working Paper Series, 2004.
Available at: http://www.bepress.com/ucbbiostat/paper160.
Accessed January 7, 2009.
17. Rabe-Hesketh S, Skrondal A, Pickles A. Reliable estimation of
generalized linear mixed models using adaptative quadrature.
Stata J 2002;2:1–21.
18. Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E. A clinical
decision support system for prevention of venous thromboem-
bolism: effect on physician behavior. JAMA 2000;283:2816–21.
19. Weingarten SR, Riedinger MS, Conner L, et al. Practice guide-
lines and reminders to reduce duration of hospital stay for
patients with chest pain: an intervention trial. Ann Intern Med
20. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving
clinical practice using clinical decision support systems: a sys-
tematic review of trial to identify features critical to success.
BMJ 2005;330:765– 8.
21. Eccles M, McColl E, Steen N, et al. Effect of computerised
evidence based guidelines on management of asthma and
angina in adults in primary care: cluster randomised trial. BMJ
22. Oppenheim M, Vidal C, Velasco F, et al. Impact of a computer-
ized alert during physician order entry on medication dosing in
patients with renal impairment. Proc AMIA Symp 2002:557–81.
23. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB,
Phillips RS. Physicians’ decisions to override computerized
drug alerts in primary care. Arch Intern Med 2003;163:2625–31.
24. Van der Sijs, Aarts J, Vulto A, Berg M. Overriding of drug safety
alerts in computerized physician order entry. J Am Med Inform
Assoc 2006;13:138 – 47.
25. Lilford RJ, Brown CA, Nicholl J. Use of process measures to
monitor the quality of clinical practice. BMJ 2007;335:648 –50.
26. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication
and related biases. Health Technol Assess 2000;4:1–115.
27. Wears RL, Berg A. Computer technology and clinical work.
210 Sellier et al., No Impact of an Alert for Drug Dosage