Effect of Alerts for Drug Dosage Adjustment in Inpatients with Renal Insufficiency

Article (PDF Available)inJournal of the American Medical Informatics Association 16(2):203-10 · March 2009with28 Reads
DOI: 10.1197/jamia.M2805 · Source: PubMed
Abstract
Medication errors constitute a major problem in all hospitals. Between 20% and 46% of prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry decreases the proportion of inappropriate prescriptions based on the renal function of inpatients. Six alternating 2-month control and intervention periods were conducted between August 2006 and August 2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one of the 24 targeted drugs that required adjustment according to estimated glomerular filtration rate (eGFR). The main outcome measure was the proportion of inappropriate first prescriptions, according to recommendation. A total of 1,122 alerts were triggered. The rate of inappropriate first prescriptions did not differ significantly between intervention and control periods (19.9% vs. 21.3%; p=0.63). The effect of intervention differed significantly between residents and senior physicians (p=0.03). Residents tended to make fewer errors in intervention versus control periods (Odds ratio 0.69; 95% confidence interval 0.41 to 1.15), whereas senior physicians tended to make more inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% confidence interval 0.91 to 3.89). Alert activation was not followed by a significant decrease in inappropriate prescriptions in our study. Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.
Research Paper
Effect of Alerts for Drug Dosage Adjustment in Inpatients with
Renal Insufficiency
ELODIE SELLIER, MD, ISABELLE COLOMBET, MD, MPH, PHD, BRIGITTE SABATIER,PHARMD, PHD,
G
AELLE BRETON,PHARMD, JULIE NIES, MS, ERIC ZAPLETAL,PHDJEAN-BENOIT ARLET, MD,
D
OMINIQUE SOMME, MD, PIERRE DURIEUX, MD, MPH
Abstract
Objectives: Medication errors constitute a major problem in all hospitals. Between 20% and 46% of
prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine
whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry
decreases the proportion of inappropriate prescriptions based on the renal function of inpatients.
Design: Six alternating 2-month control and intervention periods were conducted between August 2006 and August
2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were
included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one
of the 24 targeted drugs that required adjustment according to estimated glomerular filtration rate (eGFR).
Measurements: The main outcome measure was the proportion of inappropriate first prescriptions, according to
recommendation.
Results: A total of 1,122 alerts were triggered. The rate of inappropriate first prescriptions did not differ significantly
between intervention and control periods (19.9% vs. 21.3%; p 0.63). The effect of intervention differed significantly
between residents and senior physicians (p 0.03). Residents tended to make fewer errors in intervention versus
control periods (Odds ratio 0.69; 95% confidence interval 0.41 to 1.15), whereas senior physicians tended to make more
inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% confidence interval 0.91 to 3.89).
Conclusion: Alert activation was not followed by a significant decrease in inappropriate prescriptions in our study.
Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to
studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.
J Am Med Inform Assoc. 2009;16:203–210. DOI 10.1197/jamia.M2805.
Introduction
Adverse events constitute a major problem in all hospitals, and
the most common are attributed to drugs. In the Harvard Medical
Study Practice II,
1
adverse drug events (ADEs) were estimated to
account for 19% of iatrogenic injuries. Adverse drug events are
associated with increases in the duration of hospital stay, addi-
tional costs, and mortality.
2
Medication errors, which account for
a large proportion (28%) of ADEs, are inherently preventable.
3
More than half of these errors occur at the ordering stage.
3,4
More
specifically, dosages are inappropriate for 20% to 46% of prescrip-
tions requiring dosage adjustments based on renal function.
5–7
Computerized physician order entry (CPOE)
2,8
and the
review of all prescriptions by pharmacists
9
have been put
forward for reducing prescription errors. However, a recent
study performed in our hospital showed that pharmacy
validation produced only a moderate short-term impact on
decreasing potential prescription errors.
10
Only 26% of alerts
targeted to the prescribers resulted in a modification of the
prescription. It was suggested that development of prescrip-
tion aids for drug-dose adjustment might prevent some
errors before the intervention of pharmacists, allowing them
to concentrate on the most relevant interventions.
Indeed, computerized clinical decision support systems
(CDSS) that automatically prompt users are often associated
with increased practitioner performance. Garg et al.
11
showed that 19 of 29 drug-dosing or prescribing systems
improved practitioner performance. However, in a recent
systematic review, Chaudhry et al.,
12
evaluating efficacy of
Affiliations of the authors: Assistance Publique des Hôpitaux de
Paris-Georges Pompidou European Hospital, Medical Informatics
and Public Health Department (ES, IC, EZ, PD), Paris, France; Paris
Descartes University (IC, PD), Paris, France; Inserm (IC, JN, PD)
Paris, France; Pharmacy Department (BS, GB), Department of Inter-
nal Medicine (J-BA), Department of Geriatrics (DS), Georges Pom-
pidou European Hospital, Paris, France; MEDASYS (JN), Gif sur
Yvette Cedex, France.
Dr. Sellier was supported by a grant from the LEEM/Fédération
Hospitalière de France. J. Nies was a PhD student funded by a grant
from ANRT (CIFRE convention number: 498/2004).
The authors thank Jérôme Rosser, MD (nephrologist), Bruno Cassard
and Alexandre Stoehr (pharmacists) for developing dosage adjustment
tables, and Florence Gillaizeau (statistician) for comments during
statistical analyses.
Correspondence: Pierre Durieux, MD, MPH, Département
d’Informatique Hospitalière (DIH)-Evaluation et Gestion des Con-
naissances, Hôpital Européen Georges Pompidou, 20-40 Rue Leb-
lanc, 75908 Paris Cedex 15, France; e-mail: pierre.durieux@egp.
aphp.fr.
Received for review: 03/31/08; accepted for publication: 12/05/08
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009
203
health information technology in improving quality, pointed
out that of the 257 studies analyzed, 25% were concentrated
in four benchmark institutions. Thus, whether the results
reported would be similar for all institutions is questionable
and additional studies in other settings are needed.
We developed and implemented a system for drug dosage
adjustment integrated to the CPOE system in the two general
medical departments of our institution. The alert guided
physicians to adjust drug doses for patients suffering from
impaired renal function at the time of prescription. The main
objective of this study was to evaluate the effect of this alert
on the rate of inappropriate prescriptions. The second ob-
jective was methodological, i.e., to check whether an evalu-
ation based on routine data automatically retrieved from the
electronic health record (EHR) was feasible and reliable. We
suggested that the complete automation and computeriza-
tion of both the intervention and its evaluation would be a
condition for the long-term success of such a system.
Methods
Study Site and Setting
Our study was conducted in two medical departments of the
Georges Pompidou European Hospital, an 827-acute-bed
teaching hospital of the Assistance Publique-Hôpitaux de
Paris (metropolitan area public hospital network in Paris).
The hospital information system integrates an EHR (DxCare
®
,
MEDASYS) with CPOE. The EHR currently collates orders
and results of clinical tests and imaging procedures. The
DxCare EHR is at the center of care delivery. It is integrated
with other applications to allow the circulation of informa-
tion among wards, laboratories, and the pharmacy.
13
Orders
for laboratory tests are transmitted to the Laboratory Infor-
mation Management System, which returns the results to
the EHR. Drug prescriptions are transmitted to the Phedra
®
program, which is used by the pharmacy to manage pre-
scriptions.
All prescriptions are reviewed for interactive validation by
ward pharmacists. Four pharmacists (two senior pharma-
cists and two residents), assisted by two part-time pharmacy
students, perform these validations. The results of the phar-
macy validation can be accessed by the prescribers and/or
the nurses (depending on the pharmacist’s choice) via a
symbol inserted in front of a given prescription order line:
accepted (if the pharmacist agrees with the prescription),
refused (if the pharmacist has identified a potentially severe
prescribing error), or availability problem (if the drug
should be changed due to a problem of availability). The
physician may click on the symbol to view the pharmacist’s
comment, but is not obliged to take that comment into
account.
10
Population
This study was conducted between August 2006 and August
2007, and was based on an alternating time-series design,
with three 2-month control periods and three 2-month
intervention periods. All patients hospitalized in the Internal
Medicine Department or in the Geriatric Department and
prescribed one or more medications targeted by the alert
system were included in this study; this also included all
physicians working in these departments.
Intervention
An expert panel, including nephrologists and pharmacists,
determined which renally cleared and/or nephrotoxic med-
ications were the most ordered in our institution. Using the
Summary of Product Characteristics and expert opinion, the
panel determined the dosage adjustments according to
the estimated glomerular filtration rate (eGFR) for 24 drugs
(Table 1). The eGFR was estimated using the revised-4
component of the Modification of Diet in Renal Disease
(MDRD) study equation,
14
except that we did not take into
account the race component because it is not validated in the
European context.
15
Required dosage adjustments were
based on level of kidney function impairment, which was
divided into three categories: mild (eGFR 60 to 79 mL/min/
1.73 m
2
), moderate (15 to 59 mL/min/1.73 m
2
) and ad-
vanced (15 mL/min/1.73 m
2
). The expert panel deter-
mined the dose range and frequency for each of the
medications in each of these categories. During intervention
periods, the alert system was developed to compute these
recommendations and to automatically display the corre-
sponding alert if a targeted drug was selected for prescrip-
tion. Throughout DxCare, the alert system collected the data
transmitted from the Laboratory Information Management
System and required for computer-assisted decision making:
result of the last eGFR and its date. If the last eGFR was 60
Table 1 y List of the Drugs Included in the Study
Drug Class Molecules Recommended Adjustments*
Antibiotics Vancomycin LD SC
Teicoplanin, Gentamicin, Amikacin, Tobramycin, LD ISC
Ciprofloxacin, Levofloxacin, Norfloxacin, LD ISC
Erythromycin, Fosfomycin D
Sulfamethoxazole-Timethropim I
ACEI Ramipril, Perindropril, Captopril, Lisinopril D
B-adrenergic blocking agents Acebutolol, Bisoprolol, Sotalol D
Atenolol D 1
Antigout drugs Colchicine, Allopurinol D
Digoxin Digoxin, hemigoxin LD D1
Antidiabetes drugs Metformin† D
ACEI angiotensin converting enzyme inhibitor.
*An alert was triggered if the estimated glomerular filtration rate (eGFR) of the patient was 60 mL/min/m
2
: the recommendations could relate to
the loading dose (LD), the maintenance dose (D), the interval (I), and the need to further adjustment according to the serum concentration (SC).
†The metformin alert was triggered if eGFR 80 mL/min/m
2
and medication was contraindicated for eGFR 30 mL/min/m
2
.
204 Sellier et al., No Impact of an Alert for Drug Dosage
mL/min/1.73 m
2
(or 80 mL/min/1.73 m
2
for the met-
formin) and was measured 5 days before the prescription
was written, a reminder displayed the last result of eGFR
and its date, the dosage adjustment table, and the reference
on the prescription screen (dose adjust alert, Figure 1). The
reminder appeared just after drug selection and before
dosage selection. If no eGFR was stored or if it was per-
formed 5 days or more before the prescription, the pre-
scriber was alerted to measure the GFR before prescribing
the drug (need for eGFR alert). The physician was not
required to acknowledge or justify why he chose to ignore
the electronic recommendation.
During control periods, no alert was displayed between
drug selection and drug completion. In control periods, as
well as in intervention periods, all prescriptions for drugs
targeted in the study were validated by the ward pharma-
cist, with physicians being able to access his recommenda-
tions.
Adjudication of Orders and Study Outcomes
Dose Adjust Alerts
A log was kept each time a reminder was displayed (in
intervention periods) or should have been displayed (in
control periods) to support the adjudication of dose adjust
alerts. The reviewer pharmacist (G.B.) judged each order
independently of the ward pharmacist by examining the
dose and frequency in combination with the recommenda-
tion that was displayed. There were three possible results:
appropriate dose if the dose and interval did not exceed the
recommendations displayed or if the drug was cancelled
before completion after being recommended (metformin
was indeed contraindicated when eGFR was 30 mL/min/
m
2
), inappropriate dose if the dose or interval exceeded the
recommendations, and not applicable if the patient’s weight
was needed to calculate the adequate dose (for some antibi-
otics) and was unavailable in the database. The reviewer
pharmacist took no account of patient information, except
weight if necessary. To ensure consistency of the reviewer
pharmacist’s adjudication, a subset of 204 logs chosen at
random among the 700 first logs was blindly rated by a
physician (E.S.). The physician was unaware of the pharma-
cist’s adjudication and the study period. Cases of discor-
dance were analyzed and consensus ratings were derived by
the two raters. The inter-rater reliability was reported using
the concordance rate and Kappa statistics.
Need for eGFR Alerts
To support the adjudication of the “need for eGFR” alerts, a
log was kept for all instances in which a targeted drug was
ordered without an eGFR measurement taken within the
previous 5 days. In this case, a log was considered inappro-
priate if it was followed by the completion of the prescrip-
tion and appropriate if the prescription was cancelled.
Our primary outcome assessed the proportion of inappro-
priate prescriptions among the first prescriptions that re-
quired a dosage adjustment. A prescription was considered
a first prescription if it was the first order of a given
medication during a patient’s stay in the department, as
opposed to renewals or updates for the same medication
during his stay. The first prescription could not be influ-
enced by the ward pharmacist’s recommendations, in con-
trast to subsequent prescriptions. Thus, by including only
the first prescription, we were able to estimate the effect of
alerts prior to input by ward pharmacists.
We also investigated secondary outcomes. First, we assessed
the proportion of inappropriate prescriptions among all
prescriptions that required a dosage adjustment (excluding
prescriptions adjusted on plasmatic concentration). In this
way, we evaluated the effect of alerts in addition to recom-
mendations of the ward pharmacist. We then evaluated the
Figure 1. Example of a dose-
adjusting alert. Translation: “Your
patient had an estimated glomeru-
lar filtration rate of 10 mL/min/
1.73 m
2
on February 16, 2007, 14:04.
You can follow recommendations
above to adjust your order of cipro-
floxacin. Table, from top to bottom:
eGFR 59 mL/min/1.73 m
2
; dos-
age of 500 to 1,500 mg/day;
adjusted dosage/ordinary dos-
age ratio of 100%; eGFR of 15 to
59 mL/min/1.73 m
2
; adjusted
dosage/ordinary dosage ratio of
50% to 75%; eGFR 15 mL/
min/1.73 m
2
; adjusted dosage/
ordinary dosage ratio of 50%.”
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009 205
proportion of prescriptions for which no eGFR was avail-
able, which were followed by the cancellation of the order
(rather than its completion).
Methodological Outcome
In parallel, we also conducted a more automated process for
data extraction and validation. First, we directly selected
eligible prescriptions from the DxCare database, without
using recorded logs. Second, we ignored all prescriptions
that did not result in the drug being administered to the
patient (this method allowed us to provide the proportion of
drug dosage errors affecting patients). Third, we performed
an electronic validation using the recommendations set forth
by the expert panel. We finally assessed the proportion of
inappropriate first prescriptions and the overall proportion
of inappropriate prescriptions among drugs effectively ad-
ministered to the patient.
Statistical Analysis
Comparison of characteristics of patients and physicians
during intervention and control periods were tested using
the Chi square test and the Wilcoxon rank sum test when
appropriate. The nominal significance level for the outcomes
was 0.05 (two-sided formulation). We assessed the effective-
ness of intervention on the rate of inappropriate prescrip-
tions using a logistic regression model. We estimated the
odds ratio (OR) for a prescription to be classified as inap-
propriate and its associated 95% confidence interval (CI) in
intervention periods compared with control periods. Pre-
scriptions were not assumed to be independent due to the
hierarchical structure of data. Indeed, prescriptions (level 1)
were nested within patient and/or within physician clusters
(level 2). Patient and physician clusters were crossed at level
2: several physicians could have ordered a prescription for
one patient and a physician could have ordered prescrip-
tions for several patients. The departments (level 3) were
accounted for by including them as a dummy variable in the
model (i.e., fixed effects), because only two departments
were involved. For each outcome, we determined the patient
effect on how appropriate the prescriptions were by testing
the significance of variance between patients. We repeated
the procedure for the physician effect. The patient effect was
sometimes significant; however, this was never the case for
the physician effect. Thus, if appropriate, we used a mixed
logistic model accounting for patients as a random effect. In
this case, explanatory variables (drug category, patient gen-
der, patient age, eGFR, prescriber gender, physician’s level
and department) were tested as fixed effects. Parameters of
the model were estimated by a full maximum likelihood
method with adaptative quadrature.
16,17
In other cases, if the
patient effect was not significant, we used an ordinary
multivariable logistic model.
We repeated analyses using the data retrieved from DxCare
and the automatic validation to assess how robust the results
were and to determine any meaningful effects of the selected
outcome and validation procedure.
All analyses were performed using STATA V9.0 (StataCorp,
College Station, Texas).
Results
A total of 603 patients were included in the study, 321 in
control periods and 282 in intervention periods. Among
them, 294 (47.7 %) were ordered two or more targeted drugs.
A total of 38 physicians were involved (23 residents and 15
senior physicians), of which 29 made prescriptions in both
intervention and control periods, 3 only in intervention
periods, and 6 only in control periods. Senior physicians
were defined as all qualified physicians, including academic
physicians (professors of internal medicine and assistant
professors) and full-time non-academic physicians. No sig-
nificant differences were observed for patients’ and physi-
cians’ characteristics between the 2 groups (Tables 2 and 3).
During the study period, 1,122 logs were analyzed (Table 4).
A total of 955 logs corresponded to prescriptions for targeted
drugs that were ordered for patients with renal insufficiency
and that subsequently needed a dosage adjustment. Among
them, 707 were considered as first prescriptions. The num-
ber and distribution by drug category are presented in Table
4. The remaining 167 logs corresponded to orders of targeted
drugs without a recent eGFR measurement.
Inter-rater Reliability
The concordance rate was estimated at 90% (95% CI 0.86 to
0.94) and kappa at 0.72 (95% CI 0.61 to 0.83). Thirteen
instances of discordance were due to errors by the second
rater (E.S.). Nine were induced by the revalidation process,
which precludes the blinded physician from obtaining the
relevant data from the EHR required for validation. Seven
discordant validations were due to errors by the reviewer
pharmacist. The three drugs for which the pharmacist made
an error were systematically verified throughout the study
and corrected if necessary.
Rate of Inappropriate First Prescriptions
Seven of the 707 first prescriptions evaluated by the re-
viewer pharmacist were rated as not applicable (0.99%) and
were dropped for subsequent analyses. During intervention
periods, 19.9% of orders (59 of 297) were inappropriate
versus 21.3% (86 of 403) in control periods (unadjusted odds
0.91, 95% CI 0.63 to 1.32, p 0.63). Results according to
period are shown in Figure 2.
The rate of inappropriate prescriptions did not differ signif-
icantly between residents (62 of 288, 21.5%) and senior
physicians (24 of 115, 20.9%) in control periods (p 0.88).
However, senior physicians made more errors (24 of 82,
29.3%) than residents (35 of 215, 16.3%) in intervention
periods (p 0.01).
Table 2 y Comparison of Characteristics for
Inpatients Enrolled in Control and Intervention
Periods
Characteristics
Control
Periods
(n 321)
Intervention
Periods
(n 282) p Value
Age (yr), median (IQR) 82 (74–89) 84 (76–89) 0.12
Women, No. (%) 173 (53.9) 159 (56.4) 0.54
eGFR (mL/min/1.73 m
2
),
median (IQR)*
44 (34–52) 44 (34–52) 0.92
No. prescriptions per
patient, median (IQR)
1 (1–2) 1 (1–2) 0.83
eGFR estimated glomerular filtration rate; IQR interquartile
range.
*Data were missing for 72 patients (alerts with no GFR or result
dating from more than 5 days).
206 Sellier et al., No Impact of an Alert for Drug Dosage
In the multivariable analysis (Table 5, Model 1), the interac-
tion between intervention and prescriber level was signifi-
cant, resulting in a different odds ratio for inappropriate
prescriptions between intervention and control periods for
residents (0.69, 95% CI 0.41 to 1.15) and for senior physicians
(1.88, 95% CI 0.91 to 3.89). The odds ratio of inappropriate
prescriptions for senior physicians compared with residents
during intervention periods was 2.35 (95% CI 1.53 to 4.60,
p 0.03).
Drug category was strongly associated with whether a
prescription was inappropriate or not. Angiotensin-convert-
ing enzyme inhibitors were less prone to errors (6.7%).
Antibiotics accounted for most inappropriate prescriptions
(54.5%), as they were ordered often (26% of first prescrip-
tions) and they were classified as inappropriate in 43.4% of
cases. B-adrenergic blocking agents, antigout drugs, and
digoxin were inappropriate in 11.9%, 15.5%, and 17.3% of
cases, respectively. Eleven of the 13 first prescriptions for
metformin (84.6%) were inappropriate. Senior physicians
and residents did not order different drug classes (p 0.36).
The effect of the intervention was not associated with the
drug class (p 0.44).
There were no differences in the intervention effect between
the two departments and patient’s gender. Finally, an in-
crease in the age of the patient was associated with a
decrease in inappropriate prescriptions.
Figure 2. Rate of inappropriate first prescriptions by
study periods. A. Rates of inappropriate first prescriptions
for all physicians. B. Rates of inappropriate first prescrip-
tions for residents. C. Rates of inappropriate first prescrip-
tions for senior physicians.
Table 3 y Comparison of Characteristics for
Physicians Enrolled in Control and Intervention
Periods
Characteristics
Control
Periods
(n 35)*
Intervention
Periods
(n 32)* p Value
Women, No. (%) 20 (57) 18 (56) 0.94
Physician’s level, No. (%) 0.96
Residents 21 (60) 19 (59)
Senior physicians 14 (40) 13 (41)
No. prescriptions per
physician, median (IQR)
14 (7–23) 12 (2–22) 0.37
Department 0.63
Internal medicine 21 (60) 21 (66)
Geriatric 14 (40) 11 (34)
IQR interquartile range.
*Twenty-nine physicians made prescriptions in both intervention
and control periods, three only in intervention periods, and six only
in control periods.
Table 4 y Type of Recommendation and Drug
Category of Prescriptions Included in Control and
Intervention Periods
Type of Recommendation
Control
Periods
Intervention
Periods p Value*
Total 587 535
Need for measuring eGFR 64 103
Need for dosage adjustment 523 432
First prescriptions 406 301
Following prescriptions 117 131
Drug category†‡ 0.05
ACEI 136 (26.0) 147 (34.0)
B-adrenergic blocking agent 136 (26.0) 117 (27.1)
Antibiotic 141 (27.0) 87 (20.1)
Antigout drug 63 (12.0) 41 (9.5)
Digoxin 36 (6.9) 30 (7.0)
Antidiabetes drug 11 (2.1) 10 (2.3)
ACEI angiotensin-converting enzyme inhibitor; eGFR esti-
mated glomerular filtration rate.
*Chi-square test comparing drug categories in intervention and
control periods.
†Drug category (medications included): ACEI (ramipril, perindro-
pril, captopril, lisinopril); B-adrenergic blocking agents (acebutolol,
atenolol, bisoprolol, sotalol); antibiotics (ciprofloxacin, erythromycin,
gentamicin, vancomycin, levofloxacin, sulfamethoxazole-timethropim,
amikacin, teicoplanin, tobramycin, norfloxacin, and fosfomycin); anti-
gout drug (colchicine, allopurinol); digoxin; antidiabetes drug (met-
formin).
‡Values are expressed as number (percentage of total prescriptions
needing dosage adjustment).
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009
207
Rate of Overall Inappropriate Prescriptions
Eight of the 955 prescriptions were rated as not applicable,
and five others were adjusted on plasmatic concentration.
These 13 prescriptions (1.4%) were subsequently excluded
for analyses. During control periods, 20.4% of orders were
inappropriate (106 of 520), whereas 18.5% (78 of 422) were
inappropriate in intervention periods (unadjusted OR 0.81;
95% CI 0.51 to 1.28, p 0.37). In multivariable analysis
(Table 5, Model 2), the results were similar as previously
outlined, except for the interaction between the physician’s
level and intervention, which was not significant.
Methodological Outcomes
Models 3 and 4 in Table 5 present the results obtained from
the analyses of data retrieved exclusively from the CPOE
and whose validation was automated. Only prescriptions
that were actually administered to patients were included.
The results were similar to those obtained previously and, as
for models with the validation carried out by the pharmacist
(Models 1 and 2), the patient effect was significant only if all
prescriptions, not only the first one, were included for
analyses.
Rate of Prescriptions Cancelled if No eGFR Was
Available
In control periods, 20 of 64 logs were followed by cancella-
tion of the prescription (31.3%), in contrast to 36 of 103 logs
(35.0%) in the intervention periods. The unadjusted odds of
having an appropriate order was 1.18 (95% CI 0.61 to 2.30,
p 0.62) in intervention periods compared with control
periods.
Discussion
Considerable attention has been drawn to electronic health
record systems as a means of reducing medication errors.
We implemented alerts for drug dosing adjustment into the
daily routine use of a CPOE, through a design for interven-
tion shown to be effective in previous studies.
7,18,19
How-
ever, the intervention was not associated with a significant
decrease in the number of inappropriate prescriptions for
inpatients with renal insufficiency. The design we chose was
the best alternative to a randomized controlled trial due to
the particular features of our study. Indeed, it was not
possible to randomize physicians to intervention or control
groups due to a contamination effect. Also, it was not
appropriate to randomize units, because there were only
two units with a different recruitment of patients, which
would have led to a baseline imbalance between the two
groups. This alternative design enabled us to avoid selection
bias. Furthermore, we made substantial efforts to control for
possible confounding factors, and our analyses accounted
for the clustering of the data (multilevel models).
Moreover, we confirmed our results by various methods of
data retrieval and evaluation. We were able to retrieve all of
the relevant data from the hospital information system and
conduct an electronic validation of the prescriptions. The
overall results were similar for whichever method we used.
Although concordance and discordance cases for various
methods were not subtly evaluated, our results suggest that
it is possible to obtain a long-term indicator of physician
performance or that of the intervention effect without addi-
tional resources (i.e., log recording, manual validation).
Consequently, despite the negative results of our study, it is
conceivable that in the future the effect of these alerts may be
evaluated in other departments of our hospital.
In our study, the rate of inappropriate prescriptions during
control periods was 21.3%. This good performance may
represent a ceiling effect, which may have limited the impact
Table 5 y Adjusted Odds Ratio of Inappropriate Prescriptions According to the Method for Validation
(Pharmacist or Electronic) and to the Type of Prescriptions Included (First or All)
Variables
Model 1
Pharmacist Validation
First Prescriptions
Model 2
Pharmacist Validation
All Prescriptions
Model 3
Electronic Validation First
Prescriptions Really
Given to Patients
Model 4
Electronic Validation All
Prescriptions Really
Given to Patients
OR (CI 95%) N 700 N 942 N 673 N 843
Intervention*
Residents 0.69 (0.41–1.15) 0.70 (0.36–1.36) 0.70 (0.42–1.18) 0.75 (0.44–1.30)
Senior physicians 1.88 (0.91–3.89) 1.72 (0.92–3.23) 1.66 (0.82–3.36) 1.79 (0.85–3.75)
Patient’s age (yr) 0.98 (0.97–1.00)† 0.97 (0.95–0.99)† 0.98 (0.96–0.99)‡ 0.97 (0.95–0.99)‡
Drug category § § § §
ACEI 1.0 1.0 1.0 1.0
B-adrenergic blocking agent 2.0 (1.0–4.2) 3.4 (1.5–7.5) 3.8 (1.8–7.9) 5.3 (2.5–11.1)
Antibiotic 10.7 (5.5–20.7) 27.2 (10.7–69.3) 16.1 (7.8–33.2) 23.6 (10.3–54.2)
Antigout drug 2.5 (1.1–6.1) 3.3 (1.2–8.6) 3.2 (1.3–7.8) 3.2 (1.3–7.6)
Digoxin 3.2 (1.3–8.3) 7.3 (2.3–23.6) 4.4 (1.6–11.7) 6.4 (2.3–17.7)
Antidiabetes drug 78.0 (15.2–400.0) 527 (56–4785)
Model 1 and Model 3: ORs were estimated in an ordinary multivariable logistic regression model (no patient effect). Model 2 and Model 4: ORs
were estimated in mixed multivariable logistic regression model (significant patient effect).
ACEI angiotensin converting enzyme inhibitor; CI confidence interval; OR odds ratio.
*Due to the significance of interaction term Intervention
Physician’s level in Model 1, the odds ratio for the term Intervention was different
for residents and senior physicians. The odds ratio of inappropriate prescriptions for senior physicians in intervention versus control periods
was obtained by multiplying the odds ratio associated with the variable Intervention and the interaction term Intervention
Physician’s level.
†p 0.05.
‡p 0.01.
§p 0.001.
208 Sellier et al., No Impact of an Alert for Drug Dosage
of our intervention. Some factors may contribute to the
quality of prescriptions in our hospital: these include access,
through the CPOE, to the Vidal electronic database and the
role of the ward pharmacists. Moreover, the purpose of our
alerts was to avoid excessive doses regardless of the renal
function only, and the error rates could have been greater if
we had taken into consideration the indication of each order
(reducing consequently the recommended dosing range for
specific indication).
We developed our alert to incorporate evidence-based fea-
tures
20
associated with the success of decision support: we
used computers to deliver support, offering specific recom-
mendations and not just an assessment. Our application
generated alerts automatically as part of the clinical work-
flow, at the time and the place that the decision was made.
Indeed, if a system providing recommendations is depen-
dent on the initiative of the physicians for use, they seldom
make the required effort.
21
However, the manner chosen to
provide overdosing alerts in our study may partly explain
the negative results. Our alert was possibly not directive
enough for the physician. Indeed, in the study by Chertow et
al.,
7
the investigators implemented and evaluated a system
for adjusting drug doses for inpatients with renal insuffi-
ciency using a similar design to that reported here. During
intervention periods, the adjusted dose list, default dose
amount, and default interval were displayed to the prescrib-
ers. The intervention resulted in a significant decrease in
inappropriate dosages, from 67% to 54%. Similarly, Oppen-
heim et al.
22
implemented an alert displaying the correct
dosage of a drug only when a wrong dosage was selected by
the physician. Half of the orders generating alerts were
adjusted in response to alerts. In these two previous studies,
the exact dosage was suggested to the prescriber for each
prescription, whereas we provided an adjustment table as
rationale for advice. Also, the patients included in our study
were older than those in other studies that were investigated
with similar interventions.
7,18,19
As in a previous study,
23
we observed a significant differ-
ence of the intervention effect between residents and senior
physicians: residents tended to improve their performance
on receipt of alerts, whereas senior physicians tended to
make more errors during similar periods. Although the
difference of inappropriate prescriptions was not signifi-
cantly higher in intervention than in control periods among
senior physicians, it is worth trying to explain this unantic-
ipated effect. First, in French academic institutions, residents
write most prescriptions for hospitalized patients. Senior
physicians are likely to write prescriptions when doing the
weekly or bi-weekly review of all patients in the unit; this
review is carried out more frequently for more complicated
patients, which residents are unable to cope with, especially
young residents. Even if potentially there are situations in
which residents and senior physicians wrote different pre-
scriptions, and our multivariate analysis were unable to
adjust for these situations, it is unlikely that they occurred
differently in intervention and control periods. Second,
senior physicians may have judged that the benefit of a
chosen drug-dosing regimen outweighed the risk of an
excessive dose, or they might have disagreed with the expert
panel’s recommendations. It is also likely that senior physi-
cians preferred disregarding the alerts in favor of their own
established practice, whereas younger physicians were
keener to improve and were more receptive to new infor-
mation. Although these factors may explain the absence of
an effect of intervention among senior physicians, they fail
to explain an increase in errors. Third, we cannot exclude
that the senior physicians may have misinterpreted informa-
tion relating to the dose, which may have been unclear.
Indeed, unclear dose-related information within the context
of alerts and an absence of time could lead to the misinter-
pretation of the alerts.
24
Finally, the intervention phases may
have induced a false sense of security among senior physi-
cians, because they knew these alerts were part of a research
study.
Our study was subject to several limitations. First, the
intervention was conducted in only one hospital with its
own CPOE and health information system. Second, only two
departments were included, and thus a small number of
physicians. However, they were the most relevant depart-
ments in relation to this study, because they gathered the
oldest patients, who consequently are the most likely to
suffer from renal insufficiency. Third, our study may have
lacked power for detecting a difference between the two
groups. However, it is obvious that the alerts had no
meaningful clinical impact. Fourth, eGFR was calculated
thanks to the Modification of Diet in Renal Disease
(MDRD) formula, which is imprecise for elderly patients,
but is more accurate than the Cockroft-Gault equation.
15
There is also little evidence on drug dose recommenda-
tions in elderly patients, because the randomized con-
trolled trials often exclude elderly patients. Finally, the
MDRD formula does not accurately reflect renal function
in unsteady-state conditions (such as acute increase or
decrease of creatinine), so the alert may have delivered
inappropriate recommendations in these cases. However,
patients with these unstable conditions are more likely to
be hospitalized in intensive care or in a renal unit in our
institution. If this was not the case, the ward pharmacist
could have alerted the clinician.
We assessed the effect of the intervention as a result of
process measures rather than patient outcomes measures
because process measures are the most suitable tool for
judging quality of care.
25
Our results contrast with those of other recent reviews,
11,12
which conclude the effectiveness of CDSS for improving
physician performance. However, these reviews can be
biased by the fact that negative studies are less published
than positive ones.
26
Moreover, Chaudhry et al.
12
suggested
caution in interpreting the results of identified studies of
CDSS because of flaws in their design and analysis. Further-
more, positive studies have been carried out for the most
part by the developers of these systems. Finally, the results
of these reviews contradict other studies, which identify
high rates of alerts being overridden by physicians.
22,23
The
implementation of effective CDSS remains a challenging
task
27
and is not yet the miracle drug for improving the
performance of physicians. It is still necessary to evaluate
their impact if newly implemented in other settings and to
monitor for unanticipated effects and their underlying
causes.
Journal of the American Medical Informatics Association Volume 16 Number 2 March / April 2009 209
References
y
1. Leape LL, Brennan TA, Laird N, et al. The nature of adverse
events in hospitalised patients. Results of the Harvard Medical
Practice Study II. N Engl J Med 1991;324:377– 84.
2. Bates DW, Teich JM, Lee J, et al. The impact of computerized
physician order entry on medication error prevention. J Am
Med Inform Assoc 1999;6:313–21.
3. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse
drug events and potential adverse drug events. Implications
for prevention. ADE Prevention Study Group. JAMA 1995;
274:29–34.
4. Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin
GA. The epidemiology of prescribing errors. Arch Intern Med
2004;164:785–92.
5. Salomon L, Deray G, Jaudon MC, et al. Medication misuse in
hospitalized patient with renal impairment. Int J Qual Health
Care 2003;15:235– 40.
6. A van Dijk E, Drabbe NRG, Kruijtbosch M, De Smet P AGM.
Drug dosage adjustments according to renal function at hospital
discharge. Ann Pharmacother 2006;40:1254 60.
7. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication
dosing for inpatients with renal insufficiency. JAMA 2001;286:
283944.
8. Kaushal R, Shojania KG, Bates DW. Effect of computerized
physician order entry and clinical decision in support systems
on medication safety: a systematic review. Arch Intern Med
2003;163:1409–16.
9. Leape LL, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ,
Erickson JI. Pharmacist participation on physician rounds and
adverse drug events in the intensive care unit. JAMA 1999;282:
267–70.
10. Estellat C, Colombet I, Vautier S, Huault-Quentel J, Durieux P,
Sabatier B. Impact of pharmacy validation in a computerized
physician order entry context. Int J Qual Health Care 2007;19:
317–25.
11. Garg AX, Adhikari NK, McDonald H, et al. Effects of comput-
erized clinical decision support systems on practitioner perfor-
mance and patient outcomes. A systematic review. JAMA
2005;293:1223–37.
12. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of
health information technology on quality, efficiency and costs of
medical care. Ann Intern Med 2006;144:E12–E22.
13. Degoulet P, Marin L, Lavril M, et al. The HEGP component-
based clinical information system. Int J Med Inform 2003;69:
115–26.
14. Levey AS, Greene T, Kusek JW, Beck GJ. A simplified equation
to predict glomerular filtration rate from serum creatinine. J Am
Soc Nephrol 2000;11:155A.
15. Froissart M, Rossert J, Jacquot C, Paillard M, Houillier P.
Predictive performance of the modification of diet in renal
disease and Cockroft-Gault equations for estimating renal func-
tion. J Am Soc Nephrol 2000;16:763–73.
16. Rabe-Hesketh S, Skrondal A, Pickles A. GLLAMM manual. UC
Berkeley Division of Biostatistics Working Paper Series, 2004.
Available at: http://www.bepress.com/ucbbiostat/paper160.
Accessed January 7, 2009.
17. Rabe-Hesketh S, Skrondal A, Pickles A. Reliable estimation of
generalized linear mixed models using adaptative quadrature.
Stata J 2002;2:1–21.
18. Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E. A clinical
decision support system for prevention of venous thromboem-
bolism: effect on physician behavior. JAMA 2000;283:2816–21.
19. Weingarten SR, Riedinger MS, Conner L, et al. Practice guide-
lines and reminders to reduce duration of hospital stay for
patients with chest pain: an intervention trial. Ann Intern Med
1994;120:257–63.
20. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving
clinical practice using clinical decision support systems: a sys-
tematic review of trial to identify features critical to success.
BMJ 2005;330:765– 8.
21. Eccles M, McColl E, Steen N, et al. Effect of computerised
evidence based guidelines on management of asthma and
angina in adults in primary care: cluster randomised trial. BMJ
2002;325:941–8.
22. Oppenheim M, Vidal C, Velasco F, et al. Impact of a computer-
ized alert during physician order entry on medication dosing in
patients with renal impairment. Proc AMIA Symp 2002:557–81.
23. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB,
Phillips RS. Physicians’ decisions to override computerized
drug alerts in primary care. Arch Intern Med 2003;163:2625–31.
24. Van der Sijs, Aarts J, Vulto A, Berg M. Overriding of drug safety
alerts in computerized physician order entry. J Am Med Inform
Assoc 2006;13:138 47.
25. Lilford RJ, Brown CA, Nicholl J. Use of process measures to
monitor the quality of clinical practice. BMJ 2007;335:648 –50.
26. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication
and related biases. Health Technol Assess 2000;4:1–115.
27. Wears RL, Berg A. Computer technology and clinical work.
JAMA 2005;293:1261–3.
210 Sellier et al., No Impact of an Alert for Drug Dosage
    • "The first category reported the utilization of an e-alert without measurement of their impact on the process of care and patient or kidney out- comes6789101112. In the second group, processes of care or outcomes were measured, but e-alerting did not improve outcomes131415 . Finally, in the third set of studies, recorded clinical outcomes or quality of care indicated im- provement1617181920212223242526. "
    [Show abstract] [Hide abstract] ABSTRACT: Purpose of the review: Among hospitalized patients, acute kidney injury is common and associated with significant morbidity and risk for mortality. The use of electronic health records (EHR) for prediction and detection of this important clinical syndrome has grown in the past decade. The steering committee of the 15(th) Acute Dialysis Quality Initiative (ADQI) conference dedicated a workgroup with the task of identifying elements that may impact the course of events following Acute Kidney Injury (AKI) e-alert. Sources of information: Following an extensive, non-systematic literature search, we used a modified Delphi process to reach consensus regarding several aspects of the utilization of AKI e-alerts. Findings: Topics discussed in this workgroup included progress in evidence base practices, the characteristics of an optimal e-alert, the measures of efficacy and effectiveness, and finally what responses would be considered best practices following AKI e-alerts. Authors concluded that the current evidence for e-alert system efficacy, although growing, remains insufficient. Technology and human-related factors were found to be crucial elements of any future investigation or implementation of such tools. The group also concluded that implementation of such systems should not be done without a vigorous plan to evaluate the efficacy and effectiveness of e-alerts. Efficacy and effectiveness of e-alerts should be measured by context-specific process and patient outcomes. Finally, the group made several suggestions regarding the clinical decision support that should be considered following successful e-alert implementation. Limitations: This paper reflects the findings of a non-systematic review and expert opinion. Implications: We recommend implementation of the findings of this workgroup report for use of AKI e-alerts.
    Full-text · Article · Dec 2016
    • "Dose adjust alerts assist physicians with identifying medications that require dosage adjustment. Various studies have investigated the effect of dose adjust alerts, and except for the study of Sellier et al., these studies confirmed that dose adjust alerts improve adherence with dosing guidelines.23242526 The study of Sellier et al. had a low baseline inappropriate dosage prescribing rate of 21%, probably because of pharmacists already reviewed all prescriptions. "
    [Show abstract] [Hide abstract] ABSTRACT: To determine the prevalence, determinants, and potential clinical relevance of adherence with the Dutch dosing guideline in patients with impaired renal function at hospital discharge. Retrospective cohort study between January 2007 and July 2011. Academic teaching hospital in the Netherlands. Patients with an estimated glomerular filtration rate (eGFR) between 10-50 ml/min/1.73m2 at discharge and prescribed one or more medicines of which the dose is renal function dependent. The prevalence of adherence with the Dutch renal dosing guideline was investigated, and the influence of possible determinants, such as reporting the eGFR and severity of renal impairment (severe: eGFR<30 and moderate: eGFR 30-50 ml/min/1.73m2). Furthermore, the potential clinical relevance of non-adherence was assessed. 1327 patients were included, mean age 67 years, mean eGFR 38 ml/min/1.73m2. Adherence with the guideline was present in 53.9% (n=715) of patients. Reporting the eGFR, which was incorporated since April 2009, resulted in more adherence with the guideline: 50.7% vs. 57.0%, RR 1.12 (95% CI 1.02-1.25). Adherence was less in patients with severe renal impairment (46.0%), compared to patients with moderate renal impairment (58.1%, RR 0.79; 95% CI 0.70-0.89). 71.4% of the cases of non-adherence had the potential to cause moderate to severe harm. Required dosage adjustments in case of impaired renal function are often not performed at hospital discharge, which may cause harm to the majority of patients. Reporting the eGFR can be a small and simple first step to improve adherence with dosing guidelines.
    Full-text · Article · Jun 2015
    • "A barrier to improved prescription may be physician non-adherence to decision support alerts. If decision support systems are used, it is important to implement systems that have a minimum of nuisance alerts, work well with the electronic medical record and prescription system, have a broad user acceptance and have interruptive alerts [29]. In conclusion, we have shown that disagreements between actual used dosing and dosing as defined by expert recommendation are common in patients with moderate-to-severe renal failure, even in a setting with electronic drug prescription and automatic reporting of eGFR, and that disagreement to some extent reflects medication errors. "
    [Show abstract] [Hide abstract] ABSTRACT: In patients with impaired renal function, drug dose adjustment is often required. Non-adherence to clinical prescribing recommendations may result in severe adverse events. In previous studies, the prevalence rate of non-adherence to recommended dosing has been reported to be 19-67%. Using the clinical support system Renbase(®) as reference, we investigated the use and dosing of drugs in patients with impaired renal function in a university hospital setting using electronic prescription and automatic reporting of estimated glomerular filtration rate (eGFR). In all, 232 patients with an eGFR in the range of 10-49 ml/min/1.73m(2) were included. We identified 436 episodes with administration of renal risk drugs (prescribed to 183 patients): 410 drugs required dose adjustment according to the eGFR and 26 should be avoided. In total, the use or dosing of 66 (15%) of the 436 renal risk drugs was not in agreement with recommendations in Renbase(®) . This reflects less disagreement with expert guidelines than reported previously, indicating a possible beneficial effect of electronic prescribing and reporting of eGFR. However, we also found that disagreement to some extent reflected inappropriate drug use. We conclude that despite implementation of electronic prescribing and automated reporting of eGFR, patients with renal insufficiency may still be exposed to inappropriate drug use, with potential increased risk of adverse effects. Initiatives to reduce medication errors such as the use of electronic decision support systems should be explored. This article is protected by copyright. All rights reserved.
    Full-text · Article · Dec 2013
Show more