Computerized Prescribing Alerts and Group Academic Detailing
to Reduce the Use of Potentially Inappropriate Medications in
Steven R. Simon, MD, MPH,?wDavid H. Smith, RPh, PhD,?zAdrianne C. Feldstein, MD, MS,?z
Nancy Perrin, PhD,?z§Xiuhai Yang, MS,?zYvonne Zhou, PhD,zRichard Platt MD, MS,?wand
Stephen B. Soumerai, ScD?w
OBJECTIVES: To examine the effect of replacing drug-
specific computerized prescribing alerts with age-specific
alerts on rates of dispensing potentially inappropriate med-
ications in older people and to determine whether group
academic detailing enhances the effectiveness of these
DESIGN: Cluster-randomized trial of group academic
detailing and interrupted time-series analysis.
SETTING: Fifteen clinics of a staff-model health mainte-
PARTICIPANTS: Seven practices (113 clinicians, 24,119
patients) were randomly assigned to receive age-specific
prescribing alerts plus the academic detailing intervention;
eight practices (126 clinicians, 26,805 patients) received
alerts alone. Prior implementation of drug-specific alerts
established a downward trend in use of target medications
that served as the baseline trend for the present study.
INTERVENTION: The computerized age-specific alerts
occurred at the time of prescribing a targeted potentially
inappropriate medication (e.g., tertiary tricyclic amine anti-
depressants, long-acting benzodiazepines, propoxyphene)
and suggested an alternative medication.Clinicians at seven
sites were randomized to group academic detailing, an in-
teractive educational program delivering evidence-based
MEASUREMENTS: Number of target medications dis-
pensed per 10,000 patients per quarter, 2 years before and
1.5 years after the replacement of drug-specific with age-
RESULTS: Age-specific alerts resulted in a continuation of
the effects of the drug-specific alerts without measurable
additional effect (P5.75 for level change), but the age-
specific alerts led to fewer false-positive alerts for clinicians.
Group academic detailing did not enhance the effect of the
CONCLUSION: Age-specific alerts sustained the effec-
tiveness of drug-specific alerts to reduce potentially inap-
propriate prescribing in older people and resulted in a
considerably decreased burden of the alerts. J Am Geriatr
Soc 54:963–968, 2006.
Key words: computerized provider order entry; clinical
decision support systems; academic detailing; quality
improvement; medication errors
important target for error reduction and quality improve-
ment.1–4The expansion of electronic health records (EHRs)
and computerized provider order entry into the offices of
for technology to improve the prescribing of medications.
Previous studies have shown that computerized order entry
with clinical decision support systems can improve medi-
cation prescribing and reduce medication error rates,5,6al-
though most of these studies have been conducted in the
inpatient setting or in hospital-based ambulatory practices.
As these systems begin to diffuse into the ambulatory care
setting, evidence of their effectiveness is urgently needed
before their widespread adoption and implementation.
It was recently shown that providing prescribers with a
real-time computerized alert about certain potentially in-
appropriate medications in older people resulted in de-
creased prescribing and decreased dispensing of these
medications to elderly patients, with increased rates of use
of medications that were recommended as alternatives.7
Like many existing order entry systems, the drug-specific
edication errors and preventable adverse drug events
occur commonly in elderly patients and constitute an
Address correspondence to Steven R. Simon, MD, MPH, Department of
Ambulatory Care and Prevention, Harvard Medical School and Harvard
Pilgrim Health Care, 133 Brookline Avenue, Sixth Floor, Boston, MA 02215.
From the?HMO Research Network Center for Education and Research in
Therapeutics andwDepartment of Ambulatory Care and Prevention, Harvard
Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts;
zCenter for Health Research, Kaiser Permanente of the Northwest, Portland,
Oregon;§and Oregon Health & Science University, Portland, Oregon.
r 2006, Copyright the Authors
Journal compilation r 2006, The American Geriatrics Society
alerts in that study occurred each time a medication was
ordered that was potentially inappropriate for older people,
regardless of patient age or other clinical characteristics. It
is unknown whether age-specific alerts (occurring only for
patients in a limited age range) are more effective and
whether age-specific alerts reduce the number of false-pos-
The EHR and computerized order entry system at the
study health maintenance organization (HMO) were there-
fore modified with a new system of age-specific alerts that
to patients aged 65 and older. This study sought to examine
the effect of replacing drug-specific computerized prescrib-
ing alerts with age-specific alerts on rates of dispensing of
potentially inappropriate medications in older people and
on the burden (number) of alerts faced by prescribers.
In addition, because prescribers frequently disregard or
override prescribing alerts in a variety of settings,8this
study also sought to address the barriers to following the
recommendations of these new alerts. A randomized con-
trolled trial of intensive educational outreach, or academic
detailing,9was thus conducted to enhance the acceptance
and effectiveness of these computerized prescribing alerts.
Overall Study Design
Because the existing computerized order entry system con-
figuration would not accommodate randomization of the
prescribing alerts, an interrupted time-series analysis, one
of the strongest quasi-experimental study designs,10,11was
conducted to examine the effect of switching from drug-
specific prescribing alerts (alerts occurring each time a
medication is prescribed, regardless of patient characteris-
tics) to age-specific alerts on rates of use of potentially in-
appropriate medications in older people. This design was
chosen to estimate sudden interruptions in trends of target
medication use resulting from the transition from drug-
specific to age-specific alerts in January 2003.
At the time of implementing the age-specific alerts in
January 2003, a cluster-randomized, controlled trial of
small-group educational sessions (group academic detail-
ing) plus age-specific alerts versus age-specific alerts alone
was also conducted to reduce the use of potentially inap-
was used to ensure comparability of the experimental and
control groups by stratifying the practice sites on the basis
of baseline rates of use of two classes of the target medi-
cations (tertiary tricyclic amine antidepressants and long-
acting benzodiazepines), and each block of two clinics was
randomly allocated to intervention or control.7Table 1
shows the medications targeted in this intervention.
The institutional review board of the study HMO ap-
proved the study protocol.
Setting and Prior Intervention
The study was conducted at a nonprofit, group-model
HMO in Oregon and Washington with approximately
Since 1996, the HMO has used the EpicCare EHR
(http://www.epicsys.com/), with computerized order entry
and decision support. From November 2000 through De-
cember 2002, the EHR had a point-of-care prescribing alert
system that ‘‘popped up’’ each time a clinician ordered a
medication considered possibly harmful in older people;
these drug-specific alerts, which occurred at the time of
medication ordering regardless of the patient’s age, resulted
in reductions in the use of long-acting benzodiazepines and
tricyclic antidepressants with pronounced anticholinergic
properties.7The drug-specific alert intervention resulted in
a downward trend in the use of target medications that
served as the baseline(preintervention) trendfor the present
study of age-specific alerts.
The overall study period was November 2000 through June
2004. FromNovember2000through December2002,drug-
specific alerts were operational for the target medications.
From January 2003 through June 2004, age-specific alerts,
the study intervention, were in effect for all target medica-
tions in all practices. The group academic detailing inter-
vention occurred January through March 2003. April 2003
through June 2004 served as the postintervention period.
Study participants included all primary care clinicians (phy-
sicians, nurse practitioners, and physician assistants) at the
15 enrolled clinics and the elderly patients receiving pri-
mary care at those sites. Table 2 shows the provider and
patient composition of the study practices.
Medication dispensing was included in analyses if the
patient was aged 65 and older at the time of dispensing.
Multiple dispensing of a single target medication during a
given calendar month to the same patient was counted
once. Dispensing of a single target medication in different
months to the same patients was counted separately in each
month of its occurrence. Dispensing of target medications
from different classes (e.g., receiving both long-acting ben-
zodiazepines and tertiary amine tricyclic antidepressants)
to the same patient in the same month was counted once
for each medication class received.
Computerized Prescribing Alerts
From November 2000 through December 2002, the EHR
employed drug-specific alerts for long-acting benzodiaze-
pines and tertiary amine tricyclic antidepressants.7These
alerts resulted in a significant decrease in the use of the
targetmedications.In January2003,the drug-specificalerts
were discontinued, and at the same time a new system of
age-specific prescribing alerts was implemented. The alerts
were programmed to ‘‘pop up’’ only when both of the fol-
lowing circumstances occurred:
1. Clinician ordered one of the target medications (Table 1)
for a patient aged 65 and older.
2. The patient did not have a current supply for the same
medication from a prescription dispensed in the pre-
ceding 6 months. This condition was included so that
prescribers would not receive multiple alerts for the
same patient within a 6-month period.
SIMON ET AL.
JUNE 2006–VOL. 54, NO. 6JAGS
Each alert suggested that clinicians change the originally
intended medication order to an alternative preferred med-
ication; the order could be changed manually (retyping the
drug name) or by activating a list of medication alternatives
and choosing a medication from the menu of options.
Group Academic Detailing
The group academic detailing intervention was intended to
increase prescribers’ acceptance of the evidence-based
alerts; the protocol closely followed the key principles of
academic detailing, described in detail elsewhere.9Briefly,
theseprinciples includeconductinginterviews toinvestigate
baseline knowledge and motivation for current and pro-
posed practice patterns; establishing credibility through a
respected organizational sponsor; referencing authoritative
and unbiased sources of information; presenting both sides
of controversial issues; stimulating active physician partic-
ipation in two-way interaction; using concise and visually
appealing graphical educational materials, specifically ad-
dressing real and perceived barriers to change; and repeat-
ing and reinforcing a small number of desired behaviors.
Twenty semistructured interviews were conducted with
prescribers to characterize the incentives for and barriers to
following the instruction contained in the alerts and to
identify prescribers’ preferences regarding format and con-
tent of educational sessions to introduce new alerts in the
EHR system. Interviewees reported that they were more
likely to respond favorably to alerts highlighting quality
and safety (as opposed to cost), and they identified char-
acteristics of alerts that they viewed favorably (e.g., brief,
clear, easily navigable) and those triggering negative re-
sponses (e.g., wordy, ‘‘clunky’’).8The barriers to alert ac-
ceptance identified in these interviews became the principal
targets of the educational program, whereas the facilitating
factors frequently served as useful counterarguments to
concerns raised by participants. In addition, the results of
these interviews led to refinement of the alerts themselves,
as did discount usability testing,8a form of pilot testing.
Two respected physician idea champions, or ‘‘peer
leaders,’’12were selected to deliver the group academic de-
tailing sessions. These detailers received a 2-hour training
session covering the principles of academic detailing, an
overview of patient safety and medication errors, and the
clinical evidence underlying the recommended prescribing
practices. The detailers were provided with a list of talking
points, arguments, and counterarguments that could be in-
terspersed in the discussion and were prepared to ‘‘inocu-
late’’ the group by proactively addressing concerns that
were raised in the interviews. The group academic detailing
sessions also included an overview of the therapeutic basis
of the alerts. Both detailers rehearsed the session with re-
search staff and delivered a practice session at a clinic that
was not part of the study.
One-hour sessions were scheduled at each clinic in a
time-slot previously established for a regular departmental
meeting. At these sessions, lunch was provided, continuing
medical education credit for physician attendees was of-
fered, and visually appealing graphical educational mate-
rials were delivered, as well as a travel mug imprinted with
Table 1. Drugs Triggering Prescribing Alerts and Recommended Alternatives by Indication for Elderly Patients
Target Drug to AvoidAdverse Outcome to Avoid Recommended Alternative Agents
Tertiary tricyclic amine
Strong anticholinergic and sedating
properties. More orthostatic hypotension
than with alternatives.
Depression: nortriptyline, desipramine,
Neuropathic pain: nortriptyline
Anxiety: lorazepam, oxazepam,
paroxetine, or buspirone
Insomnia: trazodone, lorazepam,
oxazepam, or temazepam
Long half-life and active metabolites in
older adults (often several days) may result
in daytime sedation and a greater risk of
falls and fractures.
Indomethacin produces more central
nervous system side effects than other
NSAIDs. Piroxicam has an increased risk
for GI bleeding. Use NSAIDs with care in
general because of renal and GI toxicity.
Propoxyphene is no more effective than
acetaminophen but often has the side
effects of other narcotic drugs and so has
fewer indications for use.
Older persons poorly tolerate muscle
relaxants, experiencing sedation and
dizziness. Cyclobenzaprine also produces
anticholinergic side effects.
Skeletal muscle relaxants
Note: The alternative medications were chosen through discussion among the study investigators, representatives of the institution’s Pharmacy and Therapeutics
Committee, and other leaders of the organization. The alternatives represented the group’s consensus assessment of the most effective and cost-effective agents
available on the formulary at the time of the study.
NSAID5nonsteroidal antiinflammatory drug; GI5gastrointestinal.
REDUCING POTENTIALLY INAPPROPRIATE MEDICATIONS IN OLDER PEOPLE
965JAGSJUNE 2006–VOL. 54, NO. 6
the logo of the intervention program. Clinicians’ appoint-
ment times were blocked to increase attendance. Of the 113
clinicians eligible to receive academic detailing, 96 (85%)
At 4 to 7 months after the detailing session at each
intervention site, a reminder letter that reinforced the mes-
sages of the session was mailed to each clinician.
Medication-dispensing data were extracted from the auto-
mated claims system of the HMO. An analyst who was
blinded to the experimental condition of the treatment
groups ascertained the outcomes. The main outcome mea-
sure was the number of times one or more of the target
medications was dispensed per 10,000 patients per quarter.
For the evaluation of the randomized controlled trial, the
practice site was the unit of randomization, and the unit of
intervention was the clinician. The study included all avail-
able primary care clinics and clinicians. T tests were used
for continuous variables and chi-square tests for categorical
variables to assess baseline comparability of demographic
and clinical characteristics of clinicians and patients in the
intervention and control groups.
To assess the effect of group detailing in the random-
ized, controlled trial and of the alerts in the time-series
analysis, segmented regression was used with time (calen-
dar quarter) as the unit of analysis.10,13,14Segmented re-
gression models were developed for each of the target drug
groups and for the combined outcome of all target drugs,
controlling for preintervention trends. These models in-
cluded a constant term, a linear time trend term, terms to
estimate changes in the level and trend of prescribing of
each target medication after each intervention, and inter-
action terms (group by level; group by time, before alerts;
and group by time, post alerts) to test the effect of group
detailing plus alerts versus alerts alone.10Analyses were
conducted using SAS version 8.2 (SAS Institute, Inc., Cary,
NC). All analyses used intention-to-treat principles, such
to have been exposed to the intervention assigned to that
site regardless of attendance at educational sessions. Sim-
ilarly, all patients aged 65 and older were included in the
analysis of the experimental group to which their primary
care physician was assigned.
Effect of Group Academic Detailing
No apparent effect of the group academic detailing inter-
was found (Figure 1). In the practices receiving group de-
tailing plus alerts, the pre- and postintervention quarterly
rates of use of target medications per 10,000 members were
146.3 and 126.6, respectively, resulting in a decrease of
19.7 dispensed medications per 10,000 members. In com-
parison, in the practices receiving alerts alone, the pre- and
postintervention rates were 150.2 and 137.2, respectively, a
decrease of 13.0 dispensed medications per 10,000 mem-
bers per quarter. The fully adjusted segmented regression
model showed no significant difference after the interven-
tion in level change (P5.52) or slope change (P5.27) be-
tween the intervention and control practices.
After determining that there was no effect of the group
academic detailing intervention on rates of use of the target
medications, the group detailing and control arms of the
Table 2. Characteristics of Patients, Primary Care Clinicians, and Clinics Randomized to Group Detailing Plus Alerts
Versus Alerts Alone in January 2003
Plus Alerts Alerts AloneP-value
Target medication use rate, mean ? SD?
Age, mean ? SD
Chronic disease score, meanw
Primary care clinicians
Professional role, n
Age, mean ? SD
Number of sites
Outside Portland metropolitan area
Patients aged ?65
146.3 ? 8.9
74.3 ? 6.6
155.2 ? 12.1
73.6 ? 7.0
45.5 ? 7.9
45.0 ? 7.9
Note: Clinician and patient data refer to those prescribing or who were prescribed nonpreferred medications one quarter before the alert.
?Rate of use of target medications in the quarter preceding the intervention per 10,000 members, calculated as the number of patients receiving one or more of the
target medications divided by the total number of patients aged 65 and older.
wA chronic disease score, based on the method of Clark et al.23was calculated for each study patient incorporating the patient’s use of drugs for chronic disease in the
year preceding randomization.
SIMON ET AL.
JUNE 2006–VOL. 54, NO. 6JAGS
randomized trial were collapsed into a single study group,
and segmented regression analysis was used to determine
whether the age-specific alerts were associated with sudden
changes in the trend of prescribing of the target drugs in
Effect of Age-Specific Alerts
The transition in January 2003 from drug-specific alerts to
patient-specific alerts for the same target medications re-
sulted in a continuation of the established downward trend
without apparent change in the level (P5.75) or slope
(P5.22) of the time series (Figure 1). The immediate de-
crease in the rate of medication dispensing was 3.4 pre-
from the expected level based on preintervention trends.
The study had sufficient power to detect an immediate de-
crease of 21 quarterly prescriptions per 10,000 members.
For each of the target medications considered individually,
there was no evidence of any abrupt increase or decrease in
the rate of dispensing attributable to the age-specific inter-
vention, and there was no further change in the downward
slope of the trend lines for any of the medications. Analysis
of the intervention using monthly rates of medication dis-
pensing (as opposed to quarterly rates), which increases the
sample size by a factor of 3, did not materially change the
Burden of Alerts on Prescribers
In the drug-specific period (January to June 2002), 3,264
alerts came up. There were approximately 181 clinicians
during this time; therefore, it was estimated that each cli-
nician received an average of 18 alerts over a 6-month pe-
riod. Similarly, it was estimated that approximately 14
(82%) alerts per clinician occurred while ordering these
medications for patients younger than 65 (false-positive
alerts). In contrast, within a similar 6-month period during
the age-specific intervention (January to June 2004), 775
alerts occurred for approximately 194 clinicians, which is
approximately four alerts per clinician, all of which, by
design, occurred after prescribing one of the target medi-
cations to an elderly patient.
Safeprescribingof medications isan essential componentof
ensuring quality of care in older persons. Computerized
provider order entry with clinical decision support has been
promoted as one of the most promising interventions for
improving medication safety,15,16but its effectiveness in
ambulatory care remains unproven.4–6
This 5-year study of computerized decision support
tools (alerts) in a community practice setting found that
drug-specific alerts effectively reduced the use of potentially
inappropriate medications in older people. It also showed
that replacing the drug-specific alerts with alerts that oc-
curred only for patients meeting specific age criteria sus-
tained but did not enhance the effect of the drug-specific
alerts while reducing the burden of false-positive alerts on
support for the use of this strategy for reducing medication
errors and improving the quality of medication prescribing
in older patients.
Despite the effectiveness of computerized alerts for im-
proving medication prescribing in older people, room still
exists for improvement, in part because prescribers do not
always follow the recommendations of the computerized
alerts, citing their irrelevance and intrusion into the flow of
practice and patient care.8This study included a randomi-
zed controlled trial of group academic detailing to address
these and other barriers to adhering to prescribing alerts;
this educational outreach intervention made no difference
in the rates of prescribing of the target medications. It was
originally hypothesized that addressing these barriers
through educational outreach at the time of introducing a
new computerized alert system would foster physicians’
acceptance of the alerts and would lead to greater reduc-
tions in the use of target medications. Individual and group
of settings,17–19but in this study, the group academic de-
tailing was not focused on the prescribing behavior but was
instead aimed at enhancing clinicians’ acceptance of a sys-
tem intervention. Although physicians said that they would
like to learn about new medication prescribing alerts
through group educational sessions,8this moderately in-
tensive approach may not be sufficiently potent to further
enhance the effect of the computerized alerts. It remains
unknown whether intensifying the academic detailing in-
tervention or delivering it individually, instead of in the
group format, would have resulted in any measurable ef-
fect. It is also possible that physicians simply did not agree
with or did not fully understand the therapeutic basis of the
Although a variety of interventions, such as audit and
feedback,20opinion leaders,21and academic detailing,9,18
have resulted in modest improvements in prescribing, few
2001Q12001Q22001Q32001Q4 2002Q12002Q22002Q3 2002Q42003Q1 2003Q2 2003Q32003Q42004Q12004Q2
Group Detailing +
Dispensing per 10,000 persons
Figure 1. Interrupted time series showing quarterly rates of dis-
plus alert) and control (alert only) practices. In this figure, the 2-
year period preceding the intervention period corresponds to the
drug-specific alert period; the period after the intervention
(2003Q2 through 2004Q2) corresponds to the age-specific alert
period. In segmented-regression analysis, there was no effect of
the group-detailing intervention on the level (P5.52) or slope
(P5.27) of the trend line reflecting target medication use. In
analyses combining the two intervention groups, there was no
apparent change in the level (P5.75) or the slope (P5.22) in the
trend lines attributable to the age-specific alert intervention.
REDUCING POTENTIALLY INAPPROPRIATE MEDICATIONS IN OLDER PEOPLE
967JAGSJUNE 2006–VOL. 54, NO. 6
studies have identified successful strategies for sustaining
long-term effectiveness of prescribing interventions.19,22
This study showed that an age-specific computerized alert
system can sustain an established trend toward improved
prescribing for at least a year after implementation. It is
possible that the decreased alert burden associated with the
switch from drug-specific to age-specific alerts may have
contributed to this sustained effect. Whether the trend es-
tablished by the drug-specific alerts would have been con-
tinued without the institution of age-specific alerting
This study’s principal limitation is generalizability. It
was conducted in a single staff-model HMO with a com-
prehensive EHR that has for several years included com-
puterized order entry with clinical decision support.
Clinicians working within the HMO may have been more
receptive to these alerts than clinicians elsewhere.
This study showed that replacing drug-specific alerts
with alerts that take into account patient demographics can
sustain the effectiveness of the alerts at a lower burden on
clinicians. A wide range of interventions could be imple-
mented to reduce the use of potentially inappropriate med-
ications. These interventions may include direct-to-patient
approaches, formulary restrictions, and administrative re-
quirements (e.g., prior authorizations), as well as other de-
cision-support interventions aimed at prescribers. Further
study is needed to determine whether additional interven-
tions are necessary or feasible to enhance computerized or-
der entry systems and whether implementation of such
systems leads to improved medication prescribing for older
persons in other settings.
We thank Eric Diddlemeyer for assistance in manuscript
Financial Disclosures: This work was supported by a
cooperative agreement (U18 HS 11843) from the Agency
for Healthcare Research and Quality and the Harvard Pil-
grim Health Care Foundation.
Author Contributions: The original idea for the article
emerged from discussions among SRS, DHS, ACF, RP, and
SBS, who, along with NP, designed the study. XYextracted
and analyzed the medication dispensing data. YZ extracted
and tabulated the data regarding alert burden. NP served as
statistician. RP secured funding and directed the overall
research program within which this study was conducted.
SRS drafted the manuscript. All authors reviewed and ap-
provedthe finalversion of themanuscript.SRSis guarantor.
Sponsor’s Role: The funders had norole in the design of
the study, analysis of the data, interpretation of the results,
or the decision to publish.
1. Gurwitz JH, Field TS, Harrold LR et al. Incidence and preventability of ad-
verse drug events among older persons in the ambulatory setting. JAMA
2. Simon SR, Gurwitz JH. Drug therapy in the elderly. Improving quality and
access. Clin Pharmacol Ther 2003;73:387–393.
3. Simon SR, Chan KA, Soumerai SB et al. Potentially inappropriate medication
use by elderly persons in U.S. Health Maintenance Organizations 2000–2001.
J Am Geriatr Soc 2005;53:227–232.
4. Tamblyn R, Huang A, Perreault R et al. The medical office of the 21st century
(MOXXI): Effectiveness ofcomputerizeddecision-makingsupport inreducing
inappropriate prescribing in primary care. Can Med Assoc J 2003;169:
5. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order
entry and clinical decision support systems on medication safety: A systematic
review. Arch Intern Med 2003;163:1409–1416.
6. Garg AX, Adhikari NK, McDonald H et al. Effects of computerized clinical
decision support systems on practitioner performance and patient outcomes:
A systematic review. JAMA 2005;293:1223–1238.
7. Smith DH, Perrin N, Feldstein A et al. The impact of prescribing safety alerts
for the elderly in an electronic medical record: An interrupted time series
evaluation. Arch Intern Med, in press.
8. Feldstein A, Simon SR, Schneider J et al. How to design computerized
alert to ensure safe prescribing practices. Jt Comm J Qual Saf 2004;30:
to improve clinical decision making. JAMA 1990;263:549–556.
10. Wagner AK, Soumerai SB, Zhang F et al. Segmented regression analysis of
interrupted time series studies in medication use research. J Clin Pharm Ther
11. Cook TD, Campbell D. Quasi-Experimentation: Design and Analysis Issues
for Field Settings. Boston: Houghton Mifflin, 1979.
12. Finkelstein JA, Davis RL, Dowell SF et al. Reducing antibiotic use in children:
A randomized trial in 12 practices. Pediatrics 2001;108:1–7.
13. Soumerai SB, Ross-Degnan D, Gortmaker S et al. Withdrawing payment for
nonscientific drug therapy. Intended and unexpected effects of a large-scale
natural experiment. JAMA 1990;263:831–839.
14. Soumerai SB, McLaughlin TJ, Ross-Degnan D et al. Effects of a limit on
Medicaid drug-reimbursement benefits on the use of psychotropic agents and
acute mental health services by patients with schizophrenia. N Engl J Med
15. Institute of Medicine. Crossing the Quality Chasm: A New Health System for
the 21st Century. Washington, DC: National Academy Press, 2001.
16. Tierney WM, Overhage JM, Murray MD et al. Effects of computerized guide-
lines for managing heart disease in primary care. J Gen Intern Med 2003;
17. Majumdar SR, Lipton H, Soumerai SB. Evaluating and improving physician
prescribing. In: Strom B, ed. Pharmacoepidemiology, 4th Ed. New York: John
Wiley & Sons, 2005, pp 419–438.
18. Thomson O’Brien MA, Oxman AD, Davis DA et al. Educational outreach
visits: Effects on professional practice and health outcomes. Cochrane Data-
base Syst Rev 2000;(2):CD000409.
19. Simon SR, Majumdar SR, Prosser LA et al. Group versus individual
academic detailing to improve the use of antihypertensive medications in
primary care: A cluster-randomized controlled trial. Am J Med 2005;118:
20. Thomson O’Brien MA, Oxman AD, Davis DA. Audit and feedback: Effects on
professional practice and health care outcomes. Cochrane Database Syst Rev
21. Thomson O’Brien MA, Oxman AD, Davis DA. Local opinion leaders: Effects
on professional practice and health care outcomes. Cochrane Database Syst
22. Ray WA, Schaffner W, Federspiel CF. Persistence of improvement in antibiotic
prescribing in office practice. JAMA 1985;253:1774–1776.
23. Clark DO, Von Korff M, Saunders K et al. A chronic disease score with em-
pirically derived weights. Med Care 1995;33:783–795.
SIMON ET AL.
JUNE 2006–VOL. 54, NO. 6JAGS