Grand challenges in clinical decision support.

lin
m
, E
thw
b Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, USA
c Thomson Healthcare, Denver, CO, USA
and filter recommendations to the user; create an architecture for sharing executable CDS modules and services; combine recommenda-
ate interoperable, longitudinal electronic health records
(EHRs) for all patients to improve the quality of care
toward the goal of personalized medicine’’ [5] is to be
achieved, the amount and complexity of data available—
and decision support required to appropriately interpret
and respond to that data—will grow exponentially. The
net result is that there is a pressing need for high-quality
clinical decision support capabilities for clinicians, patients
* Corresponding author. Address: Center for Health Research, 3800 N.
Interstate Ave., Portland, OR 97227, USA. Fax: +1 503 335 6311.
E-mail address: dean.f.sittig@kp.org (D.F. Sittig).
Available online at www.sciencedirect.com
Journal of Biomedical Informatictions for patients with co-morbidities; prioritize CDS content development and implementation; create internet-accessible clinical deci-
sion support repositories; use freetext information to drive clinical decision support; mine large clinical databases to create new CDS.
Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality,
safety and efficiency of healthcare.
� 2007 Elsevier Inc. All rights reserved.
Keywords: Clinical decision support; Clinical information systems
1. Introduction
Multiple local, regional, and national initiatives [1] have
encouraged health care providers to implement state of the
art clinical information systems, targeting practice groups
ranging from single physician practices [2] to large inte-
grated delivery networks [3]. The efforts have aimed to cre-
and reduce waste. However, whether these efforts will
achieve these aims is uncertain. Models and pioneering
deployment efforts suggest that a high level of clinical deci-
sion support (CDS) is central to achieving these goals [4,8],
yet many EHRs do not currently include robust clinical
decision support features or functions. Furthermore, if
the goal of gathering ‘‘complex genomic profiling datad University of Pennsylvania Health System, Philadelphia, PA, USA
e Clinical Informatics Research and Development, Partners HealthCare System, Boston, MA, USA
f Elsevier Health Sciences, Philadelphia, PA, USA
g Department of Medicine (Emergency Medicine), Brigham & Women’s Hospital, Harvard Medical School, Boston, MA, USA
h Department of General Internal Medicine and Primary Care, Brigham & Women’s Hospital, Harvard Medical School, Boston, MA, USA
Received 13 August 2007
Available online 21 September 2007
Abstract
There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and main-
taining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building
process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate
and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if
patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: improve the human–computer
interface; disseminate best practices in CDS design, development, and implementation; summarize patient-level information; prioritizeGrand challenges in c
Dean F. Sittig a,b,*, Adam Wright b,h, Jero
Jonathan M. Teich f,g, Joan S. Ash b
a Department of Medical Informatics, Nor1532-0464/$ - see front matter � 2007 Elsevier Inc. All rights reserved.
doi:10.1016/j.jbi.2007.09.003ical decision support
e A. Osheroff c,d, Blackford Middleton e,
mily Campbell b, David W. Bates h
est Permanente, PC, Portland, OR, USA
www.elsevier.com/locate/yjbin
s 41 (2008) 387–392

tions to date in routine clinical use that have substan-
tially delivered on the promise to improve healthcare
edicprocesses and outcomes, though there have been an
array of successes at specific sites in individual domains
[8]. Yet even these successes have generally not been
widely replicated. There are many reasons for the lack
of diffusion of these systems. Some include ‘‘. . .the com-
plexity that arises from the nature of decision making,
the intellectual challenge of creating knowledge, technical
dimensions of delivering CDS, and social aspects of
incorporating changes into clinical care’’ [3]. In an
attempt to identify and describe the key challenges that
must be overcome if we are to achieve these anticipated
benefits, we (the authors along with several clinicians)
used an iterative, consensus-building process to generate
a list of the top 10 ‘‘grand challenges’’ in clinical decision
support. We then circulated this list via email so that
each person could rank the challenges from 1 (most
important) to 10 (least important). Ties were not
allowed.
The goal of this exercise was to further elucidate an as
yet unresolved challenge briefly described in 1994 [9] and
to help re-stimulate and focus efforts toward addressing
the most critical barriers to unlocking the full potential
of CDS. We ranked the challenges according to the
importance that they be solved if patients and organiza-
tions are to begin realizing the fullest benefits possible of
these systems. Our hope is that this list may help educate
and inspire stakeholders in a position to advance the
state of CDS technology and practice—particularly infor-
maticians world-wide and those who fund them—and as
a result, accelerate fruitful explorations. These challenges
resonate with the strategic objectives recently outlined by
an expert panel in a roadmap for national action on
CDS [3].
2. The top 10 clinical decision support grand challenges
We placed the grand challenges into three large categories:
1. Improve the effectiveness of CDS interventions
2. Create new CDS interventions
3. Disseminate existing CDS knowledge and
interventionsand consumers. In this paper, we will use a definition from
a widely used guidebook [6] of the term ‘‘clinical decision
support’’ to ‘‘refer broadly to providing clinicians or
patients with computer-generated clinical knowledge and
patient-related information, intelligently filtered or pre-
sented at appropriate times, to enhance patient care’’.
Many recognize the potential value of providing
advanced clinical decision support to participants in care
delivery [7]. Nonetheless, there are few CDS implementa-
388 D.F. Sittig et al. / Journal of BiomWithin each of these broad categories, we identified several
grand challenges, which we briefly describe below.2.1. Improve the effectiveness of CDS interventions
2.1.1. Improve the human–computer interface
We need a new or greatly improved human/computer
interface (HCI) paradigm for the presentation of clinical
decision support recommendations (both solicited and
unsolicited), one that supports and does not interrupt the
clinical workflow. Rather, the CDS should unobtrusively,
but effectively, remind clinicians of things they have truly
overlooked and support corrections, or better yet, put
key pieces of data and knowledge seamlessly into the con-
text of the workflow or clinical decision-making process, so
the right decisions are made in the first place [10,11]. Cur-
rently, unsolicited CDS alerts and reminders are often
overridden [12] for a multitude of reasons, one of which
is the poor human/computer interfaces that are currently
in use. We need new HCIs that will facilitate the process
by which CDS is made available to clinicians to help them
prevent both errors of omission and commission. Improved
HCI design may include increased sensitivity to the needs
of the current clinical scenario; provide clearer information
displays, with intrusiveness proportional to the importance
of the information; and make it easier for the clinician to
take action on the information provided.
2.1.2. Summarize patient-level information
No one can retain and process the entire content of a
complicated patient’s data; clinicians need to recall the
most important facts and conclusions pertinent to the cur-
rent situation. The CDS challenge is to intelligently and
automatically summarize all of a patient’s electronically
available clinical data, both freetext and coded, and to cre-
ate one or more brief (e.g., 1–2 page) synopses of the
patient’s pertinent past medical history, current condi-
tion(s), physiologic parameters, and current treatment(s).
These synopses should be sufficiently detailed to enable a
clinician to understand the patient’s current condition as
if she had spoken with all of the patient’s healthcare pro-
viders. The purpose of these summaries is to make all
key data needed for optimal decision-making available to
each decision maker; different summaries may be needed,
particularly for patients with complicated data, to address
the perspectives of different clinicians and workflows
[13,14]. In addition, these summaries should supply needed
data automatically to CDS applications that support such
decisions.
This summarization engine should be able to derive the
patient’s physiological state from a wide variety of data
sources and codify it as an ‘‘intermediate variable’’ which
could then be used as a trusted data item in another por-
tion of the logic, for example, ‘‘patient is on anticoagula-
tion therapy’’ or ‘‘patient is pregnant’’. As the amount of
electronically available, patient-specific, clinical informa-
tion increases, the need for clinicians to be able to under-
stand the patient’s pertinent medical history quickly and
al Informatics 41 (2008) 387–392accurately will become even more difficult and important.
Ultimately, vast amounts of data may be reduced to a sum-

reduce the number of computer-generated recommenda-
For example, a clinician may wish to follow a diabetes mel-
ediclitus guideline, but the guideline likely will not address the
fact that the patient may not only have diabetes, but also,
chronic obstructive pulmonary disease (COPD), and con-
gestive heart failure, as well. These co-morbid conditions
and existing medications may alter considerably the best-tions that a clinician or patient has to deal with to a man-
ageable number based upon an explicit value model, thus
reducing the ‘‘alert fatigue’’ that is a frequent cause of user
dissatisfaction. This challenge results from both the clini-
cian’s limited time and attention, as well as the patient’s
limited ability to accurately administer a large number of
medications or make multiple, difficult, life-style changes
at one time, for example.
2.1.4. Combine recommendations for patients with co-
morbidities
Current clinical care guidelines for condition or medica-
tion management, for the most part, ignore the fact that
the majority of elderly patients have multiple co-morbidi-
ties and medications that must be addressed by their
patient care team [15]. The challenge is to create mecha-
nisms to identify and eliminate redundant, contraindicated,
potentially discordant, or mutually exclusive guideline-
based recommendations for patients presenting with co-
morbid conditions or multiple medications. Instead, a
CDS system should present a synthesized version of the
recommendations from two or more guidelines to the clini-
cian. One of several reasons why clinical guidelines are
underutilized in practice is because they do not adequately
address these co-morbidity or polypharmacy issues [16].mary set of indicators allowing ‘at a glance’ assessment of
patient status. In addition, with better data-driven deriva-
tion and statement of a patient’s condition and related
data, automatic triggering of more extensive and more spe-
cific CDS becomes possible.
2.1.3. Prioritize and filter recommendations to the user
A robust, reliable, evidence-based CDS value model is
needed, particularly for intrusive CDS interventions. Such
a system could automatically prioritize recommendations
according to a multi-attribute utility model by combining
patient- and provider-specific data to take into account
expected mortality or morbidity reduction, patient prefer-
ences and life style, cost to the individual or organization,
effectiveness of the test or therapy, how the patient might
tolerate the recommended intervention, location in the cli-
nician’s workflow, insurance coverage, genetic and geno-
mic considerations, clinician’s past performance, and
other factors. The main challenge here is to appropriately
account for competing influences and values impacting
clinical decision making, and thus clinical decision support.
The second challenge here is to rank in priority order, and
D.F. Sittig et al. / Journal of Biompractice management of diabetes [17]. Addressing this chal-
lenge may require new combinatorial, logical, or semanticapproaches to combining and cross-checking recommenda-
tions from two or more guidelines.
2.1.5. Use freetext information to drive clinical decision
support
We need methods of extracting the clinical information
contained in the freetext portions of our electronic health
record systems into a form that would allow clinical deci-
sion support systems to access and utilize this information.
For such a system to work, it must be able to accurately
identify and classify the freetext information [18]. Auto-
mated text processing would enable more specific CDS
interventions to be presented (i.e., highly-tailored alerts
and reminders or even condition- or patient-specific order
sets) and could be used to satisfy existing clinical decision
support logic (e.g., by asserting that the patient is pregnant,
even though pregnancy is not specifically listed in the prob-
lem/condition list). This is especially important because,
according to some reports, at least 50% of the clinical infor-
mation describing a patient’s current condition and stage
of therapy resides in the freetext portions of the EHR [19].
2.2. Create new CDS interventions
2.2.1. Prioritize CDS content development and
implementation
Development and implementation of clinical decision
support content required to help clinicians and organiza-
tions deliver the highest quality, yet still reasonably priced
health care, will take many years. Deciding which content
to develop or implement first (e.g., interventions to
improve patient safety, chronic disease management, or
preventive health interventions), must be based on a multi-
tude of factors including value to patients, cost to the
health care system, availability of reliable data, difficulty
of implementation, and acceptability to clinicians and
patients, among others. While prioritization by national
interest and overall healthcare value may lead to longer
and more difficult discussions prior to some future CDS
deployment, in the long run this prioritization should
greatly facilitate the widespread use of the most valuable
CDS and lead to a much greater overall impact in the cost,
safety, and quality of healthcare. Over time, the current ad
hoc approach to local implementations might be sup-
planted by a more concerted, systematically prioritized
and executed approach.
2.2.2. Mine large clinical databases to create new CDS
There are undoubtedly many new, valuable guidelines
and CDS interventions that are waiting to be developed
and put into service, based on clinical knowledge that has
not yet been fully synthesized. We need to develop and test
new algorithms and techniques to allow researchers to
mine large clinical data repositories to expand the global
fund of clinical knowledge, which in turn underpins CDS
al Informatics 41 (2008) 387–392 389interventions that help promote improved outcomes. In
addition to the technical challenges associated with the cre-

edication, testing, and execution of these algorithms, we must
begin to address the myriad social and political challenges
facing researchers as they struggle to create or gain access
to these large clinical databases. For example, as these clin-
ical data resources begin to cross institutional and organi-
zational boundaries, much effort will be required to insure
that patient-identifiable information will remain private
and secure [20]. Similarly, a system that could ‘‘parse and
mine’’ the currently available scientific literature and iden-
tify potential clinical decision support interventions would
be quite useful. In other words, we should be able to pro-
gram our computers to ‘‘learn’’ from large aggregate dat-
abases [21].
2.3. Disseminate existing CDS knowledge and interventions
2.3.1. Disseminate best practices in CDS design,
development, and implementation
As noted earlier, some healthcare organizations have
had successful and enduring experience with CDS [8].
When these organizations are studied, common success
factors emerge, from design to communication to clini-
cal practice style to management; yet, this knowledge
is frequently not readily available to other organizations
seeking to develop CDS programs [22,23]. We need to
build on initial efforts [6,24] in developing more robust
methods to identify, describe, evaluate, collect, catalog,
synthesize and disseminate best practices for CDS
design, development, implementation, maintenance, and
evaluation. Specifically, we need measurement tools to
help us identify the most usable, economical and effec-
tive methods of implementing these CDS-related initia-
tives. This is primarily a matter of identification,
communication and education; the CDS implementation
process needs to be expressed and catalogued in a way
that allows information from successful sites to be easily
found by others. Additionally, best-practice information
also applies at the level of the individual CDS interven-
tion. For example, should we use an interruptive alert
to remind clinicians to order a pneumococcal vaccine,
or would a standing order for nurses be more effective
[24]?
The establishment of such methods for sharing our col-
lective experiences are essential for research and develop-
ment purposes—to refine and accelerate new intervention
development, and to highlight gaps and opportunities for
improvement in the knowledge-base itself. Identification
of CDS best practices implies the need for reliable measure-
ments and feedback mechanisms to assess CDS perfor-
mance [25,26], and comparisons across different
implementations of the same CDS tools and services, see
for example [27]. To accomplish this, we need to achieve
consensus on a standard taxonomy of clinical decision sup-
port interventions and outcomes that would allow us accu-
rately describe the best practices as well as compare
390 D.F. Sittig et al. / Journal of Biomoutcomes between implementations of different systems
and across organizations.2.3.2. Create an architecture for sharing executable CDS
modules and services
The goal is to create a set of standards-based interfaces
to externally maintained clinical decision support services
that any EHR could ‘‘subscribe to’’, in such a way that
healthcare organizations and practices can implement
new state of the art clinical decision support interventions
with little or no extra effort on their part [3,28]. These
knowledge modules might be designed so that they can
be loaded into a clinical information system [29], or they
might be designed to execute as a remote service, with
the local clinical system invoking them over a network
according to a standardized interface [30]. A key compo-
nent of this challenge would be to identify and standardize
the definitions of and interfaces to the data required by the
various CDS modules. In addition this architecture should
not require a specific knowledge representation scheme, but
rather encapsulate the clinical knowledge in such a way
that many different inference mechanisms could be used.
Similarly, the architecture should describe the general
intervention device used (e.g., alert, order set, intelligent
form) and its key parameters, while still allowing for exper-
imentation and commercial competition on the human/
computer interface within these broad guidelines. The vast
majority of EHR implementations across the USA have
currently implemented little if any clinical decision support
[31]. We hypothesize more ‘plug and play’ CDS applica-
tions will help overcome several of the key implementation
barriers that are currently limiting more widespread use of
CDS. Another benefit of such an architecture is the poten-
tial to greatly speed the transition from research finding to
widespread practice, a process that is estimated to take as
much as 17 years [32]. In the future, research articles and
consensus statements that have direct CDS implications
could be accompanied by a sharable CDS module in stan-
dard format.
2.3.3. Create internet-accessible clinical decision support
repositories
The challenge is to build one or more internet-accessible
repositories of high quality, evidence-based, tested, clinical
decision support knowledge modules. These interventions
and services could be easily downloaded, maintained,
locally modified, installed, and used on any Certification
Commission for Healthcare Information Technology
(CCHIT)-certified EHR product [33], using the architec-
ture described in Challenge 2.3.2, for example [34]. Such
a collection should have standards for accessibility, spon-
sorship, and trust levels, and appropriate business models
to ensure sustainability. The central repositories should
support local deployment of selected content in various
healthcare organizations and allow local customizations,
yet retain the ability to respond to on-going upgrades. For-
malized knowledge management processes and procedures
must be developed and made available to users of such a
al Informatics 41 (2008) 387–392system. These tools and techniques are essential for effec-
tive curation of knowledge assets of diverse types for differ-

heir
on
edicent stakeholders within an organization (e.g., to ensure
consistency of information for different care delivery set-
tings across rules, order sets, documentation tools, refer-
ence information, etc.) [e.g., Ref. 6, Fig. 6-3].
A related issue for managing the quality and integrity of
individual rules or other CDS interventions deployed in a
specific organization is to assure that in the aggregate the
content set performs inference and offers guidance appropri-
ately, and that when new knowledge is added to the local
CDS implementation, or local customizations introduced,
errors do not arise from myopic knowledge engineering, or
conflicts between disparate knowledge elements. Establish-
ment of such a repository is vital so each healthcare practice
and organization does not have to reinvent its own rules and
interventions, a painstaking and error-prone process. Some
material in such repositories may come from national
sources (e.g., like AHRQ’s USPSTF guidelines [35]), some
may come fromcommercial vendors, and somemaybebased
on interventions developed and uploaded by local care deliv-
ery organizations.
3. Discussion
We have presented a set of challenges around clinical
decision support, which we believe if addressed can unlock
its substantial potential. Undoubtedly there are other chal-
lenges which are important, but these clearly represent a
Table 1
Summary of the grand challenges of clinical decision support along with t
Grand challenge description
Improve the human–computer interface
Disseminate best practices in CDS design, development, and implementati
Summarize patient-level information
Prioritize and filter recommendations to the user
Create an architecture for sharing executable CDS modules and services
Combine recommendations for patients with co-morbidities
Prioritize CDS content development and implementation
Create internet-accessible clinical decision support repositories
Use freetext information to drive clinical decision support
Mine large clinical databases to create new CDS
D.F. Sittig et al. / Journal of Biompivotal set. It will take some time to address these and
the answers will likely vary somewhat, but proceeding
down this path will move things forward.
In our attempt to rank the challenges according to the
importance that they be solved in the near future if patients
and organizations are to begin realizing the fullest benefits
possible from these systems, we asked several of our col-
leagues to rank them in order from most to least important.
While the following ranked list (see Table 1) represents an
aggregate of these rankings, there was not a clear consen-
sus. This lack of consensus is best illustrated by the rela-
tively large standard deviations of the ranking scores.
This study has limitations. Only a small group of infor-
maticians were surveyed, though all are expert in clinical
decision support. Other challenges certainly exist and we
may not have considered all of them.4. Summary
Using an iterative, consensus-building process, we have
identified a set of grand challenges in clinical decision sup-
port. This list was created to educate and inspire an array
of stakeholders, including researchers and funders among
others, with the hope of stimulating further dialog and
effort in this important area. Solving these challenges is
critical if we are to achieve the full benefits of clinical deci-
sion support.
Acknowledgments
This research was supported by a Grant LM06942 from
the National Library of Medicine, National Institutes of
Health, titled Overcoming the Unintended Consequences
of Computerized Physician Order Entry Implementation.
Emily Campbell and Adam Wright were also supported by
National Library of Medicine training Grant ASMMI0031.
References
[1] Bush GW: Transforming Health Care: The President’s Health
Information Technology Plan, <http://www.whitehouse.gov/info-
cus/technology/economic_policy200404/chap3.html>; 2007 [accessed
04.06.07].
[2] Doctor’s Office Quality—Information Technology (DOQ-IT) Pro-
gram. Available from: http://www.joindoqit.com/.
mean rank and the standard deviation of that ranking (n = 9)
Mean ranking Standard deviation
2.89 2.71
3.33 1.87
3.67 2.06
4.56 2.96
5.44 2.30
5.89 2.20
6.00 2.96
6.89 1.69
7.89 1.27
8.44 3.00
al Informatics 41 (2008) 387–392 391[3] Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer
DE. A roadmap for national action on clinical decision support. J
Am Med Inform Assoc 2007;14(2):141–5.
[4] Johnston D, Pan E, Walker J, Bates DW, Middleton B. The value of
computerized provider order entry in ambulatory settings. Boston
(MA): Center for Information Technology Leadership; 2003.
[5] West M, Ginsburg GS, Huang AT, Nevins JR. Embracing the
complexity of genomic data for personalized medicine. Genome Res
2006;16(5):559–66.
[6] Osheroff JA, Pifer EA, Teich JM, Sittig DF, Jenders RA. Improving
outcomes with clinical decision support: an implementer’s guide. -
Health Information Management and Systems Society; 2005.
[7] American Health Information Community Quality Workgroup
Vision Summary January 31, 2007, <http://www.hhs.gov/healthit/
ahic/materials/qual_vision_summary.pdf>; 2007 [accessed 04.05.07].
[8] Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al.
Systematic review: impact of health information technology on
quality, efficiency, and costs of medical care. Ann Intern Med
2006;144(10):742–52.