Probability Judgment in Medicine

Abstract

Research in cognitive psychology has indicated that alternative descriptions of the same event can give rise to different probability judgments. This observation has led to the development of a descriptive account, called support theory, which assumes that the judged probability of an explicit description of an event (that lists specific possibilities) generally exceeds the judged probability of an implicit description of the same event (that does not mention specific possibilities). To investigate this assumption in medical judgment, the authors presented physicians with brief clinical scenarios describing individual patients and elicited diagnostic and prognostic probability judgments. The results showed that the physicians tended to discount unspecified possibilities, as predicted by support theory. The authors suggest that an awareness of the discrepancy between intuitive judgments and the laws of chance may provide opportunities for improving medical decision making.

Full text

227
The
Psychology
of
Decision
Making
Probability
Judgment
in
Medicine:
Discounting
Unspecified
Possibilities
DONALD
A.
REDELMEIER,
MD,
DEREK J.
KOEHLER,
PhD,
VARDA
LIBERMAN,
PhD,
AMOS
TVERSKY,
PhD
Research
in
cognitive
psychology
has
indicated
that
alternative
descriptions
of
the
same
event
can
give
rise
to
different
probability
judgments.
This
observation
has
led
to
the
de-
velopment
of
a
descriptive
account,
called
support
theory,
which
assumes
that
the
judged
probability
of
an
explicit
description
of
an
event
(that
lists
specific
possibilities)
generally
exceeds
the
judged
probability
of
an
implicit
description
of
the
same
event
(that
does
not
mention
specific
possibilities).
To
investigate
this
assumption
in
medical
judgment,
the
au-
thors
presented
physicians
with
brief
clinical
scenarios
describing
individual
patients
and
elicited
diagnostic
and
prognostic
probability
judgments.
The
results
showed
that
the
phy-
sicians
tended
to
discount
unspecified
possibilities,
as
predicted
by
support
theory.
The
authors
suggest
that
an
awareness
of
the
discrepancy
between
intuitive
judgments
and
the
laws
of
chance
may
provide
opportunities
for
improving
medical
decision
making.
Key
words:
probability
judgment;
support
theory;
unpacking
principle;
cognition.
(Med
Decis
Making
1995;15:227-230)
Medical
decisions
are
often
made
under
uncertainty.
When
evaluating
a
patient
with
chest
pain,
for
ex-
ample,
a
physician
needs
to
consider
the
possibility
that
the
patient
is
having
a
myocardial
infarction,
the
risk
of
a
serious
hemorrhage
if
thrombolytics
are
ad-
ministered,
and
the
consequences
if
thrombolytics
are
not
administered.
Uncertainty
can
sometimes
be
re-
duced
by
collecting
additional
data,
reviewing
the
sci-
entific
literature,
and
consulting
experts.
However,
it
cannot
always
be
eliminated
in
a
timely
manner.’
As
a
consequence,
action
often
depends
on
intuitive
judgments
of
the
likelihoods
of
various
possibilities.
Research
on
judgment
under
uncertainty
has
shown
that
both
laypeople
and
experts
do
not
always
follow
the
principles
of
probability
theoiy.2~3
In
particular,
alternative
representations
of
the
same
possibility
can
give
rise
to
different
probability
judgments.’
To
ac-
count
for
such
observations,
Tversky
and
Koehler’
have
developed
an
account
in
which
probability
is
assigned
not
to
events-as
in
other
models-but
rather
to
de-
scriptions
of
events,
called
hypotheses.
This
account,
called
support
theory,
assumes
that
each
hypothesis
refers
to
a
unique
event,
but
that
a
given
event
can
be
described
by
more
than
one
hypothesis.
For
example,
the
explicit
hypothesis
&dquo;death
due
to
traffic
accident,
drowning,
electrocution,
or
any
other
accident&dquo;
and
the
implicit
hypothesis
&dquo;death
due
to
an
accident&dquo;
represent
different
descriptions
of
the
same
event.
The
central
assumption
of
support
theory
is
the
unpacking
principle:
providing
a
more
detailed
de-
scription
of
an
implicit
hypothesis
generally
increases
its
judged
probability.
Thus,
the
judged
probability
of
the
explicit
description
that
lists
various
accidents
generally
exceeds
the
judged
probability
of
the
im-
plicit
description
that
does
not
mention
specific
ac-
cidents.
Like
the
measured
length
of
a
coastline,
which
increases
as
the
map
becomes
more
detailed,
the
per-
ceived
likelihood
of
an
event
increases
as
its
descrip-
tion
becomes
more
specific.
Both
memory
and
atten-
tion
contribute
to
this
effect:
unpacking
can
remind
people
of
possibilities
they
might
have
overlooked,
and
the
explicit
mentioning
of
a
possibility
may
increase
Received
January
14,
1994,
from
the
University
of
Toronto,
To-
ronto,
Ontario,
Canada
IDAR) ;
the
Wellesley
Hospital
Research
In-
stitute,
Toronto
(DAR);
Stanford
University,
Stanford,
California
(DJK,
AT);
and
the
Open
University
of
Israel,
Tel
Aviv,
Israel
(VL).
Revision
accepted
for
publication
August
18, 1994.
Supported
by
a
grant
from
the
PSI
Foundation
of
Ontario,
by
Grant
#SBR-9408684
from
the
National
Science
Foundation,
and
by
a
Fulbright
Visiting
Scholar-
ship
Award.
Dr.
Redelmeier
was
supported
by
a
career
scientist
award
from
the
Ontario
Ministry
of
Health,
Dr.
Koehler
by
a
National
Defence
Science
and
Engineering
Graduate
Fellowship,
and
Drs.
Liberman
and
Tversky
by
grant
#92-00389
from
the
United
States-
Israel
Binational
Science
Foundation.
Address
correspondence
and
reprint
requests
to
Dr.
Redelmeier:
Clinical
Epidemiology
Division,
The
Wellesley
Hospital
Research
Institute,
160
Wellesley
St.,
East,
650
Turner
Wing,
Toronto,
Ontario,
Canada
M4Y
lJ3.

228
its
salience
and
hence
its
perceived
likelihood.
In
accord
with
the
classic
theory
of
probability,
sup-
port
theory
assumes
that
the
judged
probability
of
a
hypothesis
and
of
its
complement
add
to
unity.
For
example,
the
judged
probability
of
the
hypothesis
&dquo;death
due
to
a
natural
cause&dquo;
and
that
of
the
hypothesis
&dquo;death
due
to
an
unnatural
cause&dquo;
should
sum
to
one,
even
though
each
judgment
could
be
increased
by
unpacking
the
respective
category.
The
unpacking
ef-
fect,
as
well
as
binary
complementarity,
have
been
observed
in
several
experiments
involving
nonmedical
situations s
The
present
article
explores
these
prin-
ciples
in
medical
judgments.
To
do
so,
we
presented
clinicians
with
brief
scenarios
describing
an
individual
patient
and
asked
them
to
judge
the
probabilities
of
relevant
medical
possibilities.
Unpacking
the
Residual
.
In
a
survey
of
house
officers
(n
=
59)
at
Stanford
University,
physicians
were
asked
to
review
the
fol-
lowing
scenario:
A
well-known
22-year-old
Hollywood
actress
presents
to
the
emergency
department
with
pain
in
the
right
lower
quadrant
of
her
abdomen
of
12
hours’
duration.
Her
last
normal
menstrual
period
was
four
weeks
ago.
Half
the
physicians,
selected
at
random,
were
asked
to
estimate
probabilities
for
two
diagnoses
(&dquo;gastroen-
teritis&dquo;
and
&dquo;ectopic
pregnancy&dquo;)
and
the
residual
cat-
egory
(&dquo;none
of
the
above&dquo;).
The
other
physicians
were
asked
to
estimate
probabilities
for
the
following
five
diagnoses:
the
two
diagnoses
specified
above
(&dquo;gas-
troenteritis&dquo;
and
&dquo;ectopic
pregnancy&dquo;),
three
addi-
tional
specific
diagnoses
(&dquo;appendicitis,&dquo;
&dquo;pyelone-
phritis,&dquo;
and
&dquo;pelvic
inflammatory
disease&dquo;),
and
the
residual
category
(&dquo;none
of
the
above&dquo;).
The
two
tasks
differed
only
in
that
the
residual
category
in
the
first
(short)
list
was
partially
unpacked
in
the
second
(long)
list.
All
the
physicians
were
told
that
the
patient
had
only
one
condition
and,
hence,
that
the
judged
prob-
abilities
should
add
to
100%.
Logically,
the
probability
of
the
residual
&dquo;none
of
the
above&dquo;
in
the
short
list
should
equal
the
sum
of
the
probabilities
of
the
corresponding
possibilities
in
the
long
list.
In
accord
with
the
unpacking
principle,
however,
we
found
that
the
average
probability
as-
signed
to
the
residual
in
the
short
list
was
smaller
than
the
sum
of
the
corresponding
probabilities
in
the
long
list
(50%
vs
69%,
p
<
0.005
by
Mann-Whitney
test).
As
a
consequence,
unpacking
the
residual
cat-
egory
changed
the
probabilities
assigned
to
specific
diagnoses.
For
example,
the
average
probability
as-
signed
to
&dquo;gastroenteritis&dquo;
was
substantially
higher
in
the
short
list
than
in
the
long
list
(31%
vs
16%,
p
<
0.005
by
Mann-Whitney
test).
Evidently,
unpacking
the
residual
hypothesis
reminded
physicians
of
dis-
eases
they
might
have
overlooked,
or
increased
the
salience
of
diagnoses
that
they
had
considered.
Highlighting
One
Possibility
In
the
previous
example
physicians
were
asked
to
assign
probabilities
to
a
set
of
possibilities.
Often,
how-
ever,
physicians
focus
on
a
single
possibility.
In
this
case,
they
may
be
prone
to
overestimate
the
likelihood
of
that
possibility
because
its
alternatives
are
unspec-
ified.
To
illustrate
this
point,
we
presented
the
follow-
ing
scenario
to
a
group
of
expert
physicians
(n
=
52)
at
Tel
Aviv
University:
R.G.
is
a
67-year-old
retired
farmer
who
presents
to
the
emergency
department
with
chest
pain
of
four
hours’
duration.
The
diagnosis
is
acute
myocardial
infarction.
Physical
examination
shows
no
evidence
of
pulmonary
edema,
hypotension,
or
mental
status
changes.
His
EKG
shows
ST-segment
elevation
in
the
anterior
leads,
but
no
dysrythmia
or
heart
block.
His
past
medical
history
is
unremarkable.
He
is
admitted
to
the
hospital
and
treated
in
the
usual
manner.
Consider
the
possible
out-
comes.
Each
physician
was
randomly
assigned
to
evaluate
one
of
the
following
four
prognoses
for
this
patient:
&dquo;dying
during
this
admission,&dquo;
&dquo;surviving
this
admis-
sion
but
dying
within
one
year,&dquo;
&dquo;living
for
more
than
one
year
but
less
than
ten
years,&dquo;
or
&dquo;surviving
for
more
than
ten
years.&dquo;
The
average
probabilities
as-
signed
to
these
prognoses
were
14%,
26%,
55%,
and
69%,
respectively.
According
to
standard
theory,
the
probabilities
assigned
to these
outcomes
should
sum
to
100%.
In
contrast,
the
average
judgments
added
to
164%
(95%
confidence
interval:
134%
to
194%).
As
im-
plied
by
the
unpacking
principle,
the
physicians
in
each
group
overweighted
the
possibility
that
was
ex-
plicitly
mentioned
relative
to
the
unspecified
alter-
native.
All
groups,
indeed,
overestimated
the
frequen-
cies
reported
in
the
literature.~
Notice
that
while
the
results
of
the
previous
problem
can
be
interpreted
as
a
memory
effect
(reminding
physicians
of
additional
possibilities),
the
present
results
represent
an
atten-
tion
effect
(highlighting
a
particular
interval
on
a
con-
tinuum).
Binary
Completnentarity
We
have
attributed
the
preceding
results
to
the
un-
packing
principle.
An
alternative
interpretation
is
that
people
overestimate
the
(focal)
hypothesis
that
they
are
asked
to
evaluate.
If
this
interpretation
be
correct,
the
sum
of
the
judged
probabilities
for
a
pair
of
com-
plementary
hypotheses
should exceed
one.
To
test

229
this
prediction,
we
presented
the
preceding
scenario
to
fourth-year
medical
students
(n =
149)
at
the
Uni-
versity
of
Toronto.
Half
the
participants,
selected
at
random,
were
asked
to
evaluate
the
probability
that
the
patient
would
&dquo;survive
this
hospitalization.&dquo;
The
other
half
were
asked
to
evaluate
the
probability
that
the
patient
would
&dquo;die
during
this
hospitalization.&dquo;
We
found
that
the
mean
judged
probabilities
in
the
two
groups
were
78%
and
21%,
respectively,
summing
to
99%
(95%
confidence
interval:
94%
to
104%).
As
im-
plied
by
support
theory,
judged
probabilities
add
to
100%
in
cases
with
only
two
possibilities,
and
exceed
100%
in
cases
involving
more
than
two
possibilities.
This
observation
demonstrates
that
people
overesti-
mate
what
is
specified,
not
what
is
under
evaluation.
Treabnent
Decisions
The
final
example
shows
that
the
unpacking
effect
is
not
limited
to
probability
judgments
but
can
also
extend
to
treatment
decisions.
We
asked
fourth-year
medical
students
( n
=
148)
at
the
University
of Toronto
to
consider
the
following
scenario:
M.S.
is
a
43-year-old
journalist
who
presents
to
the
emergency
department
because
of
a
fever
and
head-
ache
of
two
days’
duration.
Past
medical
history
is
re-
markable
only
for
15
years
of lupus
erythematosus,
con-
trolled
on
Tylenol
and
chronic
steroids
(prednisone
10
mg
daily).
She
does
not
look
sick.
Vital
signs
are
normal.
Physical
examination
reveals
tenderness
over
the
fron-
tal
sinuses
and
pharyngeal
erythema.
There
is
no
neck
stiffness,
tympanic
membrane
redness,
or
cervical
ad-
enopathy.
The
remainder
of
the
physical
examination
is
unremarkable
aside
from
some
degenerative
changes
in
the
small
joints
of
both
hands.
For
half
the
students,
selected
at
random,
the
scenario
was
followed
by
the
sentence:
&dquo;Obviously,
many
di-
agnoses
are
possible
given
this
limited
information,
including
CNS
vasculitis,
lupus
cerebritis,
intracranial
opportunistic
infection,
sinusitis,
and
a
subdural
hematoma.&dquo;
The
other
half
were
presented
with
a
shorter
sentence:
&dquo;Obviously,
many
diagnoses
are
pos-
sible
given
this
limited
information,
including
sinus-
itis.&dquo;
Individuals
in
both
groups
were
asked
to
indicate
whether
they
would
recommend
ordering
a
CAT
scan
of
the
head.
Logically,
there
should
be
no
difference
between
the
responses
to
the
two
versions
because
both
describe
the
same
situation.
On
the other
hand,
support
theory
suggests
that
the
possibility
of
sinusitis will
loom
larger
when
it
is
the
only
specified
diagnosis
than
when
it
is
accompanied
by
other
specified
diagnoses.
Con-
sequently,
we
expected
that
fewer
physicians
would
order
a
CAT
scan
in
the
short
version
because
the
diagnosis
of
sinusitis
does
not
normally
call
for
this
test/
Indeed,
we
found
that
fewer
respondents
rec-
ommended
a
CAT
scan
in
response
to
the
short
ver-
sion
than
in
response
to
the
long
version
(20%
vs
32%,
p
<
0.05
by
Mann-Whitney
test).
Thus,
the
unpacking
principle
applies
to
treatment
recommendations,
not
only
to
probability
judgments.
Conclusion
Subjective
assessments
of
uncertain
events
are
sometimes
necessary,
even
though
they
are
often
fal-
lible.
In
this
study
we
focused
on
a
particularly
sig-
nificant
source
of
error,
namely,
the
tendency
to
dis-
count
unspecified
possibilities.
In
the
first
problem
we
demonstrated
the
unpacking
effect
in
a
diagnostic
task
by
reminding
physicians
of
possibilities
they
might
have
overlooked.
In
the
second
problem
we
obtained
the
same
effect
in
a
prognostic
task
by
highlighting
a
specific
interval
along
a
continuum.
In
the
third
prob-
lem
we
showed
that
the
unpacking
effect
cannot
be
explained
by
overestimating
the
focal
possibility.
And
in
the
final
problem
we
illustrated
the
unpacking
effect
in
a
decision
task.
Together,
the
findings
confirm
the
main
qualitative
predictions
of
support
theory
in
med-
ical
judgment.
It
could
be
argued
that
our
respondents
believed
that
the
request
to
evaluate
a
particular
hypothesis
conveys
relevant
information
and
suggests
that
the
hypothesis
in
question
is
not
improbable.
Although
such
belief
may
contribute
to
the
unpacking
effect,
it
does
not
fully
explain
the
data.
First,
this
account
im-
plies
an
overweighting
of
the
focal
hypothesis,
con-
trary
to
the
finding
of binary
complementarity.
Second,
the
unpacking
effect
was
pronounced
in
the
myocar-
dial
infarction
example,
where
the
experts
were
in-
formed
that
other
physicians
were
evaluating
different
hypotheses.
Finally,
the
unpacking
effect
has
also
been
observed
in
non-medical
problems
where
the
re-
spondents
were
made
aware
that
the
focal
hypothesis
had
been
randomly
chosen.s
Although
there
is
no
simple
method
for
eliminating
the
unpacking
effect,
we
call
attention
to
its
presence
and
suggest
some
corrective
procedures.
First,
clini-
cians
need
to
recognize
that
judgments
under
uncer-
tainty
are
susceptible
to
error;
in
particular,
alternative
descriptions
of
the
same
situation
may
lead
to
differ-
ent
judgments.
Second,
clinicians
should
be
encour-
aged
to
unpack
broad
categories
and
compare
pos-
sibilities
at
similar
levels
of
specificity,
rather
than
compare
a
single
specific
possibility
against
an
un-
specified
set
of
alternatives.
Indeed,
unpacking
the
implicit
complement
of
a
focal
hypothesis
may
serve
as
a
useful
method
for
reducing
overconfidence.
More
generally,
a
better
understanding
of
the
cognitive
psy-
chology
underlying
medical
judgment
could
help
identify
common
biases
and
suggest
corrective
pro-
cedures.

230
References
1.
Pauker
SG,
Kopelman
RI.
How
sure
is
sure
enough?
N
Engl
J
Med.
1992;326:688-91.
2.
Kahneman
D,
Slovic
P,
Tversky
A
(eds).
Judgment
under
Uncer-
tainty:
Heuristics
and
Biases.
New
York:
Cambridge
University
Press,
1982.
3.
von
Winterfeldt
D,
Edwards
W.
Decision
Analysis
and
Behavioral
Research.
New
York:
Cambridge
University
Press,
1986.
4.
Fischhoff
B,
Slovic
P,
Lichtenstein
S.
Fault
trees:
sensitivity
of
estimated
failure
probabilities
to
problem
representation.
J
Exp
Psychol
Hum
Percept
Perform.
1978;3:552-64.
5.
Tversky
A,
Koehler
DJ.
Support
theory:
a
nonextensional
repre-
sentation
of
subjective
probability.
Psychol
Rev.
1994;101:547-67.
6.
Goldberg
RJ,
Gore
JM,
Alpert
JS,
et
al.
Cardiogenic
shock
after
acute
myocardial
infarction:
incidence
and
mortality
from
a
com-
munity-wide
perspective
1975
to
1988.
N
Engl
J Med.1991;325:1117-
22.
7.
Hourihane
J.
CT
scans
and
the
common
cold.
N
Engl
J
Med.
1994;1330:1826-7.