Content uploaded by Raymond Nickerson
Author content
All content in this area was uploaded by Raymond Nickerson on Nov 17, 2015
Content may be subject to copyright.
Review of General Psychology
1998,
Vol. 2, No. 2, 175-220Copyright 1998 by the Educational Publishing Foundation
1089-2680«8/$3.00
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Raymond S. Nickerson
Tufts University
Confirmation bias, as the term is typically used in the psychological literature, connotes
the seeking or interpreting of evidence in ways that are partial to existing beliefs,
expectations, or a hypothesis in hand. The author reviews evidence of such a bias in a
variety of guises and gives examples of its operation in several practical contexts.
Possible explanations are considered, and the question of its utility or disutility is
discussed.
When men wish to construct or support a theory, how
they torture facts into their service! (Mackay, 1852/
1932,
p.
552)
Confirmation bias is perhaps the best known and most
widely accepted notion of inferential error to come out
of the literature on human
reasoning.
(Evans,
1989,
p.
41)
If one were to attempt to identify a single
problematic aspect of human reasoning that
deserves attention above all others, the confirma-
tion bias would have to be among the candidates
for consideration. Many have written about this
bias,
and it appears to be sufficiently strong and
pervasive that one is led to wonder whether the
bias,
by
itself,
might account for a significant
fraction of the disputes, altercations, and misun-
derstandings that occur among individuals,
groups, and nations.
Confirmation bias has been used in the
psychological literature to refer to a variety of
phenomena. Here I take the term to represent a
generic concept that subsumes several more
specific ideas that connote the inappropriate
bolstering of hypotheses or beliefs whose truth
is in question.
Deliberate Versus Spontaneous
Case Building
There is an obvious difference between
impartially evaluating evidence in order to come
to an unbiased conclusion and building a case to
justify a conclusion already drawn. In the first
instance one seeks evidence on all sides of a
Correspondence concerning this article should be ad-
dressed to Raymond S. Nickerson, Department of Psychol-
ogy, Paige Hall, Tufts University, Medford, Massachusetts
02155.
Electronic mail may be sent to mickerson@infonet.
tufts.edu.
question, evaluates it as objectively as one can,
and draws the conclusion that the evidence, in
the aggregate, seems to dictate. In the second,
one selectively gathers, or gives undue weight
to,
evidence that supports one's position while
neglecting to gather, or discounting, evidence
that would tell against it.
There is a perhaps less obvious, but also
important, difference between building a case
consciously and deliberately and engaging in
case-building without being aware of doing so.
The first type of case-building is illustrated by
what attorneys and debaters do. An attorney's
job is to make a case for one or the other side of
a legal dispute. The prosecutor tries to marshal
evidence to support the contention that a crime
has been committed; the defense attorney tries
to present evidence that will support the
presumption that the defendant is innocent.
Neither is committed to an unbiased weighing of
all the evidence at hand, but each is motivated to
confirm a particular position. Debaters also
would be expected to give primary attention to
arguments that support the positions they are
defending; they might present counterargu-
ments, but would do so only for the purpose of
pointing out their weaknesses.
As the term is used in this article and, I
believe, generally by psychologists, confirma-
tion bias connotes a less explicit, less con-
sciously one-sided case-building process. It
refers usually to unwitting selectivity in the
acquisition and use of evidence. The line
between deliberate selectivity in the use of
evidence and unwitting molding of facts to fit
hypotheses or beliefs is a difficult one to draw in
practice, but the distinction is meaningful
conceptually, and confirmation bias has more to
do with the latter than with the former. The
175
176RAYMOND S. NICKERSON
assumption that people can and do engage in
case-building unwittingly, without intending to
treat evidence in a biased way or even being
aware of doing so, is fundamental to the
concept.
The question of what constitutes confirmation
of a hypothesis has been a controversial matter
among philosophers and logicians for a long
time (Salmon, 1973). The controversy is exem-
plified by Hempel's (1945) famous argument
that the observation of a white shoe is
confirmatory for the hypothesis "All ravens are
black," which can equally well be expressed in
contrapositive form as "All nonblack things are
nonravens." Goodman's (1966) claim that
evidence that something is green is equally good
evidence that it is "grue"—grue being defined
as green before a specified future date and blue
thereafter—also provides an example. A large
literature has grown up around these and similar
puzzles and paradoxes. Here this controversy is
largely ignored. It is sufficiently clear for the
purposes of this discussion that, as used in
everyday language, confirmation connotes evi-
dence that is perceived to support—to increase
the credibility of—a hypothesis.
I also make a distinction between what might
be called motivated and unmotivated forms of
confirmation bias. People may treat evidence in
a biased way when they are motivated by the
desire to defend beliefs that they wish to
maintain. (As already noted, this is not to
suggest intentional mistreatment of evidence;
one may be selective in seeking or interpreting
evidence that pertains to a belief without being
deliberately so, or even necessarily being aware
of the selectivity.) But people also may proceed
in a biased fashion even in the testing of
hypotheses or claims in which they have no
material stake or obvious personal interest. The
former case is easier to understand in common-
sense terms than the latter because one can
appreciate the tendency to treat evidence
selectively when a valued belief is at risk. But it
is less apparent why people should be partial in
their uses of evidence when they are indifferent
to the answer to a question in hand. An adequate
account of the confirmation bias must encom-
pass both cases because the existence of each is
well documented.
There are, of
course,
instances of one wishing
to disconfirm a particular hypothesis. If, for
example, one believes a hypothesis to be untrue,
one may seek evidence of that fact or give undue
weight to such evidence. But in such cases, the
hypothesis in question is someone else's
belief.
For the individual who seeks to disconfirm such
a hypothesis, a confirmation bias would be a
bias to confirm the individual's own
belief,
namely that the hypothesis in question is false.
A Long-Recognized Phenomenon
Motivated confirmation bias has long been
believed by philosophers to be an important
determinant of thought and behavior. Francis
Bacon (1620/1939) had this to say about it, for
example:
The human understanding when it has once adopted an
opinion (either as being the received opinion or as
being agreeable to itself) draws all things else to
support and agree with it. And though there be a greater
number and weight of instances to be found on the
other side, yet these it either neglects and despises, or
else by some distinction sets aside and rejects; in order
that by this great and pernicious predetermination the
authority of its former conclusions may remain
inviolate.. . . And such is the way of all superstitions,
whether in astrology, dreams, omens, divine judg-
ments, or the like; wherein men, having a delight in
such vanities, mark the events where they are fulfilled,
but where they fail, although this happened much
oftener, neglect and pass them
by.
(p. 36)
Bacon noted that philosophy and the sciences
do not escape this tendency.
The idea that people are prone to treat
evidence in biased ways if the issue in question
matters to them is an old one among psycholo-
gists also:
If we have nothing personally at stake in a dispute
between people who are strangers to us, we are
remarkably intelligent about weighing the evidence and
in reaching a rational conclusion. We can be convinced
in favor of either of the fighting parties on the basis of
good evidence. But let the fight be our own, or let our
own friends, relatives, fraternity brothers, be parties to
the fight, and we lose our ability to see any other side of
the issue than our own. .. . The more urgent the
impulse, or the closer it comes to the maintenance of
our own selves, the more difficult it becomes to be
rational and intelligent. (Thurstone, 1924, p. 101)
The data that I consider in what follows do
not challenge either the notion that people
generally like to avoid personally disquieting
information or the belief that the strength of a
bias in the interpretation of evidence increases
with the degree to which the evidence relates
directly to a dispute in which one has a personal
CONFIRMATION BIAS177
stake. They are difficult to reconcile, however,
with the view that evidence is treated in a totally
unbiased way if only one has no personal
interest in that to which it pertains.
The following discussion of this widely
recognized bias is organized in four major
sections. In the first, I review experimental
evidence of the operation of a confirmation bias.
In the second, I provide examples of the bias at
work in practical situations. The third section
notes possible theoretical explanations of the
bias that various researchers have proposed. The
fourth addresses the question of the effects of
the confirmation bias and whether it serves any
useful purposes.
Experimental Studies
A great deal of empirical evidence supports
the idea that the confirmation bias is extensive
and strong and that it appears in many guises.
The evidence also supports the view that once
one has taken a position on an issue, one's
primary purpose becomes that of defending or
justifying that position. This is to say that
regardless of whether one's treatment of evi-
dence was evenhanded before the stand was
taken, it can become highly biased afterward.
Hypothesis-Determined Information
Seeking and Interpretation
People tend to seek information that they
consider supportive of favored hypotheses or
existing beliefs and to interpret information in
ways that are partial to those hypotheses or
beliefs. Conversely, they tend not to seek and
perhaps even to avoid information that would be
considered counterindicative with respect to
those hypotheses or beliefs and supportive of
alternative possibilities (Koriat, Lichtenstein, &
Fischhoff,
1980).
Beyond seeking information that is support-
ive of an existing hypothesis or
belief,
it appears
that people often tend to seek only, or primarily,
information that will support that hypothesis or
belief in a particular way. This qualification is
necessary because it is generally found that
people seek a specific type of information that
they would expect to find, assuming the
hypothesis is true. Also, they sometimes appear
to give weight to information that is consistent
with a hypothesis but not diagnostic with respect
to it. These generalizations are illustrated by
several of
the
following experimental findings.
Restriction of attention to a favored hypoth-
esis.
If one entertains only a single possible
explanation of some event or phenomenon, one
precludes the possibility of interpreting data as
supportive of any alternative explanation. Even
if one recognizes the possibility of other
hypotheses or beliefs, perhaps being aware that
other people hold them, but is strongly commit-
ted to a particular position, one may fail to
consider the relevance of information to the
alternative positions and apply it (favorably)
only to one's own hypothesis or
belief.
Restricting attention to a single hypothesis
and failing to give appropriate consideration to
alternative hypotheses is, in the Bayesian
framework, tantamount to failing to take likeli-
hood ratios into account. The likelihood ratio is
the ratio of two conditional probabilities,
p(D\Hx)lp(p\Hj), and represents the probability
of a particular observation (or datum) if one
hypothesis is true relative to the probability of
that same observation if the other hypothesis is
true.
Typically there are several plausible
hypotheses to account for a specific observation,
so a given hypothesis would have several
likelihood ratios. The likelihood ratio for a
hypothesis and its complement, p{D\H)l
p{D\~H),
is of special interest, however,
because an observation gives one little evidence
about the probability of the truth of a hypothesis
unless the probability of that observation, given
that the hypothesis is true, is either substantially
larger or substantially smaller than the probabil-
ity of that observation, given that the hypothesis
is false.
The notion of diagnosticity reflects the
importance of considering the probability of an
observation conditional on hypotheses other
than the favored one. An observation is said to
be diagnostic with respect to a particular
hypothesis to the extent that it is consistent with
that hypothesis and not consistent, or not as
consistent, with competing hypotheses and in
particular with the complementary hypothesis.
One would consider an observation diagnostic
with respect to a hypothesis and its complement
to the degree that the likelihood ratio, p{D\H)l
p(D\~H),
differed from 1. An observation that
is consistent with more than one hypothesis is
said not to be diagnostic with respect to those
hypotheses; when one considers the probability
178RAYMOND S. NICKERSON
of an observation conditional on only a single
hypothesis, one has no way of determining
whether the observation is diagnostic.
Evidence suggests that people often do the
equivalent of considering only p(D\H) and
failing to take into account the ratio of this and
p(D\~H),
despite the fact that considering only
one of these probabilities does not provide a
legitimate basis for assessing the credibility of//
(Beyth-Marom &
Fischhoff,
1983; Doherty &
Mynatt, 1986; Doherty, Mynatt, Tweney, &
Schiavo, 1979; Griffin & Tversky, 1992; Kern &
Doherty, 1982; Troutman & Shanteau, 1977).
This tendency to focus exclusively on the case
in which the hypothesis is assumed to be true is
often referred to as a tendency toward pseudodi-
agnosticity (Doherty & Mynatt, 1986; Doherty
et al., 1979; Fischhoff & Beyth-Marom, 1983;
Kern & Doherty, 1982). Fischhoff and Beyth-
Marom (1983) have argued that much of what
has been interpreted as a confirmation bias can
be attributed to such a focus and the consequen-
tial failure to consider likelihood ratios.
Preferential treatment of evidence supporting
existing beliefs. Closely related to the restric-
tion of attention to a favored hypothesis is the
tendency to give greater weight to information
that is supportive of existing beliefs or opinions
than to information that runs counter to them.
This does not necessarily mean completely
ignoring the counterindicative information but
means being less receptive to it than to
supportive information—more likely, for ex-
ample, to seek to discredit it or to explain it
away.
Preferential treatment of evidence supporting
existing beliefs or opinions is seen in the
tendency of people to recall or produce reasons
supporting the side they favor—my-side
bias—on a controversial issue and not to recall
or produce reasons supporting the other side
(Baron, 1991, 1995; Perkins, Allen, & Hafner,
1983;
Perkins, Farady, & Bushey, 1991). It
could be either that how well people remember a
reason depends on whether it supports their
position, or that people hold a position because
they can think of more reasons to support it.
Participants in the study by Perkins, Farady, and
Bushey were capable of generating reasons for
holding a view counter to their own when
explicitly asked to do so; this finding led
Tishman, Jay, and Perkins (1993) to interpret the
failure to do so spontaneously as a motivational
problem as distinct from a cognitive limitation.
Baron (1995) found that, when asked
to
judge
the quality of arguments, many people were
likely to rate one-sided arguments higher than
two-sided arguments, suggesting that the bias is
at least partially due to common beliefs about
what makes an argument strong. In keeping with
this result, participants in a mock jury trial who
tended to use evidence selectively to build one
view of what happened expressed greater
confidence in their decisions than did those who
spontaneously tried to weigh both sides of the
case (D. Kuhn, Weinstock, & Flaton, 1994).
When children and young adults were given
evidence that was inconsistent with a theory
they favored, they often "either failed to
acknowledge discrepant evidence or attended to
it in a selective, distorting manner. Identical
evidence was interpreted one way in relation to a
favored theory and another way in relation to a
theory that was not favored" (D. Kuhn, 1989, p.
677).
Some of Kuhn's participants were unable
to indicate what evidence would be inconsistent
with their theories; some were able to generate
alternative theories when asked, but they did not
do so spontaneously. When they were asked to
recall their theories and the related evidence that
had been presented, participants were likely to
recall the evidence as being more consistent
with the theories than it actually was. The
greater perceived consistency was achieved
sometimes by inaccurate recall of theory and
sometimes by inaccurate recall of evidence.
Looking only or primarily for positive cases.
What is considerably more surprising than the
fact that people seek and interpret information in
ways that increase their confidence in favored
hypotheses and established beliefs is the fact
that they appear to seek confirmatory informa-
tion even for hypotheses in whose truth value
they have no vested interest. In their pioneering
concept-discovery experiments, Bruner, Good-
now, and Austin (1956) found that participants
often tested a hypothesized concept by choosing
only examples that would be classified as
instances of the sought-for concept if the
hypothesis were correct. This strategy precludes
discovery, in some cases, that an incorrect
hypothesis is incorrect. For example, suppose
the concept to be discovered is small circle and
one's hypothesis is small red
circle.
If one tests
the hypothesis by selecting only things that are
CONFIRMATION BIAS179
small, red, and circular, one will never discover
that the class denned by the concept includes
also small circular things that are yellow or blue.
Several investigators (Levine, 1970; Millward
& Spoehr, 1973; Taplin, 1975; Tweney et al.,
1980;
Wason & Johnson-Laird, 1972) subse-
quently observed the same behavior of partici-
pants testing only cases that are members of the
hypothesized category.
Some studies demonstrating selective testing
behavior of this sort involved a task invented by
Wason (1960) in which people were asked to
find the rule that was used to generate specified
triplets of numbers. The experimenter presented
a triplet, and the participant hypothesized the
rule that produced it. The participant then tested
the hypothesis by suggesting additional triplets
and being told, in each case, whether it was
consistent with the rule to be discovered. People
typically tested hypothesized rules by producing
only triplets that were consistent with them.
Because in most cases they did not generate any
test items that were inconsistent with the
hypothesized rule, they precluded themselves
from discovering that it was incorrect if the
triplets it prescribed constituted a subset of those
prescribed by the actual rule. Given the triplet
2-4-6, for example, people were likely to come
up with the hypothesis successive even numbers
and then proceed to test this hypothesis by
generating additional sets of successive even
numbers. If the 2-4-6 set had actually been
produced by the rule numbers increasing by 2,
numbers increasing in size, or any
three
positive
numbers, the strategy of using only sets of
successive even numbers would not reveal the
incorrectness of the hypothesis because every
test item would get a positive response.
The use only of test cases that will yield a
positive response if
a
hypothesis under consider-
ation is correct not only precludes discovering
the incorrectness of certain types of hypotheses
with a correct hypothesis, this strategy would
not yield as strongly confirmatory evidence,
logically, as would that of deliberately selecting
tests that would show the hypothesis to be
wrong, if it is wrong, and failing in the attempt.
To the extent that the strategy of looking only
for positive cases is motivated by a wish to find
confirmatory evidence, it is misguided. The
results this endeavor will yield will, at best, be
consistent with the hypothesis, but the confirma-
tory evidence they provide will not be as
compelling as would the failure of a rigorous
attempt at disconfirmation.
This point is worth emphasizing because the
psychological literature contains many refer-
ences to the confirmatory feedback a participant
gets when testing a hypothesis with a positive
case.
These references do not generally distin-
guish between confirmatory in a logical sense
and confirmatory in a psychological sense. The
results obtained by Wason (1960) and others
suggest that feedback that is typically inter-
preted by participants to be strongly confirma-
tory often is not logically confirmatory, or at
least not strongly so. The "confirmation" the
participant receives in this situation is, to some
degree, illusory. This same observation applies
to other studies mentioned in the remainder of
this article.
In an early commentary on the triplet-rule
task, Wetherick (1962) argued that the experi-
mental situation did not reveal the participants'
intentions in designating any particular triplet as
a test case. He noted that any test triplet could
either conform or not conform to the rule, as
defined by the experimenter, and it could also
either conform or not conform to the hypothesis
being considered by the participant. Any given
test case could relate to the rule and hypothesis
in combination in any of four ways: conform-
conform, conform-not conform, not conform-
conform, and not conform-not conform. Weth-
erick argued that one could not determine
whether an individual was intentionally attempt-
ing to eliminate a candidate hypothesis unless
one could distinguish between test cases that
were selected because they conformed to a
hypothesis under consideration and those that
were selected because they did not.
Suppose a participant selects the triplet 3-5-7
and is told that it is consistent with the rule (the
rule being any three numbers in ascending
order).
The participant might have chosen this
triplet because it conforms to the hypothesis
being considered, say numbers increasing by
two,
and might have taken the positive response
as evidence that the hypothesis is correct. On the
other hand, the participant could have selected
this triplet in order to eliminate one or more
possible hypotheses (e.g., even numbers
ascend-
ing; a
number,
twice the
number,
three times the
number).
In this case, the experimenter's
positive response would constitute the discon-
180RAYMOND S. NICKERSON
firming evidence (with respect to these hypoth-
eses) the participant sought.
Wetherick (1962) also pointed out that a test
triplet may logically rule out possible hypoth-
eses without people being aware of the fact
because they never considered those hypoth-
eses.
A positive answer to the triplet 3-5-7
logically eliminates even numbers ascending
and a
number,
twice the
number,
three times the
number, among other possibilities, regardless of
whether the participant thought of them. But of
course, only if the triplet was selected with the
intention of ruling out those options should its
selection be taken as an instance of a falsifica-
tion strategy. Wetherick's point was that without
knowing what people have in mind in making
the selections they do, one cannot tell whether
they are attempting to eliminate candidates from
further consideration or not.
Wason (1962, 1968/1977) responded to this
objection with further analyses of the data from
the original experiment and data from additional
experiments showing that although some partici-
pants gave evidence of understanding the
concept of falsification, many did not. Wason
summarized the findings from these experi-
ments this way: "there would appear to be
compelling evidence to indicate that even
intelligent individuals adhere to their own
hypotheses with remarkable tenacity when they
can produce confirming evidence for them"
(1968/1977, p. 313).
In other experiments in which participants
have been asked to determine which of several
hypotheses is the correct one to explain some
situation or event, they have tended to ask
questions for which the correct answer would be
yes if the hypothesis under consideration were
true (Mynatt, Doherty, & Tweney, 1977; Shak-
lee &
Fischhoff,
1982). These experiments are
among many that have been taken to reveal not
only a disinclination to test a hypothesis by
selecting tests that would show it to be false if it
is false, but also a preference for questions that
will yield a positive answer if the hypothesis is
true.
Others have noted the tendency to ask
questions for which the answer is yes if the
hypothesis being tested is correct in the context
of experiments on personality perception (Hod-
gins & Zuckerman, 1993; Schwartz, 1982;
Strohmer & Newman, 1983; Trope & Bassock,
1982,
1983; Trope, Bassock, & Alon, 1984;
Zuckerman, Knee, Hodgins, & Miyake, 1995).
Fischhoff and Beyth-Marom (1983) also noted
the possibility that participants in such experi-
ments tend to assume that the hypothesis they
are asked to test is true and select questions that
would be the least awkward to answer if that is
the case. For instance, participants asked
assumed extroverts (or introverts) questions that
extroverts (or introverts) would find particularly
easy to answer.
Overweighting positive confirmatory in-
stances. Studies of social judgment provide
evidence that people tend to overweight positive
confirmatory evidence or underweight negative
discomfirmatory evidence. Pyszczynski and
Greenberg (1987) interpreted such evidence as
supportive of the view that people generally
require less hypothesis-consistent evidence to
accept a hypothesis than hypothesis-inconsistent
information to reject a hypothesis. These
investigators argued, however, that this asymme-
try is modulated by such factors as the degree of
confidence one has in the hypothesis to begin
with and the importance attached to drawing a
correct conclusion. Although they saw the need
for accuracy as one important determinant of
hypothesis-evaluating behavior, they suggested
that other motivational factors, such as needs for
self-esteem, control, and cognitive consistency,
also play significant roles.
People can exploit others' tendency to over-
weight (psychologically) confirming evidence
and underweight disconfirming evidence for
many purposes. When the mind reader, for
example, describes one's character in more-or-
less universally valid terms, individuals who
want to believe that their minds are being read
will have little difficulty finding substantiating
evidence in what the mind reader says if they
focus on what fits and discount what does not
and if they fail to consider the possibility that
equally accurate descriptions can be produced if
their minds are not being read (Fischhoff &
Beyth-Marom, 1983; Forer, 1949; Hyman,
1977).
People who wish to believe in astrology
or the predictive power of psychics will have no
problem finding some predictions that have
turned out to be true, and this may suffice to
strengthen their belief if they fail to consider
either predictions that proved not to be accurate
or the possibility that people without the ability
to see the future could make predictions with
equally high (or low) hit rates. A confirmation
CONFIRMATION BIAS181
bias can work here in two ways: (a) people may
attend selectively to what is said that turns out to
be true, ignoring or discounting what turns out
to be false, and (b) they may consider only
p(D\H),
the probability that what was said
would be said if the seer could really see, and
fail to consider p(D\~H), the probability that
what was said would be said if the seer had no
special psychic powers. The tendency of gam-
blers to explain away their losses, thus permit-
ting themselves to believe that their chances of
winning are higher than they really are, also
illustrates the overweighting of supportive
evidence and the underweighting of opposing
evidence (Gilovich, 1983).
Seeing what one is looking for. People
sometimes see in data the patterns for which
they are looking, regardless of whether the
patterns are really there. An early study by
Kelley (1950) of the effect of expectations on
social perception found that students' percep-
tions of social qualities (e.g., relative sociability,
friendliness) of a guest lecturer were influenced
by what they had been led to expect from a prior
description of the individual. Forer (1949)
demonstrated the ease with which people could
be convinced that a personality sketch was an
accurate depiction of themselves and their
disinclination to consider how adequately the
sketch might describe others as well.
Several studies by Snyder and his colleagues
involving
the
judgment of personality traits lend
credence to the idea that the degree to which
what people see or remember corresponds to
what they are looking for exceeds the correspon-
dence as