INSIGHTFUL OR WISHFUL:
Lawyers’ Ability to Predict Case Outcomes
Charles Sturt University, Manly, New
South Wales, Australia
Pa¨r Anders Granhag
University of Gothenburg, Gothenburg,
John Jay College of Criminal Justice Elizabeth F. Loftus
University of California
Lawyers’ litigation forecasts play an integral role in the justice system. In the course of
litigation, lawyers constantly make strategic decisions and/or advise their clients on the
basis of their perceptions and predictions of case outcomes. The study investigated the
realism in predictions by a sample of attorneys (n⫽481) across the United States who
speciﬁed a minimum goal to achieve in a case set for trial. They estimated their chances
of meeting this goal by providing a conﬁdence estimate. After the cases were resolved,
case outcomes were compared with the predictions. Overall, lawyers were overconﬁdent
in their predictions, and calibration did not increase with years of legal experience.
Female lawyers were slightly better calibrated than their male counterparts and showed
evidence of less overconﬁdence. In an attempt to reduce overconﬁdence, some lawyers
were asked to generate reasons why they might not achieve their stated goals. This
manipulation did not improve calibration.
Keywords: legal decision making, case predictions, conﬁdence judgments,
Outcome Predictions in Legal Decision Making
Human cognition is not only retrospective but also prospective: People look
toward an imagined future and make various assessments and judgments on the basis
of possible future events (Flavell, 2004; Kahneman, 1973; Meeks, Hicks & Marsh,
2007). As an important part of prospective reasoning, people establish goals that they
attempt to accomplish (Gollwitzer & Schaal, 1998). These goals are associated with
conﬁdence judgments, which are deﬁned as subjective assessments of the probability
of attaining the desired goal (Koriat, 2002; Perfect & Schwartz, 2002).
In the course of regular legal practice, judgments and meta-judgments of
future goals are an important aspect of a wide range of litigation-related decisions
Jane Goodman-Delahunty, Australian Graduate School of Policing and School of Psychology,
Charles Sturt University, Manly, New South Wales, Australia; Pa¨r Anders Granhag, Department of
Psychology, University of Gothenburg, Gothenburg, Sweden; Maria Hartwig, Department of
Psychology, John Jay College of Criminal Justice, New York; Elizabeth F. Loftus, Psychology &
Social Behavior, Criminology, Law & Society, Cognitive Sciences, and School of Law, University
of California at Irvine.
This research was supported by a grant from the Fund for Dispute Resolution. We are grateful
to Patti Rognlin, Norm Thompson, Susan Rhodes, Steve Teller, and many other students for data
collection, and to Marijke Malsch, Steve Penrod, Nicole Doering, and Berenike Waubert de Puiseau
for helpful advice on the article.
Correspondence concerning this article should be addressed to Jane Goodman-Delahunty,
Australian Graduate School of Policing, Charles Sturt University, Manly Campus, P.O. Box 168,
Manly, New South Wales, 2088 Australia. E-mail: firstname.lastname@example.org
Psychology, Public Policy, and Law
2010, Vol. 16, No. 2, 133–157 © 2010 American Psychological Association
1076-8971/10/$12.00 DOI: 10.1037/a0019060
(English & Sales, 2005). From the moment when a client ﬁrst consults a lawyer
until the matter is resolved, lawyers must establish goals in a case and estimate the
likelihood that they can achieve these goals. The vast majority of lawyers
recognize that prospective judgments are integral features of their professional
expertise. For example, a survey of Dutch criminal lawyers acknowledged that
90% made predictions of this nature in some or all of their real-life cases (Malsch,
1990). The central question addressed in the present study was the degree of
accuracy in lawyers’ forecasts of case outcomes. To explore this question, we
contacted a broad national sample of U.S. lawyers who predicted their chances of
achieving their goals in real-life cases and provided conﬁdence ratings in their
Prediction of success is of paramount importance in the system for several
reasons. In the course of litigation, lawyers constantly make strategic decisions
and/or advise their clients on the basis of these predictions. Attorneys make
decisions about future courses of action, such as whether to take on a new client,
the value of a case, whether to advise the client to enter into settlement negoti-
ations, and whether to accept a settlement offer or proceed to trial. Thus, these
professional judgments by lawyers are inﬂuential in shaping the cases and the
mechanisms selected to resolve them. Clients’ choices and outcomes therefore
depend on the abilities of their counsel to make reasonably accurate forecasts
concerning case outcomes. For example, in civil cases, after depositions of key
witnesses or at the close of discovery, the parties reassess the likelihood of success
at trial in light of the impact of these events.
In summary, whether lawyers can accurately predict the outcome of a case has
practical consequences in at least three areas: (a) the lawyer’s professional
reputation and ﬁnancial success; (b) the satisfaction of the client; and (c) the
justice environment as a whole. Litigation is risky, time consuming, and expen-
sive. The consequences of judgmental errors by lawyers can be costly for lawyers
and their clients, as well as an unnecessary burden on an already overloaded
justice system. Ultimately, a lawyer’s repute is based on successful calculations of
case outcome. A lawyer who advises clients to pursue litigation without delivering
a successful outcome will not have clients for long. Likewise, a client will be most
satisﬁed with a lawyer who is accurate and realistic when detailing the potential
outcomes of the case. At the end of the day, it is the accurate predictions of the
lawyer that enable the justice system to function smoothly without the load of
cases that were not appropriately vetted by the lawyers.
Several lines of research support the proposition that lawyers’ assessments of
goals are central in the legal system. Some research singled out the attorney’s
estimate of the probability of success as the most crucial variable in shaping
decisions whether to litigate or settle a case in controversy (Schuck, 1986;
Wittman, 1988). An analysis of factors that inﬂuence assessments of outcomes in
contingent-fee cases revealed three perspectives of salience: that of the client; that
of an outside observer (e.g., a judge); and that of the lawyer, on the basis of past
experience and knowledge of how similar cases were resolved (Jay, 1989). The
lawyer’s viewpoint was regarded as the most important factor in shaping out-
comes in civil contingent-fee cases.
In civil litigation, both for plaintiffs who often retain counsel on a contingent-
fee basis and for defendants who typically do not, cost control and efﬁcient
134 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
representation may be achieved by resolving the case before trial. This can be
accomplished through negotiation, mediation, arbitration, or some other form of
alternative dispute resolution. In criminal cases, plea bargaining is routine
(Horowitz & Willging, 1984; Ross, 1970). A lawyer who cannot accurately
predict the outcome of a case or who does not thoroughly and efﬁciently
appreciate the litigation risks may ignore alternatives to trial and advise the client
to reject reasonable settlement offers. A lawyer who underestimates potential
outcomes may advise the client to accept an unreasonably lower amount in
settlement than is warranted.
The National Conference of Federal Trial Judges and the American Bar
Association conducted a survey to determine which factors were perceived by
lawyers as the most instrumental in facilitating settlement. The factor cited more
than twice as frequently as any other factor was the ability of the settlement judge
to objectively evaluate the strengths and weaknesses of a case (Brazil, 1985).
Although lawyers sometimes acknowledge that their ability to objectively assess
their cases may be suboptimal, settlement judges rely on lawyers to make these
determinations. Failure of lawyers to make proper predictions can impair settle-
ment and litigation performance (Harley, 2007).
Conﬁdence and Calibration in Lawyers’ Predictions
We contend that lawyers’ forecasts related to the achievement of important
goals play a pivotal role in practical legal decision making, but little is known
about the reliability of these predictions. To what extent are they realistic?
Psychological studies of human decision-making processes in a wide variety of
contexts have revealed that overconﬁdence is a ubiquitous phenomenon. Re-
cently, psychological researchers distinguished three forms of overconﬁdence:
overestimation, overplacement, and overprecision (Moore & Healy, 2008). Over-
estimation is the inﬂated perception of one’s ability, performance, or chance of
success. Overplacement describes the relative judgment of oneself in comparison
with others. Overprecision is “excessive certainty regarding the accuracy of one’s
beliefs” (Moore & Healy, 2008, p. 4).
Overconﬁdence has been observed in social judgments, self-predictions, and
professional predictions, in retrospective as well as prospective judgments (All-
wood & Granhag, 1999; Dunning, Grifﬁn, Milojkovic, & Ross, 1990; Lichten-
stein & Fischhoff, 1977; Paese & Feuer, 1991; Vallone, Grifﬁn, Lin, & Ross,
1990; Von Winterfeldt & Edwards, 1986). With regard to conﬁdence judgments
about achieving future goals, this calibration deﬁcit implies that decision makers
systematically overestimate their capacity to reach stated goals.
Research has identiﬁed a number of predictors of calibration (Dougherty,
2001; Gonza´lez-Vallejo & Bonham, 2007; Hogarth, 2001; Hogarth, Gibbs,
McKenzie, & Marquis, 1991). For instance, underconﬁdence or underestimation
is common when people estimate their performance on easy tasks or, in the case
of skilled individuals, on tasks on which they regard their chances of performing
well as high (Moore & Healy, 2008). Overestimation is more common on difﬁcult
135LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
Outcome feedback is an important factor in the development of well-
calibrated metacognitive judgments (Hogarth et al., 1991; Murphy & Winkler,
1971; Keren, 1987; Stone & Opel, 2000). To increase calibration accuracy,
one needs to learn whether previous predictions were correct. Although
lawyers might receive some outcome feedback, several factors bias their
judgments in the direction of overconﬁdence. First, particular professional
demands within the framework of the adversarial legal system may contribute
to this tendency: Lawyers have an ethical obligation to advocate zealously in
representing their clients. In this endeavor, lawyers must behave persuasively
and express conﬁdence in their position. Academics have noted, “lawyers need
to feel and display overconﬁdence in order to attract clients, and, later, to keep
those clients convinced that their interests are well served” (Loftus & Wa-
genaar, 1988, p. 450; see also Simon, 1988). These attitudinal demands for
high levels of conﬁdence might skew lawyers’ reasoning toward overconﬁ-
dence even when they attempt to realistically assess the likelihood of success.
Conﬁdence may also increase over time to justify commitment to a goal
(Brehm, 1956). Previous research revealed that expert predictions of the
likelihood of events were inﬂuenced by the person on whose behalf they were
working and that overconﬁdence was more prevalent when the expert had
some degree of commitment to, and emotional investment in, the outcome
(Rehm & Gadenne, 1990). More generally, lawyers may engage in wishful
thinking: They might believe a certain outcome is probable simply because of
a desire to reach it (Babad, Hills, & O’Driscoll, 1992; Kunda, 1990).
Another factor that might affect the realism of lawyers’ assessments of
future goals is perception of control (Dollinger, 2008; Langens, 2007). The
extent to which an individual believes he or she can take steps to increase the
likelihood of a desirable outcome has been shown to bias conﬁdence estimates
in those outcomes. When an event is perceived to be controllable, overcon-
ﬁdence is likely (Weinstein, 1980, 1982, 1983). This bias is linked to what
Langer (1975) called an illusion of control, deﬁned as “an expectancy of a
personal success probability inappropriately higher than the objective proba-
bility would warrant” (p. 313). In more recent studies, these results held only
in easy tasks, perhaps because the illusion of control is stronger. On perform-
ing hard tasks, people tend to perceive their performance as worse than it
really is compared with that of others (Moore & Cain, 2007). Lawyers may
underestimate or fail to appreciate the extent to which situational factors (e.g.,
unpredictable judges or juries) may rob them of the ability to control the
outcome (cf. Dunning et al., 1990).
Temporal circumstances may also inﬂuence the realism of judgments of
success. Predictions can be made long before the outcome is known or relatively
close to the outcome. The temporal sequence of predicting has been identiﬁed as
an important factor affecting conﬁdence estimates (Braun & Yaniv, 1992; Gilov-
ich, Kerr, & Husted Medvec, 1993; Hogarth & Makridakis, 1981; Rehm &
Gadenne, 1990). Events more distant in time are judged less probable than those
closer in time (Keren & Raaijmakers, 1988; Milburn, 1978; Rehm & Gadenne,
1990). Thus, lawyers’ conﬁdence levels about achieving their goals may increase
as the trial date approaches.
136 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
Empirical Research on Calibration in Lawyers’ Predictions
The ﬁndings on lawyers’ predictions in three published studies on this topic
reﬂect the general ﬁnding in judgment and decision-making research that people
display a tendency for overconﬁdence. In the ﬁrst study, criminal defense lawyers
attending legal education seminars stated a goal that they hoped to achieve on
behalf of a client and provided a conﬁdence estimate expressing the likelihood
that they would achieve this goal (e.g., a 75% chance of an acquittal). The results
showed evidence of overconﬁdence, which was replicated in a second sample of
civil defense lawyers (Loftus & Wagenaar, 1988).
A larger study of decision making by criminal lawyers working in the
context of an inquisitorial legal system in the Netherlands demonstrated that
lawyer conﬁdence was somewhat related to case outcomes: In that sample,
lawyers with high conﬁdence levels were more likely to be successful than
their counterparts whose conﬁdence estimates were lower, indicating some
predictive validity in their judgments (Malsch, 1990). The inﬂuence on
calibration of demographic variables (e.g., age, gender, years of practice,
specialization or area of expertise) was explored. Older lawyers (regardless of
years in practice) were better calibrated than younger colleagues. Female
lawyers were signiﬁcantly better calibrated than male lawyers, and specialized
lawyers were better calibrated than general practitioners. A link between type
of criminal case to metacognitive realism also emerged, in that lawyers were
more overconﬁdent in cases involving violent crimes against property and
drug-related offenses (compared with nonviolent crimes against property and
violent crimes against persons, such as assault).
Overview of the Present Research
The present research extends previous studies on realism in lawyers’ pro-
spective judgments by exploring predictions of outcome success in a large
national sample of U.S. lawyers. These lawyers represented parties in civil and
criminal cases (i.e., members of the plaintiff’s and the defense bar in civil cases
and prosecutors and defense counsel in criminal cases). We predicted that lawyers
would be prone to overconﬁdence, and we sought to further explore factors linked
to calibration in previous research (Gilovich et al., 1993; Malsch, 1989).
Assuming that we would observe more overconﬁdence than underconﬁdence,
we examined whether a cognitive intervention task would reduce overconﬁdence
and improve calibration. For example, social judgments have been corrected by
requiring participants to consider the opposite (Lord, Lepper, & Preston, 1984).
The technique we used involves encouraging participants to generate arguments
counter to their own predictions and has been used successfully in the past to
improve realism in conﬁdence judgments for general knowledge questions (Ko-
riat, Lichtenstein, & Fischhoff, 1980; Vreugdenhil, 1993). This study built on
previous research by investigating the extent to which this debiasing technique
was associated with more realistic levels of conﬁdence in the prospective judg-
ment of lawyers.
137LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
Participants were 481 litigating attorneys, including new graduates and seasoned
practitioners, from 44 states across the United States. Eligible participants were
identiﬁed by means of two sampling procedures. The majority (59%) were contacted
in person at continuing legal education (CLE) seminars. Data were collected at 19
different CLE seminars in diverse geographic regions. The remaining participants
(41%) were contacted by telephone on the basis of information sampled randomly
from local bar directories for states located in the eastern, western and midwestern
regions of the country. An introductory letter was mailed to 1,700 randomly selected
lawyers inviting them to participate in the study.
For several reasons, potential participants who were mailed a letter did not
participate. First, a group of 523 (30.8%) potential participants could not be
reached. In some cases, the letter was returned, indicating that the attorney was no
longer at the listed address; in other cases, the telephone interviewer was informed
that the attorney had moved without providing forwarding information. Some
attorneys were never available for an interview, despite ﬁve or more attempts to
reach them. This left a total of 1,177 potential participants contacted by mail.
A sizeable number of the 1,177 (272, or 23%) attorneys who were contacted
were ineligible either because their practice did not include trial work or because
they had no cases expected to come to trial within the study period. A third group
of attorneys declined to participate, either at the outset of the interview or at some
later point in the procedure. For example, 78 (6.6%) attorneys refused to disclose
their minimum goal or to express their conﬁdence in attaining their goal. In all,
859 lawyers completed the initial survey. Further attrition occurred because many
of the 859 lawyers who participated in the initial survey did not complete the
follow-up interview within the study period. A substantial number of their cases
(331, or 39%) were omitted from further analysis because the trial date in the case
selected was continued beyond the termination date for the study. A small number
of cases (46, or 5%) that were concluded within the study period were excluded
from further analysis for other reasons. In some of these cases, substitute counsel
had been appointed, so individual calibration measures were infeasible. In others,
the initially cooperative lawyers simply declined to continue participating in the
study. A handful of lawyers who changed business addresses between the time of
the initial interview and the time of the expected trial date were excluded, because
no forwarding address was available and they could not be located. Finally, a few
cases were excluded because the lawyers were unable to recall which case it was
that they had selected to discuss or because the lawyers were unable to report any
information about the case outcome. In all, follow-up interviews were completed
with 56% (481 of 859) of the lawyers who had completed the initial interview.
Survey Questionnaires and Procedures
Participants were screened for eligibility on the basis of whether they had a
case that was expected to go to trial within a speciﬁed time period (6 to 12
months). Two questionnaires were used: one for telephone interviews and one for
138 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
First, attorneys were asked to provide a code name (e.g., Explosion, Drug-
bust) for a case in litigation to help them identify that case at a later date and
enable them to protect client conﬁdentiality. Second, they were asked to provide
a brief description of the type of case, the party represented, and to answer the
following question: “What would be a win situation in terms of your minimum
goal for the outcome of this case?” Then, lawyers indicated their degree of
certainty that they would achieve the stated minimum goal, or something better,
by answering the following question: “From 0 to 100%, what is the probability
that you will achieve this outcome or something better?”
As mentioned, we included an experimental manipulation and randomly
assigned participants to one of two conditions. In the reasons condition, partici-
pants were asked to generate reasons why they might fail to achieve their
minimum goal before indicating how likely it was that this goal would be
achieved. In the no-reasons condition, participants generated reasons after the
However, lawyers in the no-reasons groups at CLEs were not invited to
consider reasons why they might not achieve their minimum goal. This procedure
was followed because when the survey questionnaires were distributed in person,
there was no way to control the order in which the participants elected to read
and/or respond to the written questions. For analysis, data from the telephone and
in-person reasons groups who generated reasons before estimating the probability
of success were collapsed to form a single “reasons” group. Similarly, data from
the telephone and in-person interview groups who did not produce reasons before
making their predictions were collapsed to form a single “no-reasons” group.
Call sheets were prepared for follow-up interviews scheduled after the date
the case was expected to come to trial. The follow-up interviews were conducted
by telephone with a structured questionnaire. A combination of open- and closed-
ended questions elicited factual information about how the case was resolved, the
negotiation process, and the ﬁnal case outcome or resolution. Other questions
sought subjective information, such as the attorneys’ perception of the outcome,
any changes the participants would make if they had the opportunity to retry the
case, and what factors affected the outcome.
Cases Included in the Study
We present data from 481 cases. More information was gathered about civil
cases (70% of the total sample) than criminal cases (30%), as shown in Table 1.
Lawyers in the Final Study Sample by Party Represented
Type of counsel Frequency %
Civil plaintiff 182 38
Civil defendant 155 32
Criminal prosecutor 79 16
Criminal defendant 65 14
Total 481 100
139LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
The 337 civil cases fell into several broad categories, the most common of which
were tort cases (personal injury/negligence, motor vehicle accidents, product
liability, malpractice, misrepresentation, etc.) constituting 57% of the civil cases.
Contract cases (construction, real estate, etc.) accounted for 19% of the civil
cases. Approximately 9% of the civil cases involved workplace disputes (discrim-
ination, worker’s compensation, etc.) and approximately 7% consisted of family
dissolutions. The balance of the civil cases (8%) involved miscellaneous contro-
versies over matters such as wills and trusts; copyright; trademark and patent law;
and bankruptcy, taxation, or administrative matters.
Of the 144 criminal cases studied, the most frequently encountered crimes
were those against persons (homicide, rape, robbery, assault and battery) com-
prising 64% of the criminal cases. Victimless crimes and crimes against property
accounted for the remaining 36% of the criminal cases studied (controlled
substances, misdemeanors, etc.).
Method of Case Resolution
Not surprisingly, cases were sometimes resolved by trial and sometimes by
settlement. The amount of time devoted to negotiation in civil cases exceeded that
in criminal cases by a factor of four. A high proportion of the lawyers (80%)
reported that settlement negotiations had taken place, and approximately three
ﬁfths of the cases (59%) were resolved by some form of out-of-court settlement.
Thirty-one percent of the cases were tried in court. An additional 10% of the cases
did not proceed to trial but were resolved through dismissal, summary judgment,
or some other legal procedural mechanism.
Responses to questions about lawyers’ minimum goals varied considerably
depending on the type of case under consideration and whether the attorney
represented a party initiating or defending against the claims. In civil cases, the
stated minimum goals were coded into two major categories. By far, the majority
of the lawyers (84%) were seeking monetary awards on behalf of their clients.
This category included cases in which the attorney represented a civil defendant
and the goal was a defense verdict. In 16% of the civil cases, the goals were
nonmonetary, such as injunctive relief, speciﬁc performance of contractual obli-
gations, a mistrial, or dismissal of the claims before trial.
In criminal cases, the goals of the lawyers were more diverse: Forty-nine
percent sought convictions of the crimes charged (the prosecution), and 25% of
the lawyers sought acquittals (the defense bar). A further 10% sought convictions
of lesser included offenses; 9% sought mistrials; and the remaining 7% sought
miscellaneous goals, such as an out-of-court resolution.
Conﬁdence in Achieving Minimum Goals
The majority of the participants expressed conﬁdence estimates that exceeded
50% (mean and median estimates of 64% and 70%, respectively). Reported case
outcomes were initially classiﬁed in one of three groups, depending on whether
the lawyers failed to meet their minimum goals, achieved their goals, or exceeded
their goals. In 32% of the cases, the ﬁnal case outcome matched the goal set by
140 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
the lawyers. Among the remaining cases, 24% of the outcomes exceeded the
lawyers’ minimum goals, and 44% of the outcomes were less satisfactory than
the minimum goals set by the lawyers. A number of lawyers who did not achieve
the predicted goal erred on the side of underconﬁdence, because their minimum
goals were too modest. However, a larger proportion of the prediction deﬁcits
came from lawyers who erred in the direction of overconﬁdence.
To investigate calibration (i.e., whether lawyers who provided high conﬁ-
dence estimates were more or less accurate in their judgment than lawyers who
provided low or moderate conﬁdence estimates), we analyzed the data within
different conﬁdence intervals. The following six conﬁdence intervals were used:
0–45%, 46–55%, 56–65%, 66–75%, 76–85% and 86–100%. The number of
participant lawyers producing conﬁdence estimates in each of the six conﬁdence
intervals is displayed in Figure 1.
As is noted in Figure 2, lawyers who provided a low conﬁdence estimate
(0–45%) underpredicted their chances of success. The mean estimated conﬁdence
of these 65 individuals was nearly 27%, but their success rate was signiﬁcantly
higher, close to 48%, t(64) ⫽3.31, p⬍.01. By comparison, lawyers whose initial
conﬁdence estimates were in the range of 46–55% and 56–65% tended to be
relatively well calibrated, t(96)⫽⫺.09, p⫽.92; and t(64) ⫽⫺1.15, p⫽.26,
For the three conﬁdence intervals above 65%, lawyers were biased in the
direction of overconﬁdence (see Figures 2): For conﬁdence intervals 66–75%,
t(118) ⫽⫺2.49, p⬍.05; for conﬁdence intervals 76–85%, t(79) ⫽⫺4.26, p⬍
.001; and for conﬁdence intervals 86–100%, t(54) ⫽⫺3.75, p⬍.001. In
summary, far more lawyers were susceptible to the overconﬁdence bias than to the
underconﬁdence bias. In general, the higher the expressed level of conﬁdence, the
greater the overconﬁdence.
Female Versus Male Attorneys
In the total survey sample, men outnumbered women by four to one (79% vs.
21%). The 361 male participants who reported years of experience were more
Figure 1. Number of participants with estimates of success in each of six conﬁ-
141LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
experienced on average (mean years of legal experience ⫽14.8, SD ⫽9.9) than
were the 90 female participants (mean years of legal experience ⫽7.1, SD ⫽4.8),
t(449) ⫽⫺7.13, p⬍.001. A comparison of initial conﬁdence estimates by gender
revealed no differences (mean conﬁdence was 65% for female lawyers and 64%
for male lawyers), t(479) ⫽0.56, p⫽.58.
As illustrated in Figure 3, consideration of the case outcomes showed a
different pattern for male versus female attorneys. Proportionately, although
Figure 2. Mean conﬁdence and achievement of minimum goal, by conﬁdence
intervals (percentage). Objective success reﬂects proportion meeting or exceeding
minimum goals within respective conﬁdence intervals. Asterisks denote signiﬁcant
differences between conﬁdence expressed and goal achievement.
Figure 3. Calibration curves for male lawyers (n⫽382) and female lawyers (n⫽99).
142 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
nonsigniﬁcantly, more female attorneys achieved their minimum goals than did
male attorneys (64% vs. 55%),
(1, N⫽481) ⫽2.40, p⫽.12.
Female attorneys whose conﬁdence estimates were below 65% had a nonsig-
niﬁcant tendency toward underconﬁdence, t(39) ⫽1.69, p⫽.10. Female lawyers
whose conﬁdence estimates ranged from 66% to 75% were well calibrated, as
indicated by the fact that the objective mean for achieved goals was 72%, t(28) ⫽
⫺0.14, p⫽89. Judgments above 75% by female lawyers were biased in the
direction of overconﬁdence, t(30) ⫽⫺2.44, p⬍.05. By comparison, male
lawyers were signiﬁcantly underconﬁdent for conﬁdence estimates below 45%,
t(49) ⫽3.18, p⬍.01; rather well calibrated for estimates between 46% and 55%,
t(79) ⫽⫺0.28, p⫽.78; and rather well calibrated for estimates between 56% and
65%, t(57) ⫽⫺1.43, p⫽.16. Judgments made by male attorneys whose
conﬁdence estimates exceeded 66% were systematically biased in the direc-
tion of overconﬁdence: For conﬁdence estimates between 66% and 75%,
t(89) ⫽⫺2.74, p⬍.01; for conﬁdence estimates between 75% and 85%,
t(61) ⫽⫺4.15, p⬍.001; and for conﬁdence estimates between 86% and
100%, t(41) ⫽⫺3.29, p⬍.01.
Thus, a major source of the gender differences was that female lawyers were
susceptible to the overconﬁdence bias only when the predicted likelihood of
success was high. Compared with their female colleagues, male lawyers were less
able to discriminate between cases in which they had a moderate probability of
success (65–75%) and those in which they had a high probability of success (over
Effects of Experience
The measure of experience used in this study was number of years in legal
practice. This number was used as a proxy for general expertise, because no
inquiries were made as to how many cases similar to the study case had the lawyer
handled previously or whether they were specialized in any area. Thus, if a lawyer
participating in the survey had several years of legal experience but selected as the
case for prediction one that was out of the ordinary in his or her practice, this
measure could be deceptive. On the other hand, expertise acquired in one
substantive legal domain may arguably transfer to another substantive domain.
The range of experience among lawyers included in the study was consider-
able, from 0 to 45 years of practice, with a mean of 13.3 years in practice (SD ⫽
9.7). Years of experience varied somewhat, depending on whether the responding
attorneys practiced civil or criminal law and the parties whom they represented.
For example, there was a clear difference in years of experience between lawyers
handling civil cases (M⫽14.9 years, SD ⫽9.9) and lawyers handling criminal
cases (M⫽8.8 years, SD ⫽7.4), t(449) ⫽6.23, p⬍.001. There were no
signiﬁcant differences between the years of experience of attorneys representing
civil plaintiffs (M⫽15.7 years, SD ⫽10.2) versus those representing civil
defendants (M⫽14.0 years, SD ⫽9.4), t(327) ⫽1.63, p⫽.10. Attorneys
prosecuting criminal cases (M⫽6.8 years, SD ⫽6.4) had signiﬁcantly less
experience than did criminal defense lawyers (M⫽11.1 years, SD ⫽8.0),
t(120) ⫽⫺3.33, p⬍.001.
143LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
If low conﬁdence or uncertainty about case outcomes were correlated with
lack of legal expertise, one might expect less experienced lawyers to express less
conﬁdence on average than the experienced lawyers. To investigate this hypoth-
esis, we categorized lawyers in two groups: one consisting of lawyers with 10 or
fewer years of legal practice (n⫽220) and one consisting of lawyers with more
than 10 years of legal practice (n⫽231). The mean estimated conﬁdence was
65% for those lawyers with 10 or fewer years of practice (SD ⫽20.5%) and 63%
for those with over 10 years of practice (SD ⫽20.3%), t(449) ⫽1.04, p⫽30. The
proportion achieving their goals did not differ between the groups (M⫽56.8% for
less experienced lawyers and M⫽55.4% for more experienced lawyers),
N⫽451) ⫽0.09, p⫽.76. The full calibration curves based on differential
experience are displayed in Figure 4.
It is interesting that, for conﬁdence levels above 60%, both lawyers with less
experience (M⫽76.5%, SD ⫽10.0%) and lawyers with more experience (M⫽
75.4%, SD ⫽10.7%) were clearly overconﬁdent: t(151) ⫽⫺3.86, p⬍.001, and
t(146) ⫽⫺4.46, p⬍.001, respectively. The pattern was replicated for high
conﬁdence estimates (over 80%): Less experienced lawyers (M⫽85.9%, SD ⫽
6.4%) were prone to an overconﬁdence bias, t(62) ⫽3.88, p⬍.001; as were more
experienced lawyers (M⫽86.0%, SD ⫽6.6%), t(57) ⫽⫺4.43, p⬍.001.
In conclusion, the data provided no support for the hypothesis that lawyers
with more practical experience are better calibrated than lawyers with less
experience. Years of practice did not count for much, at least as far as calibration
Figure 4. Calibration curves for lawyers with more than 10 years of experience
(n⫽231) and lawyers with 10 or fewer years of experience (n⫽220).
144 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
Civil Versus Criminal Cases
Data from lawyers handling 337 civil cases were examined separately to
determine whether those lawyers were better calibrated in predicting their success
than the overall sample. The mean conﬁdence estimate from this group was 65.0%
(SD ⫽18.0%), and the percentage that achieved their goals was 56.7%.
As a group, civil lawyers exhibited the same trend toward over- and under-
conﬁdence, as did the entire group of participating lawyers. For example, as can
be seen in Figure 5, lawyers who were highly conﬁdent in achieving the goals they
set (estimates exceeding 75%, mean estimate ⫽80.7%, SD ⫽6.6%), were
signiﬁcantly overconﬁdent as they achieved their goals in only 62.6% of the cases,
t(146) ⫽⫺4.52, p⬍.001.
Lawyers handling 144 criminal cases had a mean conﬁdence estimate of
62.6% (SD ⫽25.2%), and the percentage that achieved their goal was 56.9%.
There was no difference in conﬁdence estimates between civil and criminal
lawyers, t(479) ⫽1.16, p⫽.25. There was no difference in success rate between
civil and criminal lawyers,
(1, N⫽481) ⫽0.00, p⫽.96.
Conﬁdence as a Function of Temporal Distance to Trial
To determine whether lawyers’ conﬁdence levels differed as a function of the
distance in time to trial, we correlated the estimated probability of success with
the number of months elapsed between prediction and trial date. Contrary to our
expectations, the correlation was nonsigniﬁcant, r(427) ⫽.04, p⫽.40, indicating
no relation between conﬁdence levels and temporal distance to trial.
Figure 5. Calibration curves for civil cases (n⫽337) and criminal cases (n⫽144).
145LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
Calibration by Type of Case
Civil cases. Analyses of calibration by case type were possible to a limited
extent, depending on the number of cases of a similar nature included in the study.
Several categories of the tort cases in which defendant’s negligence was at issue
were aggregated into a general negligence case group (n⫽101). Approximately
50% of the lawyers responsible for these cases achieved their litigation goals,
although the mean conﬁdence estimate of the lawyers was 65%. Again, judgment
errors were concentrated on the upper end of the scale.
With respect to contract cases (n⫽65) the same trend toward underconﬁ-
dence at the lower end of the scale and overconﬁdence at the upper end of the
scale was present.
Criminal cases. Criminal cases were divided into two major categories:
crimes against property or victimless crimes (burglary, forgery, arson, possession
of controlled substances, DWIs, etc.) and crimes against persons (homicide,
assault, sexual offenses, etc.).
When crimes against property or victimless crimes were in issue, lawyers
were realistic in their estimates. Mean conﬁdence estimates were 59.1% (SD ⫽
29.3%), and the success rate was 47.9%, which did not differ signiﬁcantly,
t(47) ⫽⫺1.54, p⫽.13. When focusing on crimes against persons, the mean
conﬁdence estimate was 63.6% (SD ⫽23.1%). In this group, 61.1% achieved
their goal, t(89) ⫽⫺0.48, p⫽.63 suggesting general calibration.
Subjective Versus Objective Success and Attorney Reactions to
In the follow-up phase after the case resolved, lawyers described the case
outcome and whether they had achieved their minimum goal. They were not
explicitly reminded of their earlier stated goal. Occasionally they claimed they
had achieved their goal, even when an explicit comparison proved they had not.
To explore these mismatches, we devised two measures of success. Objective
success was measured by comparing the reported outcome of the case with the
originally stated goal. The measure was coded as a binary variable for which 1 ⫽
less satisfactory than the original goal (“a loss”) and 2 ⫽equal/better than
original goal (“a win”). A subjective success measure was obtained from the
lawyers’ self-reports as to whether they achieved their minimum goals. As is
reﬂected in Table 2, the objective success measure conﬁrmed the tendency of the
lawyers toward overconﬁdence. Lawyers’ estimated probability for success (i.e.,
their a priori estimate) was 64.3% (SD ⫽20.4%), but the objective success rate
was 56.8%, which was signiﬁcantly different, t(480) ⫽3.34, p⬍.001. The mean
subjective measure of success was 66.1%, and when conﬁdence estimates were
compared with this subjective measure of success, the overconﬁdence bias was
replaced by a tendency toward underconﬁdence, t(480) ⫽68.36, p⬍.001.
Furthermore, percentages of subjective success were signiﬁcantly higher in all
categories than percentages of objective success (see Table 2).
In the follow-up interviews, lawyers rated their feelings about the case
outcomes on a scale ranging from 1 to 5, on which 1 ⫽very disappointed,2⫽
somewhat disappointed,3⫽neutral,4⫽somewhat pleased, and 5 ⫽very
pleased. Results showed that approximately two thirds (64%) of all lawyers were
146 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
pleased or very pleased with the outcome of the case. It is interesting to note that
this number matched perfectly the estimated mean conﬁdence in achieving the
minimal goal (64%), although lawyers achieved their goals in only 57% of those
cases. Fewer than one ﬁfth of the lawyers (18%) were very disappointed or
somewhat disappointed with the case outcome, although more than twice as many
(43%) actually failed to achieve their stated minimum goal. Of the group of
lawyers who did not achieve their minimum goal (n⫽208), two thirds reported
that they were somewhat pleased or very pleased with the actual outcome.
Effects of the Debiasing Technique
We predicted that lawyers who generated reasons why they might not achieve
their litigation goals would be better calibrated. We expected to ﬁnd that gener-
ating these reasons would reduce the overconﬁdence observed in lawyers making
relatively high estimates of success. To examine this hypothesis, we compared the
212 lawyers who were asked for reasons before they provided their conﬁdence
estimate with the 269 lawyers who were not ﬁrst asked for reasons. Most lawyers
(57%, n⫽274) provided one or more reasons why they might not achieve their
litigation goals. The nature of the reasons varied widely. Some pointed to
contextual factors, such as an opponent with extensive trial experience or legal
skill, a client who was particularly unattractive, or the emotional nature of the case
Of the group who provided reasons, 41.6% provided a single reason; others
listed two or more reasons. Accordingly, the ﬁrst reason provided by lawyers was
coded and classiﬁed into one of six major categories. The largest category, “case
facts” (37.5%), included references to the weakness of the case, the emotional or
controversial nature of the case, and evidentiary problems. The next largest
category was “judge and jury issues” (24.8%) such as unpredictable fact ﬁnders.
The next distinct category was “witness factors” (14.2%) such as percipient or
expert witness problems. “Client factors” constituted a smaller group (12.5%) and
included references to unattractiveness and culpability. The next category, “court
context” (8%), comprised references to case precedent, jurisdiction, venue, court,
and geographic location. Surprisingly, the reasons least frequently cited were
Mean Prediction, Objective Success, and Subjective Success, by
Party Represented (%)
Variable All Civil
Mean estimate 64.3 65.1 65.1 72.8 50.1
Objective success 57.0 51.6 62.6 67.1 43.7
Subjective success 66.2
N481 182 155 79 66
Note. Superscripts denote signiﬁcant differences between objective and subjective suc-
(1, N⫽481) ⫽233.1.
(1, N⫽182) ⫽104.0.
(1, N⫽155) ⫽
(1, N⫽79) ⫽40.0.
(1, N⫽65) ⫽23.9.
147LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
“attorney factors,” comprising only 3.2% of the reasons stated and referring to the
skill, preparedness, or experience of the participant or opposing counsel.
Notably, 18 lawyers in the reasons group declined to generate even a single
reason why they might not achieve their minimum goal. For purposes of analysis,
we included these lawyers in the reasons group. Perhaps they contemplated some
reasons in response to our prompting. Nonetheless, no support emerged for the
hypothesis that generating reasons in advance boosted calibration. A comparison
of conﬁdence estimates in the reasons (M⫽64.3, SD ⫽19.6) and no-reasons
(M⫽64.3, SD ⫽21.1) groups revealed no signiﬁcant differences, t(479) ⫽
Figure 6 presents the calibration curves for the two groups of lawyers.
With one exception, the calibration curves for the reasons and no-reasons
lawyers are markedly similar. When initial conﬁdence estimates exceeded
60%, lawyers in both the reasons (M⫽75.7, SD ⫽10.2) and no-reasons (M⫽
76.6, SD ⫽10.7) groups both exhibited overconﬁdence, t(140) ⫽⫺3.89, p⬍
.001; and t(177) ⫽⫺4.34, p⬍.001, respectively. Calibration differed for one
conﬁdence interval; when initial conﬁdence estimates were low from the
beginning (under 45%). In this range, lawyers in the no-reasons group (M⫽
26.4, SD ⫽13.2) displayed a great deal of underconﬁdence, t(37) ⫽3.53, p⬍
.001. In contrast, lawyers in the reasons group (M⫽27.8, SD ⫽12.2) were
well calibrated, t(26) ⫽0.98, p⫽.34.
Overall, generating reasons did not accomplish what we predicted, namely
that overconﬁdence would be reduced.
Figure 6. Calibration curves for the reasons group (n⫽212) and the no-reasons
148 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
To our knowledge, this study of the ability of lawyers to predict the outcome
of their cases is the largest study of American legal practitioners of its kind
involving data drawn from actual cases. The ﬁndings extend previous research on
overconﬁdence in defense lawyers (Loftus & Wagenaar, 1988; Malsch, 1990), by
establishing that similar biases arise in predictions by criminal prosecutors and by
counsel for both plaintiffs and defendants in civil cases. Lawyers frequently made
substantial judgmental errors, showing a proclivity to overoptimism. The most
biased estimates were expressed with very high initial conﬁdence: In these
instances, lawyers were extremely overconﬁdent. These ﬁndings are consistent
with a large body of literature documenting overconﬁdence in a range of judg-
ments (theoretical explanations of miscalibration of conﬁdence are discussed in
Gigerenzer, Hoffrage, & Kleinbolting, 1991; Kahneman, Slovic, & Tversky,
1982; Moore & Healy, 2008).
With respect to the correlates of the overconﬁdence bias, certain results were
somewhat counterintuitive, such as the ﬁnding that lawyers with more experience
were not better calibrated than less experienced lawyers. By contrast, expertise
predicted calibration in Dutch criminal lawyers whose specialization was mea-
sured by the number of cases handled per year in a certain ﬁeld (Malsch, 1990).
Experience alone did not serve to calibrate either the Dutch or the North American
lawyers, although they plausibly received some form of contextual outcome
feedback that could have enhanced their calibration. Three potential explanations
to account for this ﬁnding are posited. First, more experienced lawyers may
handle cases of higher complexity and ambiguity than their less experienced
colleagues. Support for the notion that overconﬁdence is more extreme in tasks of
greater difﬁculty has emerged in several studies (Brenner, Koehler, Liberman, &
Tversky, 1996; Rehm & Gadenne, 1990). Second, a lawyer’s need for a highly
conﬁdent professional persona (Loftus & Wagenaar, 1988) may perpetuate a
tendency toward overconﬁdence over time rather than diminish it. Third, the
contextual effects of inquisitorial versus adversarial justice systems and the
differential feedback that lawyers receive over time may inﬂuence their predic-
Studies of forecasting in other professional domains showed that when the
ambiguity of the problem was reduced, forecasters relied on more objective and
fewer subjective factors in making their predictions (Braun & Yaniv, 1992).
Similarly, the ﬁnding that lawyers responsible for criminal cases involving vic-
timless crimes or crimes against property were better calibrated, compared with
lawyers responsible for cases involving crimes against persons, may be a function
of the relative ambiguity of the two types of cases.
With regard to gender, we replicated results obtained by Malsch (1990) that
female lawyers were better calibrated than their male colleagues. Male practitio-
ners were more overconﬁdent than female practitioners. These ﬁndings are in line
with gender differences observed in research on metacognition (Pallier, 2003).
Calibration did not increase as a function of time to trial. However, lawyers
selected cases that they expected to go to trial within 6–12 months. The variability
in temporal distance to trial may have been insufﬁcient to detect the predicted
inﬂuence of proximity to trial.
149LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
We applied two measures of success to determine the extent to which the
lawyers were well calibrated. One measure was the degree of correspondence
between the lawyers’ a priori goals and the reported actual case outcomes. This
was designated the objective measure of success. The subjective measure of
success was the lawyers’ self-report as to whether the a priori minimum goal was
achieved. The overconﬁdence effect disappeared and was replaced by undercon-
ﬁdence when comparing lawyers’ a priori estimates to their subjective reports of
success. This is likely due to the fact that lawyers’ own perception of success after
the case was resolved was signiﬁcantly higher than the objective measure of
success. The same self-serving bias (e.g., Babcock & Loewenstein, 1997) that
causes overconﬁdence in prospective judgments may also skew retrospective
judgments of success.
We failed to ﬁnd support for the prediction that a cognitive manipulation
(asking participants to generate reasons against achieving their minimum goal)
was related to more realistic estimates. This might be due to the anchoring and
adjustment heuristic (Tversky & Kahneman, 1974): Lawyers choose a desirable
outcome, the anchor, and thereafter make insufﬁcient adjustments for uncertainty
even when asked to generate reasons against their initial goal. Results in other
studies have also failed to show an inﬂuence on calibration of arguments gener-
ated against the chosen alternative (Allwood & Granhag, 1996) or a reduction of
unwarranted optimism in personal predictions by using pessimistic-scenario gen-
eration (Newby-Clark, Ross, Buehler, Koehler, & Grifﬁn, 2000).
Lawyers are exposed to several additional factors that may inﬂuence the
predictive validity of the speciﬁc information they have regarding an upcoming
case. Compared with professional weather forecasters, lawyers are different in
two crucial respects. First, meteorologists cannot in any way inﬂuence the
outcome of their predictions. Nothing they do can make it rain. Lawyers, on the
other hand, can behave in ways that inﬂuence the case outcome. Because they
have this opportunity, they may overestimate their own capacity and neglect the
importance of factors beyond their control (Dollinger, 2008).
Second, lawyers have a much keener interest in the goals of their predictions than
do meteorologists. Because of this, lawyers might be susceptible to overoptimism and
wishful thinking. In several studies, researchers have demonstrated that merely
imagining an event can increase subjective probability estimates that the event will
occur (Carroll, 1978; Sherman, Cialdini, Schwartzman, & Reynolds, 1985). Thus,
even before the questions about conﬁdence estimates and the reasons why lawyers
might not achieve their goals were posed, lawyers might have been biased in the
direction of overoptimism. This explanation is consistent with ﬁndings in previous
studies in which judgments about probability and likelihood were signiﬁcantly
inﬂuenced by whatever side of the issue individuals thought about ﬁrst (Hoch,
1984). Generation of a particular mental scenario may have hindered the later
generation of alternative, incompatible scenarios. People can enjoy the experience
of wishful thinking “as long as they are willing to pay the price of painful
disappointment when reality does not unfold as expected” (Babad, Hill, &
150 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
Driscoll, 1992, p. 471). In the context of litigation, we agree with Grifﬁn and
It can be argued that people’s willingness to engage in military, legal, and other
costly battles would be reduced if they had a more realistic assessment of their
chances of success. We doubt that the beneﬁts of overconﬁdence outweigh its
costs. (p. 433)
How can metacognitive realism in lawyers be improved? We failed to ﬁnd
improved calibration as a function of a cognitive debiasing intervention. Perhaps
unrealistically optimistic predictions are too closely intertwined with the profes-
sional motivation of lawyers to be overcome by a cognitive intervention. These
lawyers may also be susceptible to a general human tendency to downplay
unfavorable aspects about oneself (Fiske & Taylor, 2008) and a tendency to
interpret the past in an unrealistically favorable light (Gilbert, 2006). A lawyer’s
commitment to the client and the case may induce a self-serving bias. A self-
serving bias is the expectation of one party that exceeds the beliefs of the
opposing party (Babcock, Loewenstein, & Issacharoff, 1997). Self-serving biases
in litigation were strong predictors of nonsettlement before proceeding to court
(Loewenstein, Issacharoff, Camerer, & Babcock, 1993). In previous research on
self-serving biases, encouraging participants to consider the weakness in their
case was effective in debiasing participants (e.g., Babcock & Loewenstein, 1997).
The debiasing manipulations in these studies were similar (generating reasons
why the goal might not be achieved versus thinking about the weaknesses in one’s
own case). The discrepancy between our ﬁnding and the previous ﬁnding may be
due to the fact that the participants in our study made judgments about their own
cases rather than simulated cases assigned to them in an experimental setting. A
more profound investment in the outcome in real-life cases may increase resis-
tance to debiasing interventions.
The results of this study have practice and policy implications for lawyer–
client relationships, case management strategies, court efﬁciency, and lawyer
training and education. These policy issues can be addressed both formally and
Lawyer–client relations. Clients of lawyers who are susceptible to an
overconﬁdence bias may detrimentally rely on the advice of these legal profes-
sionals when making decisions about whether to litigate, they may choose
litigation over settlement, or they may allocate valuable resources without secur-
ing anticipated objectives. These clients are likely to experience disappointment,
frustration, or anger if the outcomes they were led to expect are not achieved.
Client dissatisfaction may increase public antipathy toward lawyers and diminish
public conﬁdence in the legal system. Lawyers who have had several experiences
with dissatisﬁed clients are likely to adopt a risk-averse strategy in managing
client relationships to avoid a negative outcome by deliberately lowering all client
expectations so that subsequent disappointment is avoided. Lawyers who are
better calibrated can avoid dissatisﬁed clients by obviating the need to reduce
client expectations to preserve their relationship. Lawyer–client relationships,
151LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
client satisfaction, general public opinions of lawyers, and conﬁdence in the legal
system may all be enhanced by well-calibrated lawyers. Clients are not well
served if their legal representatives cannot appropriately assess litigation risks or
if lawyers exercise poor judgment in advising them or deliberately lower all
Case management strategies. One implication of the present ﬁndings is
that lawyer performance can be improved by implementing case management
strategies that take into account the potential overconﬁdence biases of the litiga-
tors. Case consultations with legal peers can take place informally. For example,
in many legal ﬁrms, regular meetings are held where cases are periodically
reviewed so that the partners can manage the caseload efﬁciently and ethically.
These meetings provide ideal opportunities to obtain objective opinions from
other legal professionals in the form of third-party feedback about the strengths
and weaknesses of a case and the likelihood that the stated goals can be achieved.
Many of the most overconﬁdent lawyers will be the senior partners who may not
typically obtain third-party review or feedback in the course of their practice. Law
ﬁrms should take afﬁrmative steps to incorporate third-party feedback for the
more experienced or senior litigators in their case management systems. Legal
practitioners should also consider regularly obtaining customer feedback by
sending their clients anonymous survey questionnaires at the close of every case;
these should include questions that target the issues surrounding the management
of client expectations about the achievement of goals in a particular case.
In addition, lawyers should consider obtaining more formal external third-
party views on the likelihood of achieving their litigation goals on behalf of their
clients. It is in the early stages of litigation that patterns and expectations for the
case are established and where an intervention can have the most beneﬁcial
effects. For example, in an evaluation of the Northern California Early Neutral
Evaluation (ENE) Program (Brazil, Kahn, Newman, & Gold, 1986), attorneys
reported changing their initial assessments of the merits of their case on the basis
of a better understanding of the case’s legal and factual issues gained during the
ENE. Furthermore, participants reported greater satisfaction with the outcome of
the case and perceptions of procedural fairness regardless of the eventual reso-
lution. However, these successful ENE programs have not been widely imple-
mented (Levine, 1987; Wissler, 2002). Alternatively, lawyers can seek the input
of an experienced third-party neutral, such as an alternative dispute resolution
practitioner, mediator, or arbitrator. These professionals are often consulted
jointly by the parties in mediation, but they can also be consulted independent of
the opposing party speciﬁcally to obtain evaluative feedback of this nature.
Court efﬁciency. Overconﬁdence in difﬁcult or ambiguous cases could be
a barrier to alternative forms of dispute resolution; lawyers may pursue litigation
in cases even though the litigation objective cannot be achieved. This practice
may clog the courts with cases that lack merit and with unnecessary or protracted
litigation in cases that might be better suited to dismissal or settlement. To address
this issue, courts should consider requiring parties to appear before a third-party
neutral, such as a magistrate, judge, or mediator, for an assessment of the merits
of the case. A party who does not have an interest in a particular outcome might
be less prone to metacognitive biases that color the litigator’s decision making. In
many areas of law, trial may proceed only if mediation has been attempted and
152 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
failed. The advantage of policies of this nature is that the mediation may provide
third-party neutral feedback to the lawyers that reduces overoptimism. Thus, these
policies should be further supported. Other policies in the form of interventions
that may be effective in inhibiting unnecessary litigation should be considered
(e.g., more early neutral evaluation options and expanded use of pretrial confer-
ences in criminal as well as civil cases).
Lawyer training and education. Finally, the ﬁndings that many lawyers
are susceptible to an overconﬁdence bias have implications for legal training and
education. Information about the outcomes of this and similar studies should be
included in the relevant curricula in clinical practice courses and in professional
and ethical responsibility classes both at law school and bar admission. Better
informed lawyers will be more likely to seek third-party feedback to avoid
The key in all policy recommendations is the provision of feedback to lawyers
about the link between their professional judgment and their performance. Out-
come feedback is central to developing well-calibrated judgments (Hogarth et al.,
1991; Keren, 1987; Murphy & Winkler, 1971; Stone & Opel, 2000). To enhance
calibration precision, one needs to determine the accuracy of previous predictions.
Lawyers should be encouraged to monitor their predictions in multiple ways. Case
outcomes in the form of judicial opinions or jury verdicts are dependent on many
factors outside the control of the lawyer and provide only limited information
(win/lose); these are, thus, very blunt instruments to rely on for enhancing
calibration. More sensitive, ﬁne-grained feedback may be available from oppos-
ing counsel, judges, or an analysis of audio-visual recordings. Several avenues to
help lawyers better calibrate their metacognitive judgments are advised. Policies
requiring lawyers to consult external third parties to obtain more realistic evalu-
ations of cases and their likely outcomes may beneﬁt clients, lawyers, the justice
system, and the public interest.
Future research should examine factors that enhance lawyer calibration, such
as case-related aspects and performance feedback. If case ambiguity and case
complexity are controlled, a lawyer’s experience may prove to be a better
predictor of calibration. To date, no research has thoroughly examined this issue.
Similarly, the extent to which input from an external party improves lawyers’
realism in making case predictions and achieving their objectives needs further
investigation. However, the literature on motivated social cognition implies that
the motivation to achieve a certain case outcome may lead lawyers to discount the
assessment of the external party if it does not ﬁt with their preferred belief
This study extended previous research on the judgment, decision making, and
metacognitive realism of legal professionals. Our large sample of U.S. lawyers
showed clear evidence of unrealistic litigation goals, and our experimental ma-
nipulation with the purpose of debiasing optimistic tendencies did not have the
desired effect. Because accurate legal decision making might be in jeopardy as a
function of lawyers’ optimistic tendencies, further research ought to explore ways
to enhance calibration to reach more realistic metacognitive judgments.
153LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
Allwood, C. M., & Granhag, P. A. (1996). The effects of arguments on realism in
conﬁdence judgments. Acta Psychologica, 91, 99–119.
Allwood, C. M., & Granhag, P. A. (1999). Feelings of conﬁdence and the realism of
conﬁdence judgments in everyday life. In P. Juslin & H. Montgomery (Eds.),
Judgment and decision making: Neo-Brunswikian and process-tracing approaches
(pp. 123–146). Mahwah, NJ: Erlbaum.
Babad, E., Hills, M., & O’Driscoll, M. (1992). Factors inﬂuencing wishful thinking and
predictions of election outcomes. Basic and Applied Social Psychology, 13, 461–476.
Babcock, L., Loewenstein, G., & Issacharoff, S. (1997). Creating convergence: Debiasing
biased litigants. Law & Social Inquiry, 22, 913–925.
Babcock, L., & Loewenstein, G. (1997). Explaining bargaining impasse: The role of
self-serving biases. Journal of Economic Perspectives, 11, 109–126.
Braun, P. A., & Yaniv, L. (1992). A case study of expert judgment: Economists’
probabilities versus base-rate model forecasts. Journal of Behavioral Decision Mak-
ing, 5, 217–231.
Brazil, W. D. (1985, December). What lawyers want from judges in the settlement arena.
Federal Rules, 85–101.
Brazil, W. D., Kahn, M. A., Newman, J. P., & Gold, J. Z. (1986). Early neutral evaluation:
An experimental effort to expedite dispute resolution, Judicature, 69, 279–285.
Brehm, J. W. (1956). Post-decision changes in desirability of alternatives. Journal of
Abnormal and Social Psychology, 52, 348–359.
Brenner, L. A., Koehler, D. J., Liberman, V., & Tversky, A. (1996). Overconﬁdence in
probability and frequency judgments: A critical examination. Organizational Behav-
ior and Human Decision Processes, 65, 212–219.
Carroll, J. S. (1978). The effect of imagining an event on expectations for the event: An
interpretation in terms of the availability heuristic. Journal of Experimental Psychol-
ogy, 14, 88–96.
Dollinger, S. (2008). The illusion of control. In L. T. Benjamin, Jr. (Ed.), Favorite
activities for the teaching of psychology (pp. 201–202). Washington, DC: American
Dougherty, M. R. P. (2001). Integration of the ecological and error models of overcon-
ﬁdence using a multiple trace memory model. Journal of Experimental Psychology:
General, 130, 579–599.
Dunning, D., Grifﬁn, D. W., Milojkovic, J. D., & Ross, L. (1990). The overconﬁdence
effect in social predictions. Journal of Personality and Social Psychology, 58,
English, P., & Sales, B. (2005). More than the law: Behavioral and social facts in legal
decision making. Washington, DC: American Psychological Association.
Fiske, S. T., & Taylor, S. E. (2008). Social cognition, from brains to culture. Columbus,
OH: McGraw-Hill Higher Education.
Flavell, J. (2004). Theory-of-mind development: Retrospect and prospect. Merrill-Palmer
Quarterly, 50, 274–290.
Gigerenzer, G., Hoffrage, U., & Kleinbolting, H. (1991). Probabilistic mental models: A
Brunswikian theory of conﬁdence. Psychological Review, 98, 506–528.
Gilbert, D. (2006). Stumbling on happiness. New York: Alfred A. Knopf.
Gilovich, T., Kerr, M., & Husted Medvec, V. (1993). Effect of temporal perspective on
subjective conﬁdence. Journal of Personality and Social Psychology, 4, 552–560.
Gollwitzer, P., & Schaal, B. (1998). Metacognition in action: The importance of imple-
mentation intentions. Personality and Social Psychology Review, 2, 124–136.
154 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
Gonza´lez-Vallejo, C., & Bonham, A. (2007). Aligning conﬁdence with accuracy: Revis-
iting the role of feedback. Acta Psychologica, 125, 221–239.
Grifﬁn, D., & Tversky, A. (1992). The weighing of evidence and the determinants of
conﬁdence. Cognitive Psychology, 24, 411–435.
Harley, E. (2007). Hindsight bias in legal decision making. Social Cognition, 25, 48–63.
Hoch, J. S. (1984). Availability and interference in predictive judgment. Journal of
Experimental Psychology: Learning, Memory, & Cognition, 10, 649–662.
Hogarth, R. M. (2001). Educating intuition. Chicago: The University of Chicago Press.
Hogarth, R. M., Gibbs, B., McKenzie, C., & Marquis, M. (1991). Learning from feedback:
Exactingness and incentives. Journal of Experimental Psychology: Learning, Mem-
ory, and Cognition, 17, 734–752.
Hogarth, R. M., & Makridakis, S. (1981). Forecasting and planning: An evaluation.
Management Science, 27, 115–138.
Horowitz, I. A., & Willging, T. E. (1984). The psychology of law: Integrations and
applications. Boston: Little, Brown.
Jay, S. (1989). The dilemmas of attorney contingent fees. The Georgetown Journal of
Legal Ethics, 2, 813–884.
Kahneman, D. (1973). On the psychology of prediction. Psychological Review, 80,
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgments under uncertainty: Heuristics
and biases. Cambridge, United Kingdom: Cambridge University Press.
Keren, G. (1987). Facing uncertainty in the game of bridge: A calibration study. Orga-
nizational Behaviour and Human Decision Processes, 39, 98–114.
Keren, G., & Raaijmakers, J. G. W. (1988). On between-subjects versus within-subjects
comparisons in testing utility theory. Organizational Behavior and Human Decision
Processes, 41, 233–247.
Koriat, A. (2002). Metacognition research: An interim report. Applied metacognition (pp.
261–286). New York: Cambridge University Press.
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for conﬁdence. Journal of
Experimental Psychology: Human Learning and Memory, 6, 107–118.
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480–
Langens, T. (2007). Regulatory focus and illusions of control. Personality and Social-
Psychology Bulletin, 33, 226–237.
Langer, E. J. (1975). The illusion of control. Journal of Personality and Social psychol-
ogy, 32, 311–328.
Levine, D. (1987). Early neutral evaluation: A follow-up report. Judicature, 70, 236–240.
Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about
how much they know? Organizational Behavior and Human Performance, 20,
Loewenstein, G., Issacharoff, A., Camerer, C., & Babcock, L. (1993). Self-serving
assessments of fairness and pretrial bargaining. The Journal of Legal Studies, 22,
Loftus, E. F., & Wagenaar, W. A. (1988). Lawyers’ predictions of success. Jurimetrics
Journal, 28, 437–453.
Lord, C. G., Lepper, M. R., & Preston, E. (1984). Considering the opposite: A corrective
strategy for social judgment. Journal and Personality and Social Psychology, 47,
Malsch, M. (1989). Voorspellingen van rechterlijke beslissingen door advocaten. Neder-
landse samenvatting van “Lawyers’ predictions of judicial decisions: A study on
calibration of experts.” Unpublished doctoral dissertation, Rijksuniversiteit Leiden,
Leiden, the Netherlands.
155LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES
Malsch, M. (1990). Advocaten voorspellen de uitkomst van hun zaken [Lawyers’ predic-
tions of judicial decisions: A study on calibration of experts]. Lisse, the Netherlands:
Swets & Zeitlinger.
Meeks, J., Hicks, J., & Marsh, R. (2007). Metacognitive awareness of event-based
prospective memory. Consciousness and Cognition: An International Journal, 16,
Milburn, M. A. (1978). Sources of bias in the prediction of future events. Organizational
Behavior and Human Performance, 21, 17–26.
Moore, D., & Healy, P. (2008). The trouble with overconﬁdence. Psychological Review,
Moore, D. A., & Cain, D. M. (2007). Overconﬁdence and unconﬁdence: When and why
people underestimate (and overestimate) the competition. Organizational Behavior
and Human Decision Processes, 103, 197–213.
Murphy, A. H., & Winkler, R. L. (1971). Forecasters and probability forecasts: Some
current problems. Bulletin of American Meteorological Society, 52, 239–247.
Newby-Clark, I. R., Ross, M., Buehler, R., Koehler, D. J., & Grifﬁn, D. (2000). People
focus on optimistic scenarios and disregard pessimistic scenarios while predicting
task completion times. Journal of Experimental Psychology: Applied, 3, 171–182.
Paese, P. W., & Feuer, M. A. (1991). Decisions, actions, and the appropriateness of
conﬁdence in knowledge. Journal of Behavioral Decision Making, 4, 1–16.
Pallier, G. (2003). Gender differences in the self-assessment of accuracy on cognitive
tasks. Sex Roles, 48, 265–276.
Perfect, T., & Schwartz, B. (2002). Applied metacognition. New York: Cambridge
Rehm, J. T., & Gadenne, V. (1990). Intuitive predictions and professional forecasts:
Cognitive processes and social consequences. Oxford, England: Pergamon Press.
Ross, H. L. (1970). Settled out of court. Chicago: Aldine.
Schuck, P. H. (1986). The role of judges in settling complex cases: The Agent Orange
example. The University of Chicago Law Review, 53, 337–365.
Sherman, J. S., Cialdini, R. B., Schwartzman, D. F., & Reynolds, K. D. (1985). Imagining
an event can heighten or lower the perceived likelihood of contracting a disease: The
mediating effect of ease of imagery. Personality and Social Psychology Bulletin, 11,
Simon, W. H. (1988). Ethical discretion in lawyering. Harvard Law Review, 101,
Stone, E. R., & Opel, R. B. (2000). Training to improve calibration and discrimination:
The effects of performance and environmental feedback. Organizational Behavior
and Human Decision Processes, 83(2), 282–309.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases.
Science, 185, 1124–1131.
Vallone, R. P., Grifﬁn, D. W., Lin, S., & Ross, L. (1990). Overconﬁdent predictions of
future actions and outcomes by self and others. Journal of Personality and Social
Psychology, 58, 582–592.
Von Winterfeldt, D., & Edwards, W. (1986). Egocentrism as a source of unrealistic
optimism. Journal of Personality and Social Psychology, 53, 421–430.
Vreugdenhil, H. (1993). Conﬁdence in knowledge: Studies on the calibration of subjective
probabilities. Unpublished doctoral dissertation, University of Amsterdam, Amster-
dam, the Netherlands.
Weinstein, N. D. (1980). Unrealistic optimism about future life events. Journal of
Personality and Social Psychology, 39, 806–820.
Weinstein, N. D. (1982). Unrealistic optimism about susceptibility for health problems.
Journal of Behavioral Medicine, 5, 441–460.
156 GOODMAN-DELAHUNTY, GRANHAG, HARTWIG, AND LOFTUS
Weinstein, N. D. (1983). Reducing unrealistic optimism about illness susceptibility.
Health Psychology, 2, 11–20.
Wissler, R. L. (2002). When does familiarity breed content? A study of the role of
different forms of ADR education and experience in attorneys’ ADR recommenda-
tions. Pepperdine Dispute Resolution Law Journal, 2, 199–217.
Wittman, D. (1988). Dispute resolution, bargaining, and the selection of cases for trial: A
study of biased and unbiased data. The Journal of Legal Studies, XVII, 313–352.
Received September 14, 2009
Revision received November 23, 2009
Accepted November 30, 2009 䡲
157LAWYERS’ ABILITY TO PREDICT CASE OUTCOMES