ArticlePDF Available

Performing a Project Premortem

Authors:
  • ShadowBox LLC & MacroCognition LLC

Abstract

This publication contains reprint articles for which IEEE does not hold copyright. You may purchase this article from the Ask*IEEE Document Delivery Service at http://www.ieee.org/services/askieee/
www.hbrreprints.org
F
ORETHOUGHT
G
RIST
Performing a Project
Pre
mortem
by Gary Klein
Reprint F0709A
This document is authorized for use only by Michael Davis (MDavis@DrillScience.com). Copying or posting is an infringement of copyright. Please contact
customerservice@harvardbusiness.org or 800-988-0886 for additional copies.
harvard business review • september 2007 page 1
COPYRIGHT © 2007 HARVARD BUSINESS SCHOOL PUBLISHING CORPORATION. ALL RIGHTS RESERVED.
F
ORETHOUGHT
G
RIST
Performing a Project
Pre
mortem
by Gary Klein
Projects fail at a spectacular rate. One reason
is that too many people are reluctant to speak
up about their reservations during the all-
important planning phase. By making it safe
for dissenters who are knowledgeable about
the undertaking and worried about its weak-
nesses to speak up, you can improve a project’s
chances of success.
Research conducted in 1989 by Deborah J.
Mitchell, of the Wharton School; Jay Russo,
of Cornell; and Nancy Pennington, of the
University of Colorado, found that prospec-
tive hindsight—imagining that an event has
already occurred—increases the ability to
correctly identify reasons for future outcomes
by 30%. We have used prospective hind-
sight to devise a method called a premortem,
which helps project teams identify risks at
the outset.
A premortem is the hypothetical opposite of
a postmortem. A postmortem in a medical set-
ting allows health professionals and the family
to learn what caused a patient’s death. Every-
one benefits except, of course, the patient. A
premortem in a business setting comes at the
beginning of a project rather than the end, so
that the project can be improved rather than
autopsied. Unlike a typical critiquing session,
in which project team members are asked
what might go wrong, the premortem operates
on the assumption that the “patient” has died,
and so asks what did go wrong. The team
members’ task is to generate plausible reasons
for the project’s failure.
A typical premortem begins after the team
has been briefed on the plan. The leader starts
the exercise by informing everyone that the
project has failed spectacularly. Over the next
few minutes those in the room independently
write down every reason they can think of for
the failure—especially the kinds of things they
ordinarily wouldn’t mention as potential prob-
lems, for fear of being impolitic. For example,
in a session held at one Fortune 50–size com-
pany, an executive suggested that a billion-
dollar environmental sustainability project had
“failed” because interest waned when the CEO
retired. Another pinned the failure on a dilu-
tion of the business case after a government
agency revised its policies.
Next the leader asks each team member,
starting with the project manager, to read one
reason from his or her list; everyone states a
different reason until all have been recorded.
After the session is over, the project manager
reviews the list, looking for ways to strengthen
the plan.
In a session regarding a project to make
state-of-the-art computer algorithms available
to military air-campaign planners, a team
member who had been silent during the previ-
ous lengthy kickoff meeting volunteered that
one of the algorithms wouldn’t easily fit on cer-
tain laptop computers being used in the field.
Accordingly, the software would take hours to
run when users needed quick results. Unless
the team could find a workaround, he argued,
the project was impractical. It turned out that
the algorithm developers had already cre-
ated a powerful shortcut, which they had
been reluctant to mention. Their shortcut was
substituted, and the project went on to be
highly successful.
In a session assessing a research project in a
different organization, a senior executive sug-
gested that the project’s “failure” occurred
because there had been insufficient time to
prepare a business case prior to an upcoming
corporate review of product initiatives. During
the entire 90-minute kickoff meeting, no one
had even mentioned any time constraints. The
project manager quickly revised the plan to
take the corporate decision cycle into account.
Although many project teams engage in
prelaunch risk analysis, the premortem’s pro-
spective hindsight approach offers benefits
that other methods don’t. Indeed, the premor-
tem doesn’t just help teams to identify poten-
tial problems early on. It also reduces the kind
of damn-the-torpedoes attitude often assumed
by people who are overinvested in a project.
Moreover, in describing weaknesses that no
one else has mentioned, team members feel
valued for their intelligence and experience,
This document is authorized for use only by Michael Davis (MDavis@DrillScience.com). Copying or posting is an infringement of copyright. Please contact
customerservice@harvardbusiness.org or 800-988-0886 for additional copies.
Performing a Project Premortem
F
ORETHOUGHT
G
RIST
harvard business review • september 2007 page 2
and others learn from them. The exercise also
sensitizes the team to pick up early signs of
trouble once the project gets under way. In the
end, a premortem may be the best way to cir-
cumvent any need for a painful postmortem.
Gary Klein
(gary@decisionmaking.com) is the chief sci-
entist of Klein Associates, a division of Applied Research
Associates, in Fairborn, Ohio. He is the author of Sources
of Power: How People Make Decisions (MIT Press, 1998)
and The Power of Intuition (Doubleday, 2004).
Reprint F0709A
To order, see the next page
or call 800-988-0886 or 617-783-7500
or go to www.hbrreprints.org
This document is authorized for use only by Michael Davis (MDavis@DrillScience.com). Copying or posting is an infringement of copyright. Please contact
customerservice@harvardbusiness.org or 800-988-0886 for additional copies.
Further Reading
To Order
For
Harvard Business Review
reprints and
subscriptions, call 800-988-0886 or
617-783-7500. Go to www.hbrreprints.org
For customized and quantity orders of
Harvard Business Review article reprints,
call 617-783-7626, or e-mai
customizations@hbsp.harvard.edu
page 3
The
Harvard Business Review
Paperback Series
Here are the landmark ideas—both
contemporary and classic—that have
established Harvard Business Review as required
reading for businesspeople around the globe.
Each paperback includes eight of the leading
articles on a particular business topic. The
series includes over thirty titles, including the
following best-sellers:
Harvard Business Review on Brand
Management
Product no. 1445
Harvard Business Review on Change
Product no. 8842
Harvard Business Review on Leadership
Product no. 8834
Harvard Business Review on Managing
People
Product no. 9075
Harvard Business Review on Measuring
Corporate Performance
Product no. 8826
For a complete list of the
Harvard Business
Review paperback series, go to
www.hbrreprints.org.
This document is authorized for use only by Michael Davis (MDavis@DrillScience.com). Copying or posting is an infringement of copyright. Please contact
customerservice@harvardbusiness.org or 800-988-0886 for additional copies.
... reasons for that consequence (Klein, 2007). The exercise ensures that the collective intelligence of all participants is brought to the fore, gathering insights and perspectives beyond those of the project owner or leader. ...
... On the other hand, a Blue Team, composed of military personnel, has the mission to keep those scenarios plausible and actionable for military purposes. In late June 2022, the second series of scenarios was shared with the public. ...
... Furthermore, the creation of this community may allow for opportunities to explore and establish joint initiatives and shared understandings, thus supporting the co-ordination of the strategic foresight interventions for the Government of Portugal. During the first collaborative session that OPSI organised with PlanAPP on the 28 th of June 2022, PlanAPP announced the intention to create a Working Group for the Foresight Community. On 21 October 2022 this working group met for the first time, gathering 27 public sector organisations, and highlighted the importance and relevance of cross-sectoral collaboration regarding the use of strategic foresight. ...
Technical Report
Full-text available
This working paper discusses strategic foresight initiatives and methodologies that support decision making and process design. It highlights case studies, international benchmarks, and best practices as well as methodological recommendations and options to promote the adoption and use of strategic foresight in government. The paper is structured in four sections, each centred on a critical action to improve decision making through strategic foresight: (i) framing strategic foresight, (ii) building its fundamental components in governments, (iii) fine-tuning foresight interventions to specific contexts, and (iv) doing concrete activities to solve specific policy challenges. Given its exploratory nature, this working paper and its proposals must be seen as contributions to the ongoing debates about the use of strategic foresight for decision making in government. The ultimate purpose of this paper is to help governments become more proactive and prospective.
... Some tools for HMT and related fields have focused explicitly on their usability and usefulness for systems development. Well-known examples include the System Usability Scale (Brooke, 1996), Cognitive Walkthrough (Rieman et al., 1995), Applied Cognitive Task Analysis (Militello and Hutton, 1998), and Project Premortem (Klein, 2007). These techniques have each distilled dense theoretical concepts into methods that can be readily implemented with minimal training. ...
Thesis
Full-text available
This paper was completed as part of a comprehensive exam milestone for PhD candidacy in Human Systems Engineering at Arizona State University. Abstract: The US Department of Defense (DoD)’s future missions increasingly include Autonomous and AI-Enabled Systems (AAIS) that work in partnership with warfighters. To realize this partnership, effective Human-Machine Teaming (HMT) design needs to be integrated into the DoD acquisition of AAIS. Within the complex ecosystem of DoD acquisition, assessment through Test and Evaluation (T&E) is critical for ensuring engineering outcomes like HMT. Despite the prioritization of HMT evident in DoD strategic efforts and arguably sufficient capability to assess HMT, there are low levels of adoption of HMT assessment in DoD T&E of AAIS. To unpack and better understand the disparity between priority, capability, and adoption of HMT assessment, this narrative review explores barriers to adoption of current HMT assessment methods that can be gleaned from research, DoD guidance and policy, and the HMT community’s discourse on adoption. This synthesis led to eight challenges that are organized into themes of systems engineering policies and practices, the test workforce who would implement test methods and tools, and agreement about HMT assessment. In the long-term, achieving levels of adoption that meet the demands of HMT risks in AAIS may involve corresponding changes to policies, workforce development, cultural shifts, and development of HMT assessment tools intentionally designed to support DoD acquisition. In the near term, practical HMT assessment guidance may support DoD T&E stakeholders to adopt current methods and tools.
... Prospective hindsight-imagining that an event has already occurred and led to an undesirable outcome. Ask for the value of lower and upper bound, and explore why this bound could be wrong i.e., why could the truth be outside the range (Klein, 2007) Unreasonably wide distribution ...
Conference Paper
Full-text available
Accurate estimation is a fundamental skill in decision science, essential for effective decision making across various fields, including agriculture, environmental management, and policy development. To address the challenges of making reliable estimates, we developed an innovative online application designed for calibration training that helps students and professionals enhance their estimation skills using 90% confidence intervals. Our app provides a structured learning environment where users engage with binary and range questions, facilitating the development of their estimation capabilities. Participants are introduced to the concept of confidence intervals and trained to construct them effectively, fostering a deeper understanding of uncertainty in their estimates. We integrate insights into cognitive biases that often impede accurate estimation throughout the training. Participants learn to recognize and mitigate these biases—such as the Dunning-Kruger effect, anchoring bias, and confirmation bias—through practical examples and targeted exercises. By emphasizing the psychological factors influencing decision-making, we aim to cultivate a more nuanced understanding of how biases can skew judgment and lead to suboptimal decisions. Our app features a dual-interface design, catering to participants undergoing calibration and analysts facilitating the training. Participants (including experts, students, and stakeholders) are guided through questions that challenge their estimation skills, while analysts (lecturers, hosts, and administrators) can monitor progress and provide real-time feedback. This dual approach enhances the learning experience and promotes collaborative discussions around estimation techniques.
... To prepare all participating experts for the model parametrization session, they were subjected to calibration training to enhance their ability to make accurate estimates. Calibration training, inspired by Hubbard (2014), consists of providing participants with information on cognitive biases (e.g. the Dunning-Kruger effect (Kruger and Dunning, 1999), and the Anchoring effect (Kahneman, 2011)), instructing them in techniques to improve estimation skills (Equivalent bet (Hubbard, 2014), Klein's premortem (Klein, 2008), Start with unreasonably wide distributions), and supporting them in practicing these skills using trivia questions. Through several rounds of such questions, each followed by feedback on the accuracy of the estimates, most participants markedly increase their estimation skills. ...
Article
Full-text available
CONTEXT: Fruit yield and quality are critical determinants of the economic performance of apple orchards. However, these economic metrics are highly uncertain due to various quality-reducing factors during the growing season, and fruit growers would greatly benefit from reliable predictions. OBJECTIVE: In this study, we aim at developing a new tool to support fruit growers in anticipating yield and potential quality losses under the specific conditions of their orchards. The tool should allow application at four key time points during the growing season (at full bloom, before fruit thinning, after June drop, and four weeks before harvest) and capture uncertainty in the quality-reducing factors and the resulting yield parameters. METHODS: Using expert knowledge, we designed and parameterized the probabilistic 'ProbApple' model and conducted Monte Carlo simulations to project probability distributions for total and high-quality apple yield for a 'Gala' orchard in the German Rhineland. We compared scenarios with and without anti-hail netting to demonstrate the use of the model for predicting apple yield. RESULTS AND CONCLUSIONS: Applying the model four weeks before harvest, the median forecasted apple yield was 50.4 t/ha (25 %-quantile: 44.0; 75 %-quantile: 57.8 t/ha) with anti-hail netting and 49.3 t/ha (25 %-quantile: 42.7; 75 %-quantile: 56.5 t/ha) without. The forecasted high-quality yield was 34.9 t/ha (25 %-quantile: 27.5; 75 %-quantile: 41.6 t/ha) with the protection measure and 30.0 t/ha (25 %-quantile: 15.8; 75 %-quantile: 38.7 t/ha) without. These results are in line with commonly achieved 'Gala' apple yields in the Rhineland region.-NC license (http://creativecommons.org/licenses/by-nc/4.0/). SIGNIFICANCE: We show that ProbApple is a customizable tool for forecasting apple yield and quality, offering producers valuable insights for operational planning and informed management decisions.
... Managing an investor's subjective knowledge is likely to prove more challenging than simply providing facts and supporting data. In these cases, two techniques that may be helpful are Subjective Probability Interval Estimation (SPIES) (Lurtz, 2020) and premortem planning (Klein, 2007). SPIES is a graphical representation of all possible outcomes. ...
Article
Full-text available
This study used primary data collected during October 2022 from 2,119 U.S. retail investors to investigate how individuals were coping with the declining stock market and rising inflation. Using a path analysis, this study sought to explain the relationships between gender, financial stress, investment overconfidence, and trading behavior. First, a positive relationship was found between males and moving from stocks and bonds to cash. Next, the results indicated that females were more likely to have experienced financial stress and males were more likely to have displayed investment overconfidence. Both financial stress and investment overconfidence were positively related to moving from stocks and bonds to cash. The indirect effects of financial stress and investment overconfidence, however, were small and only partially mediated the relationship between gender and trading behavior.
Article
Full-text available
Despite diligent efforts, complications continue to occur during the placement of central venous catheters (CVCs). Healthcare Failure Mode and Effect Analysis has been promoted as a process improvement tool and this review describes the strategic application of Failure Mode and Effects Analysis (FMEA) to CVC placement. The objective is to demonstrate the utility of FMEA first as a tool for identifying quality or safety issues and second for guiding mitigation efforts.
Book
Full-text available
The Diary of a CEO
Method
Full-text available
The Targeted Premortem (TPM) is a variant of Klein's Premortem Technique, which uses prospective hindsight to proactively identify failures. This variant targets brainstorming on reasons for losing trust in AI in the context of the sociotechnical system into which it is integrated. That is, the prompts are targeted to specific evidence-based focus areas where trust has been lost in AI. This tool comes with instructions, brainstorming prompts, and additional guidance on how to analyze the outcomes of a TPM session with users, developers, and other stakeholders.
ResearchGate has not been able to resolve any references for this publication.