ArticlePDF Available

Abstract

Postmortem analysis (PMA) is a practical method for initiating knowledge management by capturing experience and improvement suggestions from completed projects. It requires little effort and quickly provides initial results, making it suitable even for small- and medium-size projects and companies. The authors describe their experiences with applying PMA techniques for collecting and analyzing experience in software organizations
Copyright ©2002 IEEE. Reprinted from IEEE Software May/June
2002.
This material is posted here with permission of the IEEE. Internal
or personal use of this material is permitted. However,
permission to reprint/republish this material for advertising or
promotional purposes or for creating new collective works for
resale or redistribution must be obtained from the IEEE by writing
to pubs-permissions@ieee.org.
By choosing to view this document, you agree to all provisions of
the copyright laws protecting it.
focus
0740-7459/02/$17.00 © 2002 IEEE
May/June 2002
IEEE SOFTWARE 43
improvement suggestions from completed
projects and works even in small- and
medium- size companies that cannot afford
extensive KM investments. However, PMA
has been mainly advocated for situations
such as completion of large projects, learning
from success, or recovering from failure.
2–4
When used appropriately, PMA ensures
that team members recognize and remember
what they learned during a project. Individ-
uals share their experiences with the team
and communicate them to other project
groups. Additionally, PMA identifies im-
provement opportunities and provides a
means to initiate sustained change.
We have applied a lightweight approach
to PMA in several projects
5,6
by focusing on
a few vital principles:
PMA should be open for participation
from the entire team and other project
stakeholders.
Goals canbut need notprovide a fo-
cus for analysis.
The PMA process comprises three
phases: preparation, data collection, and
analysis. For each phase, team members
can apply a number of fairly simple
methods, such as the KJ method (after
Japanese ethnologist Jiro Kawakita)
7
that collects and structures the data from
a group of people.
Preparation
When we conduct PMA in software com-
panies, two software process improvement
group members work as facilitators together
with two to all project team members. Facili-
tators organize the analysis, steer the discus-
sion, and document the results. They can be
employees in the company where the PMA is
conducted or external, as we are. External fa-
cilitators often have an advantage performing
the PMA because participants regard them as
more neutral and objective. However, they
might not know the company as well as inter-
nal facilitators, so preparation is important.
During the preparation phase, we walk
through the project history to better under-
stand what has happened. We review all
available documents, such as the work
breakdown structure, project plans, review
reports, and project reports.
We also determine a goal for the PMA.
Postmortem: Never Leave
a Project without It
Andreas Birk, sd&m
Torgeir Dingsøyr, Sintef Telecom and Informatics
Tor Stålhane, Norwegian University of Science and Technology
Although primarily
used for large
projects and
companies,
postmortem analysis
also offers a quick
and simple way to
initiate knowledge
management in
small- or medium-
size software
projects.
I
n every software project, the team members gain new knowledge and
experience that can benefit future projects and each members own
professional development. Unfortunately, much of this knowledge re-
mains unnoticed and is never shared between individuals or teams.
Our experience with project
postmortem analysis proves that it is an ex-
cellent method for knowledge management,
1
which captures experience and
knowledge management
Goals might be Identify major project
achievements and further improvement op-
portunities or Develop recommendations
for better schedule adherence. If a PMA
does not have a specific focus to guide our
preparation, we briefly discuss the project
with the project manager and key engineers.
We find it practical to distinguish between
two PMA types: One is a general PMA that
collects all available experience from an ac-
tivity. The other is a focused PMA for un-
derstanding and improving a projects spe-
cific activity, such as cost estimation. It helps
to explicitly state goals for both of these
PMA variants during this phase.
Data collection
In the data collection phase, we gather the
relevant project experience. Usually, project
team members and stakeholders have a
group discussion, or experience-gathering
session. We can often conduct data collec-
tion and the subsequent analysis within the
same session. You shouldnt limit experience
gathering to the projects negative aspects,
such as things to avoid in the future. Instead,
maintain a balance by identifying a projects
successful aspects, such as recommended
practices. For example, during a PMA at a
medical software company, the team realized
that the new incremental software integra-
tion process significantly improved process
control and product quality. Integration had
been so smooth that without the PMA, its
important role might have gone unnoticed.
Some techniques that we find useful for
data collection include
Semistructured interviews. The facilita-
tor prepares a list of questions, such as
What characterizes the work packages
that you estimated correctly? and
Why did we get so many changes to
the work in package X?
Facilitated group discussions. The facil-
itator leads and focuses the discussion
while documenting the main results on a
whiteboard.
KJ sessions. The participants write down
up to four positive and negative project
experiences on post-it notes. Then they
present their issues and put the notes on
a whiteboard. The participants re-
arrange all notes into groups according
to topic and discuss them.
Once the group identifies the important
topics, we must prioritize them before pro-
ceeding with the analysis. This will ensure that
we address the most significant issues first.
For example, during a PMA we per-
formed in a satellite software company, fre-
quent and late requirements changes
emerged as an important topic. A software
developer commented that during the proj-
ect, team members found it difficult to iden-
tify when the requirements had changed, so
much so that the code had to be rewritten
completely. In such situations, they made a
few wrong decisions, which reduced the
softwares quality. After this PMA session,
other project members made requirements
changes a high-priority topic for analysis.
Analysis
In this phase, as facilitators, we conduct a
feedback session in which we ask the PMA
participants: Have we understood what you
told us, and do we have all the relevant facts?
When we know that we have sufficient
and reliable data, we use Ishikawa dia-
grams
6
in a collaborative process to find the
causes for positive and negative experiences.
We draw an arrow on a whiteboard, which
we label with an experience. Then, we add
arrows with causeswhich creates a dia-
gram looking like a fishbone. In our exam-
ple from the satellite software company, we
found four causes for changing require-
ments: poor customer requirements specifi-
cation, new requirements emerging during
the project, little contact between the cus-
tomer and software company, and the soft-
ware companys poor management of re-
quirements documents.
Because PMA participants are a projects
real experts and we have time limitations,
we perform all analysis in this step.
Results and experience
Facilitators document the PMA results in a
project experience report. The report contains
A project description, including products
developed, development methods used,
and time and effort needed
The projects main problems, with de-
scriptions and Ishikawa diagrams to
show causes
The projects main successes, with de-
scriptions and Ishikawa diagrams
Once the group
identifies the
important
topics, we must
prioritize them
before
proceeding with
the analysis.
44 IEEE SOFTWARE May/June 2002
A PMA meeting transcript as an appen-
dix, to let readers see how the team dis-
cussed problems and successes
In an example from the satellite software
company, facilitators wrote a 15-page re-
port in which they documented the problem
with changing requirements with an
Ishikawa diagram that showed the four
main causes. After facilitators submit a re-
port, the knowledge management or quality
department must follow up.
In our experience, PMA is suitable when a
project reaches a milestone and when the com-
pany is looking for qualitative experience that
will help improve a similar, future project. You
should not apply PMA in situations with un-
finished activities, or when serious underlying
conflicts might remove the focus from im-
provement. If the atmosphere isnt appropriate
for discussing a projects problems, we prefer
using approaches other than PMA, such as
those outlined in Project Retrospectives: A
Handbook for Team Reviews.
2
When there
have been serious conflicts in the project, this
is more appropriate for managing the risk that
discussions degenerate into a hunt for scape-
goats. Finally, you must have enough time for
following up on PMA results.
In our experience, if teams apply PMA in
the right setting, it is an excellent step into
continuous knowledge management and im-
provement activities. It makes project team
members share and understand one anothers
perspectives, integrates individual and team
learning, and illuminates hidden conflicts. It
documents good practice and problems, and
finally, it increases job satisfaction by giving
people feedback about their work.
Performing a PMA can even improve proj-
ect cost estimation. We applied PMA to three
projects in an Internet software development
company, which all had serious cost over-
runs. The company could not allocate work-
ers with skills specific to the project. This led
to a need for coursesthe teams experts had
to act as tutors for the rest of the team and
were distracted from their roles in the proj-
ect. By performing the PMA, the company
realized the gravity of the qualification issue
and how it led to the project going over
budget. As an improvement action, a training
budget was set up on the company level in-
stead of the project level. The company no
longer charged staff qualification to the pro-
jects budget, and now views it as an invest-
ment into quality and competitive advantage.
As a result of this PMA, management real-
ized the strategic importance of staff qualifi-
cation and knowledge managementa truth
that often gets buried in the hectic rush of In-
ternet software business.
W
e received a lot of positive feed-
back from PMA participants in
different companies. Particularly,
they like that PMA offers a simple yet effec-
tive way to uncover both achievements and
improvement opportunities. One developer
at the satellite software company noted, If
you do a PMA on the project...you have to
think through things, which is a crucial
part of knowledge management. So, never
leave a project without it!
References
1. C. Collison and G. Parcell, Learning to Fly: Practical
Lessons from One of the World’s Leading Knowledge
Companies
, Capstone, New York, 2001.
2. B. Collier, T. DeMarco, and P. Fearey, A Defined
Process For Project Post Mortem Review,
IEEE Soft-
ware
, vol. 13, no. 4, July/Aug. 1996, pp. 6572.
3. N.L. Kerth,
Project Retrospectives: A Handbook for Team
Reviews
, Dorset House Publishing, New York, 2001.
4. A.J. Nolan, Learning from Success,
IEEE Software,
vol. 16 no. 1, Jan./Feb. 1999, pp. 97105.
5. T. Stålhane et al., Post MortemAn Assessment of
Two Approaches, Proc. European Software Process
Improvement
(EuroSPI 01), ICSN, Bray, Ireland.
6. T. Dingsøyr, N.B. Moe, and Ø. Nytrø, Augmenting Ex-
perience Reports with Lightweight Postmortem Re-
views,
3rd Intl Conf. Product Focused Software
Process Improvement
(Profes 01), Lecture Notes in
Computer Science, vol. 2188, Springer-Verlag, Berlin,
pp. 167181.
7. D. Straker,
A Toolbook for Quality Improvement and
Problem Solving
, Prentice Hall International, London,
1995, pp. 8998 and 117124.
May/June 2002
IEEE SOFTWARE 45
About the Authors
Andreas Birk is a consultant and software engineering professional at sd&m, software
design and management. His special interests include software engineering methods, knowl-
edge management, and software process improvement. He holds a Dr.-Ing. in software engi-
neering and a Dipl-Inform. in computer science and economics from the University of Kaiser-
slautern, Germany. He is a member of the IEEE Computer Society, ACM, and German Computer
Society. Contact him at sd&m, Industriestraße 5, D-70565 Stuttgart, Germany;
andreas.birk@sdm.de.
Torgeir Dingsøyr is a research scientist at Sintef Telecom and Informatics research foun-
dation in Trondheim, Norway. He wrote his doctoral thesis on Knowledge Management in
Medium-Sized Software Consulting Companies at the Department of Computer and Information
Science, Norwegian University of Science and Technology. Contact him at Sintef Telecom and In-
formatics, SP Andersens vei 15, NO-7465 Trondheim, Norway; torgeir.dingsoyr@sintef.no.
Tor Stålhane is a full professor of software engineering at the Norwegian University of
Science and Technology. He has a MSc in electronics, and a PhD in applied statistics from Nor-
wegian University of Science and Technology. He has worked on compiler development and
maintenance and software reliability, and on software process improvement and systems
safety. Contact him at Department of Computer and Information Science, Norwegian Univer-
sity of Science and Technology, NO-7491 Trondheim, Norway; tor.stalhane@idi.ntnu.no.
... • Post-Mortem Analysis (PMA): It is an opportunity for the team members to look back on what they did well or not. So, the group can diagnose problems and change the development process [2]. We started to apply PMA after the first three projects. ...
... A preliminary version of the work reported in this paper was presented at the 23rd International Conference on Information Visualisation (IV'2019) [64]. While that preliminary work was limited to describing the functionalities of an earlier version of IFDBMaker, which used a more primitive mechanism to encode interactive fiction in HTML5 (the use of div and span generic tags) and which lacked extensibility, in this new paper: (i) we describe the basic requirements for our approach that were elicited from our collaboration with the LEETHI research group; (ii) we analyze different proposals to the production of interactive fiction for the web and we assess these proposals with respect to the requirements elicited; (iii) we explicitly introduce the HEXIFE DSL for encoding interactive fiction, based on the assessment of the proposals analyzed, describing its development process, detailing its interactive fiction constructs, and describing how to implement this language using conventional lightweight web technologies (HTML5, CSS3, JavaScript, Web Components [31], PHP and MySQL); (iv) we describe the current version of the IFDBMaker tool, a version much more advanced than the tool described in [64] that exhibits substantial improvements in usability (e.g., the automatic management of references included in HEXIFE elements); and (v) we provide a postmortem evaluation [6,20] of the approach. ...
Article
Full-text available
This paper presents an approach to the production of web-based interactive fiction, which is grounded in the requirements posed by an expert focus group, and which integrates a domain-specific language (DSL) for interactive fiction (HEXIFE) and an authoring tool for this DSL (IFDBMaker). HEXIFE is an extension of HMTL5 that includes markup specifically devoted to different aspects of interactive fiction, such as hyperlinking, choice points, personalization, stretchtext, annotations, conditional content, and gamification. This DSL, which can be easily extended with new interactive fiction behaviors, is supported by a runtime environment based on conventional web technologies (HTML5, CSS3, JavaScript, Web Components, PHP, and MySQL). IFDBMaker, in turn, piggybacks in a widely used web-based HTML editing framework (TinyMCE) to allow the creation of interactive HEXIFE fiction through a user-friendly approach. A postmortem evaluation shows how: (i) the approach makes the feasibility of supporting interactive fiction on conventional web technologies apparent; and (ii) the approach is a feasible one to actively involve writers in the final production of interactive fiction that are distributed and played through the web.
... Students take a step back and reflect on their learning process through a different angle. In the week after the hackathon we conducted a project postmortem [34], which is a consolidated practice in agile methods [35] for reflecting about projects [36]. We gave them prompts asking what worked or did not work in: (1) the hackathon and course format; and (2) their hackathon project itself, with an additional question of what were most important topics and technologies they learned during the project. ...
Preprint
Full-text available
In 2020, due to the COVID-19 pandemic, educational activities had to be done remotely as a way to avoid the spread of the disease. What happened was not exactly a shift to an online learning model but a transition to a new approach called Emergency Remote Teaching. It is a temporary strategy to keep activities going on until it is safe again to return to the physical facilities of universities. This new setting became a challenge to both teachers and students. The lack of interaction and classroom socialization became obstacles for students to continue engaged. Before the pandemic, hackathons -- short-lived events (1 to 3 days) where participants intensively collaboration to develop software prototypes -- were starting to be explored as an alternative venue to engage students in acquiring and practicing technical skills. In this paper, we present an experience report on the usage of an online hackathon as a resource to engage students in the development of their semester project in a distributed applications course during this emergency remote teaching period. We describe details of the intervention and present an analysis of the students' perspective of the approach. One of the important findings was the efficient usage of the Discord communication tool -- already used by all students while playing games -- which helped them socialize and keep them continuously engaged in synchronous group work, "virtually collocated".
... Organizations have taken lessons learned as methods or models to improve project management processes seeking to maximize project success rate (Santos, 2014). Some techniques, such as post-project review (Gof f in et al, 2010;Harrison, 2002), the postmortem analysis projects (Birk, Dingsoyr, & Stalhane, 2002;Scott & Stålhane, 2003), and the semi-structured interview (Komi-Sirvio, Mäntyniemi, & Seppanen, 2002), are seen as ef f ective mechanisms that enable to identif y, capture, and transf er the knowledge of key lessons learned and likely to be used to benef it f uture projects. ...
Article
Full-text available
Purpose – The main objective of this article is to propose the use of a model developed by Matturo and Silva (2010) to capture knowledge in software projects based on the lessons learned.Design/methodology/approach – We carried out a qualitative research from a descriptive perspective through a single case study applied to an Enterprise Information Technology company. The company is a leader in market solutions to support customer experience management. For the data collection process, we used systematic literature review, document analysis and semi-structured interviews.Findings – The results supported project managers to better understand the storage and use of information from lessons learned in dimensioning the use of human resources and to support the estimation of new project activities. In addition, the results showed the organization's disregard for not giving due importance to the information and knowledge generated during the life cycle of a project.Research, Practical Social implications – The model allows companies to obtain new knowledge or consult existing knowledge throughout the life cycle of projects and to support project managers in the process of estimating activities and preparing budgets with greater precision, using the information from lessons learned as a support. acquired in the completed projects.Originality/value – The lack of information in the initial scope of the project and in the definition of activities in the human resource allocation process hinder the duration of the project's development activities, directly resulting in inaccurate estimates. As a result, this scenario contributes to the increased risk of deviations in terms and / or costs of software projects.
... Organizations have taken lessons learned as methods or models to improve project management processes seeking to maximize project success rate (Santos, 2014). Some techniques, such as post-project review (Goffin et al, 2010;Harrison, 2002), the postmortem analysis projects (Birk, Dingsoyr, & Stalhane, 2002 & Stålhane, 2003), and the semi-structured interview (Komi-Sirvio, Mäntyniemi, & Seppanen, 2002), are seen as effective mechanisms that enable to identify, capture, and transfer the knowledge of key lessons learned and likely to be used to benefit future projects. ...
Article
The knowledge and experience acquired during the life cycle of projects can be stored and shared among professionals through methods and models, it can being considered as possible factors for leveraging new projects. In this context, this study aims to propose the use of a model elaborated by Matturo and Silva (2010) for capturing knowledge on software projects based on lessons learned. Therefore, we adopted the qualitative research of descriptive perspective by using a single case study applied to a Technology Business Information company. The company is leader of market solutions to support customer experience management. For the process of data collection, we used systematic literature review, document analysis, and semi-structured interviews. The results of this study showed the organization's negligence in not giving due importance to the information and knowledge generated during the life cycle of a project. It is noteworthy that the model adopted in this study will help the companies to create processes to capture lessons learned generated during the life cycle of projects. As a contribution, it can be said that this model support project managers in the process of preparing activities estimations with higher accuracy.
Chapter
Benefits and reasons for going agile.
Chapter
Software development is a problem-solving activity, where ideas are combined in complex ways to create a software product that embodies new knowledge. In this endeavor, software developers constantly look for actionable knowledge to help solve the problem at hand. While knowledge management efforts in the software development domain traditionally involved technical initiatives such as knowledge repositories, experience factories, and lessons-to-learn databases, there is a growing appreciation in the software community of the role of developers' personal knowledge networks in software development. However, research is scarce on the nature of these networks, the knowledge resources accessed from these networks, and the differences, if any, between developers of different experience levels. This research seeks to fill this void. Based on a case study in a software development organization, this research explores the nature of knowledge networks of developers from a social capital perspective. Specifically, it examines the structural and relational dimensions of developers' knowledge networks, identifies the specific actionable knowledge resources accessed from these networks, and explores how entry-level and more experienced developers differ along these dimensions. The findings from the qualitative analysis, backed by limited quantitative analysis of the case study data underpin the discussion, implications for practice and future research directions.
Chapter
This chapter presents a framework for differentiated process support in large software projects. Process support can be differentiated in different levels based on the size of the development organization and the need for coordination across different levels of the organization. We have defined four main perspectives: individual, group, team, and project level, where the framework consider essential issues when planning and executing the software development processes in organizations with different levels of management. Further, a guideline is provided that suggests what is required of process support in the various organizational levels.
Conference Paper
Full-text available
Many small and medium-sized companies that develop software experience the same problems repeatedly, and have few systems in place to learn from their own mistakes as well as their own successes. Here, we propose a lightweight method to collect experience from completed software projects, and compare the results of this method to more widely applied experience reports. We find that the new method captures more information about core processes related to software development in contrast to experience reports that focus more on management processes.
Conference Paper
Full-text available
Learning from experience is the key to successes for all that develop software. Both the successes and the failures in software projects can help us to improve. Here we discuss two versions of Post Mortem Analysis (PMA) as methods for harvesting experience from completed software projects, which can be part of a larger knowledge management program. The two methods are tailored for use in small and medium size companies and are conceptually easy to apply. In addition, they require few resources compared to other methods in the field. We think that the methods are useful for companies when they need to document their knowledge, find improvement actions and as a start of systematic knowledge harvesting.
Article
Every organization experiences a range of success. In an attempt to improve business processes, many organizations focus on those projects that fail in order to `learn from their mistakes'. Whereas the author does not disagree with learning from mistakes, many organizations overlook learning from their successes. The result of any project is a direct consequence of the actions and activities used on the project. Therefore, if you believe that failure is no accident, you must believe also that success is no accident. `Learning from Success' is a study on accelerated process improvement and provides a methodology for uncovering why successful projects are indeed successful.
Article
An abstract is not available.
Article
Whereas perpetual refinement is an established norm in many businesses, the author presents a case for tapping into an organization's expertise and learning from its own successes and best practices. Why are successful projects successful? This study on rapid process improvement provides a methodology for uncovering success factors and implementing them throughout an organization
Article
Most of us pay lip service to the need for software project post mortems, but the literature offers little guidance on how to conduct them. The authors propose a tentative, standard process for conducting post mortem reviews and describe activities, roles, and artifacts of the process. The success of the post mortem-or of any learning process-demands a context that makes organizational learning possible. Management must make an honest and sincere commitment to establish this context. This commitment should take the form of a public resolution to implement risk management on subsequent projects and to make all post mortem findings input to that risk management effort. After all, lessons learned the hard way on past projects are, if nothing else, risks for future projects. Participants are empowered when they know that each issue raised during the post mortem process must be added to the risk database and evaluated methodically on each subsequent project.
Augmenting Experience Reports with Lightweight Postmortem Reviews," <i>3rd Int'l Conf. Product Focused Software Process Improvement&lt
  • T Dingsøyr
  • N B Moe
  • Ø Nytrø