Statistical analysis of deviation of actual cost from estimated cost using actual project data
ABSTRACT This paper analyzes an association of a deviation of the actual cost (measured by person-month) from the estimated cost with the quality and the productivity of software development projects. Although the obtained results themselves may not be new from the academic point of view, they could provide motivation for developers to join process improvement activities in a software company and thus become a driving force for promoting the process improvement.We show that if a project is performed faithfully under a well-organized project plan (i.e. the plan is first constructed according to the standards of good writing, and then a project is managed and controlled to meet the plan), the deviation of the actual cost from the estimated one becomes small. Next we show statistically that projects with small deviation of the cost estimate tend to achieve high quality of final products and high productivity of development teams. In this analysis, the actual project data on 37 projects at a certain company are extensively applied.
- SourceAvailable from: wustl.edu
Article: Software Engineering Economics[show abstract] [hide abstract]
ABSTRACT: This paper summarizes the current state of the art and recent trends in software engineering economics. It provides an overview of economic analysis techniques and their applicability to software engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available, the state of the art in algorithmic cost models, and the outstanding research issues in software cost estimation.IEEE Transactions on Software Engineering 02/1984; · 2.59 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: We conducted a long-term experiment to compare the costs and benefits of several different software inspection methods. These methods were applied by professional developers to a commercial software product they were creating. Because the laboratory for this experiment was a live development effort, we took special care to minimize cost and risk to the project, while maximizing our ability to gather useful data. This article has several goals: 1) to describe the experiment's design and show how we used simulation techniques to optimize it, 2) to present our results and discuss their implications for both software practitioners and researchers, and 3) to discuss several new questions raised by our findings. For each inspection, we randomly assigned three independent variables: 1) the number of reviewers on each inspection team (1, 2, or 4), 2) the number of teams inspecting the code unit (1 or 2), and 3) the requirement that defects be repaired between the first and second team's inspections. The reviewers for each inspection were randomly selected without replacement from a pool of 11 experienced software developers. The dependent variables for each inspection included inspection interval (elapsed time), total effort, and the defect detection rate. Our results showed that these treatments did not significantly influence the defect detection effectiveness, but that certain combinations of changes dramatically increased the inspection interval.01/1995
- [show abstract] [hide abstract]
ABSTRACT: From the Publisher:The complete software developer's guide to surviving projects that are "doomed to fail." In the course of a career, practically every software developer and manager will encounter projects with outrageous staffing, scheduling, budgeting, or feature constraints: projects that seem destined to fail. In the wake of re-engineering, such "Death March" projects have become a way of life in many organizations. Surviving projects that are "doomed to fail"! Negotiating the best deal up-front. Managing people and setting priorities. Choosing tools and technologies. When it's time to walk away. Now, best-selling author Edward Yourdon brings his unique technology and management insights to the worst IS projects, showing how to maximize your chances of success--and, if nothing else, how to make sure your career survives them. Yourdon walks step-by-step through the entire project life cycle, showing both managers and developers how to deal with the politics of "Death March" projects--and how to make the most of the available resources, including people, tools, processes, and technology. Learn how to negotiate for the flexibility you need, how to set priorities that make sense--and when to simply walk away. Discover how to recognize the tell-tale signs of a "Death March" project--or an organization that breeds them. If you've ever been asked to do the impossible, Death March is the book you've been waiting for.
Statistical Analysis of Deviation of Actual Cost from
Estimated Cost Using Actual Project Data
Osamu Mizuno, Tohru Kikuno
Graduate School of Engineering Science, Osaka University
1-3, Machikaneyama, Toyonaka, Osaka 560-8531, Japan
Katsumi Inagaki, Yasunari Takagi, Keishi Sakamoto
Kusatsu, Shiga 525-0035, Japan
This paper analyzes cause and effectof the deviationof actualcost (measured by person-
for developers to join process improvement activities in the software company and thus
become a driving force for promoting process improvement.
To be precise, we show that if the projects are performed faithfully under the well
organized plan(that is, the plan is first constructed according to the standards of good
writing and then projects are managed and controlled to meet the plan), then the deviation
with small deviation of the cost estimate tend to achieve high quality of final product and
high productivity of development team. In this analysis, actual project data on 37 projects
at a certain company are extensively applied.
Keywords: software development project, project plan, deviation of cost estimate, quality
and productivity, statistical analysis
This paper describes empirical research on process improvement in a certain com-
pany, which we call CompanyA for convenience. In the Company A, the software
tried to pervade the process improvement into their company. This study is a part
of process improvement activities which the SEPG has done in 1998 for the devel-
opers in their company. In the software development project, at first, the size to be
Preprint submitted to Elsevier Science26 November 1999
developed is estimated. Next a plan is constructed based on the estimate. Then the
development starts according to the plan. If the project is performed exactly as the
plan specifies, the project is regarded as successful project. However some projects
inevitably result in so-called confused projects or death march projects, in
which actual cost exceeded the estimated cost by 50%. Therefore it is strongly
desired by the SEPG to reduce the number of the confused projects.
In order to reduce the confused projects, many methods and guidelines have al-
ready been proposed. The famous methods such as COCOMO and Function
Point method aimed to make the estimate accurate.Next to improve quality and
productivity, the review activities are introduced to detect the defects in the early
stage. Then the references[8,13] insisted on the importance of constructing an
appropriate plan and utilizing it during the development. But it is clear that a good
method or guideline do not haveanyeffectif theyare utilized or applied inappropri-
ately in the development field. In order to guide appropriate application, developers
must be motivated to utilize them. Actually, Humphrey said that motivated pro-
fessionals or developers can strive for superior performance. Therefore, we should
not enforce developersto apply a newmethod before they can understand its benefit
and importance, and thus they are nicely motivated to do it.
In this paper, we take notice of the deviation of actual cost from the estimated cost
and regard projects with large cost deviation as confused projects. We introduce a
cost. On the other hand, we (including the SEPG) guess that the construction of
appropriate plan and its adherent execution is the key point to reduce confused
projects. So in order to motivate developers to construct a good plan and to execute
it, we show its benefit and importance. To sum up intuitively, we will show that
“construction of appropriate plan and its adherent execution” is a useful approach
to reduce the confused projects (with the large
?? which denotes the difference between actual cost and estimated
To be precise, we will show the following two propositions: (
structed and executed adherently, the deviation of the cost estimate,
small, and (
of the product and the productivity of the team are high. As mentioned before,
many researchers have already pointed out that appropriate plan and its adherent
execution are important for software development[8,13]. However, there are few
studies which prove the effect of appropriate plan quantitatively by using actual
development data. In this study we apply 37 project data obtained from actual de-
velopment in Company A and show the correctness of propositions
correlation analysis and test of statistical hypothesis.
?1) if the plan is con-
?? , becomes
?2) if the deviation of the cost estimate,
?? , is small, both the quality
As for the proposition
much more difficult to construct a good plan. Thus we consider plans satisfying
some standards(prepared for construction of the plan) as good plans. Hence we
make a checklist for the development plan(The detail of the checklist is described
?1, it is very difficult to define what is a good plan and is
in Section 3). Based on the checklist, we judge and evaluate the plan. For this
purpose we define a new metric
constructed adherently to the standard.
????, which indicates whether or not the plan is
Nextwe should take notice of the execution of the plan. Ideally, developersperform
the project exactly as the plan specifies. But actually various problems often disturb
the development. For example, many defects are found in the test activity, and
unexpected effort is needed to remove them. Thus we evaluate the execution of the
plan from two points of view: (1) whether the project is managed using the plan,
(2) whether the ratio of review effort to entire effort is large enough to avoid the
or not the development is performed adherently to the plan. Then we perform a
correlation analysis between the evaluation of the plan
are some extent of correlation between them.
?? (the deviation of the cost estimate). The result of analysis shows that there
project manager’s point of view, those projects never adhere to their development
plans. In this line we evaluate the resultant effects of the deviation of the cost
estimate. To be precise, we investigate the relationship of the deviation of the cost
estimateon boththequalityof thefinalproductand theproductivityofdevelopment
team. In thisanalysis we classify theprojects into twodistinctclasses using
deviation of the cost estimate):
confirmed that both the quality and the productivity of the projects in
than those of in
?includes the projects with
?(The level of significance is chosen as 0.05).
The rest of this paper is organized as follows: Section 2 describes the preliminaries
of this study, target projects, process model and fundamental data. The metrics used
in the analysis are defined in Section 3. The analysis for the deviation of the cost
estimate is performed in Section 4. It is shownthat the deviationof the cost estimate
is small if the plan is constructed adherently(to the standard) and the project is
performed or managed adherently(to the constructed plan). The analysis of effect
of the deviation of the cost estimate on the quality and productivity is shown in
Section 5. It is shown that in the project with small deviation of the cost estimate,
the quality of the delivered code is high and the productivity of the development
team is high. Finally, Section 6 summarizes this paper.
The projects targetedin this paper are the development of computer control systems
with embedded software in CompanyA. The systems analyzed are classified into
three categories: banking application, railroad application and business application.
Though we omit the details, such embedded software implements rather complex
kinds of interrupts. Furthermore, since it is delivered in the form of LSI chips,
modification of the faults after delivery is very expensive. Thus high reliability is
especially required for the embedded software.
In CompanyA, the development of such software is executed concurrently with the
design and development of system hardware. Hence it is necessary to manage the
whole project. Generally, in such a project, the specification of the software product
is strongly influenced by the restrictions of the hardware design.
However, in the case of CompanyA, modification of a specification will occur in
some specific and limited areas of the product, such as the layout of a screen or the
execution speed of the CPU. Fortunately, CompanyA can decide the interface to
the hardware and can choose the operating system itself. As a result, the content
of the requirement specification of the embedded software will be relatively stable
and only changed in a very limited way.
The 37 projects targeted in this study are categorized into three groups:
(1) 8 projects related to the banking system : ATM.
We represent and refer to them by
We represent and refer to them by
(3) 3 projects related to retail system : POS terminal.
We represent and refer to them by
The cost (that is, development effort) of these 37 projects ranges from 18.8 person-
months to 185 person-months. The average cost is 46.3 person-months.
The process improvement activity has been conducted by the software engineering
process group(SEPG) in CompanyA. Especially, the following attempts have been
carried out enthusiastically.
– Exhaustive collection of fundamental data[14,15].
– Establishing standards for activities.
? Constructing the project plan.
? Describing the development process[9,14].
As a result of these efforts, the following improvements have been observed in
quality and productivity.
– Thedevelopmentplandocumenttendsto beconstructedfaithfullytothestandard
of good writing.
– The development cost, which is one of the most important factors in the com-
pany(but unfortunately is very difficult to estimate exactly[2,5]), tends to be
– Both the quality of the delivered code and the productivity of the development
team tend to be stable and improving.
are eager to know the causes of these improvements(especially, improvement on
the quality and the productivity). This gives a strong motivation to the statistical
analysis in this paper using the data from actual projects in Company A.
On the contrary, in Company A some projects also result in so-called confused
project or death march project. Although these confused projects are excep-
tional and rare cases in Company A, the SEPG have to identify the causes of the
confusion. This gives another motivation to the statistical analysis in this paper.
2.3 Process model
In CompanyA, many kinds of computer control systems with embedded software
are developedmainly using C language. The typical products are ATMs(Automated
Teller Machine) for banking applications, POS(Point Of Sales) terminals for busi-
ness applications and ticket vending machines for railroad applications. Such prod-
ucts are developed under the development process shown in Figure1.
The process model shown in Figure1 is a standard waterfall model. As is described
in subsection 2.1, modification to the requirement specification is very rare and is
limited only to layout of screens or the speed of CPU, and thus most of the require-