Content uploaded by Masood Uzzafer
Author content
All content in this area was uploaded by Masood Uzzafer on Mar 23, 2019
Content may be subject to copyright.
A contingency estimation model for software projects
Masood Uzzafer ⁎
University of Nottingham, School of Computer Science, United Kingdom
Received 20 June 2012; received in revised form 27 November 2012; accepted 11 December 2012
Abstract
A contingency estimation model for software development projects is presented. The proposed model considers the estimated cost and the risk
of software projects to estimate contingency resources; hence, contingency estimates are correlated to the cost and risk of software projects. The
model uses a generic probabilistic representation of the estimated cost; hence, it can be deployed with any project development environment and
provides a flexible choice to software managers. Furthermore, the proposed model considers the risk tolerance of software organizations to
estimate the contingency and helps to abate the maximum impacts of risk events within the risk tolerance. The proposed model is scalable to a
portfolio of software projects. The model produces sub-additive contingency estimates which is essential to optimize a software project or a
portfolio of software projects. The results of a case-study show that the contingency estimates are comparable with the actual contingency
resources needed for the development of real software development projects.
© 2012 Elsevier Ltd. APM and IPMA. All rights reserved.
Keywords: Contingency estimation; Project management; Risk assessment; Project cost estimation
1. Introduction
Contingency resources are reserves that are set aside to manage
and handle the impact of risk events in order to protect projects
from producing undesirable results. Project managers estimate,
allocate and manage appropriate contingency resources for
effective project management. In addition, contingency resources
help project managers efficiently manage organizational resources.
Therefore, contingency estimation plays a fundamental role in
defining project management plans that produce best project
results.
Typical parameters involved in software development projects
include cost, risk, budget, schedule, quality, and specifications. The
cost is the most critical parameter of software development projects
(Pfleeger and Atlee, 2006), which is represented in man-months
units of effort. Furthermore, the risk events of software projects can
result in adverse monetary consequences that cause shortfalls in the
estimated cost of software projects (DeMarco and Lister, 2003;
Kumar, 2002). Therefore, for software projects, the contingency
resources are reserves of man-months effort which provide a buffer
against cost shortfalls due to the impacts of risk events and
safeguards software projects from producing undesirable results
(Bannerman, 2008; Janne and Lyytinen, 2000; Kitchenham et al.,
2003). The allocation of contingency resources allows software
organizations to take calculated risks; hence, the estimated
contingency resources represent the maximum risk tolerance of
an organization (Cooper et al., 2005; Khamooshi and Cioffi, 2009;
Touran, 2003).
The contingency estimation is about effective cost estimation
(Alkoffash et al., 2008; Menzies et al., 2006) and risk assessment
(Bennatan, 1994; Cooper et al., 1985; Dey et al., 1994; Wysocki,
2006). Different levels of cost commitments help to manage the
impact of different sets of risk events; hence, contingency
estimation should be correlated with the estimated cost and risk
of software development projects. Therefore, contingency estima-
tion involves modeling the cost and risk of software development
projects (Tom and Susannah, 1988).
The estimated and allocated contingency resources are not
always fully utilized because some risk events may not occur
during the development of a software project. Therefore, the
contingency resources are best managed and utilized within a
⁎Tel.: +44 971 50 4728618.
E-mail address: Keyx8muz@nottingham.edu.my.
www.elsevier.com/locate/ijproman
0263-7863/$36.00 © 2012 Elsevier Ltd. APM and IPMA. All rights reserved.
http://dx.doi.org/10.1016/j.ijproman.2012.12.002
Available online at www.sciencedirect.com
ScienceDirect
International Journal of Project Management 31 (2013) 981 –993
portfolio of software projects because only a few of the projects in
the portfolio would actually invoke the contingency reserves
(Bannerman, 2008; Kitchenham et al., 2003).
Researchers in the field of software engineering have attempted
to estimate the contingency resources needed for software projects
and have proposed various contingency estimation models.
However, the main drawback of the existing models for software
development projects is their lack of generality (Kellner et al.,
1999). Managers of software project seek flexible models that can
be easily implemented with their current project management
practices. Furthermore, some traditional contingency estimation
models do not consider the estimated cost whereas others do not
consider the risk; hence, these models generate contingency
estimates that do not correlate with the cost and risk of software
development projects.
This research work presents a contingency estimation model
that is independent of specific software development processes;
hence, it offers a flexible solution to software organizations and
managers of software projects. Furthermore, the model considers
the impact of risk events on the estimated cost; consequently, the
proposed model generates contingency estimates that are
correlated with the cost and risk of software development projects.
This research work also focuses on the scalability of the model to a
portfolio of software projects.
Furthermore, the contingency estimates should be sub-additive
which is essential to optimize a software project consisting of
multiple development tasks. Similarly, sub-additive estimates are
essential to optimize a portfolio of software projects (Uzzafer,
2010). Sub-additivity occurs when the estimate of the whole is less
than or at most equal to the combined estimate of the parts; for
example, for an estimate fof parts Aand B, the sub-additivity
property states that f(A+B)≤f(A)+f(B)(Lange, 2003; Uzzafer,
2010). The contingency resources for a software project should be
sub-additive, which explain that the contingency estimates of a
software project should be less than the sum of the contingency
resources needed for the development of each individual task of the
software project. Similarly, a portfolio of software project should
require less contingency resources then the combined contingency
resources of all the projects in the portfolio. Hence, this research
work aims to ensure that contingency estimates are sub-additive.
The proposed contingency estimation model for software
development projects is inspired by a financial contingency
estimation model. Analogies from the financial domain have
been used in the past for software development projects, see for
example the work of Kitchenham et al. (2003),Boukendour (2005)
and Costa et al. (2007). The financial contingency estimation
models use the financial risk measure (CAS, 2007). The financial
risk measure maps the impact of a set of financial risk events on the
probabilistic model of financial investments; therefore, the
estimated contingency protects financial investments against a set
of risk events (Acerbi and Drik, 2003; Acerbi et al., 2001; CAS,
2007; Morgan/Reuters, 1996; Yamai and Toshinao, 2005).
Section 2 of this paper discusses the traditional contingency
estimation models. The section describes generic (parametric and
non-parametric) probability models of cost and discusses a
parametric probabilistic model of cost of software development
projects. Furthermore, the section presents different categories of
risk and discusses a software risk measure model that measures
the impact of a set of risk events on the probabilistic model of cost
of software projects. Section 3 presents the proposed contingency
estimation model and discusses different scenarios of contingen-
cy estimations. Section 4 explains the validation procedure of the
model using different model validation techniques. Section 5
presents a case-study of the proposed contingency estimation
model on the data of real software development projects. The
results of the case-study show that the contingency estimates of
the model are comparable to the real software development
experiences. Section 6 draws some final conclusions.
2. Literature review
2.1. Traditional contingency estimation models
Fairley (1995) defines contingency as a percentile of the
distribution of estimated cost of software projects; however,
percentile values of skewed probability distributions are not
always sub-additive (Uzzafer, 2010). Kitchenham and Linkman
(1997) suggested the deployment of contingency resources only
to deal with the uncertainties of software projects; hence, their
model does not consider other types of risk. Kansala's (1997)
contingency estimation model is based on a single value
quantification of risk; however, single value representations of
risks and costs are unreliable (Fairley, 1995; Kitchenham and
Linkman, 1997; Kitchenham et al., 2003). Briand et al. (1998)
mapped the impacts of risk events onto a triangular distribution
where the mean of the triangular distribution determines the
contingency. The triangular distribution, however, suffers from
symmetric bias and generates a large mean when the maxima of
the distribution is large (David, 2008). Gillian and Robert (2001)
represented the estimated cost of software projects with Gaussian
distributions, which assumes that the cost estimates on either
side of the distribution are equally likely. Kitchenham et al.
(2003) modeled the contingency resources with a probability
distribution based on an insurance premium model. They
suggested using the contingency resources to handle risk events
that are unknown; however, the contingency resources estimated
to handle unknown risk events cannot be justified. Boukendour
(2005) used stock option theory to estimate the contingency
resources for software development projects; the model relies on
single value estimate of cost. Jantzen et al. (2006) modeled the
impact and probability of risk events using a predefined set of
percentages. This model is therefore limited to a few pre-defined
values of risk impacts and probabilities.
2.2. Software cost
The cost is the most important consideration in the software
projects (Kitchenham et al., 2003; Pfleeger and Atlee, 2006); the
estimated cost encompasses all other parameters of software
projects. The cost estimation models for software projects produces
single point estimates of costs, represented here as E.Lately,
researchers used probabilistic models (parameter, non-parameter)
to represent the cost of software projects. Kitchenham et al. (2003,
2005) used a gamma distribution (parametric model) to represent
982 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
the cost of software projects. Whereas, Fairley (1995) used the
Constructive Cost Model (COCOMO) (Boehm, 1981)forcost
estimation and constructed a non-parametric distribution through
Monte Carlo simulation to represent the cost.
The random values of costs of software development projects
are represented by Xwhich has an underlying parametric
probability distribution [Appendix A.1]. The minimum value of
Xis said to be bounded by zero, whereas the maximum value of X
is considered unbounded (Kitchenham and Linkman, 1997;
Kitchenham et al., 2003). Furthermore, the expectation of Xis
represented with EX½; the expected cost is defined as the
budgeted cost. The maximum cost a software organization is
willing to invest in a software project is the risk tolerance of the
organization and is defined as the 100ath percentile of Xwhich is
represented as x
a
. The development of software project requires
some cost (Kitchenham and Linkman, 1997), therefore, the
minimum estimated cost for the development of software projects
is defined as the 100bth percentile of Xand represented as x
b
.
Kitchenham et al. (2003, 2005) proposed using gamma
distributions, Γ(k,θ), to model the random estimated cost of
software projects such that X~Γ(k,θ)wherekand θare the
shape and spread parameters of the gamma distribution, respec-
tively. The expectation of the gamma distribution is defined as
EXeΓk;θðÞ½¼kθ(Ross, 2007). Uzzafer (submitted for
publication) explained that single value estimates of cost (E)can
be modeled with a gamma distribution by mapping Ewith the
expectation of the gamma distribution such that E↦EXeΓk;θðÞ
and then approximating the distribution parameters kand θ.A
good approximation of the shape of a gamma distribution can be
achieved by setting k= 2 and estimating the θusing E↦EX½¼kθ
(Uzzafer, submitted for publication).
In general, the managers of software development projects
assign costs and probabilities to different risk events, which
generate a sequence of cost samples and produce a discrete
non-parametric distribution of cost. Consider nsamples of cost
which forms a sequence Xiwhere iis the sample index such that
∑kPXi≤Xk
fg
¼1and the mean of the samples is defined as
EX
i
½[Appendix A.2]. The random discrete samples of cost (Xi)
can be generated using different techniques. For example, Boehm
(1991) explained that the software manager can assign each risk
event a cost estimate with an associated probability based on
the expert opinion. Whereas, Fairley (1995) used Monte Carlo
simulations to generate random samples of cost and their
probabilities.
2.3. Software risk
Researchers have adopted different expressions of risk for their
domains; in essence, risk is the possibility of loss. The central
Table 1
SEI's list of risk events for software projects.
Product engineering Development environment Program constraints
1. Requirements
a. Stability
b. Completeness
c. Clarity
d. Validity
e. Feasibility
f. Precedent
g. Scale
2. Design
a. Functionality
b. Difficulty
c. Interfaces
d. Performance
e. Testability
f. Hardware constraints
g. Non-developmental
Software
3. Code and unit test
a. Feasibility
b. Testing
c. Coding/Implementation
4. Integration and test
a. Environment
b. Product
c. System
5. Engineering specialties
a. Maintainability
b. Reliability
c. Safety
d. Security
e. Human factors
f. Specifications
1. Development process
a. Formality
b. Suitability
c. Process control
d. Familiarity
e. Product control
2. Development system
a. Capacity
b. Suitability
c. Usability
d. Familiarity
e. Reliability
f. System support
g. Deliverability
3. Management process
a. Planning
b. Project organization
c. Management experience
d. Program interfaces
4. Management methods
a. Monitoring
b. Personnel management
c. Quality assurance
d. Configuration management
5. Work environment
a. Quality Attitude
b. Cooperation
c. Communication
d. Morale
1. Resources
a. Schedule
b. Staff
c. Budget
d. Facilities
2. Contract
a. Type of contract
b. Restrictions
c. Dependencies
3. Program interfaces
a. Customer
b. Associate contractors
c. Subcontractors
d. Prime contractor
e. Corporate management
f. Vendors
g. Politics
983M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
notion of risk is that it is an event which may or may not take place;
hence, its occurrence is uncertain and it results in adverse monetary
consequences. Therefore, for software projects, the impact of risk
events adds additional cost to the expected cost (EX½)andmoves
the expected cost towards higher values of X.
2.3.1. Risk categories
Contingency resources are limited resources; while differ-
ent risk events cause varying degrees of impacts. Therefore,
software organizations manage the impact of risk events based
on their capacity of risk tolerance (Khamooshi and Cioffi,
2009; Touran, 2003). Some risk events are highly likely
to occur; whereas, certain risk events are equally likely to
occur; while, others are low probability risk events. Uzzafer
(submitted for publication) proposed three categories of risk
events for software projects namely: nominal-case, worst-case,
and extreme-case risk events.
Software Engineering Institute (SEI) established a list of
potential risk events of software development projects, shown in
Table 1, which is used for the identification of risk during the
risk assessment process (Carr et al., 1993; Higuera and Haimes,
1996; Williams et al., 1999). The risk events are grouped into
three classes which are further sub-divided into elements and
attributes. The risk assessment process identifies the risk events
and establishes the impacts and probabilities of risk events using
different techniques. Therefore, any risk event of a software
development project could be in any of the risk category based
on its impact and probability.
2.3.1.1. Nominal-case risk events. Nominal-case risk events are
usually known and are highly likely to occur during the
development lifecycle of software projects (Bannerman, 2008).
Hence, the cost impact of nominal-case risk events is represented
within the expected cost (EX½)ofsoftwareprojects.
2.3.1.2. Worst-case risk events. Worst-case risk events are
equally likely to occur; these risk events result in monetary
consequences that are within the risk tolerance. Therefore, the
cost impacts of such risk events are mapped to random values of
costs which are between the expected cost and the risk tolerance.
2.3.1.3. Extreme-case risk events. Extreme-case risk events are
low probability events with large amounts of adverse cost impacts.
Extreme-case risk events cause cost impacts that push the estimated
cost beyond the risk tolerance and results in unbounded values of
cost.
2.3.2. Software risk measure
Uzzafer (submitted for publication) presented a model to
measure the risk of software projects that relies on the
probabilistic (parametric, non-parametric) representation of
cost. The software risk measure model estimates the expectation
of maximum cost impacts due to ϑpercent worst-case risk
events nearer to risk tolerance. For a parametric model of cost
(X), the cost arising from ϑpercent worst-case risk events is
defined to be between the x
ε
(100εth percentile of X)andx
a
(100ath percentile of X), such that ε=(a−ϑ). Fig. 1 illustrates
the map of ϑpercent worst-case risk events on Xalong with the
mapping of risk categories on to Xwhere Xis represented using a
gamma distribution.
For X, the software risk measure is determined by the following
equation (Uzzafer, submitted for publication):
ρϑXjXb;a½
¼1
a−εðÞ
EXIxb≤X≤xa
fg
Ixε≤X
fg
ð1Þ
where I∙
fg is the indicator function, i.e., Iconditionfg
¼
0;condition¼fasle
1;condition¼true
n.Eq.(1) presents the expectation of random costs
between x
ε
and x
a
.
For a non-parametric model ( Xi), the cost impact of ϑ
percent worst-case risk events is defined to be between the cost
samples XYεand XYa. Therefore, the software risk measure is
the expectation of cost samples between the samples XYεand
Fig. 1. Estimated cost Xwith map of risk events.
984 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
XYa, which is defined as follows (Uzzafer, submitted for
publication):
ρϑXijXYb;Ya
½
¼1
a−εðÞ
hEX
iIXYb≤Xi≤XYa
fg
IXYε≤Xi
fg
hi
þXYbb−PXi≤XYb
−XYaPXi≤XYa
fg−aðÞÞ
þXYεε−PXi≤XYε
fg
ðÞ
ið2Þ
Fig. 2 illustrates the samples Xi; in-addition it shows the
samples XYb,XYaand XYεalong with their probabilities, i.e.,
PXi≤XYb
,PXi≤XYa
fgand PXi≤XYε
fg, respectively.
The software risk measure models, ρ
ϑ
(X|X
[b,a]
)and
ρϑXijXYb;Ya
½
, are generic and does not assume any specific
probabilistic (parametric, non-parametric) model of the cost
(Uzzafer, submitted for publication).
3. Software contingency estimation model
3.1. Towards improved contingency estimation
DeMarco and Lister (2003) hypothesize that estimation
models should be based on project management and development
practices. A survey about the use of estimation models indicated
that only 14% of organizations use these models (Heemstra,
1992; Lederer and Prasad, 1992). One reason for such a low
utilization of estimation models in the software industry is that the
traditional models for software development projects are not
generic and are dependent on specific tools and practices(Kellner
et al., 1999). Hence, the adoption of such models requires
adjusting the existing project development tools, methods and
practices which remains a challenge. This lack of generality
deprives software managers of the flexibility to choose the
development processes and methods that best fit their needs.
Furthermore, some traditional models produce estimates that
are not well correlated with the cost and risk of software projects.
In addition, no guidelines have been proposed to scale these
models to a portfolio of software projects and the sub-additivity
of the estimated contingencies remains questionable.
3.2. Contingency estimations using software risk measure
The proposed contingency estimation model is inspired from a
financial contingency estimation model (CAS, 2007). The losses
of a financial investment are on the left side of the distribution
where the financial investment reduces to lower values. However,
Fig. 3. Probability distribution of cost X.
Fig. 2. Estimated cost Xand cost samples Xi.
985M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
instead of the losses being the low percentiles of the distribution,
the random costs impacts are the high percentiles of the cost
distribution of software projects. The impact of risk events moves
the expected cost towards the higher values of costs. For a
parametric model of X, the software risk measure estimates the
expected cost due to the maximum impact of ϑpercent
worst-case risk events that are nearer the risk tolerance. Therefore,
the contingency estimate is defined as the buffer between the
expected cost of the project and the expected cost due to the
maximum impacts of risk events, which is defined as follows:
Contingency ¼ρϑXX
b;a½
−EX½
ð3Þ
The contingency estimation model, expressed in Eq. (3),
produces estimates of man-months reserves toabate the maximum
impact of ϑpercent risk events within the defined risk tolerance.
For a non-parametric model of Xi, contingency estimation model
in Eq. (3) takes the following form:
Contingency ¼ρϑXiXYb;Ya
½
−EX
i
½
ð4Þ
The proposed contingency estimation model can be easily
adopted with any project development environment, since the
model is generic for the cases of Xand Xi, and thus provides a
flexible option to software managers. The model takes a holistic
approach that enables the correlation of contingency resources
with the estimated cost and risk of software projects. Furthermore,
the validation of the model confirms that the model generates
sub-additive contingency estimates and the model is scalable to a
portfolio of software projects; these properties are discussed in
Sections 4.2 and 4.3, respectively.
Fig. 5. Probability distribution of discrete cost samples, Xi.
Fig. 4. Cumulative probability distribution of cost X.
986 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
3.2.1. Example of a parametric case
When the distribution of cost, X, is specified; for this case, the
values to x
a
,x
b
and ϑ, or alternatively, the values to a,band ϑcan
be assigned. Then the estimation of a,band x
ε
from x
a
,x
b
and ϑor
the estimation of x
a
,x
b
and x
ε
from a,band ϑare straight forward
because any 100 q
th
percentile of Xobeys the relationship q=P
[X≤x
q
]=F
X
(x
q
) and any random value x
q
adheres to x
q
=F
X
−1
(q).
These values are used in Eq. (1) to estimate ρ
ϑ
(X|X
[b,a]
), which is
then used in the contingency estimation model, expressed in
Eq. (3), for the contingency estimation.
Assume that, for a software project, Xis modeled with a
gamma distribution, i.e., X~Γ(k,θ) having expectation of EX½¼
2.5 man-months. Hence, the gamma distribution parameters are
estimated as k=2, θ=1.25 such that EX½¼kθ¼2:5.The
project manager assign values of a=0.75, b=0.25 and ε=a−ϑ=
0.65; then the estimated values of x
a
,x
b
and x
ε
are 3.3658, 1.2016
and 2.7736 man-months, respectively. The software risk measure
model, Eq. (1), then produced ρ
0.1
(X|X
[0.25,0 .75]
)= 3.0559 man-
months. This explains that the software project having expected
cost of 2.5 man-months can experience a cost of 3.0559
man-months due to the impact of worst-case risk events that
are within 10% probability of the risk tolerance of x
a
=3.3658
man-months. Therefore, the contingency for this software
project is, ρ0:1XijXi0:25;0:75½
−EX½¼
3:0559−2:5¼0:5559
man-months. This explains that a contingency reserve of 0.5559
man-months is essential to protect the software project from the
maximum impacts of worst-case risk events within the risk
tolerance. Figs. 3 and 4 shows the probability density and
cumulative probability distribution of X~Γ(k=2,θ= 1.25),
respectivley.
3.2.2. Example of a non-parametric case
When the discrete cost samples, Xi, are specified along with the
associated probabilities; for this discrete case, values are assigned
to a,band ϑand the respective samples are estimated fromXisince
any sample XYqis at Y
q
=max k:FXiXk
ðÞ≤q
. Alternatively, the
values to XYa,XYband XYεare assigned and their associated
probabilities, PXi≤XYa
fg
,PXi≤XYb
and PXi≤XYε
fg
,are
estimated. These values are then used in Eq. (2) to measure the risk
of the software project; the Eq. (4) then produces the contingency
estimate for the software project.
Consider a software project where the project manager has
estimated the cost samples, Xi, along with their probabilities based
on the expert opinion as shown in Fig. 5. The expectation of the
estimated cost samples is 2.5 man-months. Furthermore, the
project manager has estimated the minimum and maximum costs
Table 2
Selected values of b
c
from COCOMO-II.
Wi
Precedentedness 0.00
Development flexibility 0.00
Architecture/Risk resolution 0.84
Team cohesion 0.00
Process maturity 0.00
∏Wi0.84
Table 3
EAF Software cost drivers.
Cost driver Impact
RELY Required software reliability 1.15
DATA Database size 1.09
CPLX Product complexity 1.00
TIME Execution time constraint 1.11
STOR Main storage constraint 1.06
PVOL Platform volatility 0.87
PCON Personnel continuity 0.84
ACAP Analyst capability 1.00
DOCU Documentation to match lifecycle 1.13
PCAP Programmer capability 1.16
AEXP Applications experience 1.10
SITE Multi-site operations 1.17
LTEX Language and tool experience 0.91
TOOL Use of software tools 1.00
SCED Required development schedule 1.10
Product= ∏
i
(cost Drivers)
i
1.8201
Fig. 6. Cumulative distribution of discrete cost samples, Xi.
987M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
for the software projectto be the 25th (b= 0.25) an d 75th (a=0.75)
percentiles of Xi, respectively. The manager wants to abate the
impact of 10% worst-case risk events under 75th percentile, i.e.
ε=a−ϑ=0.75−0.1 = 0.65. Therefore, from the sequence Xi,the
values of XYa¼0:75 ,XYb¼0:25 and XYε¼0:65 are estimated as 3.3635,
1.2048 and 2.6104 man-months, respectively, along with
their respective probabilities which are PXi≤XYa¼0:75
¼0:756,
PXi≤XYε¼0:65
¼0:6540 and PXi≤XYb¼0:25
¼0:2540,respec-
tively. The cumulative probabilities of the discrete samples, Xi,are
shown in Fig. 6. Applying the software risk measure model from
Eq. (2) produces ρ0:1XijXi0:25;0:75½
¼3:0120 man-months.
Therefore, expected cost could reach to 3.0120 man-months due
to the impact of worst-case risk events that are within the 10%
probability of risk tolerance. Therefore, the software contingency
estimate using Eq. (4) is ρ0:1XijXi0:25;0:75½
−EX½¼
3:0120−
2:5¼0:512 man-months. This explains that a contingency of
0.512 man-months is needed to protect this software project from
the maximum impacts of the worst-case risk events within the risk
tolerance of 3.3635 man-months.
4. Model validation
Lindland et al. (1994) proposed that a model should be
feasibly validated, meaning that the validation goals should be
achieved with the specified modeling activities to the point in
which further modeling activities would not be economical.
The goals of the validation of the contingency estimation model
include the fidelity, sub-additivity and the scalability tests.
The goal of the fidelity validation checks whether the model
produces the expected output; the goal of the sub-additivity
validation ensures the sub-additive behavior of the contingency
estimates of the model; while, the goal of the scalability ensures
that the proposed model is scalable to a portfolio of software
projects. In-addition, the validation process ensures that the
Fig. 7. Estimated cost samples, Xi.
Fig. 8. Cumulative probability distribution of Xand estimated percentiles.
988 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
model is tested with different cost estimation techniques, i.e.,
COCOMO-II and bottom-up cost estimations.
4.1. Fidelity test
To achieve the goal of the fidelity test, a software project is
considered where the project manager uses the Constructive
Cost Model II (COCOMO-II) for cost estimation which is
defined as follows (Boehm et al., 2000,2010; Dillibabu and
Krishnaiah, 2005):
E¼acSbcEAF;ð5Þ
where Eis the estimated cost in man-months, Srepresents the
software size in kilo-lines of the delivered source code (KDOC),
EAF is the effort adjustment factor, which is the product
of seventeen software cost drivers, b
c
is the sum of five parameters
(Wi) and defined as bc¼1:01 þ0:01 ∑5
i¼1Wi,anda
c
is a
constant set to 2.94.
The software manager estimated the size of the project to be S=
8.62 KDOC and selected the values of Wiand EAF,whichare
shown in Tables 2 and 3, respectively. The values of Wiresulted
in b
c
= 1.01 + 0.01 × 0.84 = 1.0184. With these values the
COCOMO-II, expressed in Eq. (5),produces E¼48 man-
months of cost.
This estimated cost, E, is mapped to the expectation of
the gamma distribution such that E↦EXeΓk;θðÞ
and the
distribution parameters are estimated as k=2,θ=24 and EX½¼
224 ¼48. The project manager decided to model the cost
impacts of 10% (ϑ=0.1) worst-case risk events within the risk
tolerance. Furthermore, project manager estimated the minimum
(x
b
) and maximum (x
a
) costs to be about 24 man-months and 62.4
man-months, respectively. Using these values (x
a
=62.4, x
b
= 24,
ϑ= 0.1), the estimated values of a,band x
ε
are a= 0.7326, b=
0.2642 and x
ε
=51.34 (ε=a−ϑ=0.7326−0.1= 0.6326). Fig. 7
and 8 shows the probability density and cumulative probability
distribution of X~Γ(k=2,θ= 24), respectively. The software risk
measure model, Eq. (3), then generated ρ
0.1
(X|X
[0.2642, 0.7326]
)=
57.5 man-months. Therefore, the contingency estimate for this
software project is ρ0:1XX
0:2642;0:7326½
−EX
½
=57.5−48 =
9.5 man-months.
The contingency estimation model predicts that 9.5 man-
month of resources are needed to abate the risk of maximum
cost impacts due to 10% worst-case risk events within the risk
tolerance. This contingency estimate is approximately 19% of
the expected cost of 48 man-months of the software project,
which is in agreement with a real software development
experience, as reported by Fairley (1995), hence, the model
produces the expected output and fulfils the goal of the fidelity
validation.
4.2. Sub-additivity test
Consider a software project having estimated cost modeled
as X. The software project has nsoftware development tasks
having estimated costs of X1…Xn. Therefore, contingency
estimates of the software project are sub-additive if the
following expression holds:
ρϑXjXb;a½
−EX½:bρϑX1jX1b;a½
−EX1½þ…
þρϑXn Xn b;a½
−EXn½
ð6Þ
The expression in Eq. (6) is the sub-additive form of the
proposed contingency estimation. The estimated costs of X
could be the cost of a portfolio of software projects and X1…Xn
could be the costs of individual projects within the portfolio. A
similar expression for Xican be expressed as follows:
ρϑXijXYb;Ya
½
−EX½:bρϑX1ijX1Yb;Ya
½
−EX1i
½þ…
þρϑXniXnYb;Ya
½
−EXni
½
ð7Þ
The goal of the sub-additivity test is achieved through bottom-
up cost estimation of a software project. The project consists of
two software development tasks with individual estimated costs of
E1=10 and E2¼15 man-months. Therefore, using the bottom
up cost estimation the overall cost of the software project is the
summationofthecostsofeachtaskoftheproject,i.e. E¼E1þ
E2= 10 +15 =25 man-months. These costs are then mapped to the
expectations of the gamma distributions, such that E1↦EX1½,
E2↦EX2½and E↦EX½,whereX1andX2 represents the cost
distributions of tasks 1, 2, respectively, and Xrepresents cost
distribution of the software project. The distributions parameters
(k,θ) are then estimated as E1eΓ2;5ðÞ,E2eΓ2;7:5ðÞand
EeΓ2;12:5ðÞ. Furthermore, the values of band ϑare set at b=
0.25, ϑ=0.1 and the value of ais varied from 0.65 to 0.99 in steps
such that a= [0.65 0.75 0.85 0.95 0.99].
For this software project, the contingency estimates are
sub-additive when the contingency estimate of the project is less
than the sum of the contingency estimates of each individual task
of the software project; this inequality is expressed in Eq. (8):
ρϑXjXb;a½
−EX½:bρϑX1jX1b;a½
−EX1½
þρϑX2jX2b;a½
−EX2½ ð8Þ
Table 4 shows the contingency resources estimated at
different levels of risk tolerance. For example, at a risk
tolerance of a= 65%, the contingency estimates for task 1 and
task 2 are ρ0:1X1X10:25;0:65½
−EX1½
= 1.22 man-months and
ρ0:1X2X20:25;0:65½
−EX2½
= 3.48 man-months, respectively,
their sum is therefore ρ0:1X1X10:25;0:65½
−
EX1½ Þþ
ρ0:1X2X20:25;0:65½
−EX2½
= 4.70 man-months. The con-
tingency of the software project, however, is estimated to be
Table 4
Contingency estimates for X1, X2 and X.
aερ
ϑ
(X1|X1
[b,a]
)ρ
ϑ
(X2|X2
[b,a]
)ρ
ϑ
(X1|X1
[b,a]
)
+ρ
ϑ
(X2|X2
[b,a]
)
ρ
ϑ
(X|X
[b,a]
)
0.65 0.55 1.22 3.48 4.70 0.41
0.75 0.65 3.19 6.87 10.06 5.85
0.85 0.75 6.34 11.27 17.61 13.12
0.95 0.85 11.41 18.73 30.14 25.14
0.99 0.89 15.51 24.72 40.23 34.89
989M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
ρ0:1XX
0:25;0:65½
−EX½¼
0.41 man-months for the same
tolerance level of 65%. This shows that the contingency
required for the project is less than the combined contingency
required for all the tasks of the project. The contingency
estimates for different risk tolerance levels shows that
contingency estimates are sub-additive. These results prove
that the model generates sub-additive estimates of contingency
resources.
4.3. Scalability test
The test for the scalability of the proposed contingency
estimation model is achieved by constructing a portfolio of ten
software projects having costs of E1…E10, respectively. Using
the bottom-up cost estimation model, the estimated cost of the
portfolio, E, is the summation of the expected costs of all the
software projects in the portfolio, i.e., E¼E1þ…þE10 ¼
3693 man-months. The estimated costs of the portfolio is
mapped to the expectation of a gamma distribution such that
Ep↦EXpeΓkp;θp
. Similarly, the estimated cost of the
projects of portfolio are mapped such that Ej↦EXjeΓkj;θj
,
j∈N, respectively, where jis the project number.
The smallest and the largest of these, in terms of cost, have
estimated costs of 100 man-months and 595.23 man-months,
respectively. Table 5 illustrates the estimated costs and risk
measures of each software project assuming a= 0.75, b= 0.25,
ϑ= 10% and ε=a−ϑ= 0.75–0.1 = 0.65. In-addition, Table 5
shows the contingency estimates for each software project. For
example, the project with an estimated cost of 100 man-months
can experience random costs from x
ε
= 110.88 to x
a
= 133.93
man-months due to the impacts of 10% of worst-case risk
events near the risk tolerance of x
a
, resulting in an expected risk
of 122.61 man-months. Therefore, the contingency is estimated
to be 22.61 man-months. The sum of contingency estimates of
all the software projects is 848.37 man-months.
Furthermore, Table 5 shows that the portfolio has an
estimated cost of 3693 man-months and the risk tolerance of
4962 man-months; therefore, the estimated contingency for the
portfolio is 810.7 man-months. These results reveal that the
Table 5
Contingency of software projects.
Projects Ej(x
ε
)
j
(x
a
)
j
ρ
ϑ=0.1
(X
j
|X
j[b=0.25,a= 0.75]
) Contingency
1 100.00 110.88 133.93 122.61 22.61
2 181.09 200.36 242.41 223.35 42.26
3 240.41 265.84 323.89 294.97 54.55
4 294.67 330.48 399.56 366.11 71.44
5 343.27 381.22 462.28 421.89 78.61
6 397.69 444.62 539.70 492.44 94.74
7 454.73 503.84 612.91 555.61 100.88
8 494.02 551.51 669.58 608.90 114.88
9 591.87 654.02 791.07 722.36 130.48
10 595.23 664.97 806.03 733.12 137.89
∑3693.0 4107.8 4981.4 4541.40 848.37
Portfolio Ep(x
ε=0.65
)
p
(x
a=0.75
)
p
ρ
ϑ=0.1
(X
p
|X
p[b=0.25,a= 0.75]
)
3693.0 4081.7 4962 4503.7 810.7
Table 6
Actual, estimated costs and contingency estimates.
Project no. Actual Estimated (EX½)ρ
ϑ
(X|X
[b,a]
) Contingency= ρϑXX
b;a½
−EX½
Actual−(Contingency + EX½)%
1 446.34 276.06 337.44 61.38 32.27
2 860.64 317.71 388.35 70.64 121.61
3 592.28 857.31 1047.90 190.59 −43.48
4 592.28 499.21 610.20 110.99 −2.94
5 234.7 258.61 316.11 57.5 −25.75
6 230.48 66.74 81.57 14.83 182.55
7 127.1 101.26 123.77 22.51 2.69
8 285.6 141.38 172.81 31.43 65.27
9 72 41.59 50.83 9.24 41.65
10 314 233.53 285.45 51.92 10.00
11 681.48 1542.33 1885.30 342.97 −63.85
12 455.22 1360.09 1662.50 302.41 −72.62
13 495.5 1061.01 1296.90 235.89 -61.79
14 631 544.05 665.01 120.96 −5.11
15 433 304.57 372.29 67.72 16.31
16 499 477.58 583.76 106.18 −14.52
17 128 55.05 67.29 12.24 90.22
18 58 25.1 30.68 5.58 89.05
19 130 48.16 58.86 10.7 120.86
990 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
estimated contingency of the portfolio is less than the combined
contingency of all of the software projects of the portfolio, i.e.,
ρϑXpXpb;a½
−EXp
b∑10
j¼1ρϑXjXjb;a½
−EXj
. The con-
tingency estimation model produces sub-additive estimates of
contingency for the portfolio; hence, the scalability validation
proves that the contingency estimation model is scalable to a
portfolio of software projects.
5. Case-study of a software project
The aim of the case-study is to understand how the proposed
contingency estimation model would perform in a real software
development environment. The proposed contingency estimation
model should produce contingency estimates which are compa-
rable with the actual risk experienced during the development of
real software projects.
5.1. Case design
The data for real software projects are obtained from the study
by Lum et al. (2002). They collected data from 19 software
projects of NASA in an effort to understand the difference
between the estimated costs and actual costs of software projects.
These NASA projects used COCOMO (Boehm, 1981)for
software cost estimation and include the development of a wide
range of software applications under different platforms. The
actual and the estimated costs of the project are shown in Table 6.
Fairley (1995) reported that software development organiza-
tions allocate project resources based on the 70th percentile of
the distribution of cost. Therefore, the software risk measure
is estimated at a risk tolerance of the 75th percentile of the
estimated cost for 10% of the worst-case risk events. Therefore,
the expectation of the cost impacts attributable to risk events
will have approximately 70% probability of the distribution.
The probability of the minimum cost is assumed to be the 25th
percentile of the distribution of the cost.
For this case study, the single-point cost estimates are mapped
to the expectations of gamma distributions for cost modeling; then
contingency resources are estimated using the model expressed in
Eq. (3). In-addition, Table 6 shows the risk measures and
estimated contingencies for all the software projects. In-addition,
Table 6 shows the percentage of difference between the actual cost
and the sum of the estimated cost and contingency.
Furthermore, any project that has an actual cost which is ±
20% (Fairley, 1995) of the sum of the estimated cost and the
contingency is called a successful project and any project that
has an actual cost that is ± 100% of the sum of the estimated
cost and contingency is called an extreme project. Additionally,
the remaining projects are called challenged projects.
5.2. Case-study results
The case-study results in Table 6 shows that the actual cost
of the software project 1 is 446.34 man-months while the sum
of the estimated cost and contingency is about 337.44
man-months. Therefore, the actual cost is about 32% more
than the estimated cost and contingency combined, which
makes this project a challenged project. Similarly, the projects
3, 5, 8, 9, 11, 12, 13, 17, and 18 are challenged projects.
On the other hand, project 4 has an estimated cost and
contingency of 499.21 and 110.99 man-months, respectively;
hence, the actual cost of 592.28 man-months is about 3% less
than the sum of the estimated cost and contingency of 610.02
man-months. This is an example of a successful project;
similarly, the projects 7, 10, 14, 15 and 16 are classified as the
successful projects.
While, Software project 6 has an estimated cost of 66.74 man-
months and the estimated contingency of 14.83 man-months. The
actual cost of this project is 230.48 man-months, which is about
182% more than the estimated cost and contingency; hence this
project is classified as an extreme project. Similarly, the projects 2
and 19 show the extreme deviations of more than 100% and
hence they are extreme projects.
These NASA software projects were completed before 1994
(Madachy et al., 2006). The 1994 chaos report of Standish
Group (Haughey, 2009) showed that the software projects
completed by 1994, 16% of the software were successful, 53%
were challenged and 31% were failed projects. The results of
this case study reveals that of 19 projects, ten projects (53%)
are challenged projects, six projects (31%) are successful
projects and the remaining three projects (16%) are extreme
projects. Therefore, these results are in agreement with real
software development experiences.
6. Conclusions
Contingency estimates are critical for the successful manage-
ment and development of software projects since contingency
resources provide a buffer against software development risk.
Therefore, accurate contingency estimates would help software
managers to avoid undesirable results when managing software
development projects. This paper has presented a contingency
estimation model; the model is generic and independent of the cost
estimation and risk assessment models. Hence, it provides a
flexible choice for software organizations and managers of
software development projects. Furthermore, the model produces
contingency estimates that are correlated with the cost and risk of
software projects and considers the risk tolerance of software
organization for contingency estimations. In addition, the model is
scalable to portfolios of software projects because contingency
reserves are best utilized within a portfolio. The model generates
sub-additive contingency estimates, which is essential for manag-
ing a portfolio of software projects or managing an individual
software project consisting of different development tasks.
The proposed contingency estimation model uses the measure
of risk of software development projects. Although, any model to
measure the risk of software projects can be adopted, the proposed
contingency estimation model is demonstrated with Uzzafer
(submitted for publication) software risk measure model. Future
research work can focus on adopting different risk measuring
techniques for software development projects. For the validation
of the proposed model the estimated cost is modeled with a
991M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
gamma distribution, future research studies can adopt different
probabilistic (parametric, non-parametric) representations of cost
of software projects.
The results of the case-study confirm the Standish group's
report for the software projects completed before 1994. The
Standish group further reported that by 1998 percentage of
successful projects increased to 26, then by 2004 it was 29%
(Galorath, 2008) and by 2009 the success in the software
industry stood at 32% (Haughey, 2009). One factor which
contributed to the increase in the number of successful software
projects is the improvements in the software cost estimation
techniques. For example, COCOMO-II was introduced in the
year 2000 as a successor of COCOMO. The contingency
estimations benefits from these improvements in the cost
estimation techniques for software projects.
Appendix A
A.1
The Xhas a cumulative distribution function, F
X
(∙), such that
F
X
(x)=P[X≤x], where xis a single realization of the random
variable X. The 100qth percentile of the random estimated cost,
X, is defined as q=P[X≤x
q
]=F
X
(x
q
), where q∈(0, 1] and x
q
is
the inverse function of F
X
(x) such that F
X
−1
(q)=x
q
(Papoulis,
1991). The expectation of Xis estimated from EX½¼∫xf XxðÞ,
where f
X
(x) is the probability distribution of X(Papoulis, 1991).
A.2
For discrete samples, Xi, the probability of the kth sample,
FXiXk
ðÞ¼PXi≤Xk
fg
, suggests that Xi≤Xkifandonlyifatleast
kof the nrandom samples of the sequence are less than or equal to
Xi(Montgomery and Runger, 2007; Ross, 2007). Any 100qth
percentile of Xiis defined as the sample where PXi≤Xk
fg
¼q.A
random variable Yq =max k:FXiXk
ðÞ≤q
counts the number of
samples of the sequence that are less than or equal to the 100qth
percentile of Xi(Montgomery and Runger, 2007; Ross, 2007).
Therefore, the sample XYaof the sequence Xiis the sample for
which FXiXk
ðÞ≤a,whereais the probability of the 100 ath
percentile of Xi. Similarly, the samples XYband XYεare the
samples for which FXiXk
ðÞ≤band FXiXk
ðÞ≤ε, respectively,
where band εare the 100 bth and 100 εth percentiles of Xi.The
expectation of samples, Xi, is defined as EX
i
½¼∑kXkfXiXk
ðÞ
(Papoulis, 1991).
References
Acerbi, C., Drik, T., 2003. Expected shortfall: a natural coherent alternative to
value at risk. Economic Notes 31 (2), 379–388.Acerbi, C., Nordio, C.,
Sirtori, C., 2001. Expected shortfall as a tool for financial risk management.
(online: http://ideas.repec.org/p/arx/papers/cond-mat-0102304.html (Accessed
October 2010)).
Alkoffash, M.S., Bawaneh, M.J. Al, Rabea, A.I., 2008. Which software cost
model to choose in a particular project. Journal of Computer Science 4 (7),
606–612.
Bannerman, P., 2008. Risk and risk management in software projects: a
reassessment. The Journal of System and Software 81, 2118–2133.
Bennatan, E.M., 1994. Software Project Management. McGraw-Hill.
Boehm, B.W., 1991. Software risk management: principles and practices. IEEE
Software 8 (1), 32–41.
Boehm, B.W., 1981. Software Engineering Economics. Prentice-Hall, Englewood
Cliffs, New Jersey.
Boehm, B.W., Abts, C., Clark, B., Devnani-Chulani, S., 2010. COCOMO-II
Model Definition Manual, version 1.4. (online: http://sunset.usc.edu/
research/COCOMOII/Docs/modelman.pdf (Accessed October 2010)).
Boehm, B.W., Abts, C., Brown, W.A., Chulani, S., Clark, B.K., Horowitz, E.,
Madachy, R., Reifer, D.J., Steece, B., 2000. Software Cost Estimation with
Cocomo II. Prentice Hall.
Boukendour, S., 2005. Estimating software cost contingency using options
theory. International Conference on Information Technology: Coding and
Computing (ITCC 2005). Las Vegas, Nevada, pp. 825–828.
Briand, L.C., El Emam, K., Bomarius, F., 1998. COBRA: a hybrid method for
software cost estimation, benchmarking and risk assessment. International
Conference on Software Engineering. Kyoto, Japan, pp. 390–399.
Carr, M.J., Konda, S.L., Monarch, I., Walker, C.F., Ulrich, F.C., 1993.
Taxonomy Based Risk Identification, CMU/SEI-93-TR-006. Software
Engineering Institute, Pittsburgh, U.S.A.
Casualty Actuarial Society (CAS), 2007. Dynamic Risk Modeling Handbook.
(online: http://www.casact.org/research/drm/(Accessed October 2010)).
Cooper, D.F., MacDonald, D.H., Chapman, C.B., 1985. Risk analysis of a
construction cost estimate. International Journal of Project Management 3 (3),
141–149.
Cooper, D.F., Grey, S., Raymond, G., Walker, P., 2005. Project Risk
Management Guidelines. John Wiley & Sons; Ltd.
Costa, H.R., Barros, M.O., Travassos, G.H., 2007. Evaluating software project
portfolio risks. Journal of Systems and Software 80 (1), 16–31.
David, V., 2008. Risk Analysis: A Quantitative Guide, 3rd edition. Weily.
DeMarco, T., Lister, T., 2003. Waltzing With Bears: Managing Risk on
Software Projects. Dorset House Publishing Company.
Dillibabu, R., Krishnaiah, K., 2005. Cost estimation of a software product using
COCOMO II. 2000 model—a case study. International Journal of Project
Management 23 (4), 297–307.
Dey, P., Tabucanon, M., Ogunlana, S., 1994. Planning for project control
through risk analysis: a petroleum pipeline-laying project. International
Journal of Project Management 12 (1), 23–33.
Fairley, R., 1995. Risk Management for Software Projects. IEEE Software 11
(3), 57–67.
Galorath, D., 2008. Software project failure costs billions, better estimation &
planning can help. Galorath Incorporated. (Online: http://www.galorath.
com/wp/software-project-failure-costs-billions-better-estimation-planning-
can-help.php (10 October 2011)).
Gillian, A., Robert, A., 2001. Managing software project risk. TASSC. (Online:
http://www.tassc-solutions.com/downloads/SoftwareRisks.pdf (Accessed
October 2010)).
Haughey, D., 2009. Why software projects fail and how to make them succeed.
ProjectSmart. ProjectSmart. (online: http://www.projectsmart.com/articles/
why-software-projects-fail.php (Accessed December 2010)).
Heemstra, F.J., 1992. Software cost estimation. Information and Software
Technology 34 (10), 627–639.
Higuera, R.P., Haimes, Y.Y., 1996. Software Risk Management, CMU/SEI-96-
TR-012. Software Engineering Institute.
Janne, R., Lyytinen, K., 2000. Components of software development risk how
to address them? A project manager survey. IEEE Transactions on Software
Engineering 26 (2), 98–112.
Jantzen, K., Gillian, A., Robert, A., 2006. Estimating the effect of project risks
in software development projects. (online: http://www.tassc-solutions.com/
downloads/EstimatingRisks.pdf (Accessed October 2010)).
Kansala, K., 1997. Integrating risk assessment with cost estimation. IEEE
Software 14 (3), 61–67.
Kellner, M.I., Madachy, R.J., Raffo, D.M., 1999. Software process simulation
modeling: Why? What? How? Journal of System and Software 46, 91–105.
Khamooshi, H., Cioffi, D.F., 2009. Program risk contingency budget planning.
IEEE Transaction on Engineering Management 56 (1), 171–179.
Kitchenham, B., Linkman, S., 1997. Estimates, uncertainty and risk. IEEE
Software 3, 69–74.
992 M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
Kitchenham, B., Pickard, L., Linkman, S., Jones, P., 2005. A framework for
evaluating a software bidding model. Information and Software Technology
47 (11), 747–760.
Kitchenham, B., Pickard, L., Linkman, S., Jones, P.W., 2003. Modeling software
bidding risks. IEEE Transactions on Software Engineering 29 (6), 542–554.
Kumar, R.L., 2002. Managing risks in IT projects: an options perspective.
Information Management 40 (1), 63–74.
Lange, K., 2003. Applied Probability. Springer.
Lederer, A.L., Prasad, J., 1992. Nine management guidelines for better cost
estimating. Communications of the ACM 35 (2), 51–55.
Lindland, O.I., Sindre, G., Solvberg, A., 1994. Understanding quality in
conceptual modeling. IEEE Software 2 (11), 42–49.
Lum, K., Powell, J., Hihn, J., 2002. Validation of spacecraft software cost estimation
models for flight and ground systems. (online: http://trsnew.jpl.nasa.gov/dspace/
bitstream/2014/11968/1/02-0706.pdf (Accessed December 2010)).
Madachy, R., Boehm, B., Wu, D., 2006. Comparison and assessment of cost
models for NASA flight projects. International Forum on COCOMO and
Software Cost Modeling. 8 November 2006.
Menzies, T., Chen, Z., Hihn, J., Lum, K., 2006. Selecting best practices for effort
estimation. IEEE Transactions on Software Engineering 32 (11), 883–895.
Montgomery, D.C., Runger, G.C., 2007. Applied Statistics and Probability for
Engineers. John Wiley & Sons Inc., Hoboken, NJ.
Morgan/Reuters, 1996. J.P.RiskMetrics—Technical Document. Morgan Guaranty
Trust Company, New York.
Papoulis, A., 1991. Probability, Random Variables, and Stochastic Processes,
3rd edition. McGraw Hill Companies.
Pfleeger, S.L., Atlee, J.M., 2006. Software Engineering, Theory and Practice,
3rd edition. Pearson International Edition.
Ross, S.M., 2007. Introduction to Probability Models, 8th edition. Elsevier.
Tom, G., Susannah, F., 1988. Principles of Software Engineering Management.
Addison-Wesley.
Touran, A., 2003. Calculation of contingency in construction projects. IEEE
Transactions on Engineering Management 50 (2), 135–140.
Uzzafer, M., 2010. A pitfall of software estimated cost. Proceedings of IEEE
International Conference on Information Management and Engineering
(ICIME 2010). Chengdu, China, pp. 578–582.
Uzzafer, M., submitted for publication. Measuring risk of software projects.
Williams, R.C., Pandelios, G.J., Behrens, S.G., 1999. Software Risk Evaluation
Method Description, CMU/SEI-99-TR-029. Software Engineering Institute.
Wysocki, R.K., 2006. Effective Software Project Management. Wiley.
Yamai, Y., Toshinao, Y., 2005. Value-at risk versus expected shortfall: a
practical perspective. Journal of Banking and Finance 29, 997–1015.
993M. Uzzafer / International Journal of Project Management 31 (2013) 981–993
