Computer Security Strength & Risk:
A Quantitative Approach
A thesis presented
Stuart Edward Schechter
The Division of Engineering and Applied Sciences
in partial fulﬁllment of the requirements
for the degree of
Doctor of Philosophy
in the subject of
2004 - Stuart Edward Schechter
All rights reserved.
Thesis advisor Author
Michael D. Smith Stuart Edward Schechter
Computer Security Strength & Risk:A Quantitative Approach
When attacking a software system is only as diﬃcult as it is to obtain a vulner-
ability to exploit, the security strength of that system is equivalent to the market
price of such a vulnerability. In this dissertation I show how security strength can be
measured using market means, how these strength measures can be applied to create
models that forecast the security risk facing a system, and how the power of markets
can also be unleashed to increase security strength throughout the software develop-
ment process. In short, I provide the building blocks required for a comprehensive,
quantitative approach to increasing security strength and reducing security risk.
The importance of quantifying security strength and risk continues to grow as indi-
viduals, businesses, and governments become increasingly reliant on software systems.
The security of software deployed to date has suﬀered because these systems are de-
veloped and released without any meaningful measures of security, causing consumers
to be unable to diﬀerentiate stronger software products from weaker ones. Even if we
knew that we could make systems measurably stronger, the lack of accurate security
risk models has blurred our ability to forecast the value to be gained by strengthening
these systems. Without the tools introduced in this dissertation, those of us tasked
with making security decisions have been forced to rely on expert opinion, anecdotal
evidence, and other unproven heuristics.
1 Introduction 1
1.1 Economic approaches to security . . . . . . . . . . . . . . . . . . . . . 3
1.2 Anewapproach.............................. 5
2 What is security? 9
2.1 Threatscenarios.............................. 10
2.2 Safeguards................................. 13
2.3 Expanding and organizing threat scenarios . . . . . . . . . . . . . . . 15
2.3.1 Treesandgraphs ......................... 16
2.3.2 Limitations of threat modelling . . . . . . . . . . . . . . . . . 23
2.4 Do threats and safeguards encompass all security models? . . . . . . 24
2.5 Chaptersummary............................. 26
3 Why measuring security is hard 27
3.1 Securityrisk................................ 27
3.1.1 Annual Loss Expected (ALE) . . . . . . . . . . . . . . . . . . 29
3.1.2 Security savings (S) and beneﬁt (B) .............. 31
3.1.3 Investment return: ROI and IRR . . . . . . . . . . . . . . . . 33
3.1.4 The elusiveness of quantitative models . . . . . . . . . . . . . 35
3.2 Securitystrength ............................. 39
3.3 Chaptersummary............................. 46
4 Measuring the security strength of software 47
4.1 Securitystrength ............................. 47
4.2 Why measure software systems? . . . . . . . . . . . . . . . . . . . . . 51
4.3 Pricing vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4 Precedent for vulnerability discovery rewards . . . . . . . . . . . . . . 56
4.5 Chaptersummary............................. 60
5 Diﬀerentiating software products by security strength 62
5.1 Chaptersummary............................. 66
6 Developing strong software 68
6.1 Desirable properties of markets for defects . . . . . . . . . . . . . . . 69
6.2 Market requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.3 Simplifying Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.4 ApproachingReality ........................... 76
6.4.1 The presence of multiple defects . . . . . . . . . . . . . . . . . 76
6.4.2 Knowledge about others’ costs (part one) . . . . . . . . . . . . 77
6.4.3 The time and cost of searching . . . . . . . . . . . . . . . . . 79
6.4.4 Knowledge about others’ costs (part two) . . . . . . . . . . . . 80
6.4.5 Defect dependencies and learning about the skills of others . . 81
6.4.6 Some defects are more important than others . . . . . . . . . 85
6.5 Adversaries and the one-buyer assumption . . . . . . . . . . . . . . . 86
6.6 Delayed publication of reported defects . . . . . . . . . . . . . . . . . 88
6.7 Applying strength metrics throughout
productdevelopment ........................... 89
6.8 Chaptersummary............................. 91
7 Modelling security risk 92
7.1 An introduction to regression models . . . . . . . . . . . . . . . . . . 93
7.2 Modelling security risk . . . . . . . . . . . . . . . . . . . . . . . . . . 94
7.3 The scenario of home burglary . . . . . . . . . . . . . . . . . . . . . . 96
7.4 Regression models in computer security . . . . . . . . . . . . . . . . . 98
7.4.1 Priorwork............................. 98
7.4.2 A problem of data . . . . . . . . . . . . . . . . . . . . . . . . 100
7.5 Insider threats vs. network attacks . . . . . . . . . . . . . . . . . . . 102
7.6 The growing signiﬁcance of security strength . . . . . . . . . . . . . . 106
7.7 Chaptersummary............................. 109
8 Anticipating new threats 111
8.1 The threat of outside theft . . . . . . . . . . . . . . . . . . . . . . . . 113
8.1.1 Serialthieves ........................... 114
8.1.2 Parallelthieves .......................... 114
8.2 SerialTheft ................................ 115
8.2.1 Homogeneous Targets . . . . . . . . . . . . . . . . . . . . . . 116
8.2.2 UniqueTargets .......................... 119
8.3 ParallelTheft ............................... 124
8.4 Furtherreading .............................. 129
8.5 Chaptersummary............................. 130
9 Conclusion 131
A A brief history of fault, threat, and attack trees 135
The ﬁrst paragraph of this section is a mad-lib. Each number in the list below
describes a term of your choice which should be ﬁlled into the paragraph below it.
Please ﬁll in a term for:
(1) an inanimate object or a synonym for idiot.
(2) the name of a third-world country.
(3) the name of an academic ﬁeld not closely related to Computer Science.
(4) the name of a highly renowned graduate school.
(5) the name of a profession that pays minimum wage.
(6) the name of the place you would least like to live in.
(7) the name of a software company.
Michael D. Smith could successfully advise a (1) with a pre-school
level education from (2) to complete a doctoral degree in the study
of (3) at (4). If it had not been for his support, ﬂex-
ibility, sense of humor, and relaxed attitude I suspect that I would now be employed
as a (5) in (6), or worse might be working in the
standards compliance division of (7).
Glenn Holloway has been like a second advisor to those of us in Mike’s research
group. He has the patience to read every paper we write, the ability to quickly
understand what the paper is about, a knack for ﬁguring out how to best improve
the paper in the time available, and the attention to detail to ﬁnd the typos. His
endless knowledge of the tools of the trade, from L
EX to Visual Studio, has proved
invaluable. I suspect that having Glenn in our group reduces the time-to-graduate
for Mike’s students by at least a semester.
Michael Rabin and H. T. Kung provided invaluable advice from early in my grad-
uate career through the ﬁnal draft of this document. I especially appreciate their
encouragement to attack problems I found interesting even when these problems were
not connected to existing research projects within DEAS.
Marty Loeb was kind enough to read some of the earlier, less polished, thesis
drafts and ﬂy to Boston for the defense. Without the seminal papers he wrote with
Larry Gordon, and those of Ross Anderson, Jean Camp (my unoﬃcial ﬁfth com-
mittee member), Hal Varian, and the other founding organizers of the Workshop on
Economics and Information Security, I would likely still be looking for a dissertation
topic. I might not have discovered this research area if Ron Rivest had not taken the
time to point me in their direction.
I cannot count the number of times Susan Wieczorek, the fairy godmother of
DEAS graduate students, has waved her wand to make bureaucratic tasks and pa-
Much of the background research in risk management was performed over a sum-
mer visitation at the University of California at Berkeley that was kindly arranged
by Doug Tygar and Hal Varian.
For my grandparents and parents,
whose examples I can only aspire to follow,
and for the students of Leverett House,
who I implore not to follow mine.
How secure is a software system? How secure does a system need to be? By how
much can security be improved by putting safeguards into place?
Those of us who work to secure systems ask these questions in order to evaluate the
eﬃcacy of our security eﬀorts. We seek answers that provide measures of how eﬀective
our security eﬀorts have been in reducing risk, or that forecast the reduction in risk
that we expect from further security eﬀorts. Often this means estimating past values,
and forecasting future values, of such security risk metrics as the frequency of security
incidents or the annual cost of these incidents. To make security decisions we must
use these metrics to gauge how the choices we make will inﬂuence the eﬀectiveness of
our security strategies in reducing our security risk.
A general methodology for modelling security risks has proved elusive because the
security of a system is aﬀected not only by our actions, but by the strategic choices
of our adversaries. What’s more, security questions are approached diﬀerently when
the answers are to be presented in terms meaningful to these adversaries.
Chapter 1: Introduction 2
When an adversary asks how secure a system is, his primary concern is most likely
to be either the personal risk to his safety or freedom from attacking the system, or
the diﬃculty he will face in attempting to subvert or bypass the system’s safeguards.
An adversary will perceive a system with an additional safeguard to be more resilient
to attack only if that safeguard interferes with the plan of attack that would be used
by the adversary in the safeguard’s absence. A system’s security becomes stronger as
more time, eﬀort, or other resources are required to subvert it. From an adversary’s
perspectives, this security strength, in combination with the personal risk of the attack
to the adversary’s reputation, safety, or freedom, are the metrics of interest when
evaluating the security of a prospective target. For example, when using a cost-beneﬁt
analysis to evaluate the attractiveness of a potential target system, that system’s
security strength and the personal risk incurred in attacking that system represent
deterrent costs that must be weighed against the beneﬁts of targeting the system.
Because security strength measures the resource cost of breaching a system’s secu-
rity, it is fundamentally an economic measure. Formal methods of Computer Science,
such as information theory, have been used to address questions of security strength
when these questions can be translated into assertions of what can and cannot be
calculated. For example, the one-time pad cipher  and Shamir’s secret shar-
ing algorithm  have been proven to be secure against certain classes of attack,
regardless of the resources available to the attacker. However, the applicability of
computational approaches to security strength is severely limited. Computer systems
and networks, which make up the environment in which security mechanisms are de-
ployed and in which adversaries attack, have become too complex to be addressed
Chapter 1: Introduction 3
using formal proofs of computability.
In this dissertation I overcome these limitations by providing an economic ap-
proach for measuring the security strength of software in units of dollars. Because
this methodology does not rely on computational assumptions, it can be applied in
situations where purely computational methods are not applicable.
1.1 Economic approaches to security
Economic approaches have only recently been introduced into the study of com-
puter security. In his seminal paper, Ross Anderson  asserts that perverse incentives
to create insecure systems have caused as much harm as technological shortcomings.
Both Anderson  and Hal Varian  describe how security failings result when
those parties who are charged with protecting systems are not the parties who must
pay the consequences when security is breached. Examples include ATMs deployed
in Europe, which banks were tasked with securing but for which customers paid the
costs of ATM fraud, and distributed denial of service attacks, in which computers
secured by home users were compromised by hackers and then used to attack other’s
Other researchers have gone beyond using economics to describe problems of in-
formation security and have used economic approaches in formulating solutions. Jean
Camp and Catherine Wolfram  proposed addressing the problem of denial of ser-
vice attacks through governmental issue of vulnerability credits (similar to pollution
credits). In their proposal, owners of systems left vulnerable to network attack are
made to pay for the negative externality that results when these systems are compro-
Chapter 1: Introduction 4
mised and used to attack other systems.
Researchers have also applied economic models to security decision making and
measurement. Lawrence Gordon and Martin Loeb  have used economic models
to bound the amount that a ﬁrm should spend on security as a function of the size of
the potential loss. Kevin Soo Hoo  proposed measuring changes in security risk
by extending metrics used to gauge the absolute level of security risk. By doing so,
these metrics could be used to choose the most eﬀective bundle of safeguards possible
within one’s budget constraint.
At the ﬁrst Workshop on Economics and Information Security, many researchers
argued that the software industry should borrow from the quantitative models of
safety and reliability used in the insurance industry [16, 41, 96]. They reasoned that
if insurance companies can model the safety of a car and driver, or gauge the risk of
the failure of a safety system at a factory, then similar models could be developed to
assess software security risks. They proposed that insurers could then drive security
investment by discounting policies to ﬁrms that implemented safeguards that reduce
risk. While insurers could play an important role in managing security risks if accurate
risk models were available to them, in the past these ﬁrms have relied on actuarial
tables to gauge risk. This approach is acceptable for safety risks for which historical
data can be used to forecast future event likelihoods. However, the actions that cause
security breaches in computing systems are strategic, and the strategies available to
both the adversaries and those who secure systems are in a constant state of ﬂux.
As such, no actuarial table or risk model available today can accurately forecast the
eﬀect of choosing one security strategy over another.
Chapter 1: Introduction 5
1.2 A new approach
Past research has left open the problem of quantitatively measuring the security
strength of software and quantitatively forecasting security risk metrics. If we are to
make good security decisions today we need to be able to estimate the eﬀect each
of these choices will have in reducing our security risk in the future. Because the
security strength of a system deters attack, the inﬂuence of security strength should
be accounted for when forecasting security risks.
Most existing security risk models already take into account the deterrent eﬀect of
the personal risk incurred by an adversary who chooses to attack a system. However,
for networked software, security strength often has a greater eﬀect in deterring attack
than the presence of risks that an adversary must incur to stage an attack. After all,
there is little risk to searching software for vulnerabilities when that software can be
copied to one’s own computer and tested in private. When vulnerabilities are found,
attackers can minimize their personal risk by staging attacks remotely and rerouting
their communications in order to hide their identities.
Measures of security strength have additional uses beyond forecasting security
risk. They can be used to improve processes for making systems stronger, and they
can help consumers diﬀerentiate more secure systems from less secure ones.
This dissertation is structured to attack these problems in a bottom-up manner.
Before we can address problems of measuring and forecasting security metrics we
must ﬁrst formalize how these questions should be posed. The study of security
lacks a common language, and so I begin in Chapter 2 by introducing a lexicon and
conceptual framework for use in addressing questions of security. The terms and
Chapter 1: Introduction 6
approach are not new, but are rather the distillation of language and techniques from
a diverse body of security and risk management literature.
I then detail barriers that have limited the measurement and forecasting of security
risk and strength in Chapter 3. This background material includes relevant history of
security strength from century-old ciphers to public key cryptography. It also covers
risk management, a ﬁeld developed to model reliability and safety risks posed by
natural failures that has recently struggled to adapt to the security risks posed by
In Chapter 4, I introduce a methodology for measuring the security strength of
software systems. I argue that the strength of systems with known vulnerabilities is
negligible, and that the security strength of systems with no known vulnerabilities is
bounded by the market price to obtain a new vulnerability. The metric I propose is
the market price to discover the next new vulnerability, rather than the actual costs
expended to discover a vulnerability, because the value must be measured under
circumstances in which the next new vulnerability has yet to be found.
One immediate application of security strength metrics, their use in diﬀerenti-
ating products, is introduced in Chapter 5. This new approach, which applies the
framework of the previous chapter, is signiﬁcant because the lack of security metrics
has left consumers unable to distinguish more secure products from less secure ones.
When consumers cannot make this distinction, software developers cannot justify
investments in making their products more secure.
In Chapter 6, I integrate market methods for security measurement into the soft-
ware development process so that programs can be built to be more secure from
Chapter 1: Introduction 7
the start. The continuous measurement of security strength during the development
process enables software development ﬁrms to create incentives for developers and
testers to increase the security strength of the ﬁrm’s products. These new techniques
have the potential to transform the software development process so that trade-oﬀs
between security, cost, and time to market can be better understood and managed.
With tools in hand to measure and improve security strength, I show in Chapter 7
how to better model and forecast security risks by employing measures of security
strength. As evidence for the utility of security risk forecasting models, I explain why
these models have been successful outside the realm of computer security, against
threat scenarios where the eﬀect of security strength in deterring adversaries has
What remains is to forecast how the introduction of new threat scenarios, for
which historical data is not available, will aﬀect risk. Anticipating new threats is es-
sential because no analysis of past security incidents can prepare one for the potential
risks posed by future threat scenarios that can result from radical changes in your
adversary’s strategy or technology, and thus fall outside of existing risk models. New
and evolving threats have the potential to aﬀect a variety of systems and a potentially
catastrophic number of installations of these systems. The best we can do to prepare
is to anticipate new threats, evaluate their viability, and do our best to understand
their impact on our security risks without using historical data, as none is available.
Chapter 8 shows one approach to modelling a speciﬁc class of threat scenario: attacks
on large numbers of systems with the intent of ﬁnancial gain.
When combined, the techniques and methods presented in this dissertation can
Chapter 1: Introduction 8
be used to answer the questions with which it began. We can measure how secure
a software system is by determining the market price of a vulnerability. We can
forecast how secure a system needs to be by applying models of security risk that
employ security strength. Similarly, I show that these models can be used to gauge
how much security can be improved by putting safeguards into place.
Thus, the tools I introduce for the measurement improvement of security strength
and the models that I reﬁne to better forecast security risk can be used to create a
comprehensive quantitative approach to security. These quantitative techniques can
be immediately applied to measure and improve the strength of existing software. As
we acquire security strength statistics, we can begin to employ security risk models
to forecast risk even in newly released software.
What is security?
1. The process of identifying events that have the potential to
cause harm (or threat scenarios) and implementing safe-
guards to reduce or eliminate this potential.
2. The safeguards, or countermeasures, created and main-
tained by the security process.
At its simplest, security is the process of protecting against injury or harm. Secu-
rity also describes the safeguards, or countermeasures, put in place by that process.
In computer security, harm implies a loss of desired system properties such as conﬁ-
dentiality, integrity, or availability. The goals of security may be distinguished from
those of reliability in that they focus on preventing injury or harm resulting not only
from random acts of nature, but also from the intentional strategic actions of those
with goals counter to your own.
Used to describe a diverse and ever growing set of processes, the word ‘security’
appears to be condemned to imprecision by the weight of its own generality. If
Chapter 2: What is security? 10
security is, in the words of Bruce Schneier, ‘meaningless out of context’ [95, page
13], can a useful deﬁnition remain relevant in the face of ever changing technological
contexts? To address these matters, I have presented at the top of this chapter
a general deﬁnition of security as the combination of two sub-processes that are
themselves free of references to any speciﬁc technology or application.
In this chapter I will further reﬁne the concepts of threat scenarios and safeguards
on which this deﬁnition of security is built. I will describe existing tools for modelling
threats and discuss their limitations. Finally, I will argue that this deﬁnition is
suﬃciently general for modelling questions of security as they have changed over
2.1 Threat scenarios
1. A series of events through which a natural or intelligent
adversary (or set of adversaries) could cause harm.
2. (In Computer Security)
A series of events through which a natural or intelligent
adversary (or set of adversaries) could use the system in an
unauthorized way to cause harm, such as by compromising
the conﬁdentiality, integrity, or availability of the system’s
Like security, safeguards must be understood within a context. When presented
with a new safeguard the ﬁrst question one is likely to ask is what is it intended
to guard against? This is why the security process, as deﬁned above, begins by
describing chains of events with undesirable consequences that we wish to avert. We
Chapter 2: What is security? 11
call these chains of events threat scenarios.
In the security and risk management literature, the word threat does not refer
to the adversary who may cause harm, but to the chain of events that begin with
the adversary (who may also be known as a ‘threat source’) and end in damage or
harm [42, page 37] [79, page 21]. The word scenario is also used to describe such
chains of events [3, 6, 42, 67]. Threat scenarios, as I will call them1, provide the
context for thinking about security. By establishing a common understanding of the
events that lead to harm in a threat scenario, parties can better agree on what is at
stake and what safeguards may reduce the risk. Exploring threat scenarios by adding
detail not only helps us understand what can go wrong to cause security failures, but
also helps to ask how and why these events may occur and who may be likely to cause
them to occur.
The information security literature divides threat scenarios into three data-centric
categories, based on what desired property of the data is lost: conﬁdentiality, integrity,
or availability. These basic threat scenarios might be written as follows.
•Information is exposed to someone who should not have access to it.
•Information is modiﬁed in a manner contrary to policy.
•Authorized users are prevented from accessing information or resources in a
As we dig deeper into the who, how, and why harm may occur, the descriptions of
these threat scenarios can become much more detailed. Taking the ﬁrst basic threat
1I will occasionally use ‘threat’ alone to describe the most basic threat scenarios in which no
means of attack is speciﬁed.
Chapter 2: What is security? 12
scenario, described above, and detailing how a violation of conﬁdentiality could come
about might result in the description of the following chain of events.
An employee uses a laptop running a packet sniﬀer in order to eaves-
drop on data sent over the local area network connected to his oﬃce. He
or she reads conﬁdential management documents attached to emails as
they are in transit between managers, and then sells these documents to
If the events described above occur, then we say that security has been breached,
or that the threat scenario has been realized. The more detailed the description of
the threat scenario, the better we can can understand the likelihood of a breach, the
potential harm a breach may cause, and what we can do to prevent this scenario from
For example, a ﬁrm with little need for conﬁdentiality might examine the above
scenario and determine that the consequences do not justify additional security mea-
sures. Another organization may surmise that this speciﬁc scenario is already well
guarded against. This may be true if network jacks lead directly to switches, rather
than hubs, such that network cables only carry information that originates or ter-
minates at the machine connected to the data jack.2Another organization might
conclude that the risk posed by this threat scenario can be reduced if managers are
encouraged to share ﬁles using a secure ﬁle system, rather than an insecure email net-
work. All of these conclusions can be made because the threat scenario is described
in adequate detail.
On the other hand, dividing general threat scenarios into many unnecessarily
detailed sub-scenarios may make any form of analysis intractable. We might ask if the
2Note that while the location of the network switch may counter this speciﬁc threat scenario, it
will not safeguard against all other eavesdropping scenarios.
Chapter 2: What is security? 13
threat scenario above need only apply to situations in which an employee eavesdrops
using a laptop and his own network connection, or whether a single threat scenario
should be used to describe all network eavesdropping attacks. Similarly, we may ask
whether we need separate threat scenarios for each possible motive for the theft of
information, or whether we should simply assume the most compelling motivation we
can conceive of.
Containing the potentially exponential explosion of increasingly detailed threat
scenarios is of particular concern as past approaches to security and risk management
have failed due to the multitude of scenarios generated [59, page 7]. As we start with
more general threat descriptions and divide them into more detailed ones, we must
be sure only to make these divisions when the value of the insights gained outweighs
the cost in added complexity. Fortunately, procedures for generating and detailing
threats are well known, and are described in Section 2.3.
A policy, process, algorithm, or other measure used to prevent
or limit the damage from one or more threat scenarios.
Synonyms: countermeasure, control, security measure
Safeguards act to prevent or reduce the damage caused by the realization of one
or more threat scenarios. Safeguards are also often called countermeasures, controls,
or security measures, though the last of these terms will be used sparingly to avoid
Chapter 2: What is security? 14
confusion with the process of measuring (gauging the level of) security. Safeguards
may take any number of forms from physical barriers, to sensors, to software algo-
rithms.3Safeguards needn’t even be objects unto themselves, but may encompass
investments in improving existing policies, procedures, or processes such as design,
development, or quality assurance. Safeguards can be compliments, working more
eﬀectively in combination than when apart, or substitutes.
When adding a safeguard to a system, one must consider that an adversary might
break through the protections that the safeguard oﬀers, or ﬁnd a way to bypass the
safeguard entirely. Thus each safeguard introduced may lead to the introduction of
additional, more detailed, threat scenarios that describe events in which safeguards
are circumvented or penetrated. These new detailed scenarios may in turn lead to
the introduction of new safeguards. Security is often referred to as an ‘arms race’
because of this cycle in which new safeguards and new plans of attack are introduced
to counter each other. The process of adding safeguards, and responding to new threat
scenarios targeting those safeguards, terminates when all remaining unimplemented
safeguards are impractical or uneconomical.
Those charged with securing systems may disregard threat scenarios when the
responsibility for implementing safeguards lies outside the system’s boundaries. For
example, printers are not expected to control access to documents released into their
output trays. Rather than requiring authentication to take place at a printer before
documents are released, the onus to safeguard against printed documents reaching
the wrong hands has been placed upon organizational procedures for locating print-
3Using a single term to encompass organizational procedures and algorithms is not revolutionary.
The use of the word ‘software’ to refer to organizational procedures appears in risk management
texts as early as 1980 [105, page 20].
Chapter 2: What is security? 15
ers and distributing the documents that they print. Deﬁning clear boundaries for
the components of a system and their inputs and outputs, is known as a systems ap-
proach [105, page 11] and its necessity is widely accepted. Well-deﬁned boundaries are
essential for determining which threats a system component (itself a system) should
safeguard against, and which threats can be disregarded and countered elsewhere
within the larger system in which the component will reside. The documentation of
these delineations of responsibility is essential.
When using a systems approach one sees that security is not a single feature
one can buy in a product. Rather, the prodcut development process and products
themselves contain safeguards which, if properly combined with the safeguards of the
systems and organizations that the product is integrated into, may help to counter
threats. Careful speciﬁcation of system boundaries and responsibilities is necessary
to prevent common security failures.
2.3 Expanding and organizing threat scenarios
To choose the right safeguards to protect a system, it is necessary to understand
the threat scenarios faced by that system. The ﬁrst steps in this process are to ﬁnd
all plausible threat scenarios, add detail as necessary, and organize them so that we
can best determine the eﬀect of safeguards.
We begin the threat scenario discovery process from the simplest point possible:
enumerating the most basic threat scenarios faced by the system. These simple
threats scenarios are sometimes just called threats, as they are devoid of most scenario
speciﬁcs such as the means of attack, the motive of the adversary, or the asset targeted.
Chapter 2: What is security? 16
For example, armed robbery and check fraud would be two basic, yet distinct, threats
faced by a bank. These acts are worth distinguishing as separate basic threats as
breaking into a bank requires a very diﬀerent type of adversary, resources, and skills
than passing a false check.
The choice of basic threats is subjective and is speciﬁc to the type of system
being modelled. The occupants of a castle might diﬀerentiate the basic threat of
castle wall penetration from external sieges intended to starve the occupants and
avoid a ﬁght. For an operating system, privilege escalation and denial of service
are examples of basic threats. Basic threats need not be entirely independent. For
example, in operating system security, the scenario of privilege escalation may lead
to the distinctly diﬀerent scenario of denial of service.
2.3.1 Trees and graphs
Once you have identiﬁed basic threat scenarios, it is time to add speciﬁcs. This is
done by taking individual threat scenarios and expanding them into multiple threat
scenarios, distinguished by either the means of attack, motive, type of adversary,
or asset targeted. For example, a castle penetration scenario may be divided into
additional threat scenarios based on means of attack. Castle walls may be penetrated
by digging tunnels underneath them, using ladders or platforms to scale them, through
an open gate, or by destroying the walls and going straight through them. Each time
a threat scenario is augmented with more speciﬁcs a new threat scenario is formed.
A tree may be formed by placing the basic threat scenario at the root, placing the
augmented threat scenarios into its child nodes, and then repeating the process by
Chapter 2: What is security? 17
Figure 2.1: A tree detailing how a castle’s walls may be penetrated by an invading
expanding each of the child nodes. When nodes no longer beneﬁt from the addition
of further detail, the leaves that remain represent detailed, unique, threat scenarios.
Figure 2.1 is an example of such a tree applied to a castle and the threat of
penetration by an enemy army. The basic threat scenario appears as the root node
at the top level of our tree. The four ways we can envision the walls being penetrated
are represented by the second level of the tree, which contains the children of the root
node. If any of these child threat scenarios should be realized then the parent threat
scenario is also said to have been realized.
While we could expand each of these child nodes, I have only expanded the scenario
in which the enemy passes through the castle gate. There are a variety of ways that
the adversary could carry out the gate-penetration scenario. To illustrate this we
expand this node as we expanded the root node. Its children may represent options
available to the enemy army such as (but are not limited to) bribing the gatekeeper,
forcing the gate open, or even emulating the ancient Greeks by hiding soldiers in a
statue of a horse.
Chapter 2: What is security? 18
The relationship between the “hide in gift horse” node and its child scenarios is
diﬀerent than the other parent/child relationships in the tree. In order for the gift
horse scenario to be realized, the enemy must have the means to perform not just
one, but all of the tasks described by the child nodes. If the horse is not brought
inside the walls, or if the soldiers do not escape and open the gate, then all of the
attacker’s other steps will be for naught. Many diﬀerent representations have been
used to indicate this all or nothing requirement, from AND gates (borrowed from the
ﬁeld of computer architecture) to semicircle connectors that link the edges between
the parent and those child nodes that must be realized together. The latter is used
in Figure 2.1, with the addition of an arrow at the end of the semicircle. The arrow
represents a further requirement, that the events in each of the child nodes occur in
the sequential order given by the direction of the arrow. After all, the attack will fail
if the attackers convince the occupants to take the horse and then realize they have
yet to hide soldiers inside.4
A more contemporary example of partially completed threat tree is shown in
Figure 2.2, which shows how an outsider (or nonuser) can obtain root access to an
operating system. For each feature described in the castle example, an analogous
feature is present in this example.
Originally called fault trees , these tree-based threat enumeration techniques
were ﬁrst used to model natural threats for safety analyses. The ﬁrst application of
fault trees, in 1961, was to ensure the safety of missile launch control systems .
Fault trees were quickly adopted by the Nuclear Regulatory Commission  and
4Fans of the movie “Monty Python and the Holy Grail”  may recognize this scenario from
King Arthur’s failed castle siege.
Chapter 2: What is security? 19
Figure 2.2: A tree detailing how an outsider (nonuser) might gain access to a net-
worked operating system.
NASA . They have been applied not only in preventing failures, but also to
analyze the cause of accidents after they have occurred. They were used after the
Apollo 1 disaster, Three Mile Island, and the losses of the spaces shuttles Challenger
and Columbia [64, 12]. As fault tree analysis has been adapted to include adversarial
threat sources for use in security analysis, they have also been called threat trees [4, 62]
or attack trees [93, 94]. In addition to the graphical representation of trees, complex
trees are often represented textually as one would represent the structure of an outline
or a table of contents. Those interested in how these approaches evolved and where
they diﬀer are encouraged to read Appendix A.
Because fault trees evolved from the study of safety where the adversary (nature)
has no sense of motive, the expansion of nodes focused exclusively on the events of
the scenario and not the actors. Events were modelled as resulting from stochastic
processes. Because these systems all faced the same, consistent, natural adversary
it was possible to move and reuse tree branches from system to system. Safeguards
could be added at the very end of the process because their eﬃcacy, in the context of
safety, could be reliably estimated using historical data.
Chapter 2: What is security? 20
One point of contention when using trees to model those threat scenarios that are
posed by intelligent adversaries is where to place safeguards. Some assume the system
being analyzed is static and so all countermeasures are assumed to be existing parts
of the system . Microsoft’s Howard and LeBlanc propose mitigation circles 
be placed below the leaves of the tree, with each circle addressing the threat scenario
described by the leaf above it. They even admonish the reader from focusing on
safeguards during the threat modeling process.
Note that you should not add the mitigation circles during the threat-
modeling process. If you do you’re wasting time coming up with mitiga-
tions; Rather, you should add this extra detail later. [62, page 91]
The problem with placing safeguards below the trees is that many threat scenar-
ios involve attacks on the safeguards themselves. When Howard and LeBlanc place
encryption protocols in mitigation circles at the bottom of their trees, they leave no
room in which to expand the attacks on the encryption algorithm or protocols they
have chosen. As Anderson and Kwok have pointed out, safeguards do not eliminate
threats, but rather transform them into less plausible threats [5, page 246].
Leaving safeguards out of the trees entirely has consequences as well. In Schneier’s
example attack tree [94, Figure 21.1], in which the basic threat scenario is the opening
of a safe, both “pick lock” and “learn combo” appear as potential attacks. The
relevance of each of these nodes, and the threat scenarios they represent, depends on
the choice of a countermeasure (lock) with which to safeguard the door – a key lock
may be picked and a combination may be guessed. If all possible countermeasures are
assumed to be present, the tree will grow unwieldy. If any analysis is to be performed
it would ﬁrst be necessary to remove those scenarios that target countermeasures that
Chapter 2: What is security? 21
are not present in the system.
Instead of trees, Caelli, Longley, and Tickle  use directed graphs that inte-
grate safeguards by representing them as nodes, placed as needed, throughout the
diagram. Safeguard nodes are placed below threat nodes to prevent the threat sce-
narios represented by those nodes from being realized. Additional threat nodes can
then be placed below a safeguard node to represent attacks on that safeguard, and
so the process iterates. The graph terminates at those threat scenario nodes that
do not pose a great enough risk to justify further countermeasures. Safeguards may
be similarly integrated into a tree based approach, though Directed Acyclic Graph
(DAG) representations are more compact. Whereas a single safeguard may counter
a number of threats in a DAG, the safeguard node and all of its children must be
replicated in a tree.5
Regardless of whether scenario diagrams are trees or DAGs, integrating safeguards
throughout the representation can be beneﬁcial for understanding their eﬀect. Placing
safeguards into the diagram makes explicit the modeler’s assumptions about where
safeguards are deployed and what scenarios they are intended to counter. Adding
safeguards into the threat modelling process also increases the chance that threat
scenarios in which the safeguards are attacked or bypassed will be included in the
Figure 2.3 is threat scenario diagram representing the opening of a safe, adapted
from one of Schneier’s attack trees [94, Figure 21.1], in which safeguards have been
added in the form of transparent boxes with rounded corners. The safe being modelled
5DAGs may be converted to a tree by making copies of each child node, and all of its children,
for each of its parents.
Chapter 2: What is security? 22
Figure 2.3: A Directed Acyclic Graph (DAG) representing the threat scenarios that
result in the opening of a safe and the countermeasures used to safeguard against
contains both a combination lock and a key lock, conﬁgured so that both must be
unlocked if the door is to be opened. In addition, the safe door’s hinge is placed
on the inside of the safe to make it harder to remove or destroy. The diagram is a
Directed Acyclic Graph (DAG) and not a tree, as tree nodes cannot have multiple
parents. For example, the two diﬀerent lock safeguards share a single child node that
represents that a key, be it metallic or a combination code, could be obtained from
Safeguard nodes may also have multiple parents. For example, audits that safe-
guard a system by simulating attempts to obtain keys both discourage employees
from giving out their keys and from leaving their keys unguarded. Similarly, a pro-
hibition against granting both the combination and the key to a single employee will
help to counter two threat nodes, as it reduces the likelihood that the carelessness or
corruption of a single employee can be exploited to open both locks.
Chapter 2: What is security? 23
2.3.2 Limitations of threat modelling
No amount of historical research or brainstorming can ensure that all basic threats
will be discovered. New technologies or attack strategies can result in new threat
scenarios, or may uncover threats that had long been present but not discovered. For
example, before the link between disease and germs was established, the threat posed
by poor sanitation could neither be understood nor countered. Before the invention of
the skateboard, noise pollution was not considered a threat when architects designed
concrete steps and ramps. When settling on a set of basic threat scenarios to analyze,
it is important to also look at the rate at which new basic threats have been discovered.
Even when all the basic threats are known, there is no way to ensure that all of
a node’s child scenarios have been discovered. If the scenario of interest has been
modelled with existing threat diagrams you may ﬁnd assurance in knowing these
diagrams have stood the test of time. However, there is always the possibility that
new attacks will appear.
Finally, countermeasures are not always implemented correctly and even those
that are act only to reduce threats, not eliminate them. Keys may be guessed, well
screened employees may be bribed, and components that functioned during a million
consecutive tests may fail the next time they are used.
Despite the admitted imperfections of the security process, the better threats
and safeguards can be understood the better the eﬀectiveness of the process can be
Chapter 2: What is security? 24
2.4 Do threats and safeguards encompass all secu-
There are many reasons why one might be tempted to reject a deﬁnition of a se-
curity process consisting of discovery of threat scenarios and placement of safeguards.
One might wonder if such a model is general enough to reﬂect existing and future
security theory and practice.
One motivation for rejecting the deﬁnition of security in this thesis is the impli-
cation that one cannot reach a state of perfect and complete security. Even if perfect
countermeasures existed for each threat scenario, there is no way to know if threats
haven’t been envisioned and thus left unaddressed. The security process, as I have
described it, also bears an unfortunate resemblance to the oft-criticized ‘discover and
patch’ approach to security, in which the security process encroaches into the period
after a product’s release. One might also ask where decades of progress creating
formal security models, such as those used to prove statements about cryptographic
primitives or network protocols, ﬁt into this framework.
In fact, such formal models have always been based on scenarios. The threat
models used by cryptographers act to separate those scenarios that their algorithms
and protocols protect against from those circumstances that they cannot control.
The guarantees of formal models have always been limited to a set of known threat
scenarios used to construct the models. If new methods of attack are found outside of
those threat scenarios, the system may not be secure despite the assurances of formal
methods. In the words of Dorothy Denning , “Security models and formal methods
do not establish security. Systems are hacked outside the models’ assumptions.”
Chapter 2: What is security? 25
Given that security, especially computer security, is a constantly changing ﬁeld,
one might also ask if the deﬁnition and approach will stand the test of time. While
one cannot anticipate all future events, we can apply this security process to historical
examples to see whether it remains timely.
One might consider the security oﬀered by the safeguards of a castle in protecting
the kings and nobles of medieval times from their enemies.6One can imagine the
threat posed by having to ﬁght an advancing line of enemy knights, and how gates
and strong lower walls would initially limit the number of forces that the defense
would need to face at a time. High upper walls might be added to counter the threat
posed by the archers’ ﬂying arrows. Still more countermeasures, such as moats, were
required as an adversary might tunnel under the castle walls, scale them on a ladder
or belfry (a mobile siege tower), or attempt to destroy the walls by slamming them
with a battering ram or a large projectile. A scenario-based approach would also
remind the king that his castle could not protect him from all possible threats, such
as poisoning by his kitchen staﬀ. He’d need a food taster for that.
Castles even provide an example of how new technologies can render insecure
those systems that had seemed impenetrable when they were designed. Towards the
end of the Middle Ages, the range and size of projectiles ﬁred from tr´ebuchets and
cannons increased. Walls could no longer substitute for the combined safeguards of
a strong army and powerful weapons. Those that believe that we could completely
eliminate the problems of updating (patching) systems if only we improved the design
and testing process may also learn from this example.
6For a historical description of medieval siege tactics and countermeasures, see The Medieval
Fortress by Kaufmann and Jurga .
Chapter 2: What is security? 26
2.5 Chapter summary
In this Chapter, I deﬁned security in terms of threat scenarios and the safeguards
that are put in place to counter them. I showed how graphs, most commonly in the
form of trees, can be used to model the interaction between these threat scenarios and
safeguards. I also addressed the limitations of these approaches to threat modelling.
Finally, I argued that the deﬁnition of security provided in this chapter is generally
applicable, and will be as useful for discussing the security questions of tomorrow as
it is for understanding the problems faced by our ancestors.
Why measuring security is hard
3.1 Security risk
From home users, to ﬁrms, to governmental agencies, those of us who rely on our
systems to be secure would like to be able to forecast the risk to these systems and
the eﬀectiveness of diﬀerent security strategies in reducing these risks.
From the perspective of a business, security is an investment to be measured in
dollars saved as a result of reduced losses from security breaches, or in proﬁts from
new ventures that would be too risky to undertake without investments in security. As
a result, security modelling often falls under the control of a ﬁrm’s risk management
Government also considers information security, and the eﬀectiveness models that
guide decisions, to be a matter best addressed through risk management. The Com-
puter Security Act of 1987  mandates that government agencies should have
security plans “commensurate with the risk and magnitude of . . . harm.” The guide-
lines for such a plan take the form of the Risk Management Guide for Information
Technology Systems  from the National Institute of Standards and Technology
Chapter 3: Why measuring security is hard 28
(NIST) which deﬁnes risk management as “the process of identifying risk, assessing
risk, and taking steps to reduce risk to an acceptable level.” However, these guidelines
lack a quantitative method for modelling risk in order to asses it.
The risk management literature includes safeguards among the responses to the
risk posed by threat scenarios. These responses are avoidance, assumption, limita-
tion, and transference [69, 80, 109]. One avoids an optional risk by choosing not
to participate in the risky activity, sacriﬁcing the opportunity to beneﬁt from that
activity but safeguarding against the risk that the activity could result in a breach.
When risk is unavoidable or one chooses to accept the potential losses rather than
implement additional safeguards, one is said to assume a risk. By buying insurance
one may transfer the risk of loss to another party. If entering a contract presents a
risk, that risk may also be transferred through clauses in the terms of the contract
that assign liability to each party. Finally, one may limit risk by introducing safe-
guards that reduce the likelihood of harmful events or that limit the damage caused
when such events occur.
While the study of computer security has centered around those safeguards that
limit risk, avoidance is now a more commonly accepted security practice. For example,
ﬁrewalls can be seen as techniques for risk avoidance, as they require an organization
to forgo the beneﬁts of open networks, such as ease of access, in order to avoid risk.
The choice not to adopt electronic ballots for public elections is also the result of risk
avoidance, as is the current push to ship software with lesser used features turned oﬀ
From the risk management literature a number of metrics have evolved to measure
Chapter 3: Why measuring security is hard 29
security risks. The remainder of this section will cover a progression of these metrics,
which are summarized in Figure 3.1, and why procedures for quantitatively measuring
them have been evasive.
3.1.1 Annual Loss Expected (ALE)
The most common measure for the risk of a harmful event is Annual Loss Ex-
pected, or ALE, which is the product of the expected yearly rate of occurrence of the
event times the expected loss resulting from each occurrence.
ALE = expected rate of loss ×value of loss
The annual loss expected from all of an organization’s operations would be the
sum of the expected yearly losses that could result from each threat scenario. Unfor-
tunately, determining accurate inputs to the ALE equation is signiﬁcantly harder for
security threats than natural ones [6, 59].
If a risk manager is calculating the ALE of losses due to a natural threat source,
such as an earthquake or degradation (wear and tear) of components, she can fore-
cast the expected rate of future occurrences by using the simplest possible model,
Annual Loss Expected ALE (rate of loss) ×(value of loss)
Savings (reduction in ALE) SALEbaseline −ALEwith new safeguards
Beneﬁt B S + (proﬁt from new ventures)
Return On Investment ROI B
cost of safeguards
Internal Rate of Return IRR C0=Pn
Figure 3.1: Common metrics used by security risk managers
Chapter 3: Why measuring security is hard 30
substituting in the historical rate. This forecast must then be adjusted to account
for recent trends. Models can accurately forecast future events using historical data
only when the statistical relationships on which they are built remain stationary over
time. If the inﬂuence of the independent variables on the dependent variable changes,
the model’s forecast and error estimates will be unreliable. This stationarity require-
ment is diﬃcult to achieve when modelling events caused by strategic adversaries, as
acting counter to known models is often the dominant strategy of these adversaries.
Unlike nature, strategic adversaries learn to attack a system at its weakest point,
improve their skills over time, and thwart attempts to measure their behavior. Even
if adversaries did cooperate, historical data for human threats is lacking [76, page 63].
Between 1988 and 1991, the National Institute of Standards and Technology
(NIST) held four workshops in hopes of improving risk models for information tech-
nology. The dearth of progress is reﬂected in today’s state of the art. In October of
2001, NIST published its Risk Management Guide for Information Technology Sys-
tems , in which it recommended that event likelihoods be estimated into only
three qualitative categories: low, medium, and high. The same three categories are
recommended by the OCTAVE
approach [2, 3] from Carnegie Mellon’s Software
Engineering Institute (SEI) and CERT
Coordination Center. Unlike quantitative
likelihood estimates, these qualitative categories cannot be used to estimate Annual
Loss Expected (ALE) or other quantitative cost metrics. Upon reading the NIST
guide, the reader is left to wonder how this process can meet its requirement of
enabling senior managers to “use the least-cost approach and implement the most
appropriate controls to decrease mission risk.” [109, page 27]
Chapter 3: Why measuring security is hard 31
Gordon and Loeb  use a version of ALE that is modiﬁed for situations in
which at most one loss will occur. Thus the dollar cost of a loss is multiplied by
the likelihood of a loss, rather than the expected frequency of loss used to calculate
ALE. They model the probability that a breach will occur as a function of the dollars
invested in security. In this theoretical work, they then assume that this security
breach probability function is continuously twice diﬀerentiable (no discrete invest-
ments), that the ﬁrst derivative is negative (breach becomes less likely as investment
increases), and that the second derivative is positive (diseconomies of scale1). Their
results show upper bounds for optimal levels of security investment against a loss,
informing the risk manager of the maximum that should be spent on safeguards if all
the assumptions of the model hold. Beyond this result, the technique is not intended
to be applied for quantitative risk management decisions, such as which safeguards
3.1.2 Security savings (S) and beneﬁt (B)
In his doctoral dissertation , Kevin Soo Hoo puts aside the problem of estimat-
ing how secure any system is and focuses on measuring the beneﬁts of investments in
additional safeguards. He contends that the beneﬁts of an investment in safeguards
goes beyond the reductions in expected cost of security breaches (decreased losses).
In addition, he adds the expected proﬁts from new activities that could not have been
proﬁtably undertaken without the added security measures. We call the savings S,
and we call the sum of the savings and this new revenue the total beneﬁt, B, of a
1Diseconomies of scale means that each dollar invested in security provides a smaller fractional
reduction in security breaches than the previous dollar did.
Chapter 3: Why measuring security is hard 32
Soo Hoo uses an ALE-based methodology to calculate security savings. The
amount that can be saved by reducing the rate of successful attacks and damage
per successful attack is calculated as the decrease in annual losses that results.
S= ALEbaseline −ALEwith safeguards
Soo Hoo models the eﬀect of a safeguard as causing a fractional reduction, si, in ALE.
ALEwith safeguard i=si·ALEbaseline where 0 ≤si≤1
To keep things simple, Soo Hoo assumes that a safeguard has the same fractional
reduction on ALE regardless of the other safeguards implemented. That is:
ALEwith safeguards i,j =si·sj·ALEbaseline for all i, j
There are obvious limitations to the applicability of this model. For one, it over-
states the reduction in risk resulting from the use of safeguards that act as substitutes
for each other. For example, turning oﬀ unused local network services when these
services are already inaccessible due to a machine-level (“personal”) ﬁrewall is likely
to have a signiﬁcantly lower impact on risk than if no ﬁrewall were present. However,
in Soo Hoo’s model, removing services will result in equal fractional risk reductions
regardless of whether a ﬁrewall is already in place. The model also fails to capture
the eﬀects of complimentary safeguards. For example, an investment in an intrusion
detection system, which alerts administrators to suspicious network activity, reduces
risk only when paired with a procedure for acting on these alerts.
Chapter 3: Why measuring security is hard 33
As with previous ALE based approaches, Soo Hoo’s leaves open the question of
how to forecast the rate at which loss events will occur and how to forecast the reduc-
tions in these rates that will result from adding safeguards. Instead, his methodology
requires as its input the fractional reduction in security breaches that can be ex-
pected from implementing each of the safeguards under consideration. While Soo
Hoo cites data from the fourth and ﬁfth Computer Security Institute/FBI Computer
Security and Crime Surveys [27, 28], and believes that safeguard eﬀectiveness could
be derived from past incident data, he provides no procedure for producing these
forecasts. In lieu of a methodology and data source, his analyses rely on safeguard
eﬃcacy estimates based on “expert judgements” [59, page 52].
3.1.3 Investment return: ROI and IRR
A metric that is quickly gaining in popularity is return on security investment,
also known as security ROI [15, 21, 65, 48, 60]. Assuming that the annual beneﬁt of
a security investment will be received not only in the ﬁrst year, but in all subsequent
years, security ROI is deﬁned by Blakley  as the amount of this annual beneﬁt
over its cost. The beneﬁt is calculated as it was earlier by Soo Hoo , by adding the
expected cost savings (reduced expected loss) to the new proﬁt expected from ventures
that could not have been proﬁtably undertaken without the additional safeguards.
The cost is assumed to be incurred immediately.
Chapter 3: Why measuring security is hard 34
ROI =beneﬁt of safeguards
cost of safeguards
=(savings from safeguards) + (proﬁt from new ventures)
cost of safeguards
=ALEbaseline −ALEwith safeguards + (proﬁt from new ventures)
cost of safeguards
Gordon and Loeb  advocate that ﬁrms should discard the above ROI formula
and instead use the Internal Rate of Return (IRR, also known as the economic rate
of return) because IRR incorporates discounted cash ﬂows for investments that have
diﬀerent costs and beneﬁts in diﬀerent years. If C0is the initial cost of an investment,
and Ctand Btare the respective costs and beneﬁts in year t, one can solve for the
IRR using the following equation:
(1 + IRR)t
Though they contend that IRR is superior to ROI, Gordon and Loeb also warn
that even the correct rate of return can be used inappropriately. They caution that
rates of return not be used when comparing two investments, as an investment can
have a greater net beneﬁt but lesser rate of return. If enough cash is available to
invest in either of the options, but a ﬁrm can invest in only one (the options are
substitutes or mutually exclusive for other reasons), it would be less proﬁtable to
choose the investment with the higher rate of return over that with the greater net
Rate of return is useful for determining whether a security investment is justiﬁed
given the investor’s cost of capital. Rates of return have been growing in popularity
Chapter 3: Why measuring security is hard 35
because the metric is familiar to CFOs and others who control corporate budgets and
Unfortunately, calculating either of these rates of return requires that one ﬁrst
calculate security beneﬁt (Bor Bt) which in turn requires one to calculate ALE.
Like our other metrics, security ROI and IRR forecasts are only as accurate as the
forecasts of loss event frequencies on which they rely and today these forecasts use
best guesses rather than quantitative models.
3.1.4 The elusiveness of quantitative models
The lack of quantitative studies in computer security is attributed, in part, to a
lack of data on variables that can be shown to inﬂuence security risk.
Information collected by CERT provides data about security incidents, but not
about aspects of the larger environment, such as properties of systems that were not
victims of security incidents. Thus, these data are similar to what criminologists
would call victim studies. Broader information is obtained by the CSI/FBI survey,
but the survey’s questions were not crafted to ﬁt the needs of regression studies ,
especially not the kind that seek to model the eﬀects of diﬀerent system and safeguard
choices. Both the CERT and CSI/FBI data sources suﬀer from reporting omissions,
as many ﬁrms decide not to report security breaches in fear that they might lose
customers if breach reports were to be leaked to the public.
Surprisingly, studies using regression models have been successful in forecasting
security risk outside of software security in domains such as home security, as we
will see in Chapter 7. These studies are successful because they measure the risk to
Chapter 3: Why measuring security is hard 36
systems with a homogenous architecture (homes), with fairly homogenous safeguards
(deadbolts, alarms), from a threat that remains stationary over time (burglary). Most
importantly, we will see that the adversary at the source of the home burglary threat
is one that is deterred by the risks to his person of capture or harm during the attack,
and that the factors that indicate this personal risk are measurable. Many of the
adversaries faced by computer systems attack from a great distance, and are thus
deterred more by the strength of the system than the personal risk resulting from the
attack. Unlike indicators of personal risk, factors that aﬀect the diﬃculty of attacking
a system (or the strength of a system to make attack diﬃcult) have been diﬃcult to
Thus, regression studies in network security that excluded measures of security
strength would suﬀer from omitted variable bias. Because network attackers are often
deterred by the diﬃculty of attacking systems, and not risk, one or more indicators
of security strength must be included as independent variables if a model’s forecasts
are to be accurate enough to be useful.
In lieu of a measure of security strength, the best we can do is to substitute
independent variables that are believed to be correlated with it. For example, in
software, code complexity is often considered to be negatively correlated with se-
curity , though complexity is itself diﬃcult to measure.2If one found enough
independent variables correlated with security strength, such as complexity, security
budget, testing period, the recent rate of vulnerabilities reported, and the version
2Complexity is often measured in terms of code size. This may not be the right measure as an
increase in the size of source code due to comments may reduce, and not increase, the complexity.
An increase in the size of compiled code may be the result of an inclusion of bounds checks, which
would also be expected to make the software more secure.
Chapter 3: Why measuring security is hard 37
number, then one might be able to estimate security strength well enough to then
forecast security risks. Even if these results were at ﬁrst accurate, the moment such
a security strength forecasting methodology was published its accuracy would begin
to quickly degrade. Software vendors would use the least cost approach available to
increase their software’s security strength scores regardless of whether these actions
actually increased the actual strength of the software. Reports of testing budgets
and testing periods would be inﬂated to make testing appear more rigorous and code
would be condensed to make it appear less complex, all without any change to the
strength of the actual product.
One might attempt a regression analysis on a single system conﬁguration in order
to isolate security strength as a constant that need not be measured. If the systems
studied do not change, one might hope that their security strength would also remain
constant. However, security strength changes the moment a new vulnerability in one
of these systems is discovered. If that system were updated to repair the vulnerability
then the strength would change yet again. The approach of using a single set of
systems and versions is doomed because the systems themselves are not stationary.
Thus, historical data becomes unable to predict future security risks should a new
vulnerability be found or should the system be updated. On the other hand, if
we know the strength of a system and the incentive that would lead adversaries to
look for vulnerabilities, we could look to historical data to tell us how often new
vulnerabilities have been found and attacks staged in similar situations in the past.
In the end, there’s no substitute for a direct means to measure security strength.
An alternative approach to calculating the likelihood of a security breach might
Chapter 3: Why measuring security is hard 38
appear to be available in the tools of fault tree analysis, using the structures intro-
duced in Chapter 2. The goal of this analysis is to determine the likelihood of the
basic threat event at the root of the tree. The technique is based on the observation
that for any parent node, the probability that the event will occur is a function of the
probabilities that the child nodes will occur. Assume that the event represented by a
parent has two children, c1and c2, which occur with probabilities P(c1) and P(c2) re-
spectively, either of which trigger the parent event. The laws of probability tell us that
the probability that the parent event will occur is P(c1∨c2) = P(c1) + P(c1)P(c2|c1),
which will simplify to P(c1) + (1 −P(c1)) P(c2) if the two child events are indepen-
dent. If both child events must occur to trigger the parent event then the likelihood
of the parent event is P(c1∧c2) = P(c1)×P(c2|c1), which simpliﬁes to P(c1)×P(c2)
if the child events are independent.
If the parent event has more than two children, the analysis can be performed
iteratively over all the children of the parent so long as the laws of precedence are
respected. In other words:
P(c1∨(c2∧c3)) = P(c1) + P(c1)P(c2∧c3|c1)
=P(c1) + P(c1)P(c2|c1)P(c3|c2∧c1)
One need also account for the possibility that one has not anticipated a threat
that would result in an additional child node. The longer a given portion of a tree
has been in use, the less likely it is that new child nodes will be discovered. One
can use bayesian stochastic models, based on the previous rate of discovery of child
nodes for the node in question, to estimate the likelihood that a new child node will
Chapter 3: Why measuring security is hard 39
be discovered over a given time period. In calculating the likelihood of a security
breach at a given node, one can then combine the likelihood of a breach at known
child nodes with the likelihood of a breach at a previously undiscovered child node.
Unfortunately, this tree-based analysis does not solve the problem of estimating
probabilities, but rather pushes the problem down to the leaf nodes. The likelihood
of a breach at any given node cannot be calculated until all the leaf nodes below it are
calculated. Thus, the analysis does not eliminate the need for regression analysis or
other means of obtaining security risk statistics, but instead enables us to move the
task of calculating event probabilities down to the level of those events represented
by the leaf nodes.
3.2 Security strength
Safeguards have long been measured by how diﬃcult they are to circumvent.
Metrics of security strength attempt to quantify the time, eﬀort, and other resources
that must be consumed in order to bypass a system’s safeguards. Whereas metrics
of security risk are useful from the perspective of those the safeguards are intended
to defend, strength metrics are intended to be viewed when taking the perspective of
the adversary. For example, strength metrics might tell us how a castle’s safeguards
may increase the number of soldiers, equipment, time, or motivation required to lay
a siege with a reasonable expectation of success.
The history of security strength metrics is perhaps best understood through de-
velopment of ciphers and cryptography. A cipher is a set of rules for encoding and
decoding messages so as to ensure their conﬁdentiality and integrity. A good cipher
Chapter 3: Why measuring security is hard 40
should require little eﬀort to encode or decode a message with a key. The security
of a cipher rests on how diﬃcult it is to correctly decode a message if one does not
possess the decryption key. Threat scenarios against ciphers are often diﬀerentiated
by the amount and type of information available to the adversary. For example, a
known text attack is a threat scenario, on a symmetric cipher, in which the adversary
analyzes both the plain-text and enciphered copies of one or more messages in order
to derive the decryption key.
Many have made the mistake of assuming that a complex cipher, that appears
impenetrable to those who use it, will confound all adversaries who try to decode
it. One early example of such a mistake took place in 1586 when Mary Queen of
Scots, imprisoned by her cousin Queen Elizabeth of England, received an oﬀer of
rescue enciphered in a letter.3Rather than accept indeﬁnite imprisonment, Mary
chose to trust the strength of her cipher and risked a reply in the aﬃrmative. This
was an act of treason that could lead to almost certain death – if the letter fell into
the wrong hands and could be deciphered. The risk undertaken by Mary was very
much a function of the strength of the cipher in resisting cryptanalysis by Queen
Elizabeth’s agents. Those agents, empowered with the resources of a ruling monarch,
could dedicate more time and eﬀort to breaking the cipher than Mary’s agents had
been able to. As Mary was a threat to Elizabeth’s thrown, Elizabeth also had the
motivation to expend these resources. In the end, Elizabeth’s codebreakers were able
to decipher the message using a laborious technique known as frequency analysis.
Mary was sent to the gallows.
3The cryptographic history is from “The Code Book”  by Simon Singh, who in turn cites [45,
Chapter 3: Why measuring security is hard 41
Since the mechanisms of a cipher must be known if the cipher is to be fully
examined, a widely tested cipher cannot be a very well kept secret. Because of the
clear danger in relying on poorly tested ciphers, Auguste Kerckhoﬀs proposed in his
1883 book, La cryptographie militaire, that the strength of a cipher should rest only
on keeping the key secret, and not on the secrecy of the design of the cipher or cipher
machine. This became known as Kerckhoﬀs’ law. Even with this advance, ciphers
still came in two strengths – those that had been broken and those that had not.
The only way to test the strength of a cipher was to publish encoded messages and
challenge others to decode them.
In the ﬁrst half of the twentieth century, it was the Germans who failed to learn
from Mary’s example and failed to heed Kerckhoﬀs’ law. In 1926 they introduced
the ﬁrst version of their Enigma cipher and the Enigma machine used to encipher
and decipher messages. The machine and its cipher were complex, and their designs
were a German secret. However, by the 1930s, devices known as a bombes, invented
by Marian Rejewski of Poland, were being used to decipher messages and determine
the key used for transmission . Like Queen Elizabeth, the Polish had ample
motivation to break the cipher and were willing to dedicate more resources than the
Germans who relied on the code to protect their messages. Why, after all, would the
Germans waste resources testing such a seemingly impenetrable machine?
While the Germans increased the complexity of the Enigma machine and its cipher
for World War II, the tremendous resources of the allied forces at England’s Bletchley
Park were able to decipher the German messages for much of the war [58, 101]. The
intelligence obtained from German messages allowed the allies to avoid submarine
Chapter 3: Why measuring security is hard 42
attack, land at Normandy with a minimum of casualties, and frustrate Rommel’s
forces in Africa . While Rommel suspected that his messages were being read
by the allies, his superiors refused to believe that such a complex code could be
broken . The lesson was ﬁnally learned that security is never certain, especially
if your adversary is willing to expend more resources to test the security of your
safeguards than you are.
There remain no unbreakable ciphers outside of the one-time pad, a symmetric
cipher proven by Shannon to be information theoretically secure , but that re-
quires impractically large keys.4Rather, the developers of ﬁxed-length key ciphers
test strength by challenging themselves and others to try to ﬁnd a means of crypt-
analysis that would break the cipher. The problem faced by those who employ ciphers
is how to determine when enough eﬀort has been expended in attempting to break
the cipher before it can be deemed suitable for use. No matter how many resources
are expended in attempts to break a cipher, there is always the possibility that the
enemy can exert greater eﬀort or have better fortune in picking the right strategy to
attack the cipher.
In the 1970s, a loophole was discovered through which the problem of dedicating
testing resources to each new cipher could be bypassed. The loophole emerged with
the advent of public key cryptography, in which the key used to encrypt a message
is public knowledge and a matching private key is used to decrypt messages. As
4One time pads require one bit of randomly generated key material for each bit of data that
might be sent between the communicating parties before they next meet and can exchange new key
material. Thus, while one time pads are information theoretically secure against the scenario in
which an adversary attempts to derive the encryption key, they are vulnerable to attacks that target
key generation, key distribution, or the secure storage of keys.
Chapter 3: Why measuring security is hard 43
ﬁrst publicly described by Whitﬁeld Diﬃe and Martin Hellman 5, such schemes
required the use of a trap door function, f(x), for which it was prohibitively time
consuming or expensive to compute its inverse f−1(x). However, with knowledge of
the secret key (the trap door), a fast inverse function f−1(x) can be constructed.
The ﬁrst such trapdoor function and resulting public key encryption scheme was
the RSA scheme invented by Ron Rivest, Adi Shamir, and Leonard Adleman  in
1978.6The public key contained a composite number that was the product of two
large prime numbers (factors). The private key could be constructed if one knew
the prime factors. If the adversary could factor the composite number in the key, he
could recreate the private key and the cipher would be broken. The inventors of the
RSA scheme could not rule out the possibility that it would be possible to decrypt
the message without knowing the private key and without the ability to factor the
composite in the public key. Still, RSA remains the most commonly used public key
In 1979, Michael Rabin introduced his public key encryption scheme which also
used a composite of two large primes as the public key, and for which the factoriza-
tion of the composite served as the private key . Rabin went further and proved
that if one could build a machine to decrypt randomly chosen messages, one could
use the same machine to factor composites. Thus, if used within these constraints,
breaking Rabin’s cryptosystem was proven to be as hard as factoring the composite
that composed the public key. The problem of eﬃciently factoring composites is one
5Public key cryptography was ﬁrst conceived in 1970 as ‘non-secret encryption’ by James Ellis [38,
39] as part of his classiﬁed research within the British Communications-Electronics Security Group
6Once again, this discovery was preceded by classiﬁed work at the British CESG, this time by
Cliﬀord Cocks [26, 39].
Chapter 3: Why measuring security is hard 44
that has intrigued mathematicians, and later computer scientists, for ages. Thus, the
security strength of Rabin’s encryption scheme had already been subject to decades
of testing by some of the world’s most talented, highly motivated, individuals. Shaﬁ
Goldwasser and Sylvio Micali  would later devise an encryption algorithm that
reduced the security of all encrypted messages (not just randomly generated ones)
to the strength of long studied computational problems. In addition to the factor-
ing problem, they added the assumption that it is diﬃcult to distinguish quadratic
residuosity modulo a composite with unknown factors .
Security measurement beneﬁted from the advent of cryptosystems that were es-
tablished to be as diﬃcult to break as a known computational problem. One could
estimate the amount of time required to break the system with current technology
and knowledge by using computational complexity analysis, or even by simulating a
portion of the algorithm. Thus, one could answer the question of how much money
your adversary would need to spend on equipment and how long he would have to
work in order to break your cryptosystem using existing technology and algorithms.
One could also crudely estimate the eﬀect of advancing technology in reducing the
cost of future attacks on the cryptosystem. Moore’s law, which predicts exponential
decline in computing costs over time, could be used to model the decline in the cost of
computation. Crude models were also created to predict advances in factoring algo-
rithms (critical to many cryptosystems) based on past progress in the search for faster
algorithms. The necessity of these models demonstrates that even strength results
in cryptography, a ﬁeld known for formal and precise results, depend on uncertain
factors such as human innovation in overcoming unsolved problems. Unless scientists
Chapter 3: Why measuring security is hard 45
prove that there exist problems that are signiﬁcantly more expensive to solve than to
verify (such as by proving P6=N P )7, the strength of cryptosystems will continue to
depend, in part, on how well the computational problems on which they are based
have been tested. Regardless, cryptographic models will remain subject to new forms
of attack outside the models on which they were built, as was discovered with the
advent of timing and other side channel attacks.
While cryptography has advanced greatly through the reduction of cryptographic
problems to known computational problems, few other security safeguards can beneﬁt
from this approach. As a result, the progress in the theory of cryptography has far
outpaced that of most other security research. Public key cryptography has also far
outpaced the development of the public key infrastructure required to support it.
Almost a quarter of a century after the invention of RSA, almost all the systems
that use it rely on infrastructure for certifying the identity of public key holders that
experts consider unacceptably weak [40, 55].
Outside of cryptography, estimating the strength of a safeguard still requires ex-
tensive examination. It is a common lament that testing cannot prove the absence of
any security ﬂaws, and that at best it can probabilistically show the expected time
or eﬀort required to ﬁnd the next ﬂaw. However, given that any measure of secu-
rity strength in real-world systems will contain uncertainty, testing can still play an
important role in reducing and measuring this uncertainty.
If the strength of a system depends on how well the system has been tested, how
does one measure it? Some security experts track the security strength of software
7That is to say, proving that there exist problems which take exponentially more time to solve
than it takes to verify that their solution is correct.
Chapter 3: Why measuring security is hard 46
by watching the rate at which ﬂaws are discovered and reported, look at which fea-
tures are included, or examine the reputation and size of the manufacturer. However,
little evidence has been published to support any of these metrics. The problem is
even more complex for systems with human components (organizations) as people
introduce a level of nondeterminism into the system that makes security strength
impossible to measure with the traditional computer science tools of logic and com-
putational complexity alone.
3.3 Chapter summary
In this chapter we reviewed a number of security risk metrics: Annual Loss Ex-
pected (ALE), security savings (S), security beneﬁt (B), and Return On Investment
(ROI and IRR). While the values of these metrics may be calculated after the fact
(ex-post) from historical data, forecasting future values has proved problematic. The
crux of the problem is that the frequency and severity of incidents has been chang-
ing over time, and is not likely to remain stationary given the ﬂux inherent to an
environment driven by advancing technology.
We also reviewed the problems inherent in measuring security strength. In essence,
problems of security strength come down to measuring how hard it is for the adversary
to violate your security requirements. Measuring time, eﬀort, and other resources is
the domain of economics. In the following chapter, I’ll explain how, using economic
principles, markets can be used to to the measure security strength of software sys-
Measuring the security strength of
4.1 Security strength
For any given threat scenario, a system is only as strong as it is diﬃcult for
an adversary to cause the scenario to be realized. Diﬃculty not only implies the
adversary’s eﬀort but also his or her need to obtain other resources, such as time and
equipment. The sum of the resource costs incurred in breaking through safeguards
to realize a threat scenario is called the cost-to-break .
We saw in Chapter 3 that, when the security of a system rests on a well stud-
ied computational problem, cost-to-break is often measured in units of computation.
The amount of computation required to accomplish that adversary’s goals is rarely
known ahead of time, but is instead probabilistic. For example, when guessing a
password or using a randomized factoring algorithm against a public key cryptosys-
tem, there is an extremely small, but non-zero, probability that the adversary will
succeed immediately based on lucky guesswork. Because the amount of computation
required to breach security is probabilistic, it is the expected cost-to-break that must
Chapter 4: Measuring the security strength of software 48
Solving computational problems is only one of the many ways to breach the secu-
rity of real, more complicated, systems, especially those systems composed of people
with motives and ﬂaws of their own. While the cost of equipment, eﬀort, and time
may not be measurable with the same level of formal precision as the cost of compu-
tation, we need not abandon measurement. Measuring the current cost of equipment
may be as simple as shopping around to ﬁnd its market price. Estimating the future
cost of equipment may be nearly as straightforward. The cost of computation has
declined at a rate very close to that predicted by Moore’s law. Like equipment, time
and eﬀort can also be measured in units of dollars or other currency. If the amount of
time and skill required to breach security is known, we can use labor market statistics
to estimate its dollar cost.
One of the diﬃculties in measuring cost-to-break is that it is a function of your
adversaries’ costs, not your own. It’s impossible to research the skill of every potential
adversary, the value they place on their time, and the other resource costs they require
to realize a threat scenario. What’s more, the adversaries themselves are not likely to
know how much time and resources they will need to expend to breach security. This
is especially true if breaching the security of a system requires that the adversary ﬁnd
a vulnerability to exploit.
To understand why even the adversary may not be able to measure his own costs,
assume that ﬁnding a vulnerability is an essential step in breaching the security of
a system. There are a series of tasks the adversary can perform to look for vul-
nerabilities, from inspecting code to writing and executing tests. To maximize his
Chapter 4: Measuring the security strength of software 49
0.2 0.4 0.6 0.8 1
0.2 0.4 0.6 0.8 1
Cost as fraction of reward for reporting vulnerability
Probability vulnerability found
Figure 4.1: A cumulative probability distribution that represents an individual’s be-
liefs of the likelihood that he or she can ﬁnd a vulnerability in a system (the y axis)
for the cost represented by the x axis.
productivity, the adversary will start with the tasks that have the greatest chance of
success in ﬁnding a vulnerability per unit cost. Diminishing expected returns result
because the tasks with the highest expected proﬁtability are executed ﬁrst. We can
see in Figure 4.1 that individuals may perceive cost-to-break not as a single value,
but a cumulative probability distribution. The chance of success in ﬁnding a vulnera-
bility increases with total investment (the ﬁrst derivative is positive), but the chance
of success for each additional dollar invested is smaller than for the previous dollar
(the second derivative is negative).
Economically rational individuals will only perform tasks so long as their expected
return is greater than their expected cost. If an individual believes a vulnerability
is worth rdollars, the cost of task iwill be ci, and the probability that task iwill
result in the discovery of a vulnerability is pi, then he will perform the task only when
ci≤pi·r, or equivalently when ci
pi≤r. In fact, while a risk-neutral individual will
Chapter 4: Measuring the security strength of software 50
continue to perform tasks when ci
pi< r and be indiﬀerent to performing tasks when
pi=r, a risk-averse individual will not search for vulnerabilities when ci
pi> r −²for
some positive measure of risk aversion ².
There is no reason to believe an individual will expend the same amount of re-
sources to ﬁnd a vulnerability as he would if he knew the true cost-to-break. Percep-
tions may be updated after one task is completed and before choosing to start the
next task, though updating estimates of ciand picannot guarantee that they will be
accurate. It is the perceived expectation of the cost-to-break, not the true cost, that
determines how many resources adversaries will be willing to spend to ﬁnd vulnera-
bilities in a system. Thus, a metric of security strength that incorporates perceptions
of cost-to-break may not only be as valid as a measure that includes only true costs,
it may be more valid.
Security is not the only ﬁeld in which perception can often trump reality. The
theory of investment tells us the true value of a ﬁrm is the net present value of the
income stream that the ﬁrm will return to its investors. However, a ﬁrm with a higher
perceived value will be more likely to obtain debt ﬁnancing, make deals, and retain
employees. Such a company will thus be more likely to succeed, and return money
to its investors, than a ﬁrm that is identical except for perceptions. This is why it is
impossible to separate perceptions from true value. The true value of a ﬁrm can never
be known so long as it is a going concern with an uncertain future. The perceived
value of a publicly traded ﬁrm is usually readily available and falls within a tight
range between the market bid and ask price.
It should not be surprising that a market-based approach could also be used
Chapter 4: Measuring the security strength of software 51
for security. The ﬁrst such proposal was presented by Jean Camp and Catherine
Wolfram  in the context of securing individual computers from being used in
denial of service attacks. In their work, insecurity in networked computers is treated
as an externality much like pollution in power plants.
A market approach is also eﬀective for measuring the strength of software .
This will be the topic of the remainder of the chapter.
4.2 Why measure software systems?
Quantifying the security of physical components and systems is more diﬃcult than
doing so for software. No two physical components can ever be shown to be truly
identical. There is always a risk that defects will occur in a replica that did not occur
in the original.
I focus on software systems because they are more suitable for security strength
measurement. The ease with which software systems may be copied increases the ease
of measurement, as well as the value of measurement. Software, like all information
goods, has a negligible copy cost and each copy maintains the exact same qualities as
the original. Verifying that two information goods are identical replicas is also trivial.
After a program has undergone testing or security strength measurement, you can be
assured that your copy is indeed a replica of the software that was measured. The
ability to amortize the costs of measuring security strength over a large number of
copies is essential to making secure software aﬀordable.
It is also possible to verify that a software program hasn’t changed since you last
used it. Unlike mechanical systems or systems with human components, software does
Chapter 4: Measuring the security strength of software 52
not degrade or otherwise change over time. Changes, such as upgrades and patches,
may be controlled by the system’s administrator. While the security strength of a
system changes when new ﬂaws and vulnerabilities arise, the software itself can be
veriﬁed as remaining unchanged.
Measuring software alone implies system boundaries that delineate human com-
ponents as external to the system, or part of the larger system in which the software is
deployed. To make this delineation we must deﬁne how the system interacts with the
outside world, including its users. We must specify what each type of user is and isn’t
allowed to do. The value of deﬁning these boundaries is that we can then measure
security strength of the software system exclusive of the people and external systems
that will diﬀer at each location in which it is installed. Because software systems may
be more formally speciﬁed than organizational systems, the threat scenarios they face
can also be more clearly deﬁned.
The quantity and wide availability of software tools that exploit vulnerabilities
to attack systems, which often appear very soon after vulnerabilities are publicized,
implies that the cost-to-break of systems with known vulnerabilities is quite low. It
is more interesting to estimate the security strength of systems in which there are no
known vulnerabilities than to measure the negligible security strength for systems that
do have known vulnerabilities. In order to realize a threat scenario against a system
with no known vulnerabilities, a new vulnerability must be found and a technique
to exploit the vulnerability to breach security (known simply as an exploit) must be
developed. The cost-to-break is thus dominated by the cost to ﬁnd a vulnerability
and create an exploit. Since primitive exploits are often required in order to prove
Chapter 4: Measuring the security strength of software 53
the existence of a vulnerability, these costs may be considered together as the cost of
ﬁnding a vulnerability.1
An adversary intent on breaching the security of a computer system may search
for a vulnerability himself or he may exploit the labor market to pay someone else
to do it. If we assume the worst, the individual with the lowest cost of ﬁnding and
exploiting a vulnerability is the adversary with the most to gain from this knowledge,
or someone that this adversary trusts enough to do business with.
4.3 Pricing vulnerabilities
The market price to ﬁnd and report a vulnerability, or MPV, estimates the cost of
ﬁnding a vulnerability in a software system as measured by a market in which anyone,
including those we might consider to be our adversaries, can participate. The MPV
bid price is the highest price oﬀered to buy a previously unreported vulnerability. The
MPV ask price is the lowest price at which a seller is asking to report a vulnerability.
Transactions occur when the bid price reaches the ask price. The MPV at all other
times is a range bounded at the bottom by the bid price and at the top by the ask
For any given threat scenario, a ﬁrm can establish an MPV bid price for vulner-
abilities that could be exploited by an adversary to realize that threat. One way
to establish this bid price is to make a public oﬀer to pay a reward to anyone who
1It is possible for the cost of exploiting a vulnerability to exceed the cost of discovering it. An
extreme example of such a vulnerability is a back-door guarded by public key cryptography. That is,
a vulnerability intentionally placed in the code in order to grant access to anyone who knows a secret
key. While such a vulnerability may be easy to ﬁnd, writing an exploit without knowledge of the
secret key requires one to accomplish the monumental task of breaking the public key cryptosystem.
Chapter 4: Measuring the security strength of software 54
reports a new vulnerability of this type. If the reward oﬀer is common knowledge to
those who have the skills to ﬁnd vulnerabilities then the test will lead to one of the
following three states:
1. Someone has accepted the oﬀer and reported a vulnerability.
2. One or more vulnerabilities have been found but not reported.
3. No vulnerabilities have been found.
For sake of example and without loss of generality, consider what happens if the
reporting reward (MPV bid price) is set at
In state (1) we have failed to establish a lower bound on the security strength of
the system. A vulnerability was found and reported for less than
10,000, and there’s
no reason to believe that another vulnerability wouldn’t be found for the same, or
even a lesser, price. The MPV bid price is no longer bounded as the reward has been
claimed. To restore the bound, the vulnerability must be repaired and a new reward
While it is impossible to know that no vulnerabilities have been found, and estab-
lish that one is in state (3) and not state (2), the lower bound on market price of a
vulnerability is the same in both states.
The only individuals who can distinguish state (2) from state (3) are those who
discover one or more vulnerabilities (or believed they could ﬁnd one for the reward
price) but decide not to report them. State (2) may occur when an individual has
a more lucrative use for the vulnerability information. However, by not reporting
the vulnerability and declining the reward one pays an opportunity cost of
Chapter 4: Measuring the security strength of software 55
In other words,
10,000 is sacriﬁced by anyone who could report a vulnerability but
decided not to and thus
10,000 remains a valid lower bound on market price of a
In state (3) the system remains secure despite an oﬀer of
10,000 for a means to
break it. So long as all potential adversaries (and those who would ﬁnd vulnerabilities
on behalf of those adversaries) are aware of the oﬀer,
10,000 is now a lower bound on
the price at which one would ﬁnd and report a vulnerability if oﬀered the opportunity.
Making the existence and amount of the reporting reward common knowledge
among all those who might ﬁnd vulnerabilities may appear to be a tall order. Solutions
include exchanges similar to stock exchanges. After all, while the current asking price
of a share of IBM may not be common knowledge, the means for anyone interested in
transacting in IBM shares to get the price and make a trade is. Just as press releases
and analyst reports give investors the information they need to trade in equities,
similar intelligence sources can guide security testers to ﬁnd the best systems for
them to investigate. In lieu of a full market, reward postings from a small set of
security ﬁrms that administer reward oﬀers may be suﬃcient to establish a market
given the size of the security research community.
Given the low price of personal computers, and their availability to those with the
skills to test software for security vulnerabilities, equipment resource costs for testing
software systems should be negligible in most cases. There will always be exceptions,
such as software that requires a large number of CPUs, extremely high network band-
width, expensive peripherals, or data which the software developer cannot distribute
to testers without violating another party’s copyright.
Chapter 4: Measuring the security strength of software 56
One set of costs of attack that are not accounted for by the market price of a
vulnerability are the personal risks the adversary undertakes by staging the attack.
These personal risks, which may include reputation damage, personal harm, or incar-
ceration, are separate from security strength because they can’t be measured in the
4.4 Precedent for vulnerability discovery rewards
Recent history already contains a number of examples where ﬁrms have oﬀered
rewards to those who ﬁnd means of breaching the security one of the ﬁrm’s products.
One of the longest running and most reputable set of challenges is run by RSA
While providing no formal metrics, RSA Security  contends that one of the
goals of its
10,000 Secret-Key Challenge is to “quantify the security oﬀered by. .. ciphers
with keys of various sizes.” The threat scenario they address is made abundantly clear.
For each contest, the unknown plaintext message is preceded by three
known blocks of text that contain the 24-character phrase “The unknown
message is:”. While the mystery text that follows will clearly be known
to a few employees of RSA Security, the secret key itself used for the
encryption was generated at random and never revealed to the challenge
administrators. The goal of each contest is for participants to recover the
secret randomly-generated key that was used in the encryption.
RSA Security also oﬀers a factoring challenge [84, 86]. The problem of factoring
is chosen because data encrypted with the RSA algorithm may be decrypted by
anyone who can factor the composite contained in the public key. They contend
this challenge is helpful “in choosing suitable key lengths for an appropriate level of
Chapter 4: Measuring the security strength of software 57
security.” Rewards range from
10,000 for factoring a 576-bit composite to
for factoring a 2048-bit composite .
The RSA factoring challenges only address one threat scenario faced by users of
the RSA cryptosystems. There is no proof that the RSA cryptosystem is as diﬃcult
to break as factoring composite keys. Threat scenarios that bypass factoring to break
the cryptosystem are not addressed by the challenge. Threat scenarios in which keys
are not generated randomly, or in which the RSA cipher fails to provide security when
used under certain conditions, are not addressed. For example, Don Coppersmith 
has shown messages encoded multiple times for multiple receivers may be decoded if
the receivers use RSA public keys in which the publicly known exponent is a small
number. This vulnerability cannot be reported for a reward under the rules of the
RSA factoring contest.
To RSA’s credit, the rules for both of their contests are clear and well deﬁned,
and evidence of past pay-outs establish the credibility that the reward oﬀer will be
honored. The publicity these contests have received ensure that a large fraction of
the individuals with the potential to succeed in the challenge know about it.
Unfortunately, there is no centralized clearinghouse for these challenges and so the
costs of discovering the contests, as well as reading a non-standardized set of contest
rules, still pose signiﬁcant barriers to participation.
Finally, the rewards oﬀered by RSA Security, while signiﬁcant, are not likely to
spur a great deal of research. A criminal armed with the ability to factor RSA public
keys could make millions, if not billions, of dollars with that knowledge. The fame,
and security consulting fortunes, destined for anyone who ﬁnds a way to factor 2048-
Chapter 4: Measuring the security strength of software 58
bit composites far outweigh the value of the
Other security challenges have made the ﬂaws of RSA’s contests seem small. Per-
haps the most notorious was run by the Secure Digital Music Initiative (SDMI) to
test the integrity of their music watermarking schemes. The challenge was launched
by the release of an Open Letter to the Digital Community  from SDMI.
The Secure Digita