ArticlePDF Available

Policy Analysis: A Systematic Approach to Supporting Policymaking in the Public Sector

Authors:

Abstract and Figures

This paper describes a systematic process for examining complex public policy choices that has been developed and refined over the past 50 years and is often called policy analysis. Its purpose is to assist policymakers in choosing preferred courses of action by clarifying the problem, outlining the alternative solutions and displaying tradeoffs among their consequences. In most real-world policy situations there are many possible alternatives, many uncertainties, many stakeholders and many consequences of interest. Also, there is usually no single decisionmaker and little chance of obtaining agreement on a single set of preferences among the consequences. As a result, there is no way to identify an optimal solution. Instead, policy analysis uses a variety of tools to develop relevant information and present it to the parties involved in the policymaking process in a manner that helps them come to a decision. It is a problem-oriented approach that does not presume a model structure for assessing the consequences of a policy or ranking the alternatives. The paper provides a brief history of policy analysis, describes the most important elements of the policy analysis process, provides an illustrative example of the use of the approach and suggests directions for future developments that can enrich the approach and increase the chances for successful use of the results. Copyright © 2000 John Wiley & Sons, Ltd.
Content may be subject to copyright.
JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS
J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
Policy Analysis: A Systematic Approach to Supporting
Policymaking in the Public Sector
WARREN E. WALKER
a,b,
*
a
RAND Europe
,
Leiden
,
Netherlands
b
Delft University of Technology
,
Delft
,
Netherlands
ABSTRACT
This paper describes a systematic process for examining complex public policy choices that has been developed and
refined over the past 50 years and is often called policy analysis. Its purpose is to assist policymakers in choosing
preferred courses of action by clarifying the problem, outlining the alternative solutions and displaying tradeoffs
among their consequences. In most real-world policy situations there are many possible alternatives, many
uncertainties, many stakeholders and many consequences of interest. Also, there is usually no single decisionmaker
and little chance of obtaining agreement on a single set of preferences among the consequences. As a result, there
is no way to identify an optimal solution. Instead, policy analysis uses a variety of tools to develop relevant
information and present it to the parties involved in the policymaking process in a manner that helps them come
to a decision. It is a problem-oriented approach that does not presume a model structure for assessing the
consequences of a policy or ranking the alternatives. The paper provides a brief history of policy analysis,
describes the most important elements of the policy analysis process, provides an illustrative example of the use
of the approach and suggests directions for future developments that can enrich the approach and increase the
chances for successful use of the results. Copyright © 2000 John Wiley & Sons, Ltd.
KEY WORDS
: decisionmaking; government; planning; policymaking
1. THE POLICYMAKING PROBLEM
The world is undergoing rapid changes. The fu-
ture is uncertain. Policymakers are faced with
policy alternatives that are often numerous, di-
verse and produce multiple consequences that are
far-reaching yet difficult to anticipate (let alone
predict). Different groups perceive and value dif-
ferent consequences differently. Nevertheless,
public policymakers have a responsibility to de-
velop and implement policies that have the best
chance of contributing to the health, safety and
well-being of their constituencies.
Given this context, policymaking is not easy.
Uncertainties abound. Data are limited. Simply
identifying the key policy issues is a difficult task
and one does not have the luxury of ignoring
certain topics because they are too messy or in-
tractable. However, without analysis, important
policy choices are based on hunches and
guesses sometimes with regrettable results.
Over the past 50 years, policy analysts in the
United States and Europe have developed a sys-
tems-based approach and a set of tools for exam-
ining public policy issues that illuminate the
uncertainties and their implications for policy-
making, that identify tradeoffs among the alterna-
tive policies and that support the policymaking
process.
2. HISTORY OF POLICY ANALYSIS
Policy analysis has its roots in operations re-
search. It evolved from operations research (in the
late 1940s and early 1950s) through systems anal-
ysis (in the late 1950s and early 1960s) to policy
analysis in problem-oriented work for govern-
ments carried out at the RAND Corporation and
other applied research organizations in the 1960s
and 1970s. Miser (1980) and Majone (1985) de-
scribe this evolution. In the beginning, operations
research techniques were applied to problems in
which there were few parameters and a clearly
defined single objective function to be optimized
(e.g. aircraft design and placement of radar instal-
lations). Gradually, the problems being analysed
became broader and the contexts more complex.
* Correspondence to: RAND Europe, Newtonweg 1,
2333 CP Leiden, Netherlands. E-mail: warren@rand.org
Copyright © 2000 John Wiley & Sons, Ltd. Recei6ed
23
December
1999
Accepted
18
May
2000
W.E. WALKER
12
Health, housing, transportation and criminal jus-
tice policies were being analysed. Single objectives
(e.g. cost minimization or single variable perfor-
mance maximization) were replaced by the need
to consider tradeoffs among multiple (and con-
flicting) objectives (e.g. the impacts on health, the
economy and the environment, and the distribu-
tional impacts on different social or economic
groups). Non-quantifiable and subjective consid-
erations had to be considered in the analysis
(Schlesinger, 1967 provided an early discussion of
this issue). Optimization was replaced by satis-
ficing. Simon (1969, pp. 64–65) defined satisficing
to mean finding an acceptable or satisfactory
solution to a problem instead of an optimal solu-
tion. He said that satisficing was necessary be-
cause ‘in the real world we usually do not have a
choice between satisfactory and optimal solutions,
for we only rarely have a method of finding the
optimum’. Uncertainty became a more important
element in the analysis. And the tools (and their
associated disciplines) needed to deal with the
increased breadth and uncertainty expanded from
an initial focus on mathematical modelling to
include surveys, focus groups, scenario develop-
ment and gaming.
The policy analysis process has been applied to
a wide variety of problems. Miser and Quade
(1985, ch. 3) provide examples of some of these,
including
improving blood availability and utilization,
improving fire protection,
protecting an estuary from flooding, and
providing energy for the future.
More generally, the policy analysis approach has
been used in the formulation of policies at the
national level, including national security policies,
transportation policies and water management
policies. Other examples that illustrate the ap-
proach can be found in a variety of publications,
including Drake et al. (1972), House (1982),
Mood (1983, ch. 20) and Pollock et al. (1994).
More recently, RAND Europe has used the ap-
proach in a range of studies, including
an examination policies for improving the
Dutch river dikes (Walker et al., 1994),
an examination of options for reducing the
negative impacts of road freight transport in
the Netherlands (Hillestad et al., 1996),
an examination of policy options for improv-
ing maritime safety in the North Sea (Walker
et al., 1998), and
an examination of infrastructure options for
the Netherlands’ civil aviation system (RAND
Europe, 1997a).
Section 6 uses the last study to illustrate the policy
analysis process.
3. THE CONTEXT FOR POLICY ANALYSIS
Public policy analysis is a rational, systematic
approach to making policy choices in the public
sector. It is a process that generates information
on the consequences that would follow the adop-
tion of various policies. It uses a variety of tools
to develop this information and to present it to
the parties involved in the policymaking process
in a manner that helps them come to a decision. It
is more an art than a science since ‘it draws on
intuition as much as on method’ (Bardach, 1996,
p. 1). And, as Heineman et al. (1990) state: ‘As
long as human dignity and meaning exist as im-
portant values, social science cannot achieve the
rigor of the physical sciences because it is impossi-
ble to separate human beliefs from the context
and process of analysis’. Nevertheless, policy
analysis uses the scientific method. This means
that
the work is open and explicit,
the work is objective and empirically based,
the work is consistent with existing knowledge,
and
the results are verifiable and reproducible.
Its purpose is to assist policymakers in choosing a
course of action from among complex alternatives
under uncertain conditions.
The word ‘assist’ emphasizes that policy analy-
sis is used by policymakers as a decision aid, just
as check lists, advisors and horoscopes can be
used as decision aids. Policy analysis is not meant
to replace the judgment of the policymakers (any
more than an X-ray or a blood test is meant to
replace the judgment of medical doctors). Rather,
the goal is to provide a better basis for the
exercise of that judgment by helping to clarify the
problem, presenting the alternatives and compar-
ing their consequences in terms of the relevant
costs and benefits.
The word ‘complex’ means that the policy being
examined deals with a system that includes peo-
ple, social structures, portions of nature, equip-
ment and organizations; the system being studied
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 13
Two sets of external forces act on the system:
external forces outside the control of the actors in
the policy domain and policy changes. Both sets of
forces are developments outside the system that
can affect the structure of the system (and, hence,
the outcomes of interest to policymakers and
other stakeholders). These developments involve a
great deal of uncertainty. The external forces
themselves are highly uncertain. They include the
economic environment, technology developments
and the preferences and behaviour of people. The
policy changes are not uncertain, but their effects
on the structure of the system are. Typically,
scenarios are the analytical tools that are used to
represent and deal with these uncertainties. Each
scenario is a description of one possible future
state of the system. Scenarios do not forecast
what will happen in the future; rather they indi-
cate what can happen. Also, scenarios do not
include complete descriptions of the future sys-
tem; they include only factors that might strongly
affect the outcomes of interest.
Policies are the set of forces within the control
of the actors in the policy domain that affect the
structure and performance of the system. Loosely
speaking, a policy is a set of actions taken by a
government to control the system, to help solve
problems within it or caused by it, or to help
obtain benefits from it. In speaking about na-
tional policies, the problems and benefits gener-
ally relate to broad national goals — for example,
tradeoffs among national environmental, social
and economic goals. A goal is a generalized,
contains so many variables, feedback loops and
interactions that it is difficult to project the conse-
quences of a policy change. Also, the alternatives
are often numerous, involving mixtures of differ-
ent technologies and management policies and
producing multiple consequences that are difficult
to anticipate, let alone predict.
The word ‘uncertain’ emphasizes that the
choices must be made on the basis of incomplete
knowledge about alternatives that do not yet
physically exist, for a future world that is un-
known and largely unknowable.
Policy analysis is performed in government, at
all levels; in independent policy research institu-
tions, both for-profit and not-for-profit; and in
various consulting firms. It is not a way of solving
a specific problem, but is a general approach to
problem solving. It is not a specific methodology,
but it makes use of a variety of methodologies
(including multicriteria decision analysis) in the
context of a generic framework. Most important,
it is a process, each step of which is critical to the
success of a study and must be linked to the
policymakers, to other stakeholders and to the
policymaking process.
The approach is built around an integral system
description of a policy field (see Figure 1). At the
heart of the system description is a system model
(not necessarily a computer model) that represents
the policy domain. The system model clarifies the
system by (1) defining its boundaries and (2)
defining its structure the elements and the
links, flows and relationships among them.
Figure 1. Elements in the policy analysis approach.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
14
non-quantitative policy objective (e.g. ‘reduce air
pollution’ or ‘ensure traffic safety’). Policy ac-
tions are intended to help meet the goals.
For each policy goal, criteria are used to mea-
sure the degree to which policy actions can help
to reach the goal. These criteria are directly re-
lated to the outcomes produced by the system.
Those system outcomes that are related to the
policy goals and objectives are called outcomes of
interest. Unfortunately, although a policy action
may be designed with a single goal in mind, it
will seldom have an affect on only one outcome
of interest. Policy choices, therefore, depend not
only on measuring the outcomes of interest rela-
tive to the policy goals and objectives, but identi-
fying the preferences of the various stakeholders
and identifying tradeoffs among the outcomes of
interest given these various sets of preferences.
The exploration of the effects of alternative poli-
cies on the full range of the outcomes of interest
under a variety of scenarios and the examination
of tradeoffs among the policies requires a struc-
tured analytical process that supports the policy-
making process.
4. THE STEPS OF POLICY ANALYSIS
The policy analysis process generally involves
performing the same set of logical steps (see, for
example, Walker et al., 1979, p. 70 and Findeisen
and Quade, 1985, p. 123). Most projects include
only a subset of the steps. The steps are not
always performed in the same order and there is
usually feedback among the steps. The steps are
summarized in Figure 2 and briefly described
below.
Step 1. Identify the problem. This step sets the
boundaries for what follows. It involves identify-
ing the questions or issues involved, fixing the
context within which the issues are to be
analysed and the policies will have to function,
clarifying constraints on possible courses of ac-
tion, identifying the people who will be affected
by the policy decision, discovering the major op-
erative factors and deciding on the initial ap-
proach.
Step 2. Identify the objectives of the new policy.
Loosely speaking, a policy is a set of actions
taken to solve a problem. The policymaker has
certain objectives that, if met, would ‘solve’ the
problem. In this step, the policy objectives are
Figure 2. Steps in a policy analysis study.
determined. (Most public policy problems in-
volve multiple objectives, some of which conflict
with others.)
Step 3. Decide on criteria (measures of perfor-
mance and cost) with which to evaluate alternative
policies. Determining the degree to which a pol-
icy meets an objective involves measurement.
This step involves identifying consequences of a
policy that can be estimated (quantitatively or
qualitatively) and that are directly related to the
objectives. It also involves identifying the costs
(negative benefits) that would be produced by a
policy and how they are to be estimated.
Step 4. Select the alternative policies to be eval-
uated. This step specifies the policies whose con-
sequences are to be estimated. It is important to
include as many as stand any chance of being
worthwhile. If a policy is not included in this
step, it will never be examined, so there is no
way of knowing how good it may be. The cur-
rent policy should be included as the ‘base case’
in order to determine how much of an improve-
ment can be expected from the other alternatives.
Step 5. Analyse each alternative. This means
determining the consequences that are likely to
follow if the alternative is actually implemented,
where the consequences are measured in terms of
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 15
the criteria chosen in Step 3. This step usually
involves using a model or models of the system.
This step is usually performed for each of several
possible future worlds (scenarios).
Step 6. Compare the alternatives in terms of
projected costs and effects. This step involves ex-
amining the estimated costs and effects for each of
the scenarios, making tradeoffs among them and
choosing a preferred alternative (which is robust
against the possible futures). If none of the alter-
natives examined so far is good enough to be
implemented (or if new aspects of the problem
have been found, or the analysis has led to new
alternatives), return to Step 4.
Step 7. Implement the chosen alternative. This
step involves obtaining acceptance of the new
procedures (both within and outside the govern-
ment), training people to use them and perform-
ing other tasks to put the policy into effect.
Step 8. Monitor and evaluate the results. This
step is necessary to make sure that the policy is
actually accomplishing its intended objectives. If
it is not, the policy may have to be modified or a
new study performed.
The individual steps in the process are de-
scribed in detail by Findeisen and Quade (1985)
and by Quade (1989, Chapter 4). An example that
illustrates the process is given in Section 6, below.
First, however, some general comments about a
few of the steps.
Steps 1–3 are probably the most important in
the entire process. Together, they can be referred
to as ‘formulating the problem’. The remainder of
the steps can be referred to as ‘solving the prob-
lem’. Russell Ackoff once said: ‘We fail more
often because we solve the wrong problem than
because we get the wrong solution to the right
problem’. This means that a great deal of effort
should be devoted to these three steps. In fact,
some of the projects RAND Europe carries out
deal exclusively with these three steps and the
projects are viewed by the client as being very
successful. Often, however, analysts give these
steps short shrift. Many times, the problem state-
ment given to the analyst is accepted without
question. For example, the problem originally
posed to our project team examining freight trans-
port policy options was to find the best ways to
shift freight off the highways and onto other
modes. However, realizing that this was more of a
solution statement than a problem statement, we
asked the client why freight should be shifted off
the highways. This question revealed that the goal
was really to reduce the negative effects of road
freight transport. But, there are ways of reducing
the negative effects of road freight transport be-
sides shifting it off the highways, such as making
better use of the existing infrastructure and truck
fleet. The goal of the research was, therefore,
widened to include these other possibilities and
the research eventually revealed that the modal
shift options were the least cost-effective ways of
reaching the goal.
In Steps 2 and 3, there is often little effort made
to identify the objectives of the various stakehold-
ers or to identify policy impacts spanning the
concerns of all affected groups. In many cases,
qualitative impacts are ignored and only quantita-
tive impacts are assessed. It is important to realize
that qualitative analysis can be a valid scientific
endeavour and numerical estimates of impacts are
not necessary for making policy decisions. For
example, in our analysis of civil aviation in-
frastructure options, we estimated aircraft noise
and exhaust emissions quantitatively, while we
estimated the effects on safety and natural settings
qualitatively. Failure to take into account the
interests of all stakeholders will often lead to the
results of the study being ignored by policymakers
or attacked by stakeholders.
There are two important rules for carrying out
Step 4. First include the existing situation as the
base case. Second, include as many alternatives as
stand any chance at all of being worthwhile. Do
not exclude an alternative merely because it seems
impractical or runs contrary to past practice. Per-
sonal judgments on such issues should be with-
held. The analysis will show whether the benefits
to be derived outweigh the cost of making such
radical changes or indeed, of making any
changes at all.
Step 5, which usually involves the use of mod-
els, is only one step in the process and generally
not the most important step. Some analysts act as
if the model is more important than the problem
they are trying to solve. But the truth is that
models are merely the tools of the policy analyst,
much as brushes are tools of the artist — they are
a means to an end, not the end in itself. It is easy
for an analyst to become more interested in the
model than in the problem itself. Focusing atten-
tion on the mechanics of the computation or on
the technical relationships in the model may ne-
glect important questions that should be raised in
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
16
the study. Modellers may thus find out a great
deal about inferences that can be drawn from the
model, but very little about the question they are
trying to answer. The policy analyst, however,
must keep his work problem oriented, remember-
ing that his primary job is to solve a problem, not
build a model.
This leads to two important principles of policy
modelling.
1. Fit the model to the problem,not the problem to
the model. Every analyst has his own sets of
tools and techniques that he is likely to want
to apply to a problem. Sometimes these tools
are appropriate and sometimes they are not. It
is not uncommon for an analyst to make
assumptions that will fit the problem to the
tool he wants to use, rather than to search for
the appropriate tool (or develop a new one).
2. Use the simplest model that will do the job. The
analyst must keep in mind that he is going to
have to explain his results and methodology to
a policymaker who will generally not be famil-
iar with advanced mathematics. The simpler
the model the easier it will be to explain and
the better the chance that the policymaker will
understand the analysis. As Quade points out:
‘The most convincing analysis is one that a
nontechnician can think through’ (Quade,
1989, pp. 362–363).
All of the steps of a policy analysis study should
be carried out in close cooperation with the rele-
vant policymakers and should be connected to the
policymaking process. In fact, a policy analysis
study should be a partnership between analysts
and policymakers. This is often not the case in
practice. In many policy analysis studies the ana-
lyst or consulting firm performs the first six steps
on their own — perhaps interacting with the poli-
cymakers at a monthly or quarterly steering
group meeting. But, in many policy analysis stud-
ies, the results do not get used. The studies may
have been well designed, the models elegant and
the reports produced impressive, but from a pol-
icy point of view, the study is not a success unless
the results are used by the policymakers.
There are many reasons why the results do not
get used. But one of the major reasons is that the
policymaker(s) did not understand how the results
were obtained, or did not agree with the way one
or more of the steps were carried out. I believe
that the single most important factor in determin-
ing whether or not the results of a study are used
is the relationship between the analyst(s) and the
policymaker(s). A policy analysis study should be
a joint effort of the analyst(s) and the policymak-
er(s) a partnership in the true sense of the
word. In this partnership there should be a clear
division of responsibility and differentiation of
roles. The analyst(s) should do the data analysis
and the modelling and should present information
to the policymaker(s) in a manner that facilitates
the evaluation of the alternative policies and the
choice of the one to be implemented. The policy-
maker(s) should play a major role in defining the
objectives, identifying the constraints on feasible
solutions, choosing the policy to be implemented
and supporting the implementation effort.
In most of the RAND Europe policy analysis
studies previously cited, the work ended with the
presentation to policymakers of the effects of each
of the policy options. That is, we carried out only
Steps 1–5. Remember that the policy analyst’s job
is to provide decision support to policymakers,
not to make the decisions. By carefully and con-
scientiously carrying out Steps 1–5, we supply
much of the information that is needed by the
interested parties so that a good policy choice can
be made.
5. PRESENTATION OF RESULTS
Once the impacts have been assessed, a major
difficulty still remains: synthesizing the numerous
and diverse impacts and presenting the results in a
way that facilitates the comparison and ranking
of the tactics. Many approaches have been devel-
oped for this purpose. Most of these are aggregate
approaches. In an aggregate approach, each im-
pact is weighted by its relative importance and
combined into some single, commensurate unit
such as money, worth, or utility. Decisionmakers
then use this aggregate measure to compare
alternatives.
However, there are drawbacks to using an ag-
gregate approach. First, the aggregation process
loses considerable information: For example, it
suppresses the fact that Policy A has environmen-
tal problems whereas Policy B has financial prob-
lems and Policy C has safety problems.
Second, any single measure of worth depends
strongly on the weights given to the different
impacts when they were combined and the as-
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 17
sumptions used to get them into commensurate
units. Unfortunately, these crucial weights and
assumptions are often implicit or highly specula-
tive. They may impose on the decisionmakers a
value scheme bearing little relation to their con-
cerns. For example, traditional cost-benefit
analyses implicitly assume that a euro’s worth of
one kind of benefit has the same value as a
euro’s worth of another; yet in many public de-
cisions, monetarily equivalent but otherwise dis-
similar benefits would be valued differently by
society. Also, in converting disparate impacts
into monetary values, speculative assumptions
must sometimes be made, such as: What is the
value of a person’s life? How much money is
one dead bird worth? Are a million dead birds
worth a million times one dead bird?
Third, the aggregate techniques are intended
to help an individual decisionmaker choose a
single preferred alternative the one that best
reflects his/her values (importance weights). Seri-
ous practical and theoretical problems arise
when there are multiple stakeholders and even
multiple decisionmakers. The practical problems
include the need to answer the following ques-
tions: Whose values get used (the issue of inter-
personal comparison of values) and what
relative weight does the group give to the pref-
erences of different individuals (the issue of eq-
uity)? The theoretical problem associated with
these questions is that it has been proved that
there is no rational procedure for combining in-
dividual rankings into a group ranking that
does not explicitly include interpersonal com-
parison of preferences. To make this comparison
and to address the issue of equity, full consider-
ation of the original impacts appears essential.
We generally use a disaggregate approach in
which the impacts of tactics are presented in the
form of tables that Hammond et al. (1999) call
consequences tables and that I call scorecards.
Each column of a scorecard represents an im-
pact and each row represents a policy option.
An entire row shows all of the impacts of a
single option; an entire column shows each op-
tion’s value for a single impact. Numbers or
words appear in each cell of the scorecard to
convey whatever is known about the size and
direction of the impact in absolute terms, i.e.
without comparison between cells. In comparing
the tactics, each stakeholder and decisionmaker
can assign whatever weight he/she deems appro-
priate to each impact. Explicit consideration of
weighting thus becomes central to the decision
process itself, as it should be. Prior analysis can
consider the full range of possible impacts, using
the most natural description for each impact.
Therefore, some effects can be described in
monetary terms and others in physical units;
some can be assessed with quantitative estimates
(e.g. air pollutant emissions) and others with
qualitative comparisons (e.g. ‘the stakeholder ac-
ceptability for this tactic is high’). A notional
scorecard is shown in Figure 3.
A disadvantage of this approach is that the
amount of detail can make it difficult for the
decisionmakers to see patterns or draw conclu-
sions. To aid decisionmakers in recognizing pat-
terns and trading off disparate impacts
colouring of the boxes of the scorecard can be
used, e.g. blue for best policy option for this
impact, yellow for intermediate and red for
worst. This shows the ranking of the impact
values across rows, for each column indepen-
dently of all other columns. Tradeoffs among
impacts can be made by making comparisons
between columns. Another decision aid is to
prepare a summary scorecard for each impact
category (e.g. a safety scorecard, an environment
scorecard, an economy scorecard, etc.).
The scorecard approach has several advan-
tages. It makes it possible to present a wide
range of impacts and permits each stakeholder
and decisionmaker to give each one whatever
weight he/she deems appropriate. Applying some
method where impacts are weighted requires ad-
ditional calculations or the use of specific pro-
grams. It helps them to see the comparative
strengths and weaknesses of the various policy
options, to consider impacts that cannot be ex-
pressed in numerical terms and to change their
set of weights and note the effect that this
would have on their final choices. In the case of
multiple decisionmakers, the scorecard has the
additional advantage of not requiring explicit
agreement on weights for different social values.
It is generally much easier for a group of deci-
sionmakers to determine which alternative they
prefer (perhaps for different reasons) than what
weights to assign to the various impacts. A
more complete discussion of scorecards can be
found in Goeller et al. (1977, pp. 10–13); Miser
and Quade (1985, pp. 89–l08) also illustrate
their use.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
18
Figure 3. A notional scorecard.
6. A POLICY ANALYSIS CASE STUDY:
THE TNLI PROJECT
6.1. Introduction
The TNLI (Toekomstige Nederlandse Lucht6aart
Infrastructuur or Future of Dutch Civil Aviation
Infrastructure) project was a broad policy study
focused on ways of coping with the projected
growth in air transport demands in the Nether-
lands. The project was carried out during the
period August 1995–October 1996 by RAND
Europe for three Dutch ministries: the Ministry of
Transport, Public Works and Water Manage-
ment, the Ministry of Housing, Spatial Planning
and Environment and the Ministry of Economic
Affairs. With the support and active participation
of staff members of these ministries, a set of
policy analysis tools was designed and built to
assess the impacts of the various policy options
and identify promising options to help achieve the
policy goals.
The following subsection describes the problem
addressed and the overall approach of the project.
Subsections after that describe the scenarios that
were developed, the criteria that were used for
assessing the effects of the infrastructure options,
the impact assessment models that were used to
assess the impacts of the options, the options
themselves and the results of the impact assess-
ment. For a complete description of the project
and its results, see RAND Europe (1997a).
6.2. Problem statement, context and overview of
project
In 1996, the Dutch Government was in the midst
of a reassessment of its policies related to civil
aviation caused in large part by the fact that the
growth in commercial civil aviation traffic in the
Netherlands had exceeded expectations. One of
the inputs to the reassessment was an analysis
that investigated possible future developments of
demand and capacity of air transport, alternative
infrastructure options that could be implemented
to meet these future situations and societal de-
mands and expectations that would influence the
choice among those alternatives.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 19
Air transport benefits the Dutch economy in
many ways. Approximately 70000 people are di-
rectly employed by airports and airlines in the
Netherlands. The availability of a wide range of
high quality air services in the country also helps
create a favorable investment climate and pro-
motes international trade. Nevertheless, the direct
and indirect environmental effects of air transport
at both the local and global levels are increasingly
becoming problems that need to be addressed. In
addition, regions close to airports suffer other
negative effects, such as surface traffic congestion
and aircraft noise. These negative effects tend to
increase as the air traffic increases. These facts,
together with discussions with members of the
client organizations, resulted in the following
statement of the question to be addressed by the
TNLI project.
The demand for infrastructure for ci6il a6ia-
tion transport in the Netherlands may continue
to increase.Acti6ities related to ci6il a6iation
ha6e social,economic,safety,en6ironmental,
spatial,accessibility,and cost consequences.
The question the nation must answer is
whether or not to accommodate the demand in
light of these consequences,and,if so,how.
The commercial aviation system of the Nether-
lands can be divided into two parts: the supply
system and the demand system. The main ele-
ments of the supply system are the airlines, the air
traffic control system and the national and re-
gional airports. The infrastructure options
analysed all related to changes in the supply of
airport capacity. The demands considered in the
analysis are passenger and cargo demands for
transport to, from, or through the Netherlands.
The planning horizon for the project was chosen
to be the year 2025 and beyond (we refer to this
time horizon as 2025+).
The project used the policy analysis approach
described above to clarify the problem, outline the
alternative solutions and display tradeoffs among
their consequences. The policy analysis approach
as applied in the TNLI project is shown in Figure
4.
After the problem was formulated and the sys-
tem described (as presented above), we proceeded
to carry out the following steps:
defining scenarios,
defining criteria (impacts),
defining policy options (tactics),
building impact assessment models, and
describing results.
These efforts are summarized below.
6.3. Scenarios
The evaluation of alternative infrastructure op-
tions needs to take place in the context of the
future world in which they will have to function.
But the future is uncertain. One way to deal with
this uncertainty is to construct alternative possible
scenarios and look for options that perform
Figure 4. Research approach on TNLI project.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
20
reasonably well in some or all of them. A scenario
is a description of a hypothetical future state of
the world, including a consideration of the major
uncertainties encountered in moving far into the
future. The scenarios do not predict what will
happen in the future; rather they are plausible
descriptions of what can happen. They pay atten-
tion to developments within the system and to
those outside the system that affect the system,
excluding the infrastructure options to be exam-
ined. We constructed five scenarios for a year
2025+that focus specifically on (1) the world of
civil aviation and (2) changes both inside and
outside the civil aviation system that are relevant
for making policy decision about investments in
civil aviation related infrastructure in the Nether-
lands. They are not an inventory of all the
changes affecting civil aviation that can occur in
the future encompassing all possible changes
in five scenarios is impossible. However, they
cover the range of plausible future demands; each
scenario, although a single point in the array of
possible futures, represents a family of changes
that fall within the category represented by the
scenario.
The five scenarios differ in terms of the assump-
tions that are made about (1) worldwide growth
of civil aviation, (2) the configuration of the civil
aviation system in Europe, (3) civil aviation poli-
cies within the European Union, (4) the develop-
ment of competing transportation systems, (5)
airport capacity in Europe and (6) aircraft tech-
nology. Table I gives an overview of the five
scenarios. For more details on the scenarios and
their development, see RAND Europe (1997b).
6.4. Criteria for assessing the effects of the
infrastructure options
During the initial phases of the project we held
roundtable meetings with representatives of many
Table II. Categories and examples of criteria used in
the analysis
System aspects
Interference with the use of airspace for military
purposes
Effects of weather
Mobility aspects
Interference of airport-related traffic with local
non-airport-related transportation activities
Integration with the high speed train network
Economic aspects
Employment
Gross value added
En6ironmental aspects
Emissions from landings and takeoffs
Ground access emissions
Noise
Third-party risk
Visual intrusion
Spatial aspects
Land needed for the infrastructure option
Land becoming available for noise-sensitive functions
Costs of infrastructure options
Pre-construction costs
Construction costs
Additional costs
of the stakeholder groups (e.g. Dutch airport
owners and users, national environmental interest
groups). Based on these meetings, examination of
the literature and additional internal and external
discussions, we identified a set of quantitative and
qualitative criteria. The resulting criteria were
used in the analysis to assess a particular ‘case
(combination of infrastructure option and sce-
nario). Table II gives an overview of the cate-
gories and types of criteria that were used in the
analysis. Appendix D of RAND Europe (1997a)
provides a complete list of the criteria used.
Table I. Scenarios for 2025+for the TNLI project
Scenario no. Worldwide civil aviation Role of the Netherlands Annual passenger Annual tonnes of cargo
movements
1 Growth One of six hubs 103 million 7.7 million
No hubGrowth2 2.3 million40 million
3 Growth One of ten hubs 64 million 4.6 million
Downturn4 10.0 million82 millionOne of three hubs
5 Downturn No hub 14 million 0.8 million
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 21
6.5. Infrastructure options
Based on meetings with the client organizations
and on other sources and discussions, we specified
a set of infrastructure options to be examined in
the analysis. Options that varied the number of
regional airports were excluded, since we assumed
that the role of regional airports in the Nether-
lands was unlikely to grow in a world dominated
by hub-and-spoke airline networks. An infrastruc-
ture option was, therefore, defined to be a config-
uration of national airport(s), where a national
airport is defined as an airport that is able to
accommodate large volumes of passengers and/or
cargo and that can function as a hub.
The infrastructure options were defined in
terms of the following key characteristics:
the number of national airports in the Nether-
lands (one or two),
in case there are two national airports, whether
the traffic at the airports is allocated based on:
type of traffic (intra-European versus intercon-
tinental), hub-carrier, or purpose (pure cargo
versus mainly passengers),
the location of (the parts of) the airport
(densely populated area; sparsely populated
area; in the sea; in a lake; at the border).
In assessing the effects of an infrastructure option
for a particular scenario, we assumed that all
demand for air transport in that scenario would
be accommodated; i.e. the national airport(s)
would be sized to meet the national demands.
This means that no demand would be shifted
from national to regional airports because of ca-
pacity constraints. It also means that the physical
and/or environmental capacity of an airport, in
terms of aircraft movements, passenger move-
ments and tonnes of cargo, is not part of the
description of an infrastructure option.
The fourteen primary infrastructure options we
examined are described below and summarized in
Figure 5.
6.5.1. 1
4
.N
2
popa,N
2
lake,N
2
sea and N
2
spa
There are two completely separate national air-
ports in the Netherlands, which are not linked by
a dedicated high speed people/cargo mover. The
Figure 5. Infrastructure options examined.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
22
two national airports are connected by public
transport. One of the two national airports is
Schiphol. The second airport is located in the sea,
in a lake, or on land (in a populated area or a
sparsely populated area). Traffic is allocated be-
tween the airports based either on type of traffic
(intra-European versus intercontinental) or hub-
carrier (one hub at each airport).
6.5.2. 5
6
.N
2
cargo
(
popa and lake
)
There are two completely separate national airports
in the Netherlands, which are not linked by a
dedicated high speed people/cargo mover. The two
national airports are connected by public transport.
One of the two national airports is Schiphol. The
second airport is built to handle only cargo and can
be built in a populated area or in a lake. All of the
cargo is intercontinental cargo and only full
freighters are used at the cargo airport. There is no
express cargo handled at the cargo airport.
6.5.3. 7
9
.RRpopa,RRlake and RRsea
The terminals remain in the current location at
Schiphol. The runways at Schiphol with the highest
noise effects are closed down. Remote runways are
located at a water location, or on land. The remote
runways are connected to the terminals by an
underground people/cargo mover, capable of very
high speeds. There is no other landside connection
besides the people/cargo mover. The noisiest air-
craft and all night flights are assumed to use the
remote runways.
6.5.4. 10
.N
1
popa
All demand is accommodated at Schiphol (located
in a densely populated area). Facilities are enlarged
as needed in the scenario.
6.5.5. 11
.N
1
lake
Schiphol is closed and there is a single national
airport located on an artificial island in a lake. The
airport handles all types of transport and is con-
nected to the mainland by a land bridge.
6.5.6. 12
.N
1
sea
Schiphol is closed and there is a single national
airport located on an artificial island in the sea. The
airport handles all types of transport and is con-
nected to the mainland by a land bridge.
6.5.7. 13
.N
1
spa
Schiphol is closed and there is a single national
airport in a sparsely populated area, non-eco-
nomic center. The airport handles all types of
transport.
6.5.8. 14
.N
1
border
Both Schiphol and Zaventem (Brussels) airports
are closed or have a reduced role. There is a single
national airport located on land somewhere on
the Dutch/Belgian border. The airport handles all
types of transport. For assessing this option, we
assumed that the direct economic costs and bene-
fits are split between the Netherlands and
Belgium.
6.6. Impact assessment models
Impacts of the various infrastructure options
were estimated using both qualitative and quanti-
tative (computer) models. The models were de-
signed to explore the implications of un-
certainties in the future world of civil aviation
for policy decisions regarding civil aviation in-
frastructure, to provide a level playing field for
comparing alternative infrastructure options and
to provide insights into the costs and benefits of
the infrastructure options for the various scenar-
ios. Computer models were developed for esti-
mating demand, economic effects, costs, noise,
aircraft exhaust emissions and ground access
emissions associated with the scenarios and in-
frastructure options. Figure 6 shows how the
computer models fit together conceptually.
The demand model produces a set of numbers
for a scenario that represent estimates of future
demands based on the assumptions for that sce-
nario. The demands are given in terms of the
number of aircraft movements, tonnes of cargo,
aircraft mix, number of origin/destination (O/D),
transfer, business and leisure passengers and the
share of these passengers carried by the home
carrier and by foreign carriers.
Figure 6. Interrelationships among computer models.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 23
The model used to estimate national economic
effects uses the demand model’s estimates of pas-
senger, cargo and aircraft movement volumes to
estimate the effects of the resulting civil aviation
activity on employment, value-added and demand
for land, office space and land transport. The
model considers the economic picture painted by
the scenario, as well as the specifics of the in-
frastructure option (the number, location and
function of national airports). Both direct and
indirect employment effects are estimated. The
direct effects are those jobs that are directly re-
lated to aviation activities in the Netherlands.
Indirect effects are jobs created by intermediary
deliveries and by the attractiveness of an airport
and its location.
The noise model and the aircraft emissions
model use the number of aircraft movements by
aircraft type from the demand model to estimate
their impacts (an indicator for the ground surface
within a specified noise immission contour
1
and
total exhaust emissions of the landing and takeoff
cycle of all flights up to 10000 feet, respectively).
The ground access emissions model uses the
demand figures to estimate the emissions gener-
ated by O/D passengers, employees and cargo
travelling to and from the airport.
The cost model estimates the financial cost of
providing capacity at the national airport(s) to
meet the demand estimated for a given case.
The numbers produced by the models are not
important in themselves. It is the relative differ-
ences in those numbers that are important. The
most important objective of the assessment of
infrastructure options was to provide insights into
the relative performance of the options in each
scenario. In other words, for a given scenario and
measure of performance, the assessment was in-
tended to shed light on which options performed
best. Our intention was to be able to make the
following sorts of statements: for some given level
and composition of demand (i.e. a scenario), Op-
tion A is better than Option B in terms of cost
and environmental effects, but not as good in
terms of its economic effects. The quantitative
information from the computer models and the
information from the qualitative models were
placed on scorecards that were used to compare
the relative advantages and disadvantages of the
various infrastructure options.
6.7. Assessment of infrastructure options
The assessment of each infrastructure option was
carried out for each of the five scenarios. Each
combination of a scenario and an option was
defined to be a separate case. The assessment of a
case on the set of qualitative and quantitative
criteria was done relative to a comparison point.
Two major insights were developed in carrying
out the assessment. First, the relative ranking of
the infrastructure options on each of the perfor-
mance measures was found not to differ by sce-
nario. Second, the important policy conclusions
about the fourteen infrastructure options could be
summarized by discussing them in terms of six
‘themes’.
The first insight is based on grouping the five
scenarios into two categories: scenarios that have
high growth in demand for air travel in the
Netherlands and those that do not. In the two
low/no growth scenarios (Scenarios 2 and 5), the
air transport demands do not exceed the existing
capacity constraints at Schiphol, so there would
be little need to change existing policy or build
new infrastructure in order to accommodate the
demand. Therefore, the detailed results of the
assessments of the infrastructure options for these
two scenarios could be ignored. In the remaining
three scenarios, the relative performance of the
infrastructure options for a given criterion re-
mains virtually unchanged. So, presenting the re-
sults of the assessment for any one of the
scenarios would be sufficient.
We grouped the assessments of infrastructure
options into six themes: a land airport, remote
runways, two national airports, a border airport,
a water airport and a separate cargo airport. As
an illustration, the assessment of a water airport is
presented below.
A water airport located in a lake or the sea
would drastically reduce or fully eliminate the
problems associated with third-party risk, aircraft
noise and local emissions. It would also eliminate
the problem of urban sprawl around the runways
that might constrain future airport expansion.
But, if there were a landside connection to the
mainland, a water airport might be a way to
1
Noise immissions are the total noise individuals experi-
ence on the ground from overflying aircraft. Noise
emissions are the sounds produced by the aircraft en-
gines. Noise immissions are affected by such things as
the distance between the individual and the aircraft,
weather and wind conditions, and whether it is day or
night.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
24
Figure 7. (a) Impacts from a single airport in a populated area and in the sea. (b) Impacts from two national
airports in populated areas. (c) Impacts from a national airport in a polluted area and a second in the sea.
stimulate regional economic development. Figure
7(a), (b) and (c) shows how noise and ground
access emissions could be reduced by a sea loca-
tion. (The noise bars for the sea location and land
location are shaded differently to indicate that,
while noise is emitted around the sea location, it
would probably not cause much nuisance or
annoyance.)
The major drawback to building a new airport
at a water location is the cost of doing so. Build-
ing an airport at a water location is significantly
more expensive than building a comparable air-
port at a land location. A water location is also
more vulnerable to the weather than a land loca-
tion. Also, because airport employees cannot live
close to the airport, their commuting times would
increase. This has negative repercussions for the
environment as well. In addition, an airport at a
water location can cause new ecological problems.
Finally, locating a railway station near a water
location would require additional infrastructure
investments.
In summary, a water location has the potential
to completely eliminate the problems associated
with aircraft noise and third-party risk. However,
the scale of such construction would cause new
environmental problems. Such an airport would
also be very expensive. Thus, the reasonableness
of such an option depends on the value attached
by society to eliminating aircraft noise and third-
party risk. Also, if demand is to be accommo-
dated and no land is available for a new airport
or expansion of Schiphol is ruled out, an airport
in the sea or a lake may be the only viable option.
6.8. Use of the project’s results
Because the policy analysis approach that was
used in the project was clear and straightforward,
the public and government officials were quickly
able to understand the information it provided.
The results have proved to be very useful, primar-
ily to identify the infrastructure options that
should be examined more carefully and those that
should not be investigated further. For example,
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 25
prior to the project, the Netherlands was seriously
considering the possibility of building a second
national airport. However, our analysis of four
such options (infrastructure options 1–4 in Figure
5) showed that the requirements included very
high passenger demand, a high-speed link between
the two airports, a hub carrier at each airport and
lack of excess capacity at competing airports.
Subsequent attention has focused on an airport
located on an artificial island in the North Sea
(infrastructure option 12) and Schiphol expansion
(infrastructure option 10). As described above, an
airport in the sea would drastically reduce or
totally eliminate the problems associated with
third-party risk, aircraft noise and local emissions.
However, it would be extremely expensive. By
contrast, expanding Schiphol is the least costly
and least disruptive option. All other infrastruc-
ture options would necessitate a redesign of
airspace. Also, when compared to building an
entirely new airport on a virgin site, the expansion
of Schiphol is less disruptive in terms of new
intrusion on natural settings and new visual intru-
sion. Its expansion would require less new con-
struction than any other option, which also means
that expanded Schiphol would have all the prob-
lems of a large airport located in a densely popu-
lated area. For example, it would expose large
numbers of people to aircraft noise. Third-party
risk could also increase with an increase in air-
craft movements and airport-related traffic would
interfere with other road users around the airport,
exacerbating an already problematic situation.
The project has been followed up by several
more detailed studies (including engineering stud-
ies and environmental impact studies), which have
concentrated on these two options.
7. FINAL OBSERVATIONS
Many operations research tools, such as decision
analysis, simulation and optimization, have been
very successful. They are still useful and relevant
as tools in many contexts. However, the complex
and uncertain world of public policymaking re-
quires applying what in marketing theory terms
might be called customer-based marketing, rather
than technology-based marketing. In technology-
based marketing, the product is designed first and
a market/customer sought out second. Customer-
based marketing seeks to understand the customer
first and then create the product second. The key
factor in choosing which style of marketing to do
is the cost of failure. When the cost of a failed
offering is relatively low, technical marketing is
preferred. When costs are high (as in the case of a
new automobile), customer marketing is pre-
ferred. So, in providing decision support for pub-
lic policymakers, a choice must be made: are you
involved in technology-based marketing (the ap-
proach generally followed by methodologists) or
customer-based marketing? For most real-world
policy problems, a customer-based approach is
more appropriate. This means starting with the
problem, not with the product. The policy analy-
sis process is customer based.
The profession of operations research is alive
and well and its problem solving tools are needed
and used more than ever. But it would contribute
to its image and success if it expanded its frame of
reference and re-defined its role in the decision-
making process. Since one of the primary goals of
operations research is to help decisionmakers
when they have to make a choice among alterna-
tive options, its tools are perfectly suited to policy
analysis. But it would be more successful if it paid
more attention to the steps of policy analysis in
addition to the modelling step (i.e. if it were
customer based rather than technology based).
For most real-world decision problems, the steps
before modelling are characterized by an imagina-
tive and somewhat unstructured exploration of
the objectives of the decisionmakers, the roles of
other stakeholders, the scope of the policy do-
main, the major uncertainties and the search for
promising alternatives. These steps set the agenda
for the subsequent data collection, the develop-
ment of scenarios and the building of models to
be used in the evaluation of the alternative op-
tions. In the steps after modelling, the scientific
insights obtained in the analysis have to be trans-
ferred into the political arena. They, therefore,
have to be presented to the decisionmakers in
terms that they understand and can use; that is, in
a form that enables them to draw the relevant
conclusions, to select a strategy and to present the
decision to the wider group of stakeholders. The
policy analysis approach presented in this paper
seeks to solve problems by carrying out these
steps and by using appropriate tools from a large
toolkit rather than focusing on a single tool.
Many research opportunities still remain within
the various steps in the process and in tool de-
velopment. In fact, policy analysis as currently
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
W.E. WALKER
26
practiced has many critics and still has many
pitfalls and limitations to be overcome. Laurence
Lynn (1999) discusses many of these criticisms,
pitfalls and limitations. Most are focused on the
need to move policy analysis away from a ‘ratio-
nal’, technical, decisionmaker focus toward a
more qualitative, soft-science, participative focus.
As Alice Rivlin (1971) states: Policy analysts have
to learn that ‘educators, doctors, civil ser-
vants,. . . even generals’ are ‘knowledgeable, nec-
essary, and not always wrong’. And Kathleen
Archibald (Majone and Quade, 1980, pp. 193
194) writes that ‘when it comes to examining
pitfalls [of policy analysis], we find that the most
serious pitfalls will not be circumvented by greater
rigor or improved technical skills. Competencies
usually considered ‘‘softer’’ — imagination, judg-
ment, interpretive skills — are just as important’.
The major challenges for traditional policy ana-
lysts, therefore, are
to make the linkages between the policy analy-
sis process and the policymaking process more
effective,
to improve the integration of stakeholders and
other actors besides decisionmakers into the
policy analysis process, and
to add more ‘soft science’ tools (such as semi-
nar games Kahan et al., 1992 and process
management de Bruijn and ten Heuvelhof,
1993) to their methodological toolkits.
I believe that, as a group, the policy research
community already possesses the best set of tools
and qualifications to help society address the
seemingly intractable problems caused by the
complexity, uncertainty and system interconnec-
tivity facing public policymakers at this point in
the development of civilization. There is a great
need for people who can provide sound guidance
and advice. Many professionals, ranging from
architects to philosophers and journalists, believe
that they can solve the social problems of the
world. But I truly believe that we are uniquely
positioned to help improve public policy and the
quality of life of people throughout the world and
we should try to do so.
I cannot express my message to persons inter-
ested in providing decision support to public sec-
tor decisionmakers any better than Donald Scho¨n
did when he wrote:
In the varied topography of professional
practice there is a high, hard ground where
practitioners can make effective use of re-
search-based theory and technique, and there
is a swampy lowland where situations are
confusing ‘messes’ incapable of technical so-
lution. The difficulty is that the problems of
the high ground, however great their techni-
cal interest, are often relatively unimportant
to clients or to the larger society, while in the
swamp are the problems of greatest human
concern. (Scho¨n, 1983)
I urge those researchers who are truly interested
in providing decision support to the public sector
to pack their toolboxes with as wide a variety of
tools as they can master and spend some more
time with me down in the swamp. The rewards,
both personal and professional, can be immense.
REFERENCES
Bardach E. 1996. The Eight-Step Path of Policy Analy-
sis
(
A Handbook for Practice
)
. Berkeley Academic
Press: Berkeley, CA.
de Bruijn JA, ten Heuvelhof EF. 1993. Managing envi-
ronmental policy networks. In The En6ironment
:
To-
wards a Sustainable Future. Sdu Publishers: The
Hague; 85–109.
Drake AW, Keeney RL, Morse PM (eds). 1972. Analy-
sis of Public Systems. MIT Press: Cambridge, MA.
Findeisen W, Quade ES. 1985. The methodology of
systems analysis: an introduction and overview. In
Handbook of Systems Analysis
:
O6er6iew of Uses,
Procedures,Applications and Practice, Chapter 4,
Miser HJ, Quade ES (eds). Elsevier: New York.
Goeller BF, Abrahamse AF, Bigelow JH, Bolten JG, de
Ferranti DM, De Haven JC, Kirkwood TF, Petr-
uschell RL. 1977. Protecting an Estuary from
FloodsA Policy Analysis of the Oosterschelde, vol.
I. RAND: Santa Monica, CA, Summary Report.
R-
2121
/
1
-NETH..
Hammond JS, Keeney RL, Raiffa H. 1999. Smart
Choices
:
A Practical Guide to Making Better Deci-
sions. Harvard Business School Press: Boston, MA.
Heineman RA, Bluhm WT, Peterson SA, Kearny
EN. 1990. The World of the Policy Analyst
:
Rational-
ity,Values and Politics. Chatham House: Chatham,
NJ.
Hillestad RJ, Walker WE, Carrillo MJ, Bolten JG,
Twaalfhoven PGJ, van de Riet OAWT. 1996. FOR-
WARD Freight Options for Road,Water,and Rail
for the Dutch
:
Final Report. MR-736-EAC/VW.
RAND: Santa Monica, CA.
House PW. 1982. The Art of Public Policy Analysis,
Sage Library of Social Research, vol. 135. Sage Publi-
cations: Beverly Hills, CA.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
POLICYMAKING IN THE PUBLIC SECTOR 27
Kahan JP, Setear JK, Bitzinger MM, Coleman SB,
Feinleib JA. 1992. De6eloping Games of Local Drug
Policy. N-3395-DPRC. RAND: Santa Monica, CA.
Lynn LEJR. 1999. A place at the table: policy analysis,
its postpositive critics, and the future of practice.
Journal of Policy Analysis and Management 18(3):
411–424.
Majone G. 1985. Systems analysis: a genetic approach.
In Handbook of Systems Analysis
:
O6er6iew of Uses,
Procedures,Applications and Practice. Chapter 2,
Miser HJ, Quade ES (eds). Elsevier: New York.
Majone G, Quade ES (eds). 1980. Pitfalls of Analysis.
John Wiley & Sons: New York.
Miser HJ. 1980. Operations research and systems anal-
ysis. Science 209: 139–146.
Miser HJ, Quade ES (eds). 1985. Handbook of Systems
Analysis
:
O6er6iew of Uses,Procedures,Applications
and Practice. Elsevier Science Publishing Co: New
York.
Mood AM. 1983. Introduction to Policy Analysis.
North-Holland: New York.
Pollock SM, Rothkopf MH, Barnett A (eds). 1994.
Operations Research and the Public Sector,Hand-
books in Operations Research and Management Sci-
ence, vol. 6. North-Holland: New York.
Quade ES. 1989. Analysis for Public Decisions (3rd
edn). Elsevier Science Publishers: New York.
RAND Europe. 1997a. A Policy Analysis of Ci6il A6ia-
tion Infrastructure Options in the Netherlands. DRU-
1512-VW/VROM/EZ. Leiden.
RAND Europe. 1997b. Scenarios for Examining In-
frastructure Options in the Netherlands. DRU-1513-
VW/VROM/EZ. Leiden.
Schlesinger JR. 1967. On Relating Non-Technical Ele -
ments to System Studies, P-3545. RAND: Santa
Monica, CA.
Scho¨ n DA. 1983. The Reflecti6e Practitioner
:
How Pro-
fessionals Think in Action. Basic Books: New York.
Simon HA. 1969. The Sciences of the Artificial. MIT
Press: Cambridge, MA.
Walker WE, Abrahamse A, Bolten J, Kahan JP, van de
Riet O, Kok M, den Braber M. 1994. A policy
analysis of Dutch river dike improvements: trading
off safety, cost, and environmental impacts. Opera-
tions Research 42(5): 823–836.
Walker WE, Chaiken JM, Ignall EJ (eds). 1979. Fire
Department Deployment Analysis
:
A Public Policy
Analysis Case Study. Elsevier North–Holland: New
York.
Walker WE, Po¨ yho¨ nen M, de Jong JH, van der Tak C.
1998. POLSSSPolicy for Sea Shipping Safety
:
Cost-Effecti6eness Analysis. RE-98.006.1. RAND
Europe: Leiden.
Copyright © 2000 John Wiley & Sons, Ltd. J.Multi-Crit.Decis.Anal.9: 11 27 (2000)
... In this case study-based paper, we investigate three collaborative activities aimed at finding innovative coastal policy solutions in the Netherlands. Our analysis leans on social-ecological systems theory (Redman, Morgan Grove, and Kuby 2004;Berkes, Colding, and Folke 2008), policy analysis (Walker 2000;Enserink et al. 2010;Walker 2013, 2001) and transdisciplinary learning (Jahn, Bergmann, and Keil 2012;Bergmann et al. 2012) through stakeholder engagement. The theoretical promise of such engagement in a complex coastal system (elaborated in Section 3) serves as a framework for identifying the different types of outcomes and the knowledge necessary to resolve the local coastal problems. ...
... The concept of the environment as linked, nested and interacting systems of humans and natural ecosystems, emphasises that humans are part ofand not apart fromnature (Berkes and Folke 1998). Policy analysis rests on systems thinking (Slinger, Taljaard, and D'Hont 2020), and offers purposeful, systematic and structured analytical approaches to investigate the effect of policy activities on their real-world contexts (Walker 2000;Enserink et al. 2010;Walker 2013, 2001). Transdisciplinary approaches recognise varied sources of knowledge as relevant in establishing system understanding, including environmental and social science, practice, local knowledge and governance knowledge (Jahn, Bergmann, and Keil 2012;Bergmann et al. 2012). ...
... It is acknowledged in the stages model that human beliefs are inseparable from the context and process of analysis (Walker, 2000); however, these beliefs are not the focus of the stages model. The ACF (Weible and Sabatier, 2007;Cairney, 2020) was needed to enhance the analysis on manifestations of land policy processes, specifically for its premise of belief systems (policy actors' beliefs in shaping land policies). ...
... The model is an input-output model of the policy process, which presupposes a set of policy demands (input) that are then processed by the political system into laws, programs, and the goods and services provided by government (output) (Birkland, 2016). The stages model is a problem-solving approach that is divided into steps for defining the problem and steps for solving it (Walker, 2000). ...
Article
Land policies are formulated with the goal of addressing land use management challenges. Therefore, a thorough investigation is required to assess effectiveness of land policy processes. The unknown land use policy effectiveness is how and where the formulation and identification of land use problems affect the throughput of policy implementation. The main objective of this paper is to assess the effectiveness of land policy processes using models of public policy analysis. The study analyses how the stages model could be useful for analysis, and in which areas the advocacy coalition framework (ACF) could enhance the analysis. For comparative insights, land policy cases of Ethiopia and Rwanda were used. Data on land policy processes in the two cases were gathered from literature and data collected from field. The analysis with the two models shows that the effectiveness of executing land use policy processes does not only rely on the conventional cycle and sequence of land use policy implementation steps (i.e. identification of the problem, formulation of the solution, execution of the solution). Instead, during the problem formulation certain dynamics occur which may prevent finding the right and only solution. The stages model is useful for analyzing manifestations as they occur along processes, and the ACF is required as a valuable model to enhance analysis by understanding the cause of manifestations. This study would impact the research method for analyzing effectiveness of land policy processes and improving practices of policy making on the basis of good land governance and public administration perspective.
... Non-monetary benefits and effects can be accounted for separately in a decision framework such as policy analysis with the use of scorecards (Walker, 2000) (see also Step F.1). In occasions, it is not possible, or there is not enough time and resources to produce a quantification of certain relevant effects, for example the impact on endangered migratory bird species when a reservoir for flood water is built. ...
... Once the impact assessment Steps C, the adaptation Steps D, and the economic appraisal Steps E are complete, the necessary information is available to support a comprehensive decision process. Affirmed decisional frameworks/tools that enable this are: cost-effectiveness analysis, multi-criteria analysis, policy analysis with scorecards (Walker, 2000), or multi-actor multi-criteria analysis (Macharis et al., 2012). Use of these frameworks should be concerted, and count on the involvement of as many relevant stakeholders as possible. ...
Technical Report
Full-text available
Adaptation to climate change receives little if any attention during the phase of planning and appraisal of investments into infrastructure at the Member State and European level. Recently, efforts have been made to assist project planners with incorporating considerations of adaptation into their workflow, but no guidelines yet address adaptation projects in their own right. Because adaptation projects face specific challenges, such as the necessity of dealing with large uncertainties, we endeavoured in this Deliverable of ECONADAPT to fill this gap and produce guidelines tailored to the appraisal of adaptation projects. To enable the identification of the key steps and challenges in the appraisal, we conducted two case studies, appraising adaptation projects in the Vltava river, Czech Republic, and in Bilbao, Spain. From these we distilled the lessons learned into the guidelines here presented, which aim to address practitioners, and are therefore as straightforward and free of technical jargon as possible. The guidelines are structured in 22 steps for the practitioner to follow, divided in the areas of: context analysis; hazard assessment; impact assessment; adaptation; economic assessment; and decision-making with consideration of stakeholders. Each step is explained in a small section of typically half to two pages, containing: a brief overview of the problem; a display of the methods available to tackle it; a brief account of what was done in the ECONADAPT case studies, and what can be learned from them; recommendations about good practices. In addition, we have compiled summary tables of the steps, aimed to provide: 1) an impression at a glance of all that needs to be accomplished in the adaptation appraisal; 2) a schematic map with the minimal amount of information that the practitioner should keep in mind at any moment. Main finding 1: the appraisal, and wider evaluation and of the possible options for adaptation is by its own nature a comprehensive and multidisciplinary exercise. The practitioner should count on (access to) a range of expertise to carry out the exercise. Main finding 2: it is possible to summarize the essential aspects of the appraisal in a set of steps that should be carefully considered and at least inspire the practice.
... The decision to choose specific variables was made under the data uncertainty. Walker views decision-making as choosing among alternatives in order to change system outcomes in a desired way [18]. ...
Preprint
Full-text available
The paper presents the results of the study, the purpose and practical input of which was the analysis and evaluation of municipal governments, including the identification of municipalities best developed in socioeconomic terms in conditions of data uncertainty. To this end, factors with a significant impact on the concept of territorial development were identified. For the purpose of the analysis, the variables were adopted that were related to various spheres of municipal development, including the social, economic, infrastructural, environmental and administrative ones. The study used such methods and exploration techniques as: desk research analysis, critical analysis of the literature on the subject, methods of economic analysis and methods of multidimensional comparative analysis. The uncertainty of data was caused, among other things, by the fact that the study did not take into account the effect of barriers hampering development processes and focused exclusively on variables whose highest values indicated the most effective use of development stimulating factors. Moreover, the diversification of the municipalities' endogenous potential was not included in the considerations. The above reasons seem to confirm the study results that found two municipalities: Kołobrzeg and Świnoujście to be the best developed in socioeconomic terms. In the period under analysis the values of the diagnostic variables of both municipalities were the most similar to the benchmark municipality. The explanation for the results can be found, among others, in the high endogenic potential of both municipalities, especially in comparison with other coastal municipalities of the West Pomeranian Voivodship. The study contributes to further in-depth research on the best method of selecting variables for analyzing the development of municipalities. Future research in this field should be focused on identifying the best method of selecting diagnostic features where some features are uncertain or imprecise.
... Sea level rise projections data have been derived from Losada et al. (2014), who regionalised the latest IPCC RCP-specific data for the Spanish coast. ...
Technical Report
Full-text available
In this deliverable we present the results of the appraisals conducted in the two case studies of Work Package 6 of ECONADAPT. For the Vltava (Czech Republic) and the Bilbao (Spain) case studies, we assessed the benefits and costs of investment to adapt to climate-induced variations in flooding. In these appraisals, we emphasize the methodologies that allow addressing the multiple sources of uncertainty that typically characterize appraisals of climate change adaptation investments. By carrying out these two case studies we intend to explore the methodological challenges connected with dealing with the large and diverse uncertainties that are typical of investments in adaptation to climate change. So far, adaptation to future climate change, and the relevant uncertainties have rarely been explicitly included in economic appraisal studies, even when the investments under study have multi-decadal life times that make them particularly vulnerable to long-term climatic and socio-economic change. Damages result from extreme events that may become more frequent over time. It is therefore important to quantify the risks posed by changing future conditions, without and with the investment in adaptation. We have carried out simulations in a cascade of models, spanning several disciplines, from climate, to hydrology, to flood risk, through to economic modeling, and produced economic appraisals mostly in terms of the Net Present Values of the adaptation investments planned. Further, for the Bilbao case, results are also expressed in terms of Value-at-Risk, and of Expected Shortfall of the investment, two risk measures that are often used in financial risk analysis. A final table of this Deliverable 6.3 provides a qualitative account of the relative importance of each source of uncertainty on the results of the two case study appraisals.
... The well-known steps of the rational model or the well-known cycle of public policy making (Anderson, 1975) that represents a rather idealized process was in various scholar's writings analyzed (Walker, 2000) as: ...
Chapter
Full-text available
The Covid-19 crisis is a unique challenge that transcends national borders of many countries. Its immediate life-threatening effects for certain patients and its contagiousness left no time for executives to reflect upon the measures that would be efficient enough to tame the issue. Conventional tools such as the Parliamentary working, the consultation period, etc. cannot be of great assistance and immediate, flexible forms of management, such as governmental committees and task forces of experts and other stakeholders, are instead recommended, so as to determine in a "sense making approach" a viable provisional solution. Administrations need to acquaint themselves with experimentation and "trial-and-error", to combine the "regulation" and the "execution", to "adopt" and "adapt" to new conditions with a new mindset. A "small-wins" approach is of great assistance as it is a step-by-step methodology in which administrations gain knowledge and capitalize on what works and what does not for handling the situation. This new working methodology in unchartered waters challenges the conventional governmental working and takes power from the stable institutional framework, transferring it to more flexible forms of governance. Leadership, novel staff arrangements, better use of e-tools to settle team working and service delivery, and all this combined with political responsibility and accountability, are of great importance in dealing with Covid-19, proving the necessity to dispose of mature democracies in times of crisis.
... Our first admonition is not really tentative: it is about seeing the world in "system" terms. That idea was stressed with the advent of systems analysis and policy analysis [12][13][14]. Oddly, the field of System Dynamics developed in parallel [15][16][17] with these, with surprisingly limited interactions. In any case, the profound significance of system thinking is often not visible in modern-day policy studies. ...
Chapter
This paper describes a possible agenda for changing the teaching and conduct of policy analysis to better reflect the centrality of complex adaptive systems and the potential role of new analytic methods, including computational social science (CSS). The agenda identifies desirable changes in four categories: (1) world view when conceiving and posing problems, (2) the basis for reasoning and inference, (3) analytic style, and (4) the character of models and model-based analysis. The paper’s intent is to obtain reactions and suggestions from others in the CSS community to help refine the ideas.
... Policy analysis refers to the art and science of determining which public policy, among alternatives, will most likely achieve a determined set of goals. Policy analysis aims to assist policymakers in choosing desired courses of action by illuminating the problem, outlining the alternative solutions, and displaying tradeoffs among their consequences (Walker, 2000). ...
Article
Full-text available
This study provides a quantitative assessment of the likely economy-wide impacts of tariff cuts and productivity improvements in agriculture for Sri Lanka. A static, multisector, Computable General Equilibrium (CGE) model, using the Supply and Use Table (SUT) of 2010, was employed highlighting specified agricultural sub-sectors and their interactions with other production sectors in the economy. Constructing a CGE model entailed the development of a Social Accounting Matrix (SAM) to represent the Sri Lankan economy. The SUT, household income and expenditure survey, economic stat of the Department of Census and Statistics, and economic data library of the Central Bank of Sri Lanka were used to develop the SAM. The SAM was used to calibrate the CGE model. Coding and operationalization of the CGE model were executed using the PATH solver of the General Algebraic Modeling System software using a modified version of the standard CGE model. Production was specified as a Constant Elasticity of Substitution (CES) production function whereas consumption was specified as a Linear Expenditure System (LES). Using the HIES data, the LES was estimated using a seemingly unrelated regression model. The CGE model included a representative household, two factors of production i.e., labor and capital, commodities, activities, the government, savings, and investment and trade. Productivity improvements in the selected agricultural subsectors lead to a significant positive response in the paddy, vegetables, coconut growing, and livestock sectors. However, productivity improvements in the specified agricultural sectors lead to a decline in demand for labor because of improved primary factor productivity and the decline of market prices. A cut in prevailing tariffs in agricultural industries shows negative impacts on households as a whole.
... The policy analysis methodology presented by Walker (2000) was used as guidance when discussing the appropriate policy mix. This was further informed by proposals in relevant literature, and by the opinions of the interviewees. ...
Conference Paper
Full-text available
Extending the lifetime of products is seen as a key objective for realising the vision of a Circular Economy. One way to increase the lifespan of products is to enable more repair activities. However, consumers encounter a variety of barriers for repairs, prompting public authorities in Europe and the US to adopt or propose policies in support of consumer repairs. Sweden has recently adopted a circular economy action plan, where increasing the number of consumer repairs is a stated objective. However, Sweden has so far only adopted a few repair policies, most notably through the tax reliefs for the repair sector that were implemented in 2017. The aim of this contribution is to research how Sweden could develop a more comprehensive policy mix for promoting consumer repairs, also by taking into consideration initiatives from other countries and regions. The research is based on a literature review and semistructured interviews with policymakers and other relevant actors in Sweden, Europe and the US. The study shows that a lot of interesting initiatives aiming at increasing repairs are currently being proposed. The new requirements related to repairs, developed within the European Union’s (EU) Ecodesign Directive, have been positively received but the process is cumbersome and it will take time before their full effect becomes evident. Initiatives, such as the French repairability index and the French repair fund will create incentives for the producers to design more repairable products and make it easier for consumers to repair. On the same track, the Repair Network of Vienna with its repair vouchers makes repairs cheaper and more trustworthy. Also, the US policy proposals on right-to-repair laws would help to create an open market for repairs for a lot of products. Sweden has the possibility to gain knowledge through the implementation of similar policies, and by considering new policies suggested in literature and by the interviewees. Thus there is potential for Sweden to be a frontrunner in creating a more resource efficient society through increased repair activity. Concluding, some preliminary proposals for a future policy mix are presented.
Thesis
Full-text available
Poverty alleviation has been the South African (SA) government’s central policy issue since Apartheid without success. Hence, this study selected a paradigm shift. Instead of focusing on poverty alleviation, the study constructed a positive mirror image of poverty: prosperity attainment. Notably, the SA government has been using job creation as a policy instrument in vain. Some countries have enacted the Technical Vocational Education and Training (TVET) model and used it as a central policy mechanism for poverty alleviation. Yet poverty alleviation is the means to an end, and the end should be prosperity. In South Africa, there is an absence of a holistic policy framework for TVET implementation across basic education, tertiary education, and industry, which makes coordination difficult. Hence, this study explored TVET policies from China, Finland, Germany and Nigeria to assess the best-fit elements. The primary objective of this study was to explore how TVET can be an instrument for prosperity realisation. The challenges that inhibit prosperity are neither linear nor straightforward. Hence, this research explored TVET policies using a qualitative research approach to bring a deep and robust understanding of the complex issue under investigation. This research was ontologically driven by the notion that what transpires in the world is not equivalent to what people see. Therefore, the study adopted critical realism (CR) as a philosophical worldview. Moreover, the study used the case study research strategy to answer the research questions and a systematic review of documents to collect data, which it analysed inductively using thematic analysis. The findings suggest that TVET may enable good governance when curricula are infused with a nationalist ideology, spiritual culture and ethics. Moreover, the results suggest that TVET could enable environmental sustainability by facilitating ecological revolution through incorporating environmental sustainability education. Moreover, a TVET curriculum may likewise facilitate economic sustainability by enabling economic development through entrepreneurship. Lastly, TVET could advance social sustainability by enabling employability, which entails self-employment and industrial employment. The synthesis of findings leads to the conclusion that TVET could enable sustainable socio-economic development, thus enabling prosperity. Based on the findings and conclusions of this study, the researcher recommends that the South African government considers reintegrating basic and higher education to prevent fragmented governance of the TVET model. Bringing the two departments under one ministry could ensure effective and coherent pathways and implementation of TVET. The study further recommends that the National Treasury conducts an expenditure analysis and cost modelling exercise to ensure equitable funding of TVET in rural areas, urban areas, and previously disadvantaged communities. Additionally, the government should consider scrapping the pre-specified generic curriculum across the country and tailoring the curriculum content per student (considering the knowledge and skills the students already possess)—enhancing the students’ career prospects cognitively and innovatively while reducing the learning period.
Chapter
There is a broad consensus on the bad state of the biosphere. There is also agreement on the direction in which the relation between society and nature must be transformed. This relation must be made sustainable.
Article
Postpositivist critics have brought a new stridency to the ongoing discourse about the nature, applications, and usefulness of policy analysis. Regrettably, their critique is based on a decontextualized caricature, virtually a parody, of policy analysis training and practice. Their assertions are chilling but false, ideological rather than analytical, and detached from the inconvenient realities of policy making and management. Far from being narrowly technocratic and scientistic, policy analysis is dedicated to improving the craft of governance. It is fueled by intuition, argument, and ethical promptings; clearly engaged with the world of political action; and often identified with interests and values otherwise unrepresented at the table. Q-methodology and other approaches to values identification and analysis can be important contributors to policy analysis practice, but postpositivists have a very long way to go if they are to be relevant to the practical challenges of democratic governance that arise in the many roles that working policy analysts perform. (C) 1999 by the Association for Public Policy Analysis and Management.
Article
This book presents leading recent studies on the application of formal modeling for improved delivery of public services. It very significantly updates and extends the type of material found in "Operations Research for Public Systems" (MIT Press, 1967), which began to organize studies about public systems. Most of the chapters can be read with or without detailed consideration of their technical content. For clarity and compactness, much of the intermediate mathematical detail is referenced to other sources. The editors and authors have striven to make it possible for administrators, who may have limited analytic backgrounds, to use this book to develop their own views on the place of formal analysis in system planning and operation.Contents: "Introduction, " Alvin W. Drake, Ralph L. Keeney, and Philip M. Morse; "From Inside the System, " Frederick O'R. Hayes; "Analysis and Urban Government, " Peter L. Szanton; "Public Systems Analysis: A Consultant's View, " Martin L. Ernst; "Afterthoughts on Four Urban Systems Studies Performed with Small Cities, " W. Edward Cushen; "A Critique of Formal Analysis in Public Decision Making, " Ralph L. Keeney and Howard Raiffa; "Quantitative Models in Public Administration: Some Educational Needs, " Alvin W. Drake; "The New York City Fire Project, " Edward H. Blum; "Emergency Ambulance Transportation, " Keith A. Stevenson; "Improving the Effectiveness of New York City's 911, " Richard C. Larson; "Methods for Allocating Urban Emergency Units, " Jan M. Chaiken and Richard C. Larson; "Blood Bank Inventory Control, " John B. Jennings; "Library Models, " Philip M. Morse; "Efficient Operation of Runways, " Amedeo Odoni; "Post Office Mail Processing Operations, " Charles C. McBride; "Driver Accident Models and Their Use in Policy Evaluation, " Joseph Ferreira, Jr.; "Analysis of a Total Criminal Justice System, " Alfred Blumstein and Richard C. Larson; "Water Quality Management, " David H. Marks; "A Rational Approach for Government Decisions concerning Air Pollution, " Howard M. Ellis and Ralph L. Keeney; "Analysis in Health Planning, " Robert N. Grosse; "Puerto Rico's Citizen Feedback System, " John D. C. Little, Chandler H. Stevens, and Peter Tropp; "A Study of the Education Process: The Structure of a Lesson, " R. W. Revans; "Operations Research in University Planning, " Robert M. Oliver; "Use of Decision Analysis in Airport Development for Mexico City, " Richard deNeufville and Ralph L. Keeney.