ArticlePDF Available

An interdisciplinary framework for participatory modeling design and evaluation - What makes models effective participatory decision tools?

Authors:

Abstract

Increased requirements for public involvement in water resources management (WRM) over the past century have stimulated the development of more collaborative decision-making methods. Participatory modeling (PM) uses computer models to inform and engage stakeholders in the planning process in order to influence collaborative decisions in WRM. Past evaluations of participatory models focused on process and final outcomes, yet, were hindered by diversity of purpose and inconsistent documentation. This paper presents a two-stage framework for evaluating PM based on mechanisms for improving model effectiveness as participatory tools. The five dimensions characterize the ‘who, when, how, and why' of each participatory effort (stage 1). Models are evaluated as “boundary objects,” a concept used to describe tools that bridge understanding and translate different bodies of knowledge to improve credibility, salience, and legitimacy (stage 2). This evaluation framework is applied to five existing case studies from the literature. Though the goals of participation can be diverse, the novel contribution of the two-stage proposed framework is the flexibility it has to evaluate a wide range of cases that differ in scope, modeling approach, and participatory context. Also, the evaluation criteria provide a structured vocabulary based on clear mechanisms that extend beyond previous process- and outcome-based evaluations. Effective models are those that take advantage of mechanisms that facilitate dialogue and resolution and improve the accessibility and applicability of technical knowledge. Furthermore, the framework can help build more complete records and systematic documentation of evidence to help standardize the field of PM.
1
RESEARCH ARTICLE
10.1002/2016WR019373
An interdisciplinary framework for participatory modeling
design and evaluation—What makes models effective
participatory decision tools?
2
Stefanie M. Falconi AQ11
1
and Richard N. Palmer
2
3
1
Department of Geography and Environmental Engineering, The Johns Hopkins University, Baltimore, Maryland, USA,
4
2
Department of Civil and Environmental Engineering, University of Massachusetts, Amherst, Massachusetts, USA
5
Abstract Increased requirements for public involvementinwaterresourcesmanagement(WRM)overthe
6
past century have stimulated the development of more collaborative decision-making methods. Participatory
7
modeling (PM) uses computer models to inform and engage stakeholders in the planning process in order to
8
influence collaborative decisions in WRM. Past evaluations of participatory models focused on process and final
9
outcomes, yet, were hindered by diversity of purpose andinconsistentdocumentation. This paper presents a
10
two-stage framework for evaluating PM based on mechanismsforimprovingmodeleffectiveness as participatory
11
tools. The five dimensions characterize the ‘‘who, when, how, and why’’ of each pa rtici patory eff ort ( stage 1 ). Mo dels
12
are evaluated as ‘‘boundary objects,’’ a concept used to describe tools that bridge understanding and translate dif-
13
ferent bodies of knowledge to improve credibility, salience, and legitimacy (stage 2). This evaluation framework is
14
applied to five existing case studies from the literature. Though the goals of participation can be diverse, the novel
15
contribution of the two-stage proposed framework is the flexibility it has to evaluate a wide range of cases that dif-
16
fer in scope, modeling approach, and participatory context. Also, the evaluation criteria provide a structured
17
vocabulary based on clear mechanisms that extend beyond previous process-based and outcome-based evalua-
18
tions. Effective models are those that take advantage of mechanisms that facilitate dialogue and resolution and
19
improve the accessibility and applicability of technical knowledge. Furthermore, the framework can help build
20
more complete records and systematic documentation of evidence to help standardize the field of PM.
21
22
23
1. Introduction
24
A fundamental challenge in natural resources management is the integration of technical information and
25
effective public participation, a key component in democratic decision-making [National Research Council
26
(NRC), 2008]. While it is generally accepted that technical knowledge can lead to more informed and effec-
27
tive decisions, empirical studies also show that stakeholder inclusion/exclusion in decisions ‘‘rests on the
28
rules of engagement of stakeholders and the practices regarding the availability and accessibility of knowl-
29
edge’’ [Lemos, 2008]. This evidence highlights the need to improve practices of how scientific and technical
30
information are used and how stakeholders are engaged in democratic processes that support open
31
debates about alternatives. This paper discusses the role of models in addressing these challenges.
32
In the United States (USA), public participation has a long history in WRM beginning in the 1920s [Creighton
33
and Langsdale, 2009]. In the second half of the twentieth century, decentralized and participatory manage-
34
ment increased when project funding responsibilities shifted to state and local government resources
35
[Priscoli, 1989, 2004]. By then, USA’s landmark environmental laws greatly expanded the rights of citizens to
36
participate in decision-making [Davis et al., 1975; Ertel and Koch, 1976; Ertel, 1979]. These important laws
37
included the National Environmental Protection Act (1969), the Federal Clean Water Act (1972), the
38
Endangered Species Act (1973), and the policy document Principles and Standards for Planning (1973) [U.S.
39
Congress, 1969; U.S. Environmental Protection Agency, 1972; U.S. Congress, 1973; U.S. Water Resources Council,
40
1973]. Increased participation resulted in conflicts that eventually prompted federal agencies to invest in
41
alternative dispute resolution techniques to improve stakeholder involvement.
42
Emphasis on stakeholder participation is also a prominent feature of the European Water Framework Direc-
43
tive [Commission of the European Communities, 2000] and the international discourse on water policy—
Key Points:
!Paper presents a two-stage
framework that captures the
interdisciplinary nature of
participatory models
!The two-stage framework is applied
to five case studies to evaluate
mechanisms that improve model
effectiveness as participatory tools
!Effective models are those which
improve mechanisms for increasing
credibility, salience, and legitimacy of
technical knowledge
Supporting Information:
!Supporting Information S1
Correspondence to:
S. M. Falconi,
stefanie.falconi@jhu.edu
Citation:
Falconi, S. M. and R. N. Palmer (2017),
An interdisciplinary framework for
participatory modeling design and
evaluation—What makes models
effective participatory decision tools?,
Water Resour. Res.,53, doi:10.1002/
2016WR019373.
Received 21 JUN 2016
Accepted 18 JAN 2017
Accepted article online 24 JAN 2017
V
C2017. American Geophysical Union.
All Rights Reserved.
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 1
Water Resources Research
PUBLICATIONS
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 1
ID: vedhanarayanan.m Time: 22:01 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Revised proofs are sent only in the case of extensive corrections upon request
44
Integrated Water Resources Management (IWRM) [UN-Water, 2008]. Moreover, the guiding tenets of the
45
1992 Dublin Principles on Water and Sustainable Development, the basis for Agenda 21 at the Rio Earth
46
Summit, placed participation at the center of discussions on environmental management [United Nations,
47
1992; Global Water Partnership-TEC, 2004]. IWRM emerged as the dominant paradigm for managing water
48
worldwide through the democratization of decisions with an emphasis on three pillars: efficiency, sustain-
49
ability, and equity [UN-Water, 2008; Mukhtarov, 2009]. The Global Water Partnership echoes the USA’s Princi-
50
ples and Standards for Planning (1973) when it defines IWRM as ‘‘a process which promotes the coordinated
51
development and management of water, land, and related resources, in order to maximize the resultant
52
economic and social welfare in an equitable manner without compromising the sustainability of vital eco-
53
systems.’’ As interest in IWRM grows, the question of how to best engage stakeholders to improve adaptive
54
management and institutional capacity remains [Jeffrey and Gearey, 2006; Medema et al., 2008].
55
This paper differs from past studies in that it identifies mechanisms to improve model effectiveness in par-
56
ticipatory efforts derived from five empirical case studies. The unique contribution of this paper is a two-
57
stage analysis framework that characterizes the five dimensions of participation (stage 1) and then evalu-
58
ates participatory models based on the concept of ‘‘boundary objects’’ (stage 2). Boundary objects are
59
defined as translation devices at the interface of different organizations or groups that can act as bridges to
60
facilitate mutual understanding and cooperation [Huvila et al., 2014]. The goal of our analysis framework is
61
not to evaluate the outcomes of participatory modeling (PM), nor is it to identify the characteristics of the
62
‘‘best’’ models; instead, our objective is to propose and categorize common mechanisms that promote mod-
63
el effectiveness across a wide range of participatory decision-making contexts, forms of participation, and
64
types of computer models. We argue that a structured vocabulary and a multidimensional, interdisciplinary
65
evaluation framework can improve both the design and the documentation of participatory models in
66
water resources conflicts.
67
This paper is organized as follows: section 2 outlines challenges in evaluating participatory models and
68
reviews evaluation methods; section 3 provides a motivation for the proposed framework; section 4 outlines
69
stage 1 of our framework and characterizes the five dimensions of participation as they relate to models;
70
section 5 presents the concept of models as boundary objects, describes characteristics of effective PM, and
71
lays out corresponding criteria for stage 2 of the evaluation framework; section 6 uses the two-stage pro-
72
cess to evaluate five case studies; section 7 provides lessons from the evaluation framework; and section
73
8 summarizes and concludes the use of models for participation in WRM.
74
2. Challenges in Evaluating Participatory Models
75
The combination of public participation and models creates several challenges for the evaluation of PM
76
efforts. Indeed, the complexity of WRM problems, given their tendency to present unclear problem state-
77
ments, diverse stakeholders, contentious and/or multiple objectives, and negotiated solutions [Rittel and
78
Webber, 1973; Liebman, 1976; Pidd, 1999; Xiang, 2013], requires the interdisciplinary critical thinking and
79
problem-solving skills that have been identified as necessary to address other environmental problems
80
[Wolman, 1977].
81
Thirty years ago, Rogers and Fiering [1986] presented an analysis of the infrequent application of systems
82
models in government and real-world projects; they concluded that ‘‘models are not usually concerned
83
with what decision-makers care about’’ [Rogers and Fiering, 1986]. They argued for more practical models
84
that demonstrated greater flexibility in assumptions and applications. Their assessment remains accurate
85
today. Several past studies have tried to evaluate the benefits of models when used in participatory efforts.
86
However, designing an evaluation process has proven difficult, as real water conflicts have uncontrollable
87
variables and do not allow a ‘‘with’’ and ‘‘without’’ model comparison, so determining their effectiveness
88
requires careful interpretation of past case studies. The diversity of existing PM approaches complicates
89
evaluations (individual case and comparative studies), which are necessary to advance the field of PM. Previ-
90
ous work in PM has emphasized the evaluation of processes, intermediate outcomes, and resources out-
91
comes, and their distinction is presented next.
92
Creighton and Langsdale [2009] review 21 water supply and drought case studies for efficacy and identify
93
PM characteristics that are critical to the participation process. They recognize the limitations of docu-
94
mented case studies and recommend the need to address in future research how the level and method of
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 2
ID: vedhanarayanan.m Time: 22:01 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 2
95
stakeholder involvement affect: (1) the credibility and adequacy of the computer model, (2) the political
96
acceptance of the recommendations, and (3) the stakeholders’ ability to participate in the model building
97
exercise. Michaud [2009] provides methods for analysts to evaluate outcomes of PM efforts based on partici-
98
pant interviews and surveys. Michaud’s evaluation method relies on the similarity of the comparative case
99
studies, but his conclusions do not provide a structure for characterizing the purpose or nature of the partici-
100
patory effort. Carr et al. [2012] provide an extensive evaluation of participation (without models) based on the
101
classification of process, intermediate, and resource management outcomes assessments. We discuss these
102
three classifications in more detail, since their distinctions are critical in building our evaluation framework.
103
Process evaluations focus on how participation was conducted in terms of factors such as accountability,
104
agenda, facilitation, and clear ground rules under the assumption that processes that do well on those fac-
105
tors are more likely to yield good outcomes. Management outcomes evaluations are retrospective in nature
106
and gauge how well the ultimate objective (e.g., ecological, economic) was achieved. Evaluations based on
107
resource management outcomes can be more challenging and more limited than evaluations based on
108
process or intermediate outcomes because there is no control against which to measure the results (e.g.,
109
water shortage severity, environmental damage) and other factors outside of participation could influence
110
ultimate outcomes (e.g., regulatory or political changes). Intermediate outcomes are the incremental bene-
111
fits of participation, often unplanned, that can build trust, communication, networks, agreements, institu-
112
tional capacity, and other intangible outcomes.
113
Carr et al. argue that intermediate outcomes evaluations, though often overlooked, should play a more sig-
114
nificant evaluative role because their benefits become visible before ultimate outcomes. For this study,
115
intermediate outcomes are arguably the most interesting since they are directly indicative of the participa-
116
tory process and mechanisms.
117
3. Motivation
118
Taken together, Creighton and Langsdale [2009], Michaud [2009], and Carr et al. [2012] provide evidence of
119
the value of intermediate outcomes and the need to look beyond ultimate outcomes. These studies demon-
120
strate the need to evaluate PM, despite diversity in PM approaches, the wide range of goals of PM, and oth-
121
er inherent challenges mentioned. In response, we propose PM evaluation criteria based on process
122
mechanisms and on their relation to intermediate outcomes.
123
Public participation has been interpreted inconsistently within IWRM [Conca, 2006; Abers and Keck, 2013].
124
The field of PM emerged from the need to integrate stakeholder engagement in an open and transparent
125
discussion facilitated by structured processes that address the needs and objectives of collaborative
126
decision-making [Dreyer and Renn, 2011]. As democratization and public participation brought new actors
127
to the negotiating table, new tools, and methods were required to incorporate diverse knowledge and per-
128
spectives into the mathematical language and formal logic of models [Palmer et al., 2013]. These models
129
needed an interdisciplinary perspective to make the scientific knowledge and the insights they communi-
130
cate accessible and relevant in the policy process. Nevertheless, the disciplinary differences and the variety
131
of participatory contexts in which models are applied posed specific challenges. As a result, systematic eval-
132
uations cannot be based on a single disciplinary metric.
133
4. Stage One: Dimensions of Participation
134
The National Resource Council [NRC, 2008] identified five dimensions of public participation in environmen-
135
tal decision-making:
136
1. Who is involved?
137
2. At what stage in the planning process are the participants involved?
138
3. What is the degree and effort of involvement of the participants and organizer?
139
4. What extent of power and/or influence do the participants have in the decision process? and,
140
5. What are the goals and purposes guiding the participation process?
141
The following sections explore how to characterize each of these dimensions and their relationship to
142
modeling.
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 3
ID: vedhanarayanan.m Time: 22:01 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 3
143
4.1. Dimensions 1 and 2: Identifying Stakeholders, and Stages of Participation
144
Defining the appropriate public participants in public decision process is often challenging and rarely
145
straightforward. The goals of participation (see section 4.3) typically guide which participants are needed to
146
be successful (e.g., technical experts, civil society, interests groups). Public participation is an iterative pro-
147
cess that can take significant commitment and effort from everyone involved. Organizing public meetings
148
at the initial stages of the planning process is effective in identifying appropriate stakeholders. However,
149
analysts must be aware that stakeholder inclusion or exclusion may foster sociopolitical polarization that
150
can affect the long-term impacts and success of the planning process. There is no definitive prescription for
151
identifying and engaging stakeholders, but suggested methods exist to aid in balancing between inclusive-
152
ness and determining effective group size. Grimble and Wellard [1997] and Colfer et al. [1999] suggest meth-
153
ods for formally identifying, selecting, and engaging appropriate stakeholders. Herath [2004] provides a
154
systematic stakeholder selection process through focus groups and demographic analysis combined with a
155
snowball effect where early stakeholders suggest the subsequent stakeholders.
156
The timing of stakeholder and decision-maker engagement must also be defined. Simple classifications
157
(such as a priori, during, and posterior participation) have been suggested [Mendoza and Martins, 2006], as
158
well as more detailed classifications based on the five stages of model development [Hare, 2011]: (1) data
159
collection, (2) model definition, (3) model construction, (4) model verification and validation, and (5) model
160
application. In practice, clearly defined stages such as those described in Hare [2011] are often combined.
161
Examples that fall under one of the five-stage classification can be found in the literature. Models are used
162
to inform stakeholders about the system and the interactions between system components [Gilbert and
163
Bankes, 2002; Barreteau et al., 2010]. Models are also used as scenario-analysis tools for policymakers [Rajan
164
and Shibasaki, 2000; Verburg et al., 2004; Ligtenberg et al., 2009; Webler et al., 2011]. Occasionally participants
165
are involved in all five stages, and researchers cogenerate model inputs, structure, and results [Bousquet
166
et al., 1998; Hare et al., 2003; Lynam et al., 2007]. The problem, however, is that frequently, descriptions of
167
model development in the literature do not carefully identify when stakeholders were engaged. Instead,
168
researchers commonly use vague language and use the term ‘‘participatory modeling’’ in a generic manner
169
without specifying when or how participation took place [e.g., see Vennix et al., 1999].
170
4.2. Dimensions 3 and 4: Degree of Involvement and Extent of Influence
171
Degree of involvement and extent of influence are constructed scales of how participants’ engagement
172
translates into influence on decisions. For example, Arnstein’s ladder of participation [1969] ranks various
173
forms of public participation on an eight-step ladder from least to most influential. Generally speaking, Arn-
174
stein’s ladder and other constructed scales [see Cornwall, 2008] rank together the degree of involvement
175
and the extent of influence of stakeholder participation. However, these two factors are not easily separable,
176
because the degree and effort of involvement by participants and organizers often determines how much
177
stakeholders influence decisions.
178
For our case comparison, we apply the International Association for Public Participation’s influence and
179
involvement characterization to discuss these dimensions. These characteristics are deemed appropriate for
180
this context because, different levels of participation are appropriate for different resource management
181
problems as the characteristics relate to other constructed scales without ranking them (Table T11).
182
We note that among the several studies reviewed, many lacked explicit statements on these participatory
183
dimensions. This omission is problematic given that greater involvement and influence in the decision pro-
184
cess is precisely what makes stakeholders interested in participating. Even though researchers do not
185
always control participants’ influence and involvement in the final decisions, and despite the fact that a
186
project’s time frame may be too brief to observe its influence on outcomes, researchers must not omit
187
reporting on these two dimensions. Stating explicit targets for the stakeholder engagement process will
188
both facilitate studies’ (self) assessment and help study organizers to set clear expectations for all of those
189
involved.
190
4.3. Dimension 5: Goal and Purpose of Participation
191
The primary goals and purposes for public participation can range from simply informing stakeholders to
192
supporting meaningful dialogue that drives the decision-making process. Objectives are also influenced by
193
the unique context and setting of each case. Models cannot be expected to support processes for which
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 4
ID: vedhanarayanan.m Time: 22:01 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 4
194
they are not designed; hence, each case should be evaluated relative to its stated purpose. Empirical
195
reviews of PM studies show the importance of creating an initial clear purpose, because poorly designed
196
approaches can result in actions that address contradictory objectives [Dreyer and Renn, 2011].
197
Renn [2008] notes that participatory objectives are not consistent between disciplines and that ‘‘conflicts
198
about the best structure of a participatory process arise from overt or latent adherence to one or another
199
concept [or approach].’’ An anthropologist, economist, sociologist, and engineer may have different per-
200
spectives on how best to select participants and how best to define model structure. Renn’s [2008] analysis
201
provides significant insight into the disciplinary and theoretical underpinnings of: (1) how problems are for-
202
mulated, (2) who is considered an important stakeholder, and (3) what the objectives and rationales for par-
203
ticipation are. These disciplinary perspectives and methodological paradigms impact the objectives of PM
204
and create a wide range of participatory efforts. Ultimately a ‘‘researcher’s perception predetermines which
205
problems are perceived, how they are perceived, and approaches toward the research’’ [Prell et al., 2007].
206
Since disciplinary perspectives impact problem definition, ‘‘the choice of a modeling paradigm might result
207
in the exclusion of many relevant bodies of learning’’ [Hisschem
oller et al., 2001].
208
5. Stage Two: Attributes of Successful Participatory Models
209
5.1. Models as Boundary Objects
210
The concept of models as ‘‘boundary objects’’ is introduced to evaluate their effectiveness as collaborative
211
tools. Boundary objects are defined as interactive and adaptable objects that enable mutual understanding
212
and collaboration in diverse group efforts. A map can be a boundary object that facilitates a group of people
213
meeting for dinner; likewise, models can enable participants to converge on a strategy in water management
214
conflicts. Previous research on boundary objects establishes their role in helping users to: (1) establish a
215
shared syntax or language; (2) identify differences, concerns, and relationships clearly; and (3) transform their
216
current collective knowledge toward an agreement of facts through discussion, negotiation, and careful scruti-
217
ny of what they know [Star and Griesemer, 1989; Carlile, 2002]. Research has shown that boundary objects
218
employ diverse types of knowledge in problem-solving [Star, 1989], innovation [Carlile, 2002], cooperation
219
and consensus building [Fuller, 2009], and collective solutions to environmental problems [Bacic et al., 2006].
220
5.2. Attributes of Successful Participatory Models
221
In this research, criteria are derived from boundary objects research to illustrate a novel application of the
222
concept to evaluate participatory models. Based on Cash et al. [2003] research in sustainable sciences field,
Table 1. Levels of Involvement and Extent of Influence in Public Participation
a
Inform Consult Involve Collaborate Empower
Level of influence
public
participation
To provide the public with
balanced and objective
information to assist in
understanding the
problem, alternatives,
opportunities, and/or
solutions
To obtain public feedback
on analysis, alterna-
tives, and/or decisions
To work directly with the
public throughout the
process to ensure that
public concerns and
aspirations are consis-
tently understood and
considered
To partner with the public
in each aspect of
decision-making includ-
ing the development of
alternatives and the
identification of the
preferred solution
To place final decision-
making in the hands of
the public
Promise to the
public
We will keep you
informed
We will keep you
informed, listen to and
acknowledge concerns
and aspirations, and
provide feedback on
how public input influ-
enced the decision
We will work with you to
ensure that your con-
cerns and aspirations
are directly reflected in
the alternatives devel-
oped and provide feed-
back on how public
input influenced the
decision
We will seek your direct
advice, recommenda-
tions, and innovation in
formulating solutions
and incorporate these
into the decisions to
the maximum extent
possible
We will implement what
you decide
Typical techniques
applied
Fact sheets Public comment Workshops Citizen advisory
committees
Citizen juries
Web sites Focus groups Deliberative polling Consensus building Ballots
Open houses Surveys Participatory decision-
making
Delegated decisions
Public meetings
a
Adaptation from International Association for Public Participation [2005].
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 5
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 5
223
three metrics of success for boundary objects in public policy relative to their ability to convey scientific
224
knowledge: (1) credibility, (2) salience, and (3) legitimacy. Table T22 summarizes how each case study rates
225
based on these three. While these metrics are derived from characteristics described by Cash et al. [2003],
226
these ideas are also present in the list of intermediate outcomes (referenced as ‘‘products of process’’) given
227
by Carr et al. [2012]. In the present study, we expand these characteristics into actual criteria. Each is
228
addressed in the subcategories of metrics that we developed (Table 2) based on extensive reading of other
229
case studies and their relative success. The mechanisms presented are not intended to be an exhaustive
230
list, but a list is provided as a descriptive guide. Each criterion is described and justified, and the five metrics
231
are used to gauge the success of a case study in meeting each criterion.
232
The concept of credibility [Cash et al., 2002] is related to the quality of information. Here we develop the
233
idea of credibility specifically for models in the context of PM. A model that fails to capture a system with
234
an appropriate level of accuracy, precision, unambiguity, completeness, and complexity will be judged less
235
credible. Discussing a model’s credibility will uncover knowledge gaps, identify crucial issues, and reveal
236
any discrepancies in the way stakeholder define the problem. A model’s credibility is based on its contents
237
being derived from trusted sources. If judged to be credible, models can provide a common language for
Table 2. Three Evaluation Criteria Further Subdivided into 15 Metrics for Judging PM Success
Criteria Description
Case study evaluation
Zimbabwe ACT-ACF Las Vegas Solomon Isl. Senegal
Credibility
C1 Identifies knowledge gaps, crucial issues,
and discrepancies in problem
understanding
High High Medium High High
C2 Builds shared understanding of facts and
language as the starting point for
discussions
High High High High High
C3 Uses data/information derived from
trusted sources
Medium High High High High
C4 Provides means for stakeholder inclusion
in relevant dialogues
Medium High Low Medium High
C5 Promotes communication, enhanced
credibility and accountability of
information, and bridges gaps in new
perspectives
Medium High Medium High High
Salience
S1 Builds effective and frequent communi-
cation channels in a two-way
dialogue
High High Low Medium Medium
S2 Translates information and technology
results to address end-user needs
Medium High Medium High Medium
S3 Incorporates diverse knowledge from a
range of users
Low Medium Low High High
S4 Results in a single-text document as an
agreement on a set of facts and a
platform for coproduction
High High Medium Medium High
S5 Helps link relevant questions to end
users to answers model can
accurately provide
Medium High Low High High
Legitimacy
L1 Provides open and transparent criteria
for decisions and rules of conduct
High Medium High High High
L2 Allows real-time criticisms, feedback, and
update mechanisms
High High High Medium High
L3 Accommodates new information/prefer-
ences through model flexibility and
acts as educational tool to users
High High High Medium Medium
L4 Analyzes alternative scenarios to create a
collaborative environment and
converge on solutions
High High High High High
L5 Elucidates decision process through
forum, provides insights rather than
‘‘optimal’’ solutions
High High Medium High High
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 6
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 6
238
detailed discussions of the system under study, enhance and deepen stakeholders’ discussions, and encour-
239
age identification of which information is considered reliable and which is less reliable.
240
The concept of salience [Cash et al., 2002] is related to the relevance of the information—in this context,
241
information that is provided by a model—to the needs of stakeholders and decision makers. In this context,
242
a model is salient when coproduction between stakeholders and analysts allows stakeholders to use the
243
model to improve their understanding of the system and articulate the questions they want the model to
244
answer. Salient models build effective and frequent communication channels between the stakeholders
245
and allow diverse users to interact. A salient model ideally provides a platform where information can be
246
coproduced by stakeholders and model builders, so that the model becomes a repository for agreed upon
247
facts, described as a ‘‘single-text negotiation document’’ [Bourget, 2011].
248
The concept of legitimacy [Cash et al., 2002] involves participants’ trust in the neutrality of the information,
249
organizations, and/or processes. Translating the concept into the context of models used in participatory
250
exercises, legitimacy involves participants’ trust in the neutrality of model outputs. A legitimate model is fair
251
in its treatment of stakeholders’ opposing views and divergent values, unbiased in its representation of
252
preferences and interests. Legitimate models provide transparent and clear assumptions. Such models are
253
highly flexible, allowing for real-time changes and feedback, and can easily change assumptions and algo-
254
rithms. Legitimate models are capable of serving as educational tools for stakeholders and policy-makers
255
because they were cogenerated with analysts and not constructed in isolation. They also generate insights
256
such as new understanding of the system, not just answers. Lack of confidence in a model by participants is
257
not uncommon until participants can see the potential value of models as analytical tools. Confidence
258
comes from understanding the process and confirming model outputs with intuition or past experience
259
rather than what may seem ‘‘magical answers.’
260
By establishing 15 metrics, stage two of our framework transforms the previously identified characteristics
261
of credibility, salience, and legitimacy into three evaluation criteria that may be used to capture ‘‘success.’
262
Success is judged by how well a model increases understanding across disciplines, enables two-way dia-
263
logue, enhances legitimacy of decisions, and builds cooperation in its various forms (e.g., consensus, shared
264
vision, end to stalemate). These are important components in building trust, a key factor in institutional
265
cooperation and governance in the common-pool resource literature [Ostrom, 1999].
266
6. Case Studies
267
6.1. Methods and Case Selection
268
Evidence from case studies has several advantages given the depth of insights it provides on the underlying
269
drivers, even though broad conclusions are limited by the small sample size [Srinivasan et al., 2012]. These
270
five cases are not intended to be representative of all participatory models type; the focus is limited to well-
271
recognized, structured, quantitative models within systems analysis methods (in a parallel analysis of sys-
272
tems models described in Rogers and Fiering [1986] and Rogers [1978]).
273
The following selection criteria were applied in selecting the five case studies: the study needed at least
274
one peer-reviewed published paper; planning studies were desired that were based on realistic scenarios
275
and made use of a computer model; the degree of involvement was representative of one of the five cate-
276
gories (Table 1); stakeholder involvement was documented in enough detail and not limited to model
277
outputs.
278
Furthermore, the five case studies have a breadth of modeling approaches and diversity in participatory
279
context that revealed wide variation in the effects we expected to observe among PM approaches. The dis-
280
cussion of each case study is organized as follows: First, we present background for each case study and a
281
description of the participatory dimensions and model characteristics. Then, we evaluate the relative merits
282
of each model and the summary of the two-stage evaluation based on the evidence provided.
283
6.2. Community-Based Forest Management in Zimbabwe
284
6.2.1. Case Background
285
The first case study is a community-based, forest management study in the midlands of Zimbabwe. It com-
286
bines participation with traditional optimization methods using multicriteria decision analysis (MCDA) tech-
287
niques [Mendoza and Prabhu, 2005]. Organizers from the University of Illinois Urbana-Champaign held six
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 7
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 7
288
meetings with district-level government personnel from varied backgrounds that had at least 10 years of
289
knowledge of the problem. The researchers led the problem-structuring phase (preference and value elicita-
290
tion) using cognitive mapping and influence diagrams and the problem-solving phase using MCDA prioriti-
291
zation techniques.
292
MCDA was used to obtain diverse participant knowledge, to elicit participant preferences, and to aggregate
293
individual choices to evaluate, prioritize, and aggregate the indicators identified by participants. Soft-
294
systems methods involved the Collaborative Vision Exploration Workbench (Co-View) software that involves
295
a simulation framework for integrating objectives and other components of strategic planning, mainly
296
Strengths, Weaknesses, Opportunities and Threats (SWOT) indicators. Co-View allowed for a scenario-based
297
analysis of the problem, giving participants an appreciation for the model’s potential as an analytical tool to
298
compare various policies. Also, the authors claim participants developed a level of trust for the model, since
299
they perceived it as a transparent tool capable of ‘‘generating insights rather than magical answers’
300
[Mendoza and Prabhu, 2005].
301
6.2.2. Participatory Dimensions and Effective Model Characteristics
302
The purpose of the model was to create transparency within participatory processes as a means to build
303
trust, confidence, and integrity in the planning process. More specifically, transparency and trust came from
304
a model built with stakeholders to evaluate openly the cumulative impact of proposed management action
305
plans. Mendoza and Prabhu led the organizing team. The core participants were six district-level govern-
306
ment personnel, including two district foresters, one agricultural extension officer, one social scientist, and
307
an expert on gender issues who was also a provincial officer.
308
Participants were engaged in three stages: model formulation and structuring, model building, and model
309
use. Participants identified problems, prioritized preferences using MCDA techniques, and quantified pro-
310
cess indicators via SWOT. The degree of involvement was moderate but the level of influence was to ‘‘consult’’
311
based on the typology described in Table 1. While the MCDA was used to involve participants’ diverse con-
312
tributions, elicit preferences, and aggregate individual choices, in the absence of any statement of its role, if
313
any, in the actual management decisions is unclear.
314
In building credibility, the analyst began engagement with a free thinking exercise on cognitive mapping to
315
get stakeholders to frame the problem in terms relevant to them and then used the model to narrow the
316
scope. The model allowed participants to iteratively test and discuss how each management plan met
317
agreed upon purposes and specific goals. The iterative process had two positive effects that improved the
318
model’s saliency. First, it clarified ideas and helped focus the debate. Second, model runs generated rapport
319
as participants gained appreciation for the trade-offs between goal optimization and costs of executing
320
plans. The low and medium scores were given because despite the title—which proposes a ‘‘community-
321
based forest management’’—there were no stakeholders who were not government personnel.
322
The model served as an organizing platform where ideas, opinions, and divergent views were debated and
323
validated by those participating, so that the exchange of ideas was mutual and instructive. The designed
324
plans under a collaborative environment proved central to strengthening legitimacy. The participants recog-
325
nized the benefits and value of the model’s ‘‘what-if’’ and ‘‘if-then’’ features for analyzing policies, because
326
they allowed them to develop a realistic and insightful perspective that built their trust in the model.
327
6.2.3. Evaluation of Relative Merits and Shortfalls of PM Effort
328
Acknowledging the multiple objectives in resource management, this case made use of MCDA to encour-
329
age a systematic arrangement of criteria and decisions, organizing priorities and evaluating the multiple
330
objectives. MCDA quantified each element that influenced the decisions, and generated insight into desir-
331
able and undesirable strategies and action plans.
332
There are two common pitfalls to optimization that the case study avoided, at least partially. First, a com-
333
mon criticism of optimization is that the ‘‘best’’ optimal solution is determined by an analytical structure
334
that may not fully capture the complexity of the actual problem under investigation. Second, optimization
335
models can be particularly compatible with problems dominated by economic objectives and the assump-
336
tions of rational choice theory [Hisschem
oller et al., 2001; Rogers et al., 2009]. The authors, hence, adapted
337
the optimization model to prioritize and aggregate options but integrated the modeling process with cog-
338
nitive mapping to engage stakeholders early on during the problem-structuring phase, and elicited from
339
the participants’ assessment qualitative and semiquantitative indicators.
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 8
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 8
340
These adaptations resulted in a mixed model that was useful for stakeholders’ purposes and interests since
341
they contributed to the subtleties regarding assumptions of the modeling framework and the criteria by
342
which action plans would be assessed. During the Co-View simulations, decision elements demanded surro-
343
gate measures (i.e., quantifiable values), so that improved plans could be judged based on the final man-
344
agement goal of ‘‘increased total revenue.’’ That information was combined with the Co-View model that
345
simulated how the chosen criteria affected the ultimate goal. The authors recognized that this revenue opti-
346
mization was a simplification that made the analysis tractable and that it would pose significant challenges
347
if the final indicator of success had been strictly qualitative in nature. Nevertheless, the integrated methods
348
allowed participants to assess and gain insights into the plan strategies. In the end, with assistance from the
349
model, participants realized the delicate balance between achieving an objective and its costs.
350
The two-stage evaluation was limited given the insufficient records to completely characterize this work. An
351
additional working paper [Mendoza and Prabhu, 2002] reports on a postcompletion valuation process with
352
20 participant interviews. Based on their valuation follow-up, study authors assert that the intended pur-
353
pose of building trust through a transparent process was achieved. Authors also assert that another advan-
354
tage observed was the ability to introduce flexibility to systems analysis to balance analytical solutions with
355
other important end-user concerns. However, documentation and material made available lacked specificity
356
about whether the model influenced any management decisions.
357
6.3. Shared Vision Model for the Tri-State Water Conflict in the ACT-ACF River Basin
358
6.3.1. Case Background
359
The Alabama-Coosa-Tallapoosa and Apalachicola-Chattahoochee-Flint (ACT-ACF) river basins made use of a
360
Shared Vision Model (SVM) as part of a Shared Vision Planning (SVP) process to codevelop simulation mod-
361
els by representatives of federal and state government [Palmer, 1998]. To avoid a federal court lawsuit initi-
362
ated by the state of Alabama over water allocations made by the state of Georgia, the US Army Corps of
363
Engineers (USACE) and the states of Alabama, Florida, and Georgia adopted a Memorandum of Agreement
364
to conduct a $13.5 million, 6 year, Comprehensive Study. A direct consequence of the Comprehensive
365
Study, and its most notable legacy, was an interstate water compact ratified by the U.S. Congress and
366
signed into law by the governors of the three states and the USA president.
367
SVP includes three processes: (1) traditional water resources planning, (2) structured public participation,
368
and (3) collaborative computer modeling [Bourget, 2011; Werick and Palmer, 2011; Palmer et al., 2013]. The
369
SVM was a systems dynamic model (SDM) built in STELLA that simulated a complex network of rivers and
370
the services they provide, including navigation, flood control, environmental services, and Atlanta’s growing
371
water demand from Lake Lanier. Five models were developed and modified over time to improve commu-
372
nication and build rapport between the core participants and the research team. The final model allowed
373
the working group to evaluate and formulate new alternatives. The model’s user interface was constructed
374
to allow modification of control variables and parameters. This feature gave participants freedom to formu-
375
late, evaluate, and refine alternatives, including combined operating features, which resulted in a new set of
376
options not previously considered [Hamlet et al., 1996].
377
6.3.2. Participatory Dimensions and Effective Model Characteristics
378
The broad purpose of the study was to solve a multistate/federal conflict over water management by identi-
379
fying innovative reservoir operating policies that met the needs of the region. More specific goals included:
380
(1) create a catalog and repository for important data (hydrologic information, demand/supply data, etc.),
381
(2) characterize the physical features of the basin, (3) document the system’s operating policies, (4) evaluate
382
alternatives, (5) illustrate trade-offs between system objectives and seek compromise solutions, and (6)
383
expand the number of people who understand the system’s operation.
384
The models were built by researchers from the University of Washington in consultation with representa-
385
tives of the states of Georgia, Florida, and Alabama, and the USACE Institute of Water Resources. The core
386
participants of the study were members of an Executive Coordinating Committee created for the long-term
387
planning of the interstate watershed composed of two representatives from each of the state governors
388
and two representatives from the USACE. There was also a separate committee of four members composing
389
the Technical Coordination Group (TCG), one from each of the state agencies and the USACE.
390
The study was originally commissioned to avoid a lawsuit between the study partners. A memorandum of
391
understanding (MOU) was used in staying the lawsuit, and the participants were engaged in all five stages
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 9
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 9
392
of participation as part of the MOU. This meant that from the problem definition, to data gathering during
393
model construction, to running simulations for validation, participants were actively engaged in attempting
394
to reach an agreement.
395
There was a high degree of involvement by participants, categorized as collaborative. Initially, the TCG
396
defined large scopes of the work but failed to agree on overarching measures of performance for reservoir
397
operating rules and water allocation policies. Over time, instead of formulating specific reservoir operations,
398
they requested a more flexible SVM that could facilitate the analysis of new alternatives. Participants had
399
very high involvement in model building and could make recommendations, but their influence was low as
400
final decisions were the purview of the governors and legislators.
401
The model was widely accepted as an accurate and credible representation of the system, but only after a
402
series of tests to compare the reservoir levels and estimated flows. Comparative tests were conducted
403
between the STELLA model and the HEC5 model previously used. The SVM proved to be more accurate and
404
better suited to account for multireservoir operating rules. The research team emphasized active participa-
405
tion, incorporating stakeholder critiques in new versions of the model. This resulted in an SVM that was
406
salient and some members of the TCG became skilled in modifying the assumptions of the model in order
407
to answer their questions.
408
The model was widely recognized for its legitimacy among stakeholders. These stakeholders included the
409
primary management agencies and expert consulting agencies such as the U.S. Fish and Wildlife Service
410
and the Nature Conservancy (TNC). Another example of the improved collaborative environment is TNC’s
411
effort to develop metrics that supported the ranking of alternative reservoir operating policies making use
412
of the SVM in partnership with researchers at the University of Washington.
413
6.3.3. Evaluation of Relative Merits and Shortfalls of PM Effort
414
SDMs create platforms for enriching discussion between analysts and users based on visual interfaces that
415
clarify the relationships among variables, alternative future scenarios, and the possible trade-offs. Modern
416
SDMs have evolved from the early 1960s when FORTRAN was the modeling language of choice, to object-
417
oriented environments (e.g., STELLA, POWERSIM, VENSIM) and programming languages designed specifi-
418
cally for making dynamic simulations more accessible. Accessibility leads to significant consequences. For
419
example, in this case clarifying that increasing Atlanta water use would in fact not substantially reduce base
420
flow to other major rivers (the Flint River), which was a widespread belief. While STELLA is object oriented, it
421
does require basic programing skills. Thus, the STELLA model was integrated with an easy-to-use Excel inter-
422
face, which allowed stakeholders to participate in characterizing the problem and appropriately defining
423
feedback loops while also tailoring model outputs through Excel.
424
The two-stage evaluation shows the SVM was not an effective boundary object for its intended long-term
425
management purpose. While the insights generated by the SVM built consensus, the process also managed
426
to insulate the core participants from influencing decisions when their role was seen as advisors rather than
427
decision-makers. The model was effective in achieving the more moderate research goals. Despite some ini-
428
tial resistance, the SVM modeling platform contributed to improved dialogue as core participants engaged
429
in repeated testing of hypotheses and virtual experimentation, making the SVM widely trusted. To some
430
extent the SVM is still used today [Leitman, 2008; Leitman and Kiker, 2015]. Nevertheless, although partici-
431
pants joined to determine a common purpose, the effort did not anticipate that the interstate Compact
432
would establish water allocation agreements outside the Comprehensive Study. The role of the core partici-
433
pants was effectively transformed into that of advisors who could not execute plans when key actors and
434
political tides shifted.
435
6.4. Water Management Alternatives in Las Vegas, Nevada
436
6.4.1. Case Background
437
The third case study is the development of a model to evaluate alternatives to extend the life of the water
438
supply for the city of Las Vegas [Stave, 2003]. The author and team at the University of Nevada, Las Vegas
439
developed an SDM to compare the relative merits of different policy options. The SDM engaged participants
440
to re-evaluate the starting conditions or change their preferences in an open exploration of policy options.
441
During ten workshops the team tested the effectiveness of the SDM. Different alternatives for water conser-
442
vation (indoor/outdoor use, hotel use, etc.) were modeled, and, as the stakeholders gained confidence, they
443
made suggestions on new hypotheses to test, although the model did not permit the addition of such new
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 10
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 10
444
alternatives. The results provided several insights counter to stakeholders’ intuitions. For example, casino
445
and hotel water use was not as significant as residential use, and small changes in per capita demand were
446
as effective as large supply increases.
447
6.4.2. Participatory Dimensions and Effective Model Characteristics
448
The primary purpose of the model was to test the effectiveness of an SDM to support stakeholder learning
449
and discussion but was not intended as a decision support tool. During workshops, participants were
450
engaged in evaluating the relative merits of extending the year in which demand would outstrip supply
451
(denoted as the ‘‘crossing-point’’) based on the given policy options available.
452
The study organizers included researchers at the University of Nevada, Las Vegas in consultation with the
453
Southern Nevada Water Authority and the U.S. Bureau of Reclamation. The core participants were recruited
454
for research purposes and comprised 83 community members and residents who ranged widely in age and
455
profession. The stages of participation were compartmentalized. In the first two stages, experts provided
456
data and defined the problem, but they did not take part in the subsequent workshops. Then, participant
457
groups were involved in model validation and analysis of acceptable policy options. All 10 workshops fea-
458
tured different participants each time.
459
The degree of involvement and extent of influence in this case study were intentionally established to be low.
460
The degree of involvement falls in the inform category, given that the workshops were 2 day events, yet
461
featured new attendees each time. This is in contrast with other case studies presented that attempted to
462
maintain the same core participants throughout the duration of the study. As a communication tool for the
463
public, participants were engaged in the later stages of the model’s use to learn about policies that changed
464
future demand/supply outcomes, yet this engagement was not intended to change any policy decisions.
465
The model built credibility by providing a shared language and starting point for discussion. However, it was
466
not inclusive because it did not reflect stakeholders’ engagement in problem definition or in alternative
467
generation. The model demonstrated poor saliency because its design did not allow participants to apply
468
their knowledge in defining alternatives or addressing questions/problems relevant to their specific situa-
469
tion. Instead, participants were given a predefined problem to solve and five policy options to solve it; two
470
options were new alternative supplies and three options addressed water demand reduction or
471
conservation.
472
The simplicity of the model contributed to its legitimacy as participants quickly grasped the effects of
473
changing parameters, trade-offs, and policies to move the supply/demand crossing point. The model’s suc-
474
cess in building legitimacy was self-reinforcing. As participants gained confidence, trust in the usefulness
475
and results of the model increased and fueled greater interest and engagement from participants. When
476
the model challenged their views on the problem, they saw this as an incentive to ask more questions and
477
make new suggestions. Legitimacy was reduced, however, by the inability to introduce stakeholder-
478
designed alternatives.
479
6.4.3. Evaluation on Relative Merits and Shortfalls of PM Effort
480
Simulation models are favored in stakeholder engagement because of their ability to covalidate model
481
building and to structure and improve the content and timing of discussions [Jenkins-Smith and Sabatier,
482
1994; Dwyer and Stave, 2008]. Several SDM platforms use object-oriented environments and are more visual
483
in nature. These effective characteristics of simulation models have led to their growing application in
484
resource management such as: Nandalal and Simonovic [2003] who use system dynamics in water conflict
485
resolution for a hypothetical collaborative decision-making case; Palmer et al. [1999] in river-basin planning;
486
Stave [2002] for public participation in transportation and air quality management decision-making; and
487
Bolson and Broad [2012] for a South Florida regional water management model. Winz and Brierley [2007]
488
present an in-depth review of theoretical and practical developments of SDMs in the past 40 years, which,
489
along with other cited authors herein, suggest that SDMs are particularly suitable for stakeholder participa-
490
tion and a favored method among analysts.
491
In the Las Vegas case, participants were presented with visuals of the decision variables, parameters, con-
492
straints, feedback loops, and outputs that enriched the discussion. Stave [2003] suggests that the model
493
altered the participants’ understanding of the fundamental drivers of water conflicts. Participants were sur-
494
prised with the finding that solely increasing supply was not the best policy option. The simulations provid-
495
ed several counterintuitive results that made for lively discussions. Consequently, the opportunity surfaced
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 11
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 11
496
among participants to reconcile starting assumptions and to become receptive to possible solutions that
497
were not deemed acceptable before.
498
The two-stage evaluation shows that the Las Vegas SDM was able to achieve its modest purpose, but any
499
larger impact was limited by a narrow scope of a proof-of-concept engagement. Our conclusion is congru-
500
ent with follow-up communication with the author. The primary published paper states explicitly that par-
501
ticipation in the modeling process was opened to community participants at the last stage of policy
502
analysis and not in earlier stages such as problem definition or model construction. The goal was to inform
503
participants by engaging them in the modeling effort to understand how different policies affected the
504
crossing-point of demand and supply. The model’s simple and intuitive nature helped participants under-
505
stand the problem of demand/supply from a new perspective and appreciate the results even when they
506
did not align with their own paradigm. This is apparent from participants’ ability to engage and the fact
507
that they proposed new testing scenarios in real time, which implies that they had developed a new under-
508
standing and insight into the system. While the model was credible, participants had no influence on deci-
509
sions or policies, and experts rather than stakeholders determined the relevant model and the policy
510
options. Thus, the legitimacy and saliency were affected
511
6.5. Water Resource Allocation in the Solomon Islands
512
6.5.1. Case Background
513
In the Solomon Islands case, local managers of the Kongulai water catchment make use of a water resource
514
allocation model constructed to support decisions [Chan et al., 2010]. The catchment is approximately
515
50 km
2
, is located upstream of the capital city Honiara, and provides about 60% of the capital’s water sup-
516
ply. Participants included landholders, local government officials, and donor and nongovernmental organi-
517
zations. Historic conflicts exist between the clans living in the catchment and the Solomon Island Water
518
Authorities (SIWA), including protests over inadequate or late royalty payments and sabotage of the water
519
infrastructure on the part of the landowners. SIWA’s past community engagement efforts were limited and
520
unsuccessful. Over the two-and-a-half-year study, the research team engaged participants in an inclusive
521
and culturally sensitive manner to avoid resurfacing of past tensions.
522
Stakeholder groups were first invited to participate separately to describe their challenges. The research
523
team used these descriptions to build conceptual diagrams, which participants reviewed before analysts
524
built the Bayesian model. Emphasis was initially placed on problem formulation. Then, a Bayesian Network
525
(BN) was used to generate and analyze defensible scenarios informing catchment management planning.
526
Five months later, a small representative group of participants (about one-third of the original group) met
527
for a second time to edit, verify, and comment on the final BN. The organizers recruited a cultural guide to
528
facilitate workshops. Also, to be culturally sensitive and promote inclusion, several of the workshops were
529
divided into groups based on clans and gender.
530
6.5.2. Participatory Dimensions and Effective Model Characteristics
531
The purpose of this study was to improve information and data collection on water use to prioritize poten-
532
tial management actions. The model was not intended to address negotiations over previous conflicts
533
regarding resource royalty payments. The project was organized by researchers from two different Austra-
534
lian Universities in collaboration with the Solomon Water Authority, the Australian Water Resources Facility,
535
and the AusAID research initiative within the International Water Centre. The core participants include com-
536
munity landowners (two subclans living within and below the catchment area), government agencies, and
537
donor and nongovernmental organizations. A cultural guide who also served as an interpreter was recruited
538
to help conduct the participatory research within the appropriate context.
539
The stages of participation in this case were fragmented. The first stage emphasized engaging participants
540
in building conceptual diagrams based on their problem definition. However, to avoid the resurfacing of
541
past tensions, stakeholder groups were engaged separately by local community, government representa-
542
tives, and donor and nongovernmental organizations. Subsequently, the diagrams were merged to build a
543
BN and a smaller, yet representative, group of participants was asked to join in a second iteration for edit-
544
ing, verifying, and commenting on the final BN model.
545
The degree of involvement can be characterized as involved in general, with some groups invited to be collabora-
546
tive given the level of information elicited from those participants. Participants were engaged to define impor-
547
tant stages like establishing the problem and ‘‘common terms of reference,’’ and, for a smaller group, evaluating
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 12
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 12
548
policy alternatives. This case study was different in how it involved stakeholders in a more fragmented manner,
549
effectively separating who was involved in model inputs and outputs. As a result, the extent of influencing deci-
550
sions is different for managers and all other stakeholders.Managerswereexpectedtoremainwiththemodel
551
after the research team left, so their engagement and influence was important. The model enabled managers
552
to prioritize interventions. The same influence cannotbestatedforthecommunitygroupsandNGOs;theyused
553
the model at a conceptual level as an information tool rather than a decision tool.
554
The deliberate inclusiveness in the early stages positively impacted the credibility of the model. Despite past
555
acrimonious relations, diverse stakeholders were able to engage in constructive dialogue and establish
556
common ground on the facts. The fragmented nature of the stages of participation, however, left some
557
doubts about the extent of inclusion of participants in relevant and influential dialogue later on.
558
Saliency was judged on how well the model captured diverse knowledge and relevant questions. In this
559
respect, the ability to bring people together to communicate and agree on facts showed that given the
560
right environment and platform, participants found common ground to overcome differences to collabo-
561
rate in new and unexpected ways.
562
The model’s legitimacy ranked high in most criteria, since providing transparency and accountability in the
563
process also improved relationships. The model created a ‘‘shared vision’’ over water plans and played a
564
role in establishing interventions and priorities in infrastructure, making catchment and supply manage-
565
ment decisions, and, to some extent, influencing the behavior of the water users. These positive changes in
566
participant relationships reinforced themselves and made the modeling effort more legitimate.
567
6.5.3. Evaluation of Relative Merits and Shortfalls of PM Effort
568
BN modeling has found many applications in participatory ecological risk assessment because of its graphical
569
representations and its ability to integrate stakeholders’ conceptual diagrams of the problem definition. Cas-
570
telletti and Soncini-Sessa [2007] and Uusitalo [2007] provide other examples of BN in environmental applica-
571
tions. BN models are graphical models that represent probabilistic systems operating under uncertainty. They
572
have the ability to automate probability updates with new observations, providing improved model accuracy
573
with new iterations. Another advantage is the low formal data requirement of BNs, which makes them suit-
574
able in limited data contexts such as this one. In this case study, the limited availability of data, high uncertain-
575
ty, and the initial incomplete understanding of the system persuaded researchers to use a BN model.
576
Initially, a ‘‘staged knowledge-building process’’ was used to elicit knowledge and views from participants, which
577
were later combined with quantitative data about the physical characteristics of the catchment area. The con-
578
cept diagrams, developed separately from different groups and perspectives, were later merged to create one
579
comprehensive network diagram. In subsequent sessions,theanalystsintroducedperspectivesfromprevious
580
groups for comparison and to underscore common views. This step proved to be a significant challenge; how-
581
ever, once progress was made in individual groups, the groups were reunited in a follow-up session to simplify
582
the model within smaller representative subgroups. In the final workshop, a small group of water professionals
583
with technical backgrounds was summoned to determine the initial conditional probabilities. The model was
584
crucial in creating a single-text document agreeing on the set of facts; however, the fragmented nature of par-
585
ticipation left unclear the extent to which all representatives influenced decisions in later stages.
586
The two-stage evaluation showed that BN modeling effort was successful at meeting its purpose of
587
enabling managers to prioritize interventions. A separate publication [Hoverman et al., 2011] by the authors
588
that evaluated the participatory effort provides several examples of their success, such as the benefits of
589
bringing transparency and accountability to the process, resulting in improved credibility and legitimacy of
590
the model. Though not an original study goal, an independent forum composed of the same participants
591
came together to deliberate on logging issues indirectly related to water. Government agents facilitated the
592
forum and requested model outputs prior to making decisions. The unexpected outcomes of the study (i.e.,
593
the creation of an independent group that used the model for other deliberations) are a testament to the
594
saliency and legitimacy brought by the model building process, whereby people learned to work creatively
595
and collaboratively in new and unexpected ways.
596
6.6. Regional Planning in the Senegal River Valley
597
6.6.1. Case Background
598
The fifth study was set in the Senegal River Delta and applied the Common-pool Resources and Multi-Agent
599
Systems (CORMAS) modeling platform that was previously developed by the CIRAD Center (Cooperation
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 13
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 13
600
Internationale en Recherche Agronomique pour le De"veloppement). CORMAS uses direct and extensive stake-
601
holder engagement workshops [Le Page et al., 2012] and provides a framework for developing simulation
602
models of cooperation among agents and institutions that manage Common Pool Resources (CPR)
603
[Bousquet et al., 1998]. The CORMAS modeling tool arose from a rich literature developed by CIRAD
604
researchers in the 1990s [Le Page et al., 2012]. In the most cited case study of CORMAS, D’Aquino et al.
605
[2002] use a modeling platform to integrate participants’ knowledge and stimulate collective learning by
606
having participants build a ‘‘shared’’ model of land-use problems and possible solutions. Three study sites
607
spanning 2500 km
2
and with a combined population of 40,000 in the Senegal River Delta made use of role-
608
playing games and an agent-based model (ABM) to develop sustainable land-use management strategies.
609
The model building took 2 years and involved several 3 day workshops with local communities. The entire
610
project spanned 10 years. Local community representatives (i.e., farmers, hunters, fisherman, breeders) and
611
public institutions were engaged in several workshops led by CIRAD’s research team. Researchers allowed
612
stakeholders to play the role of an agent (e.g., farmer, breeder, etc.) and actively deliberate and decide on
613
each stage of the model building process. In later planning stages, role-playing games were modeled and
614
made into ABM and geographical information systems (GIS) tools. The ABM provided quick and systematic
615
assessments of management options, while GIS created a visual integration of input and output data. The
616
self-design process elicited from participants the most crucial elements and stakeholders to include in the
617
analysis, and they identified the incentives, constraints, and challenges faced by each stakeholder. This con-
618
sensus meant that later in the process these elements would be more difficult to contest when tensions or
619
conflict over actions escalated. As stakeholders designed new self-governing rules for monitoring and regu-
620
lating, access and use of resource was more widely understood.
621
6.6.2. Participatory Dimensions and Effective Model Characteristics
622
The purpose of participation was to create autonomous and empowered communities that could improve
623
land-use planning by means of stakeholder-driven, bottom-up simulations of decisions and self-governing
624
regulation alternatives. CORMAS’ programming language is object-oriented, creating visual models that are
625
built to improve the use, access, and transfer of technical information in land-use problems. Organizers
626
included the CIRAD research team in collaboration with one rural community council. The intended core
627
participants were public institutions and local community representatives, including farmers, hunters, and
628
fisherman, all from various ethnic groups.
629
Since this model was a self-designed model, the stakeholders were engaged in all five stages of participa-
630
tion. This includes development of the problem definition and model building through the role-playing
631
games and model validation and use during scenario-based analysis. The degree of involvement was charac-
632
terized as high with goals of empowering participants to make decisions and self-govern over agreed upon
633
rules. However, the extent of influence was moderate and small scale at first. The community felt empow-
634
ered to design and regulate their own land-use rules. In later years, community members had the ability to
635
create and negotiate land-use plans. Eventually, an assessment carried out a decade later showed that even
636
these small successes led to the adoption of self-design models (e.g., CORMAS) for other national level proj-
637
ects and laws.
638
Credibility was judged high based on the consensus built through the modeling efforts. The stakeholder
639
meetings and self-design model resulted in a formal proposal drafted with agreed upon recommendations
640
and management actions that acts as a single-text document. The ABM effort built rapport by eliciting
641
locals to share their values, water use, and risks tolerance and aversions. The concept of saliency also scored
642
high based on the ability to translate the modeling platform into an agreement on facts despite diverse
643
knowledge.
644
The model’s legitimacy was judged based on participants’ ability to create new rules for access to resources,
645
design innovative collective rules, and organize in order to monitor new land-use regulations. These
646
changes also helped achieve the principle goals of the study. Unique to this modeling platform was the lev-
647
el of autonomy given to participants to determine priorities based on their interactions. This is reflected in
648
the high scores given in credibility, saliency, and legitimacy criteria.
649
6.6.3. Evaluation of Relative Merits and Shortfalls of PM Effort
650
An ABM is defined by an individualistic, as opposed to systemic, approach to modeling. ABMs are com-
651
posed of autonomous agents, their environment, and the properties that emerge from their complex and
652
dynamic interactions [Bonabeau, 2002]. As this case study shows, ABM simulations take advantage of a
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 14
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 14
653
flexible model structure to create stakeholder-driven, bottom-up modeling platforms. ABM makes no
654
assumptions of ‘‘rationality’’—there are no assumptions of homogeneous actors, perfect information, nor
655
perfect economic efficiency [Parker et al., 2003; Gerst et al., 2012]. Instead, ABMs are well suited to represent
656
heterogeneous and bounded rational agents that are autonomous in their decisions and interactions [Bona-
657
beau, 2002; Yang et al., 2009; Gerst et al., 2012]. Also evident from this case is the model flexibility that allows
658
agents’ behaviors and environments to be influenced by affiliations and interactions, or feedback mecha-
659
nisms that cause them to learn and evolve. This has made ABM popular in the common-pool resource litera-
660
ture, which embraces more nuanced representations of physical-social interactions [Bousquet and Le Page,
661
2004; Janssen and Ostrom, 2006]. These and other characteristics of ABMs have expanded their application
662
in the last 20 years to a wide range of disciplines including anthropology, economics, ecology, engineering,
663
and natural resource management [Bonabeau, 2002; Niazi and Hussain, 2011].
664
The flexibility of ABMs is beneficial in answering questions related to the ‘‘institutional rules [that] may direct
665
individuals to act in the benefit of the collective’’ [Parker et al., 2003]. In this project, CORMAS became a tool
666
for exploring collective actions since each participant represented an agent and they learned about emer-
667
gent characteristics of effective self-governing institutions [Ostrom, 1993, 1999; Janssen and Ostrom, 2006].
668
By engaging participants in a simulation of month-to-month decisions based on alternatives to meet indi-
669
vidual needs, the ABM revealed their preferences, risk tolerance, and motives for their decisions, and
670
allowed the participants to discuss these openly. Since ABMs are built from the perspective of agent units,
671
they can learn and adapt through a heuristic decision process. The CORMAS model was used as an explor-
672
atory tool to develop a testing technique for alternative hypotheses or candidate explanations. During initial
673
workshops, organizers made use of the agent unit approach to capture, through role-playing games, more
674
realistic agent behaviors. Each participant was associated with a constituent unit in the model that made
675
decisions based on needs, preferences, and motives [Bousquet and Le Page, 2004; Castella et al., 2005; Yang
676
et al., 2009]. The project’s scope was limited by the lack of local technical involvement that left the finished
677
model like an orphaned tool. It was unclear who would use it after the organizing team was gone.
678
The two-stage evaluation shows the project purpose was achieved successfully. Despite an initial moderate
679
level of influence, engagement was sustained in the long term, and a survey carried out 10 years later
680
showed that the model enabled members to build capacity and autonomy in future dealings at regional
681
and national levels [D’Aquino and Papazian, 2014]. The Senegal Delta study’s greatest benefit came from
682
the ‘‘dynamic’’ discussions and ‘‘wide-ranging analysis’’ of collective decisions rather than numerical results.
683
Stakeholders were learning how to manage and self-regulate. The greatest strengths and shortfalls both
684
came from the self-design method. The strengths lie in the autonomous nature of the model that allowed
685
participants to establish the agenda and priorities, providing opportunities for change as new perspectives
686
from other constituents were included. Also, the process generated collaborative solutions that had not
687
seemed viable or favorable before. Conversely, the project’s focus on self-design meant that no technical
688
experts were included and the influence of the project was initially limited and would only manifest many
689
years later.
690
6.7. Limitations of the Case Study Analysis
691
This study assumes, based on the premise of IWRM and the laws and regulations previously identified, that
692
inclusive participation is beneficial and desired in resource management. However, while there is no conclu-
693
sive evidence against participation, there is an ongoing debate on the value and capacity of participation in
694
resource management [Lubell, 2004; Koontz and Thomas, 2006; Muro and Jeffrey, 2008; Reed, 2008].
695
This analysis is based on studies published in peer-reviewed journals and on the evidence and assertions
696
presented by the authors. This introduces a bias toward cases that are published based on the experience,
697
interpretation, and information as reported by the authors. Published cases are written from one point of
698
view, which may or may not reflect the opinions of participants. Authors’ optimistic perspectives on the
699
benefits of participation can influence reporting [Carr et al., 2012]. To minimize these identified biases, addi-
700
tional evidence was used: (1) self-evaluations in supplementary papers, (2) third-party evaluations, and (3)
701
direct author communication. The list of evidence for each study is summarized in supporting information
702
material 1.
703
Lastly, the papers documenting the case studies selected in this chapter were not prepared for the purpose
704
of our analysis. As a result, some information relevant to our evaluation may be missing or unavailable to
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 15
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 15
Table 3. Summary of Five Dimensions Characterizing each Participatory Efforts and Examples of Metrics in Which Each Case Scored High
Case study Participatory dimensions summary Two-stage evaluation summary
Community-based
forest
management in
Zimbabwe
Participants— Successful: Creation of a flexibility optimization model using
mixed methods to improve model saliency
St a k e h o l d e r s :Six district-level government personnel: two district
foresters, one agricultural extension officer, one social scientist, and
one provincial officer who is also an expert on gender issues
Or ga ni zi n g t e am :University research team and a facilitator
Stages of participation—Model formulation, problem definition and
identification, model building and use, model validation by quantifying
SWOT and process indicators
Satisfactory: Model was successful at building consensus and
achieving purpose, however, it is unclear if or how this
influenced anything
Degree of involvement—Consultation and moderate
Level of influence—No indication that any decisions were influenced Needs improvement: Provide a more complete record and docu-
mentation of the short-term or long-term benefits of the effort
Purpose—Broad process goal: Reach consensus on action plans based
on transparency to build trust, confidence, and integrity in planning
process
Shared vision
modeling for
ACT-ACF water
basin
Participants— Successful: The model was salient and some core members
became skilled in modifying the assumptions of the model in
order to answer their questions. To some extent the model is
still used today
St a k e h o l d e r s :Representatives from Alabama, Florida, Georgia, and
USACE
Or ga ni zi n g t e am :University research team and Institute of Water
Resources
Stages of participation—Problem definition, model construction, data
gathering, simulation runs, verification and validation of models
Satisfactory: Effectiveness in achieving the more moderate
research goals. The model platform engaged in repeated
virtual experimentation building trust and saliency, however, it
had little influence in decisions
Degree of involvement—Collaborative and created new policy
alternatives
Level of influence—Advisors had trouble executing plans later on when
the politics changed and final decision went to state governors
Needs improvement: The model lost saliency when political tides
shifted. This effectively turned the participants into advisors
who were unable to influence decisions
Purpose—Support water allocation and new reservoir operation policies
Water management
alternatives in
Las Vegas
Participants— Successful: The simple and intuitive nature of the model.
Participants acquired a new systems perspective that allowed
them to accept results even when counterintuitive
St a k e h o l d e r :Intended users were residents and community members.
Expert opinions were solicited from Southern Nevada Water Authority
and U.S. Bureau of Reclamation
Or ga ni zi n g t e am :University research team in consultation with U.S.
Bureau of Reclamation and water authorities
Satisfactory: The model was made into an effective tool that is
salient and legitimate in the policy process
Stages of participation—Experts were engaged in problem definition,
public participants were engaged in model use, analysis, and validation
of acceptable policy options
Degree of involvement—Inform Needs improvement: Incorporation of stakeholder knowledge
and feedback to improve outcomes and increasing the level of
influence in decisions
Level of influence—As a communication tool not intended to change
decisions
Purpose—Primarily used as a stakeholder learning support tool that
allowed them to evaluate the relative merits of different policy options
to resolve future problems of demand exceeding supply
Solomon Islands’
water resource
allocation
Participants— Successful: Creation of dialogue and collaboration despite past
acrimonious relationships among participants. Careful thinking
on how to be sensitive to age, gender, and cultural differences
to achieve inclusiveness
St a k e h o l d e r s :Community landowners, government agencies, and donor
and nongovernmental organizations. A cultural guide and interpreter
helped conduct the participatory research
Or ga ni zi n g t e am :Solomon Water Authority and University research team
in collaboration with Australian Water Resources Facility, and AusAID in
the International Water Centre
Satisfactory: The model was crucial in creating consensus on a set
of facts; however, the fragmented nature of participation left
some questions regarding credibility when not everyone was
included in relevant dialogues at later stages
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 16
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 16
705
us. While this limits our analysis and conclusions, nonetheless, there is a good deal of information from
706
which it is possible to draw some conclusions with confidence. Indeed, the lack of standardized data, docu-
707
mentation, and reporting practices is the basic motivations for this study. Conclusions are limited to the
708
boundary object criteria we evaluated. Despite these limitations, important documented evidence from
709
case studies provides valuable insights into the mechanisms for tackling real world resource problems [Srini-
710
vasan et al., 2012].
711
7. Lessons from Case Studies Evaluation Framework
712
An evaluation based on new disciplinary boundaries was necessary to capture the interdisciplinary nature
713
of PM. The five dimensions emphasize clarity in characterizing questions of who, when, how, and why for
714
each effort, while the concept of models as boundary objects provides a criterion for evaluating the models’
715
credibility, salience, and legitimacy. Table T33 provides a summary of the two-stage evaluation process. Four
716
lessons on the mechanisms for model effectiveness are drawn based on the proposed evaluation
717
framework:
718
First, the effectiveness of the five participatory models was independent of their (diverse) technical charac-
719
teristics. Models developed by scientists for other technicians are intended for experts rather than stake-
720
holders and are generally more complex, data-intensive, and sophisticated. They are typically judged by
721
their scientific accuracy. Conversely, models in the policy context are intended for a diverse audience and
722
must weigh the benefits of increased complexity against the costs, namely the possibility, of alienating non-
723
technical participants [Webler et al., 2011]. These are best judged by their ability to capture scientific facts
724
and render them useful—through mechanisms like those developed—to both technical and nontechnical
725
participants. Previous comparative studies indicate that complexity can often obscure transparency, limit
726
model accessibility, and lead to information asymmetries that can undermine the participatory process and
727
support insulated decision-making [Mendoza and Prabhu, 2005; Lemos et al., 2010]. It is clear from the stage
728
2 assessments (Table 3) that modelers in all five case studies were aware of the need to produce under-
729
standable models, although the evaluations on particular subcriteria suggest that stronger two-way dia-
730
logue between modelers and the public could improve salience. Prior method comparison studies [Hobbs
731
et al., 1992] reinforce our conclusion that how the model is used—the interactions and dealings among par-
732
ticipants and model—is as important as the specifics of the modeling method and is consistent with the
733
NRC findings that direct interactions between participants increase the effectiveness of future engagements
734
by building mutual understanding, trust, and dealings [NRC, 2008].
Table 3. (continued)
Case study Participatory dimensions summary Two-stage evaluation summary
Stages of participation—Fragmented: local community, government
representatives, NGOs, and donor organizations in problem definition,
a smaller representative group engaged in model use and validation
Degree of involvement—Collaborative
Level of influence—Government agencies used the model for prioritiz-
ing interventions. Model was not intended to resolve negotiation
conflicts
Needs improvement: Inclusion of community landowners and
other stakeholders in later stages of model building and use
Regional land-use in
Senegal River
Delta
Participants— Successful: Achieved the goal of empowerment and building
adaptive capacity among community members. Created a
single-text negotiating agreement
St a k e h o l d e r s :Public institutions and local community representatives
from various ethnic groups
Or ga ni zi n g t e am :One rural community council and CIRAD research team
Stages of participation—Problem definition developed in role-playing
games, model building and validation, and model use in scenario-
based analysis
Satisfactory: The direct short-term benefits of the modeling effort
are vaguely explained. Long-term, it ias difficult to identify the
model’s influence on national level policies, who ‘‘owned’’ the
model, or how was it accessible to anyone
Degree of involvement—Empowerment
Level of influence—Local community members created self-governing
rules for land-use at small scale. With time it empowered members to
create land-use plans
Needs improvement: Providing a two-way dialogue between
experts and core participants to create mutual learning
Purpose—Improve land use strategies and planning by empowering
communities
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 17
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 17
735
Second, models with greater flexibility had enhanced relevance to case study participants, in part because
736
these models evolved to become more complete and sometimes even more complex only as the partici-
737
pants’ involvement became more sophisticated; they were thus designed around the decision-making pro-
738
cess. Model completeness and flexibility here refer to the factors that were deemed important, and how
739
they were measured and integrated into the model platform. In the Senegal Delta case, the self-design
740
aspects of the model resulted in a simple and highly flexible model. With each role-playing game, the mod-
741
el was adapted to include new information identified as relevant by the participants. Given the diversity of
742
participants and their limited previous experience with modeling, the use of a simple, self-designed model
743
was crucial to facilitate dialogue. In the ACT-ACF case study, the representatives from each State and the
744
USACE were midlevel to high-level water managers, who appreciated increased accuracy and completeness
745
despite its accompanying complexity if the model captured system nuances. Managers’ understanding of
746
the system and model improved making the model more relevant as they requested greater sophistication
747
to address their specific questions. The evaluation criteria show the balance between completeness and
748
flexibility as they relate to the mechanisms of relevance and credibility.
749
Third, the inherently interdisciplinary and largely nontechnical tasks of PM (e.g., identifying stakeholders,
750
level of stakeholder involvement) will affect the model’s ability to build credibility, saliency, and legitimacy.
751
Participants’ interactions—e.g., consensus, trust, information exchange—are highly sensitive to group rep-
752
resentation and size. The case studies suggest that a balance must be achieved between establishing a
753
broad representative group and finding a group size that enhances coherent and productive dialogue
754
between participants. In Zimbabwe, six representative participants ensured a quick and efficient model
755
building and engagement process. However, it is unknown whether the PM effort influenced any decisions
756
and, therefore, whether the model, however effective, was anything more than an exercise with no real con-
757
sequences. Conversely, the Solomon Island case study engaged more than 99 stakeholders; but, due to past
758
social conflicts, workshops were initially held separately. This resulted in a more culturally sensitive environ-
759
ment associated with age and gender roles that was inclusive, though it was time-intensive, and the authors
760
acknowledged the ‘‘unwieldy’’ effects of large groups in reaching consensus. The evaluation highlights the
761
nature of this balance. Model effectiveness in improving legitimacy, credibility, and saliency criteria is highly
762
dependent on the working group size and how size influences the dynamics of group interactions.
763
Finally, the boundary object lens of the evaluation framework reveals how computer models themselves—
764
as distinct from group facilitation, mediation, or other participatory techniques—can catalyze engagement
765
in negotiation environments. During workshops in the Las Vegas case study, participants accessed SDM out-
766
puts to compare the effects of five possible policy solutions to a water resources problem. And in the ACT-
767
ACF case study, the model had a user interface designed to let participants themselves change model con-
768
trol parameters and features. Advances in computing technology such as fast computation speeds and the
769
ease with which we can now create and share information in graphic form improve the participatory experi-
770
ence. Real-time user feedback has tangible benefits, as it allows changes to model formulation, quick and
771
easy policy trade-off comparisons, and timely deliberations. Advancements in computer technology have
772
boosted the ability of models to contribute to the evolving needs of fast-paced negotiation environments
773
that rely on quick information exchange. This builds on two previous studies: first, that effective science for
774
decision-making allows changes in how problems are defined and framed before providing solutions [Cash
775
et al., 2003], and; second, that models can be central to creating negotiated solutions from participants’
776
dynamic interactions [Bousquet and Le Page, 2004; Dwyer and Stave, 2008].
777
8. Summary and Conclusions
778
Over the last 30 years, changes in ‘‘traditional’’ WRM models opened a debate among experts about how to
779
create a more accessible modeling paradigm that is transparent, flexible, timely, and relevant to the needs
780
of a diverse public [Loucks and French, 1985; Rogers and Fiering, 1986; Lund and Palmer, 1997; Simonovic and
781
Fahmy, 1999; Mendoza and Prabhu, 2005]. Traditional models lacked the ‘‘explicit recognition of various
782
interests and pressures [that] are part of the process used to generate the alternative candidate’’ policies
783
[Rogers and Fiering, 1986].
784
The novel contribution of the proposed two-stage evaluation framework is threefold. First, the two-stage
785
proposed framework provides a structured vocabulary on clear mechanisms that capture the technical and
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 18
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 18
786
social nature of PM based on previously established concepts of participation (stage 1) and boundary
787
objects (stage 2). Second, the criteria we have developed extend beyond the process-based and outcome-
788
based evaluation of the previously cited literature given that we identify several challenges regarding par-
789
ticipation and the accessibility and applicability of technical knowledge that need to be considered prior to
790
outcomes. Third, the framework allows flexibility to evaluate a wide range of cases, even when the effort’s
791
stated objectives for participation are quite different in scope or discipline.
792
Finally, any comprehensive assessment of the value of PM requires systematic documentation, evaluations,
793
and comparative studies as documented evidence of previous trials, failures, and successes. Part of the
794
motivation and challenge of this work is the limited data available and lack of standard documentation. The
795
framework presented can help structure more complete records and systematically document evidence.
796
The diversity of efforts in the literature suggest that PM has been a practice dominated by ‘‘trial-and-error’’
797
and ‘‘learning-by-doing,’’ that needs standard documentation for the field to evolve and consolidate
798
findings.
799
800
References
801
Abers, R. N., and M. E. Keck (2013), Practical Authority: Agency and Institutional Change in Brazilian Water Politics, Oxford Univ. Press,
802
New York.
803
Arnstein, S. (1969), A ladder of citizen participation, J. Am. Inst. Plann.,34(4), 216–224.
804
Bacic, I. L. Z., D. G. Rossiter, and A. K. Bregt (2006), Using spatial information to improve collective understanding of shared environmental
805
problems at watershed level, Landscape Urban Plann.,77(1–2), 54–66, doi:10.1016/j.landurbplan.2005.01.005.
806
Barreteau, O., P. Bots, and K. Daniell (2010), A framework for clarifying ‘‘Participation’’ in participatory researc h to prevent its rejection for
807
the wrong reasons, Ecol. Soc.,15(2). AQ1
808
Bolson, J., and K. Broad (2012), Early adoption of climate information: Lessons learned from South Florida water resource management,
809
Weather Clim. Soc.,5(3), 266–281, doi:10.1175/WCAS-D-12-00002.1.
810
Bonabeau, E. (2002), Agent-based modeling: Methods and techniques for simulating human systems, Proc. Natl. Acad. Sci. U. S. A.,99(Suppl.
811
3), 7280–7287, doi:10.1073/pnas.082080899.
812
Bourget, L. (Ed.) (2011), Converging Water: Integrating Collaborative Modeling With Participatory Processes to Make Water Resources Decisions.AQ2
813
Bousquet, F., and C. Le Page (2004), Multi-agent simulations and ecosystem management: A review, Ecol. Modell.,176(3–4), 313–332, doi:
814
10.1016/j.ecolmodel.2004.01.011.
815
Bousquet, F., I. Bakam, H. Proton, and C. Le Page (1998), Cormas: Common-pool resources and multi-agent systems, Tasks Methods Appl.
816
Artif. Intell., 826–837. AQ3
817
Carlile, P. R. (2002), Product view of knowledge objects in new boundaries: Boundary development, Organ. Sci.,13(4), 442–455, doi:
818
10.1287/orsc.13.4.442.2953.
819
Carr, G., G. Bl
oschl, and D. P. Loucks (2012), Evaluating participation in water resource management: A review, Water Resour. Res.,48,
820
W11401, doi:10.1029/2011WR011662.
821
Cash, D., W. C. Clark, F. Alcock, N. Dickson, N. Eckley, and J. J
ager (2002), Salience, credibility, legitimacy and boundaries: Linking research,
822
assessment and decision making, KSG Work. Pap. Ser. RWP02-046, (November), 25, doi:10.2139/ssrn.372280.
823
Cash, D. W., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, D. H. Guston, J. J
ager, and R. B. Mitchell (2003), Knowledge systems for sustain-
824
able development, Proc. Natl. Acad. Sci. U. S. A.,100(14), 8086–8091, doi:10.1073/pnas.1231332100.
825
Castella, J., T. N. Trung, and S. Boissau (2005), Participatory simulation of land-use changes in the Northern Mountains of Vietnam: The com-
826
bined use of an agent-based model, a role-playing game, and a geographic information system, Ecol. Soc.,10(1).
827
Castelletti, A., and R. Soncini-Sessa (2007), Bayesian networks and participatory modelling in water resource management, Environ. Model.
828
Software,22(8), 1075–1088, doi:10.1016/j.envsoft.2006.06.003.
829
Chan, T., H. Ross, S. Hoverman, and B. Powell (2010), Participatory development of a Bayesian network model for catchment-based water
830
resource management, Water Resour. Res.,46, W07544, doi:10.1029/2009WR008848.
831
Colfer, C. J. P., et al. (1999), The BAG: Basic Assessment Guide for Human Well-Being, Cent. for Int. For. Res. AQ4
832
Commission of the European Communities (2000), Directive 2000/60/EC, Eur. Parliament, Counc. of the Eur. Union.
833
Conca, K. (2006), Governing Water: Contentious Transnational Politics and Global Institution Building, MIT Press, Cambridge.
834
Cornwall, A. (2008), Unpacking ‘‘Participation’’: Models, meanings and practices, Community Dev. J.,43(3), 269–283, doi:10.1093/cdj/bsn010.
835
Creighton, J. L., and S. Langsdale (2009), Analysis of Process Issues in Shared Vision Planning Cases.
836
D’Aquino, P., and H. Papazian (2014), An inclusionary strategy reaching empowering outcomes ten years after a two-year participatory
837
land uses management, Environ. Manage. Sustainable Dev.,3(2), 154–181, doi:10.5296/emsd.v3i2.6595.
838
D’Aquino, P., C. Le Page, F. Bousquet, and A. Bah (2002), A novel mediating participatory modelling: The self-desig n process to accompany
839
collective decision making, Int. J. Agric. Resour. Gov. Ecol.,2(1), 59–74, doi:10.1504/IJARGE.2002.000022.
840
Davis, A., J. Anderson, and R. Gough (1975), Alternative Information and Interaction Approaches to Public Participation in Water Resources
841
Decision Making.
842
Dreyer, M., and O. Renn (2011), Participatory approaches to modelling for improved learning and decision-making in natural resource gov-
843
ernance: An editorial, Environ. Policy Gov.,21(6), 379–385.
844
Dwyer, M., and K. A. Stave (2008), Group model building wins: The results of a comparative analysis, in Proceedings of the 2008 International
845
Conference of the System Dynamics Society, pp. 1–17.
846
Ertel, M. (1979), A survey research evaluation of citizen participation strategies, Water Resour. Res.,15(4).
847
Ertel, M., and S. G. Koch (1976), Citizen Participation in Comprehensive Water Resources Planning.
848
Fuller, B. W. (2009), Surprising cooperation despite apparently irreconcilable differences: Agricultural water use efficiency and CALFED,
849
Environ. Sci. Policy,12(6), 663–673, doi:10.1016/j.envsci.2009.03.004.
850
Gerst, M. D., P. Wang, A. Roventini, G. Fagiolo, G. Dosi, R. B. Howarth, and M. E. Borsuk (2012), Environmental Modelling & Software Agent-
851
based modeling of climate policy: An introduction to the ENGAGE multi-level model framework, Environ. Model. Software, 1–14.
Acknowledgments
This work was supported by the NSF
funding (STS grant 1457080) and the
Faculty for the Future Fellowship. This
funding is gratefully acknowledged.
We would like to thank William Werick
for contributions on early versions of
this paper and arguing for the need of
a two-stage framework. Also, we thank
G. Cambareri and the anonymous
reviewers for their insightful
comments and suggestions. All data
reviewed to evaluate each case study
in this paper comes from sources listed
in the supporting information material
and properly cited in the reference list.
The work and views in this paper are
those of the authors and do not
necessarily express the views of the
sponsors.
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 19
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 19
852
Gilbert, N., and S. Bankes (2002), Platforms and methods for agent-based modeling., Proc. Natl. Acad. Sci. U. S. A.,99(Suppl. 3), 7197–7198,
853
doi:10.1073/pnas.072079499.
854
Global Water Partnership-TEC (2004), ‘‘Integrated Water Resources Management (IWRM) and Water Efficiency Plans by 2005’’ Why, What and
855
How?
856
Grimble, R., and K. Wellard (1997), Stakeholder methodologies in natural resource management: A review of principles, contexts, experien-
857
ces and opportunities, Agric. Syst.,55(2), 173–193, doi:10.1016/S0308-521X(97)00006-1.
858
Hamlet, A. F., et al. (1996), Simulating Basinwide Alternatives Using the ACT-ACF Shared Vision Models.
859
Hare, M. (2011), Forms of participatory modelling and its potential for widespread adoption in the water sector, Environ. Policy Gov.,21(6),
860
386–402, doi:10.1002/eet.590.
861
Hare, M., R. A. Letcher, and A. J. Jakeman (2003), Participatory modelling in natural resource management: A comparison of four case stud-
862
ies, Integr. Assess.,4(2), 62–72, doi:10.1076/iaij.4.2.62.16706.
863
Herath, G. (2004), Incorporating community objectives in improved wetland management: The use of the analytic hierarchy process,
864
J. Environ. Manage.,70(3), 263–273, doi:10.1016/j.jenvman.2003.12.011.
865
Hisschem
oller, M., R. Tol, and P. Vellinga (2001), The relevance of participatory approaches in integrated environmental assessment, Integr.
866
Assess.,2(2), 57–72.
867
Hobbs, B. F., V. Chankong, and W. Hamadeh (1992), Does choice of multicriteria method matter? An experiment in water resources plan-
868
ning, Water Resour. Res.,28(7), 1767–1779.
869
Hoverman, S., H. Ross, T. Chan, and B. Powell (2011), Social learning through participatory integrated catchment risk assessment in the
870
Solomon Islands, Ecol. Soc.,16(2).
871
Huvila, I., T. D. Anderson, E. H. Jansen, P. McKenzie, L. Westbrook, and A. Worrall (2014), Boundary objects in information science research:
872
An approach for explicating connections between collections, cultures and communities, in Proceedings of the American Society for
873
Information Science and Technology, vol. 51, pp. 1–4.
874
International Association for Public Participation (2005), Public participation spectrum. [Available at http://www.iap2.org.] AQ5
875
Janssen, M. A., and E. Ostrom (2006), Empirically based agent-based models, Ecol. Soc.,11(2), 37.
876
Jeffrey, P. J., and M. Gearey (2006), Integrated water resources management: Lost on the road from ambition to realisation?, Water Sci.
877
Technol.,53(1), 1–8, doi:10.2166/wst.2006.001.
878
Jenkins-Smith, H. C., and P. A. Sabatier (1994), Evaluating the advocacy coalition framework, J. Public Policy,14(2), 175–203, doi:10.1017/
879
S0143814X00007431.
880
Koontz, T., and C. Thomas (2006), What do we know and need to know about the environmental outcomes AQ6of collaborative management?,
881
Public Adm. Rev.
882
Leitman, S. (2008), Lessons learned from transboundary management efforts in the Apalachicola-Chattahoochee-Flint Basin, USA, in Trans-
883
boundary Water Resources: A Foundation for Regional Stability in Central Asia, pp. 195–208. AQ7
884
Leitman, S., and G. A. Kiker (2015), Development and comparison of integrated river/reservoir models in the Apalachicola-Chattahoochee-
885
Flint basin, USA, Environ. Syst. Decis.,35(3), 410–423, doi:10.1007/s10669-015-9564-3.
886
Lemos, M. C. (2008), Whose water is it anyway? Water management, knowledge, and equity in Northeast Brazil, in Water, Place, and Equity,
887
pp. 249–270.
888
Lemos, M. C., A. Bell, N. Engle, R. Formiga-Johnsson, and D. Nelson (2010), Technical knowledge and water resources management: A com-
889
parative study of river basin councils, Brazil, Water Resour. Res.,46, W06523, doi:10.1029/2009WR007949.
890
Liebman, J. C. (1976), Some simple-minded observations on the role of optimization in public systems decision-making, Interfaces (Provi-
891
dence),6(4), 102–108, doi:10.1287/inte.6.4.102.
892
Ligtenberg, A., A. Beulens, D. Kettenis, A. K. Bregt, and M. Wachowicz (2009), Simulating knowledge sharing in spatial planning: An agent-
893
based approach, Environ. Plann. B,36(4), 644–663, doi:10.1068/b33059.
894
Loucks, D. P., and P. N. French (1985), Interactive data management for resource planning and analysis, Water Resour. Res.,21(2), 131–142.
895
Lubell, M. (2004), Collaborative environmental institutions: All talk and no action?, J. Policy Anal. Manage.,23(3), 549–573, doi:10.1002/
896
pam.20026.
897
Lund, J. R., and R. N. Palmer (1997), Water resource system modeling for conflict resolution, Water Resour. Updat.,3(Holling 1978), 70–82.
898
Lynam, T., W. De Jong, and D. Sheil (2007), A review of tools for incorporating community knowledge, preferences, and values into decision
899
making in natural resources management, Ecol. Soc.,12(1), 5.
900
Medema, W., B. S. Mcintosh, and P. J. Jeffrey (2008), From premise to practice: A critical assessment of integrated water resources manage-
901
ment and adaptive management approaches in the water sector, Ecol. Soc.,13(2), 29.
902
Mendoza, G. A., and H. Martins (2006), Multi-criteria decision analysis in natural resource management: A critical review of methods and
903
new modelling paradigms, For. Ecol. Manage.,230(1–3), 1–22, doi:10.1016/j.foreco.2006.03.023.
904
Mendoza, G. A., and R. Prabhu (2002), Enhancing Participatory Planning of Community-Managed Forests Using Problem Structuring Models
905
and Approaches: Experiences from a Case Study.
906
Mendoza, G. A., and R. Prabhu (2005), Combining participatory modeling and multi-criteria analysis for community-based forest manage-
907
ment, For. Ecol. Manage.,207(1–2), 145–156, doi:10.1016/j.foreco.2004.10.024.
908
Michaud, W. R. (2009), Performance Measures to Assess the Benefits of Shared Vision Planning and Other Collaborative Modeling Processes.
909
Mukhtarov, F. (2009), The hegemony of integrated water resources management: A study of policy translation in England, Turkey, and
910
Kazakhstan. AQ8
911
Muro, M., and P. J. Jeffrey (2008), A critical review of the theory and application of social learning in participatory natural resource manage-
912
ment processes, J. Environ. Plann. Manage.
913
Nandalal, K. D. W., and S. P. Simonovic (2003), Resolving conflicts in water sharing: A systemic approach, Water Resour. Res.,39(12), 1362,
914
doi:10.1029/2003WR002172.
915
National Research Council (2008), Public Participation in Environmental Assessment and Decision Making, edited by T. Dietz and P. C. Stern,
916
The Natl. Acad. Press, Washington, D. C.
917
Niazi, M., and A. Hussain (2011), Agent-based computing from multi-agent systems to agent-based models: A visual survey, Scientometrics,
918
89(2), 479–499, doi:10.1007/s11192-011-0468-9.
919
Ostrom, E. (1993), Design principles in Long-Enduring Irrigation Institutions, Water Resour. Res.,29(7), 1907–1912.
920
Ostrom, E. (1999), Self-governance and forest resources, in CIFOR Occasional Paper 20, vol. 62.
921
Le Page, C., N. Becu, P. Bommel, and F. Bousquet (2012), Participatory agent-based simulation for renewable resource management: The
922
role of the cormas simulation platform to nurture a community of practice, J. Artif. Soc. Soc. Simul.,15(1), 10.
923
Palmer, R. N. (1998), A history of shared vision modeling in the ACT-ACF comprehensive study: A modeler’s perspective.
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 20
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 20
924
Palmer, R. N., W. Werick, A. MacEwan, and A. Woods (1999), Modeling water resources opportunities, challenges and trade-offs: The use of
925
shared vision modeling for negotiation and conflict resolution, in Proceeding of the Water Resource Planning and Management Confer-
926
ence, pp. 1–13, Am. Soc. of Civil Eng.
927
Palmer, R. N., H. E. Cardwell, M. A. Lorie, and W. Werick (2013), Disciplined planning, structured participation, and collaborative modeling—
928
Applying shared vision planning to water resources, JAWRA J. Am. Water Resour. Assoc.,49(3), 614–628, doi:10.1111/jaw r.12067.
929
Parker, D. C., S. M. Manson, M. A. Janssen, M. J. Hoffmann, and P. Deadman (2003), Multi-agent systems for the simulation of land-use and
930
land-cover change: A review, Ann. Assoc. Am. Geogr.,93(2), 314–337, doi:10.1111/1467-8306.9302004.
931
Pidd, M. (1999), Just modeling through: A rough guide to modeling, Interfaces (Providence),29(2), 118–132.
932
Posner, S. M., E. McKenzie, and T. H. Ricketts (2016), Policy impacts of ecosystem services knowledge, Proc. Natl. Acad. Sci. U. S. A.,113(7),
933
1760–1765, doi:10.1073/pnas.1502452113. AQ9
934
Prell, C., K. Hubacek, M. S. Reed, C. Quinn, N. Jin, J. O. E. Holden, T. I. M. Burt, M. Kirby, and J. A. N. Sendzimir (2007), If you have a hammer
935
everything looks like a nail: ‘‘Traditional’’ versus participatory model building, Interdiscip. Sci. Rev.,32(3), 1–20, doi:10.1179/
936
030801807X211720.
937
Priscoli, J. D. (1989), Public involvement, conflict management: Means to EQ and social objectives, J. Water Resour. Plann. Manage.,115(1),
938
31–41.
939
Priscoli, J. D. (2004), What is public participation in water resources management and why is it important?, Water Int.,29(2), 221–227, doi:
940
10.1080/02508060408691771.
941
Rajan, K., and R. Shibasaki (2000), A GIS based integrated land use/cover change model to study human-land interactions, Int. Arch. Photo-
942
gramm. Remote Sens.,33(B7/3; Part 7), 1212–1219.
943
Reed, M. S. (2008), Stakeholder participation for environmental management: A literature review, Biol. Conserv.,141(10), 2417–2431, doi:
944
10.1016/j.biocon.2008.07.014.
945
Renn, O. (2008), Risk Governance: Coping with Uncertainty in a Complex World, Earthscan, London.
946
Rittel, H. W. J., and M. M. Webber (1973), Dilemmas in a general theory of planning, Polit. Sci.,4(2), 155–169.
947
Rogers, P. P. (1978), On the Choice of the ‘‘Appropriate Model’’ for water resources planning and management, Water Resour. Res.,14(6),
948
1003–1010.
949
Rogers, P. P., and M. B. Fiering (1986), Use of systems analysis in water management, Water Resour. Res.,22(9), 146–158.
950
Rogers, P. P., L. MacDonnell, and P. Lydon (2009), Political decision making: Real decisions in real political context, in The Evolution of Water
951
Resource Planning and Decision Making, edited by C. Russell and D. D. Baumann. AQ10
952
Simonovic, S. P., and H. Fahmy (1999), A new modeling approach for water resources policy analysis, Water Resour. Res.,35(1), 295–304.
953
Srinivasan, V., E. F. Lambin, S. M. Gorelick, B. H. Thompson, and S. Rozelle (2012), The nature and causes of the global water crisis: Syn-
954
dromes from a meta-analysis of coupled human-water studies, Water Resour. Res.,48,W10516,doi:10.1029/2011WR011087.
955
Star, S. L. (1989), The structure of ill-structured solutions: Boundary objects and heterogeneous distributed problem solving, in Distributed
956
Artifidal Intelligence, 2, edited by M. Huhns, pp. 37–54, Morgan Kaufmann, San Francisco, Calif.
957
Star, S. L., and J. R. Griesemer (1989), Institutional Ecology, ‘Translations’ and boundary objects: Amateurs and professionals in Berkeley’s
958
Museum of Vertebrate Zoology, 1907–39, Soc. Stud. Sci.,19(3), 387–420, doi:10.1177/0306312890190030 01.
959
Stave, K. A. (2002), Using system dynamics to improve public participation in environmental decisions, Syst. Dyn. Rev.,18(2), 139–167, doi:
960
10.1002/sdr.237.
961
Stave, K. A. (2003), A system dynamics model to facilitate public understanding of water management options in Las Vegas, Nevada, J.
962
Environ. Manage.,67(4), 303–313, doi:10.1016/S0301-4797(02)00205-0.
963
U.S. Congress (1969), The National Environmental Policy Act of 1969.
964
U.S. Congress (1973), The Endangered Species Act of 1973.
965
U.S. Environmental Protection Agency (1972), The Federal Clean Water Act.
966
U.S. Water Resources Council (1973), Principles and Standards for Planning Water and Related Land Resources.
967
UN-Water (2008), Status Report on Integrated Water Resources Management and Water Efficiency Plans.
968
United Nations (1992), Agenda 21: Programme of Action for Sustainable Development.
969
Uusitalo, L. (2007), Advantages and challenges of Bayesian networks in environmental modelling, Ecol. Modell.,203(3–4), 312–318, doi:
970
10.1016/j.ecolmodel.2006.11.033.
971
Vennix, J. A. M., J. Wright, and F. Prize (1999), Group model-building: Tackling messy problems, Syst. Dyn. Rev.,15(4), 379–401.
972
Verburg, P. H., P. P. Schot, M. J. Dijst, and A. Veldkamp (2004), Land use change modelling: Current practice and research priorities, GeoJour-
973
nal,61(4), 309–324, doi:10.1007/s10708-004-4946-y.
974
Webler, T., S. Tuler, and T. Dietz (2011), Modellers’ and outreach professionals’ views on the role of models in watershed management,
975
Environ. Policy Gov.,21(6), 472–486, doi:10.1002/eet.587.
976
Werick, W., and R. N. Palmer (2011), When Should Shared Vision Planning Be Used?
977
Werick, W., A. M. Keyes, R. N. Palmer, and J. Lund (1994), Virtual drought and shared visions: Some innovations from the National Drought
978
Study, in Proceedings of the Conference on Drought Management in the Changing West: New Directions for Water Policy, Int. Drought
979
Cent. Rep.
980
Winz, I., and G. Brierley (2007), The use of system dynamics simulation in integrated water resources management, in Proceedings of the
981
25th International Conference of the System Dynamics Society, pp. 1–26.
982
Wolman, M. G. (1977), Interdisciplinary education: A continuing experiment, Science,198(4319), 800–804.
983
Xiang, W.-N. (2013), Working with wicked problems in socio-ecological systems: Awareness, acceptance, and adaptation, Landscape Urban
984
Plann.,110, 1–4, doi:10.1016/j.landurbplan.2012.11.006.
985
Yang, Y.-C. E., X. Cai, and D. M. Stipanovic
´(2009), A decentralized optimization algorithm for multiagent system-based watershed manage-
986
ment, Water Resour. Res.,45, W08430, doi:10.1029/2008WR007634.
987
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 21
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Water Resources Research 10.1002/2016WR019373
FALCONI AND PALMER MODELS AS EFFECTIVE PART. DECISION TOOLS 21
AQ1: Please provide page range for Barreteau et al. [2010], Castella et al. [2005], Hoverman et al.
[2011], and Huvila et al. [2014].
AQ2: Please provide publisher name and location for Bourget [2011], Creighton and Langsdale
[2009], Davis et al. [1975], Dwyer and Stave [2008], Ertel and Koch [1976], Global Water
Partnership-TEC [2004], Hamlet et al. [1996], Mendoza and Prabhu [2002], Michaud [2009],
Ostrom [1999], U.S. Congress [1969], U.S. Congress [1973], U.S. Environmental Protection
Aagency [1972], U.S. Water Resources Council [1973], UN-Water [2008], United Nations [1992],
Werick and Palmer [2011], and Winz and Brierley [2007].
AQ3: Please provide volume for Bousquet et al. [1998] and Gerst et al. [2012].
AQ4: Please provide publisher location for Colfer et al. [1999], Commission of the European Com-
munities [2000], Palmer et al. [1999], and Werick et al. [1994].
AQ5: Please provide sponsor name and location for International Association for Public Participa-
tion [2005].
AQ6: Please provide volume and page range for Koontz and Thomas [2006] and Muro and Jeffrey
[2008].
AQ7: Please provide editors name and publisher name and location for Leitman [2008] and
Lemos [2008].
AQ8: Please provide complete details for Mukhtarov [2009] and Palmer [1998].
AQ9: Please note that Posner et al. [2016] and Werick et al. [1994] do not appear in the text.
Please provide text citations or delete from the Reference list.
AQ10: Please provide publisher name and location and page range for Rogers et al. [2009].
AQ11: Please confirm that given names (red) and surnames/family names (green) have been
identified correctly.
AQ12: Author: Please complete the Publication Fee Consent Form included with your article and
return to the Production Editor with your proofs. Please use any version of Adobe Acrobat Pro or
Reader 11 or higher to edit the form. Reader can be downloaded free at: http://www.adobe.com/
products/acrobat/readstep2.html.
J_ID: WRCR Customer A_ID: WRCR22486 Cadmus Art: WRCR22486 Ed. Ref. No.: 2016WR019373 Date: 6-February-17 Stage: Page: 22
ID: vedhanarayanan.m Time: 22:02 I Path: //chenas03/Cenpro/Ap plicationFiles/Journals/Wiley/WRCR/Vol00000/170019/Comp/APPFile/JW-WRCR170019
Falconi’s department affiliation had a name change as of the submission. It is now:
Department of Environmental Health and Engineering. All other information is correct.
... In participatory MCA as well as in other participatory evaluation methods the identification of group decision-making participants is obviously a fundamental step of the process and represents one of the most debated topics among scholars and practitioners (e.g. Abelson et al., 2003;Burton, 2009;Cass, 2006;Davies et al., 2005;Falconi and Palmer, 2017;Glucker et al., 2013;Gregory, 2000;Petts, 1999;Sewell and Coppock, 1977a;Weale, 2001). In general, participatory processes can span from small group decision-making procedures, focussing on specific and circumscribed problems and affecting a limited number of individuals, to negotiation processes on complex and uncertain social and policy problems, having implications for a very wide area and a large number of people (Kilgour et al., 2010). ...
... On one hand, as highlighted in the literature on participatory and deliberative evaluation (e.g. Falconi and Palmer, 2017;Hare, 2011), it is sensible to assume that different problems require a different level of involvement of participants and different engagement techniques. On the other hand, also concerning these important aspects, only in a few of the examined articles (e.g. ...
... The selection of this small number of participants and groups is also fraught with difficulties and challenges and implies largely arbitrary considerations on a number of interdependent factors, including the nature of the problem at hand, its geographical boundaries, and the dimensions of the problem to prioritise (Kahane et al., 2013;Stirling, 2006). There is a persistent risk of missing stakeholder groups that instead should have been included and also the danger of reinforcing existing patterns of social and political disparities (Falconi and Palmer, 2017). Indeed, in many cases, the choice of which stakeholders to involve may lean (intentionally or unintentionally) towards the most organised, and often most powerful groups, that have consolidated themselves as a public presence (Kahane et al., 2013). ...
Article
Full-text available
Over the past few decades, a number of participatory multi-criteria analysis methods, combining deliberative procedures with multiple decision criteria assessment techniques, have been developed to tackle complex policy problems. However, several important aspects of such methods, including the way in which different and often contrasting viewpoints should be included in the analysis, appear to have been largely neglected by previous studies. Possible problems and drawbacks that may hamper the applicability and feasibility of multi-actor multi-criteria exercises and the utility and reliability of their outcomes also deserve further investigation. This article seeks to fill this knowledge gap by proposing a conceptual framework and classification scheme that illustrates the different possible approaches for identifying the key elements of the multi-criteria problem (i.e. options, objectives/criteria, weights and scores), while dealing with different points of view. It also discusses the potential advantages, disadvantages and issues of each approach and ultimately defines the overarching factors that should orientate the selection of one specific approach over the others.
... According to the Cambridge Dictionary, evaluation is defined as the "process of judging the quality, importance, amount, or value of something" [34]. Applying this definition to the context of this paper, there is opportunity to better understand the quality, importance, and value of PMs [35]. This shifts the focus from solely one aspect of PM to a more holistic consideration of the whole PM process (e.g., knowledge integration and learning, technical systems model development, participatory and integrated planning, etc), providing opportunity for further knowledge on which aspects of PM are particularly important for policy decisions and community learning, as well as the ongoing improvement of PM methods [36,37]. ...
... Evaluators are relied upon to address questions on the effectiveness of investments in local, state, and national programs, as well as to better explain if observed outcomes were (or were not) as planned, and how unintended consequences can be addressed [38][39][40][41]. There may be various motivations for conducting an evaluation of PM programs including the desire to improve and share knowledge on good practice for PM, quantitatively and qualitatively report on project impacts, as well as to assess the value of PM for future work [24,35]. Evaluations also keep the modellers, funders, and other stakeholders of interest held accountable for demonstrating outcomes, as well as to provide merit to the work being evaluated [24]. ...
... Evaluations that comprehensively capture the complex (e.g., uncertain, dynamic) nature of PM can be difficult [42], as embedding participatory approaches in systems modelling and simulation creates several challenges [35]. For instance, the focus of PM outcomes is often still on the knowledge integration and learning process rather than the multi-value perspectives integrated within the participatory process used to develop the models [31]. ...
Article
Full-text available
Introduction Systems modelling and simulation can improve understanding of complex systems to support decision making, better managing system challenges. Advances in technology have facilitated accessibility of modelling by diverse stakeholders, allowing them to engage with and contribute to the development of systems models (participatory modelling). However, despite its increasing applications across a range of disciplines, there is a growing need to improve evaluation efforts to effectively report on the quality, importance, and value of participatory modelling. This paper aims to identify and assess evaluation frameworks, criteria, and/or processes, as well as to synthesize the findings into a comprehensive multi-scale framework for participatory modelling programs. Materials and methods A scoping review approach was utilized, which involved a systematic literature search via Scopus in consultation with experts to identify and appraise records that described an evaluation framework, criteria, and/or process in the context of participatory modelling. This scoping review is registered with the Open Science Framework. Results The review identified 11 studies, which varied in evaluation purposes, terminologies, levels of examination, and time points. The review of studies highlighted areas of overlap and opportunities for further development, which prompted the development of a comprehensive multi-scale evaluation framework to assess participatory modelling programs across disciplines and systems modelling methods. The framework consists of four categories ( Feasibility , Value , Change/Action , Sustainability ) with 30 evaluation criteria, broken down across project-, individual-, group- and system-level impacts. Discussion & conclusion The presented novel framework brings together a significant knowledge base into a flexible, cross-sectoral evaluation effort that considers the whole participatory modelling process. Developed through the rigorous synthesis of multidisciplinary expertise from existing studies, the application of the framework can provide the opportunity to understand practical future implications such as which aspects are particularly important for policy decisions, community learning, and the ongoing improvement of participatory modelling methods.
... While landscape simulation modeling can be an effective tool to communicate multiple dimensions of environmental issues, which can aid policy discussion with a wide range of stakeholders [37] (e.g., problem definition, social and environmental objectives, modeling assumptions and strategies, policy framework, etc.), there are important considerations that emerged from our case study for streamlining the modeling process and stakeholder engagement: ...
Article
Full-text available
This study describes a collaborative modeling process deployed at the Cantareira Water Supply System (CWSS) in São Paulo City Metropolitan Area, Brazil. The CWSS faces challenges for meeting the increasing water demand, while land-use and climate change and their combined effect on its water cycle and balance have created a complex water resources management problem. Through a stakeholder engagement process—involving scientists and policymakers, the water utility company, and state administration—environmental simulation models were developed to elicit and represent multiple environmental, economic, and policy perspectives, developing a mutual language to communicate and establish common goals of water resources management. Study outputs include estimation of biophysical and economic benefits associated with prioritized native vegetation restoration activities in the source watersheds. These outputs are deployed in support of landscape planning and the decision process integrating multiple stakeholder perspectives in São Paulo state administration, the water utility company, and municipalities.
... Furthermore, identifying stakeholders by decision making power and personal impact metrics (Dhirasasna andSahin, 2019, Eden andAckermann, 2021) in addition to content areas knowldege, provides critical information about stakeholder-level barriers for change, e.g., low-interest, high-power individuals . Finally, such stakeholder classifications provide better information about the levels of involvement stakeholders should have in the project (Falconi and Palmer, 2017). ...
... Of course, widely applied and well-tested simulation modelling of the soil-water-atmosphere-plant system is a de facto illustration of an interdisciplinary effort, as soil scientists, hydrologists, climatologists and agronomists and ecologists have to provide basic data for the models (e.g. White et al., 2013;Kroes et al., 2017;Holzworth et al., 2018;Bieger et al., 2017;Falconi and Palmer, 2017;De Vries et al., 2022). Modelling is therefore a key methodology when assessing ecosystem services. ...
Article
Full-text available
The previous rather abstract debate about sustainable development has been focused by the introduction of the United Nations (UN) Sustainable Development Goals (SDGs) in 2015 and the related European Union (EU) Green Deal (GD) in 2019. Restricting attention to agriculture, proposed targets and indicators are, however, not specific enough to allow a focus for developing innovative and sustainable management practices. Clarity is needed because farmers are suspicious of governmental actions. To confront these problems, the European Commission (EC) has presented the Mission concept that requires joint learning between farmers, scientists and citizens. For the soil mission, “living labs” are proposed that should evolve into “lighthouses” when environmental thresholds for each of at least six land-related ecosystem services are met. This presents “wicked” problems that can be “tamed” by measuring indicators for ecosystem services that are associated with the land-related SDGs in a given living lab. Thresholds with a character that is occasionally regional are needed to separate the “good” from the “not yet good enough”. Contributions by the soil to ecosystem services can be expressed by assessing soil health. By introducing the mission concept, the policy arena challenges the research community to rise to the occasion by developing effective interaction models with farmers and citizens that can be the foundation for innovative and effective environmental rules and regulations. We argue and illustrate with a specific example, that establishing Living Labs can be an important, if not essential, contribution to realizing the lofty goals of the SDGs and the Green Deal as they relate to agriculture.
... As we built our model, using MURAL and Zoom, the 'visual' nature of the process helped us to represent the connections between abstract concepts, any of which could be changed by proposals from any stakeholder [93]. Used in this way, our FCM, in keeping with the literature, helped stakeholders to: (1) create a shared language, allowing for diverse knowledge and different disciplines to work together by creating a 'model' together on MURAL and in iterations afterwards; (2) clearly identify points of difference or similarity of relationships within and between stakeholder mental models; and (3) "transform their current collective knowledge toward an agreement of facts through discussion, negotiation, and careful scrutiny of what they know" to move towards innovation, cooperation, and consensus building [94][95][96]. PM and the process we used provided a model by which we could encourage individuals to share their perspectives on a collective issue of interest, forming an explicit group mental model as the object around which the discussion could be mediated [75,97,98]. Even if stakeholders did not completely agree with the model, it is more difficult to ignore the model "because they built the model themselves" [43,99]. ...
Article
Full-text available
Regenerative agriculture (RegenAg) can help landholders attune their agricultural practices to the natural design of the earth’s cycles and support systems. The adoption of RegenAg, however, hinges not only on a good understanding of biophysical processes but perhaps more importantly on deep-seated values and beliefs which can become an obstacle for triggering widespread transitions towards synergistic relationships with the land. We designed and facilitated a Participatory Modelling exercise with RegenAg stakeholders in Australia—the aim was to provide a blueprint of how challenges and opportunities could be collaboratively explored in alignment with landholders’ personal views and perspectives. Fuzzy Cognitive Maps (FCM) were used to unpack and formalise landholder perspectives into a semi-quantitative shared ‘mental model’ of the barriers and enablers for adoption of RegenAg practices and to subsequently identify actions that might close the gap between the two. Five dominant narratives which encode the key drivers and pain points in the system were identified and extracted from the FCM as a way to promote the internalisation of outcomes and lessons from the engagement. The Participatory Modelling exercise revealed some of the key drivers of RegenAg in Australia, highlighting the complex forces at work and the need for coordinated actions at the institutional, social, and individual levels, across long timescales (decades). Such actions are necessary for RegenAg to play a greater role in local and regional economies and to embed balancing relationships within systems currently reliant on conventional agriculture with few internal incentives to change. Our methods and findings are relevant not only for those seeking to promote the adoption of RegenAg in Australia but also for governments and agriculturalists seeking to take a behaviorally attuned stance to engage with landholders on issues of sustainable and resilient agriculture. More broadly, the participatory process reported here demonstrates the use of bespoke virtual elicitation methods that were designed to collaborate with stakeholders under COVID-19 lockdown restrictions.
... Multiple examples in water and natural resources management (e.g. Falconi and Palmer 2017;Wheeler, Robinson, and Bark 2018) suggest that models foster engagement by participants in a governance process through deliberation and negotiation. Discussion of PM and governance is beyond the scope of this paper. ...
Article
This work provides an analysis for the civil engineering community of the practice of participatory modelling (PM), reviewing the advances that environmental researchers and practitioners have made over 20 years, providing key references, case studies, and practical guidelines. Past consultation methods have proven inadequate to build trust with communities, and have led to development of PM to improve engagement. Three lessons from PM are emphasised: (1) listen to stakeholders to better understand the system, (2) collaborate with stakeholders to better model the system, and (3) co-decide on actions to better empower and engage stakeholders. Advice on the key aspects to consider when designing a PM process is summarised. The challenges and obstacles to progress are analysed for PM in civil engineering applications. PM appears to be of greater value in larger projects involving complex socio-technical systems. The incorporation of PM within civil engineering work will be more useful to society when civil engineers understand better the operation of PM.
... This space must also allow for individuals to share a common language toward meeting specific goals. Through participatory modeling processes, individuals co-create the goal statement and use modeling in some way to define group actions (see [28,29] for examples). One group action could be data collection; hence our interest in integrating CS with PM ( Figure 1). ...
Article
Full-text available
Stakeholder engagement and participation is often an essential ingredient for successful environmental conservation and management. Including stakeholders in participatory environmental research has been an increasingly recognized necessity for understanding the complex nature of social–ecological systems (SES). The public is also essential to help structure environmental problems and decide on management interventions. As a result, new inclusive approaches to scientific research have emerged, such as Citizen Science. While there have been many climate change-related citizen science projects, in this paper, we provide an overview of a specific type of citizen science project. More specifically, we describe a participatory modeling approach to citizen science which can support climate change research.
Article
Objective: The aim of this study was to examine safety-related contamination threats and risks to health-care workers (HCWs) due to the reuse of personal protective equipment (PPE) among emergency department (ED) personnel. Methods: We used a Participatory Design (PD) approach to conduct task analysis (TA) of PPE use and reuse. TA identified the steps, risks, and protective behaviors involved in PPE reuse. We used the Centers for Disease Control and Prevention (CDC) guidance for PPE donning and doffing specifying the recommended task order. Then, we convened subject matter experts (SMEs) with relevant backgrounds in Patient Safety, Human Factors and Emergency Medicine to iteratively identify and map the tasks, risks, and protective behaviors involved in the PPE use and reuse. Results: Two emerging threats were associated with behaviors in donning, doffing, and re-using PPE: (i) direct exposure to contaminant, and (ii) transmission/spread of contaminant. Protective behaviors included: hand hygiene, not touching the patient-facing surface of PPE, and ensuring a proper fit and closure of all PPE ties and materials. Conclusions: TA was helpful revealed that the procedure for donning and doffing of re-used PPE does not protect ED personnel from contaminant spread and risk of exposure, even with protective behaviors present (e.g., hand hygiene, respirator use, etc.). Future work should make more apparent the underlying risks associated with PPE use and reuse.
Article
Systems Intelligence (SI) can contribute to the design and practice of Participatory Modelling (PM) by paying attention to the interplay of the ‘soft’ socio-emotional system created by the actors involved and the dynamics created by their interactions and the ‘hard’ structure of the process. Here, we argue that by combining the perspective of SI with the four functions of PM (normative, substantive, instrumental, and educational), we can strengthen a collaborative and positive PM process, systematically designed to create socio-emotional decisions that stakeholders bring out into a wider system with them. This entails drawing from the four functions of PM, (normative, substantive, instrumental, and educational). To provide a blueprint of how each function might be achieved, we examine, through a transdisciplinary lens, the characteristics of each function, the sub-components and practical suggestions of how that might be applied in a PM context. Our main focus is to encourage a systems-based approach to achieving these functions, thereby avoiding piecemeal solutions, so we explore how the perspective of Systems Intelligence provides a lens and organizing structure to consider, design and facilitate PM. SI can help us to conceptualize and design PM, as it understands the central role of people within a dynamic system, a key starting point for those looking to design or direct their own PM process or for those searching (researchers, practitioners, or policymakers) for long-term solutions to problems of socio-ecological systems (SES). We look at how these two fields, PM and SI, might combine in practice, and suggest several promising areas of study to explore further. These insights will be of use to PM facilitators and researchers, as well as others using participatory methods in addressing SES challenges, particularly those encouraging the adoption of systemic perspectives, like Systems Intelligence.
Article
Full-text available
Research about ecosystem services (ES) often aims to generate knowledge that influences policies and institutions for conservation and human development. However, we have limited understanding of how decision-makers use ES knowledge or what factors facilitate use. Here we address this gap and report on, to our knowledge, the first quantitative analysis of the factors and conditions that explain the policy impact of ES knowledge. We analyze a global sample of cases where similar ES knowledge was generated and applied to decision-making. We first test whether attributes of ES knowledge themselves predict different measures of impact on decisions. We find that legitimacy of knowledge is more often associated with impact than either the credibility or salience of the knowledge. We also examine whether predictor variables related to the science-to-policy process and the contextual conditions of a case are significant in predicting impact. Our findings indicate that, although many factors are important, attributes of the knowledge and aspects of the science-to-policy process that enhance legitimacy best explain the impact of ES science on decision-making. Our results are consistent with both theory and previous qualitative assessments in suggesting that the attributes and perceptions of scientific knowledge and process within which knowledge is coproduced are important determinants of whether that knowledge leads to action.
Article
The National Environmental Policy Act of 1969 was enacted on 1 January 1970 in an effort to specify maintenance methodology for the environment of the United States and to protect natural resources while allowing for health and welfare concerns for an evergrowing population. It created a Council on Environmental Quality, which generates information for the executive branch of the government to be transmitted to the legislative branch to facilitate the best possible system and relationship between the people and the land of the United States.
Article
Multi-criteria decision analysis (MCDA) is an umbrella approach that has been applied to a wide range of natural resource management situations. This paper has two purposes. First, it aims to provide a critical review of MCDA methods applied to forest and other natural resource management. The review seeks to layout the nature of the models, their inherent strengths and limitations. Models are categorized based on different classification schemes and are reviewed by describing their general characteristics, approaches, and fundamental properties. The review goes beyond traditional MCDA techniques; it describes new modelling approaches to forest management. The second purpose is to describe new MCDA paradigms aimed at addressing the inherent complexity of managing forest ecosystems, particularly with respect to multiple criteria, multi-stakeholders, and lack of information. Comments about, and critical analysis of, the limitations of traditional models are made to point out the need for, and propose a call to, a new way of thinking about MCDA as they are applied to forest and natural resource management planning. These new perspectives do not undermine the value of traditional methods; rather they point to a shift in emphasis-from methods for problem solving to methods for problem structuring. (c) 2006 Elsevier B.V. All rights reserved.
Article
Group model-building here refers to a system dynamics model-building process in which a client group is deeply involved in the process of model construction. The problem that is modelled can be reasonably well defined, but it can also take the form of an ill-defined or messy problem, i.e., a situation in which opinions in a management team differ considerably These messy managerial situations are difficult to handle, primarily because thus far little theoretical work has been conducted to shed more light on the question why these messy situations exist and why it may be difficult for a management team to reach agreement. This article fills this theoretical gap by drawing on literature from sociology, (social) psychology and small-group research. Insights from this literature are discussed and translated into guidelines for conducting Group Model-Building projects for messy problems. The article ends with the conclusion that system dynamicists should include Group Model-Building and facilitation training in their teaching programs. Copyright (C) 1999 John Wiley & Sons, Ltd.
Chapter
This chapter explores the changes in Brazil’s water management through the emergence of a new paradigm. It pushes two aspects of water management, namely: To reframe water as a common good with economic value, and to create participatory institutions that will help in the development of water management. The chapter posits the value of knowledge and research in providing clear paths for stakeholders to be able to make informed decisions regarding effective water management. It explores how Brazil’s water systems pose significant challenges to policymakers in equity terms, and what we can learn from their case and history, concluding with the ability to reform water management and policies being critically reliant on personal belief systems and worldviews. Evidence is shown of the possibility of more equitable, transparent, and accountable systems that are more beneficial when compared to previously exclusionary, clientelistic systems.