ResearchPDF Available

Abstract and Figures

Citizen and stakeholder participation are often expected to improve the outcomes of public governance. Little attention has been paid so far to whether and under what circumstances the outputs of participatory processes are actually taken up by policy decisions and get implemented. This study reports on findings from a case survey meta-analysis of 143 cases of public environmental decision-making across Europe, North America, Australia and New Zealand. The paper asks: (1) What kinds of outputs do participatory decision-making processes produce? (2) What are the key contextual conditions under which binding political decisions take up the outcomes of participatory processes? (3) What are the key process features that determine whether binding policy outputs emerge? This study takes us further towards understanding the 'fate' of participatory decision making in environmental governance and beyond.
Content may be subject to copyright.
Working Paper 10 | February 2021
From Collaboration to
Policymaking: How Collaborative
and Participatory Decisions
Actually Change Policy (or not)
Edward Challies,
Nicolas W. Jager,
Jens Newig,
Elisa Kochskämper
Maren Preuss
ConstDelib Working Paper Series, no. 10 / 2021
COST Action CA17135
COST Action 17135, in short ConstDelib, focuses on “Constitution-making and deliberative
democracy” and carries out research with the support of the European Union’s Horizon
2020 Framework Program. The Action is active until 2022 and includes members from over
40 countries. Its main aim is to bring together researchers, public servants, elected
officials, experts, citizens, participatory consultants and civil society organizations to
discuss and reflect on the democratic challenge of reforming a constitution through
democratic deliberation.
The Working Paper Series
The ConstDelib Working Paper Series publishes work in progress aiming to contribute to
the advancement of knowledge in the broader field of democratic deliberation and in
particular with reference to Constitution-making. The works cut across a wide range of
research areas, demonstrating both the depth and breadth of research being undertaken
by members of the COST Action. We also offer the opportunity for researchers outside the
Action to publish with us their work related to the topic of the Action.
Managing Editor:
Sergiu Gherghina (University of Glasgow)
Editorial Board:
Venetia Argyropoulou (European University Cyprus)
Paul Blokker (University of Bologna)
Marie Dufrasne (University Saint-Louis Brussels)
Raphaël Kies (University of Luxembourg)
Sergiu Mișcoiu (Babes-Bolyai University Cluj)
Monika Mokre (Austrian Academy of Sciences)
Ioannis Papadopoulos (University of Macedonia)
Min Reuchamps (Catholic University of Louvain)
Yanina Welp (Albert Hirschman Centre on Democracy, Graduate Institute)
Oya Yeğen (Sabanci University)
Logo and cover design:
Alexandros Kyriakidis (University of Macedonia)
ConstDelib Working Paper Series, no. 10 / 2021
Citizen and stakeholder participation are often expected to improve the outcomes of
public governance. Little attention has been paid so far to whether and under what
circumstances the outputs of participatory processes are actually taken up by policy
decisions and get implemented. This study reports on findings from a case survey meta-
analysis of 143 cases of public environmental decision-making across Europe, North
America, Australia and New Zealand. The paper asks: (1) What kinds of outputs do
participatory decision-making processes produce? (2) What are the key contextual
conditions under which binding political decisions take up the outcomes of participatory
processes? (3) What are the key process features that determine whether binding policy
outputs emerge? This study takes us further towards understanding the ‘fate’ of
participatory decision making in environmental governance and beyond.
decision-making, participatory democracy, policy outputs, environmental governance
Edward Challies is a Senior Lecturer with the School of Earth and Environment, University
of Canterbury, New Zealand. E-mail:
Nicolas W. Jager is a senior researcher at the chair of Ecological Economics at Carl von
Ossietzky Universität Oldenburg, Germany. E-mail:
Jens Newig is professor of governance and sustainability and head of the Institute of
Sustainability Governance at Leuphana University Lüneburg, Germany. He is co-affiliated
with Leuphana’s Center for the Study of Democracy. E-mail:
Elisa Kochskämper is a senior researcher in the department on Institutional change and
regional public goods at the Leibniz Institute of Research on Society and Space in Erkner,
Germany. Email:
Maren Preuss is a power market analyst at Aurora Energy Research. She studied
Environmental Policy at Leuphana University Lüneburg, Germany. E-mail:
To cite this paper
Challies, Edward, Jager, Nicolas W., Newig, Jens, Kochskämper, Elisa & Preuss, Maren.
2021. From Collaboration to Policymaking: How Collaborative and Participatory Decisions
Actually Change Policy (or not),February, ConstDelib Working Paper Series, no. 10, pp. 1-
All views expressed in this paper are those of the authors and do not represent the views
of COST Action CA17135, COST Association, or the EU. The copyright rests with the authors.
Introduction: From Collaboration to Policymaking?
Democratic innovations, such as various forms of collaborative governance and citizen and
stakeholder participation are often credited with producing better environmental policy outputs
and outcomes than top-down, technocratic decision-making (Emerson and Nabatchi 2015;
Koontz and Thomas 2006; Newig and Kvarda 2012; Jager et al. 2020). However, relatively little
attention has been paid so far in the literature to how far collaborative and participatory outputs
actually inform policy decisions and get implemented (Birnbaum 2016; Font et al. 2018, Koontz
2005). This study contributes to addressing this gap, and asks if, how, and under what conditions
the outputs of collaborative governance processes are taken up in politically binding decisions
and policy. The study analyzes 143 cases of participatory environmental governance, examining
key aspects of the socio-political context in which processes occur, the characteristics of
processes themselves, and the nature of the outputs produced, in order to trace the fate of
participatory outputs. With this analysis we seek to build on the work of Font and colleagues
(Font et al. 2018; Font et al. 2016), who examined the fate of proposals originating from 39
participatory processes across 25 Spanish municipalities. Drawing inspiration from this research,
and from the work of others who have studied the policy impacts of participation processes (e.g.
Busenberg 2000; Koontz 2005, 2006; Koontz et al. 2019), we take in a wider variety of
participatory decision-making processes that have played out in a range of national settings,
addressing diverse environmental issues.
We find that it tends to be participatory outputs that draw heavily on knowledge and the
experiences of participants, and in which participants have a strong say in the content and shape
of the output, that are taken up in political decisions. Our results also suggest that high levels of
communication alone, controlling for participant influence and knowledge integration, have a
negative effect on the political uptake of participatory outputs. These findings (1) provide
additional nuance and insight on the fate of outputs generated by means of public participation,
thus (2) contributing new empirical evidence to inform ongoing debates about the legitimacy
and instrumental value of participatory governance and democratic innovations (see Newig and
Fritsch 2009a).
Conceptual framework
In this study, our unit of analysis is the public environmental decision-making process (DMP). We
are interested in processes that are to a greater or lesser extent participatory or collaborative
(Newig et al. 2013), and which aim to reach a collectively binding decision on issues concerning
From Collaboration to Policymaking
the environment. A DMP may be initiated in a ‘top-down’ or a ‘bottom-up’ fashion, and may
comprise a single process or several related (sub-) processes (e.g. public hearings, task forces,
round tables, citizen advisory committees etc.). This broad framing intentionally captures a wide
variety of governance modes and ‘degrees’ of participation or collaboration in planning,
licensing, rule-making, impact assessment and other forms of public policy-making.
We understand public and stakeholder participation as a multi-dimensional concept that
may vary along at least three dimensions (Fung 2006; Newig et al. 2018):
o Communicative intensity: The nature, direction and intensity of information flows (e.g. one-
way provision of information or consultation versus two-way dialogue and collaborative
development of preferences);
o Participant influence: The extent to which participants are able to shape or determine the
decisions taken i.e. the ‘participatory output’;
o Access: The extent to which a range of stakeholders and other actors are included in, or are
able to participate in, the process (e.g. participation by a small number of selected experts,
representatives of organized groups, or citizens versus participation by the general public).
In this sense, a policymaking or planning process involving some form of public or stakeholder
participation may be understood as producing a sequence of outputs (e.g. decisions, plans,
proposals) with more or less binding character that may in turn be implemented and
subsequently have some impact on the environment (see Figure 1). According to this conceptual
model a participatory process, which may itself comprise multiple sub-processes or fora, is
embedded within a wider public DMP. This wider DMP of course plays out in a particular socio-
political and environmental context, aspects of which are likely to shape the dynamics of
participation and decision-making. For the purposes of the present analysis, we identify the end
of a participatory process as the point at which a participatory output is produced.
This participatory output (which could be a decision, a proposal, or a set of
recommendations) may have varying degrees of legal bindingness, ranging from a simply
informative character, to semi-binding forms of commissioned advice, to legally binding
resolutions or decisions. In many cases, however, participatory processes have a consultative or
advisory character (Bingham 2010; Fung 2006; Klijn et al. 2012; Koontz 2005) without a direct
legal mandate. Eventually, the political decision itself may or may not be implemented. Factors
that can hinder implementation include a lack of cooperation among implementing actors, and
the overruling of political decisions through litigation and court rulings (Coglianese 1997;
Edward Challies et. al
Kochskämper et al. 2016). Where a political decision is implemented, the environmental impact
of this in terms of actual observable change in environmental quality may be delayed or difficult
to recognize, and attribution of environmental change to a specific political decision is not
straightforward (Ferraro 2009; Koontz et al. 2019).
Figure 1: A participatory decision-making process model including outputs and implementation
While implementation and subsequent impact of environmental policy are important areas of
research, we focus in this study on a critical and under-researched intermediate step between
participatory or collaborative governance, and the adoption of policy namely, the uptake of
participatory outputs in political decisions. In particular, we analyze how and under what
conditions participatory outputs are taken up in political decisions. Evidence on factors
influencing the uptake of participatory decisions in the political process is sparse (Font et al.
2018). Building on Font et al. (2018), we distinguish potential influencing factors in three
categories: contextual, process-related, and output-related factors.
Contextual factors
Contextual factors relate to socio-political and environmental aspects of the policymaking
setting within which a given process takes place (Ingram 2011). Different societies and
communities have different dispositions towards public participation in political processes,
From Collaboration to Policymaking
influenced by the political culture of the decision-making environment. While some communities
may be very amenable to and familiar with public participation, others may be less accustomed
to participation, and less comfortable with the idea of involving the wider public directly in the
political process (Newman et al. 2004). It is plausible that in places with a more established
culture of participation, participatory decisions will be taken more seriously and accepted to a
greater degree by the authorities. We therefore test the following hypothesis:
H1: Where there is a political culture of participatory decision-making, participatory outputs are
more likely to be taken up in political decisions.
Apart from the political culture towards participation, social capital within a given community
in terms of the degree of shared norms and trust among government actors and the public
(Newig et al. 2013) may play an important role in the adoption of a participatory output (Klijn
et al. 2010; Oh and Bush 2014). Close, trustful relationships among networks of stakeholders,
authorities and the wider public may foster sincere engagement and collaboration, enhance
mutual monitoring and social control, and increase the detection of non-compliance (Leach and
Pelkey 2001; Ostrom 1990). In light of this, we therefore hypothesize that:
H2: Where social capital among authorities and the public is high, participatory outputs are more
likely to be taken up in political decisions.
A further contextual factor, which we expect to be related to the foregoing factors, is connected
to the decision-making level in the sense of the administrative level at which a given process plays
out. This administrative scalar dimension has been identified as influential in shaping how
participation plays out (Newig, Schulz and Jager 2016). As Dahl (1994) argued, participation may
be easier to organize on more local levels, which potentially allow for the representation of a
larger proportion of the community. In such a local-level setting, the adoption of participatory
outputs may be more likely given closer relations between the public and the responsible
authorities. We therefore test the following hypothesis:
H3: Where a participatory process takes place at a lower, more local level, participatory outputs
are more likely to be taken up in political decisions.
Edward Challies et. al
Beyond the aforementioned contextual factors, and relating more to the particular decision-
making environment, we suggest that certain problem settings will have a bearing on the
success of participatory processes and the adoption of participatory outputs. In particular we
assume that so-called NIMBY (not in my backyard) situations represent a specific kind of problem
setting that implies considerable barriers to rational dialogue and consensus building (Schively
2007).In such contexts, the interests of a particular group of actors are to be weighed against a
wider collective interest, often leading to highly contentious proposals and intractable situations
(Fischer 1993). Under these circumstances, two trajectories appear possible: On the one hand, a
participatory output may entail a mutually acceptable solution to the tensions between
particular and collective interests, which may make adoption of the output more likely. On the
other hand, a participatory output may emerge out of a conflictual process, which proposes a
sub-optimal or unworkable solution. This participatory output may then be overruled or
disregarded by a subsequent political decision if it reflects too strongly the particular interests
of a few affected parties or otherwise fails to overcome deadlock or deliver progress on an
urgent or important issue. While both general scenarios may be plausible, we test here the
following negative hypothesis, given the often particularly challenging nature of NIMBY settings:
H4: Where a NIMBY situation characterizes the issue at stake in a participatory process,
participatory outputs are less likely to be taken up in political decisions.
Process-related factors
A second set of variables that might help explain the fate of participatory outputs relates to key
features of the participatory process itself. It is reasonable to assume that the qualities of a
participatory process will impact on the uptake of the output to emerge from the process in a
subsequent political decision (Beierle and Cayford 2002; Font et al. 2018; Koontz 2006). We
assess the qualifying characteristics of participatory processes according to the three
dimensions identified above i.e. communicative intensity, participant influence, and access to
the process. In terms of communicative intensity, it is possible that a process characterized by
intensive two-way dialogue among participants, and free flow of information, will deliver an
output that is more reflective of the shared interests of participants, and more likely to be
accepted by all parties, including the responsible authority. However, there is also potential for
highly intensive processes to arrive at proposals or decisions that are not feasible for authorities
From Collaboration to Policymaking
to adopt in a given political or economic climate. For the present analysis, however, we test the
following hypothesis:
H5: Where communication in a participatory process is more intensive, participatory outputs are
more likely to be taken up in political decisions.
The second feature of participatory processes that may affect the uptake of participatory
outputs relates to participant influence, in the sense of the degree to which participants are able
to shape or determine the participatory output (Newig et al. 2018). A high degree of participant
influence may have a positive effect on uptake of the output if the output reflects the values and
interests of participants and stakeholders, and therefore commands widespread public
acceptance and approval (Kochskämper et al. 2018). This is perhaps even more likely where
authorities have deliberately delegated authority to participants and afforded influence. In cases
where participants may have asserted a high degree of influence against the intended or
anticipated function of the process, it is possible that this relationship could work in the opposite
direction against uptake of the participatory output. Here we test the following positive
H6: Where participants have more influence in determining the participatory output,
participatory outputs are more likely to be taken up in political decisions.
Access to the participatory process is a further process-related factor that might influence uptake
of the output. Participants may be recruited into a decision-making process or gain access to a
process in a variety of ways. Some processes may involve participation by a small number of
strategically selected experts or representatives of key groups who are invited to be part of the
process (Fung 2006). Other processes may be relatively open and inclusive, whereby participants
self-select and freely opt in to the process. While both modes of participant selection can be
effective depending on the nature of the issue at hand and the purpose of the participatory
process, we suggest that a more targeted participant selection, wherein the responsible
authority maintains control over who participates, may be more likely to generate an output that
is consistent with the aims and priorities of the CA and is therefore subsequently taken up or
adopted (Koontz & Moore Johnson 2004). We hypothesize that:
Edward Challies et. al
H7: Where access to a participatory process is more open or unrestricted, participatory outputs
are less likely to be taken up in political decisions.
As the decision-making processes analyzed in this study are oriented towards addressing an
environmental problem, we also investigate the importance of relevant knowledge provided,
elicited, and aggregated during the process. The knowledge in question may relate to the
resource or issue at hand, or to the socio-political context in which the decision will need to be
implemented (Ulibarri 2015). The key factor of interest here, though, is the extent to which the
process involved structured methods for the incorporation of knowledge relevant to addressing
the environmental issue at hand. A participatory output that is produced through such
knowledge integration methods is likely to be more fit-for-purpose (Newig et al. 2018), and
therefore more likely to be taken up in a political decision. We test the following hypothesis:
H8: Where participant knowledge is deliberately integrated into a participatory process,
participatory outputs are more likely to be taken up in political decisions.
Beyond key characteristics of the process itself, we suggest that the commitment of the
responsible authority to supporting and maintaining the process may be a decisive factor in the
fate of process outputs (Busenberg 2000). Greater commitment on the part of the authority is
likely to mean that the process is sufficiently resourced and supported to reach a solution that is
acceptable from the perspective of the responsible authority. In general, we contend, an
authority that is highly committed to a participatory process is more likely to also be committed
to adopting the output of that process. We hypothesize that:
H9: Where the responsible authority is more committed to maintaining the participatory
process, participatory outputs are more likely to be taken up in political decisions.
Output-related factors
Finally, the nature of the participatory output itself may impact on its uptake in a political
decision. Following Font et al. (2018), we assume that an output that implies divergence from the
status quo may have a lower chance of being taken up in political decisions. This is plausible
because adoption of such outputs may demand that authorities acknowledge a need to change
(or even admit to being out of step with public sentiment), and because departure from business
From Collaboration to Policymaking
as usual typically requires considerable investment in new expertise, systems and processes
(Barbier 2011). In contrast, outputs that mainly reiterate existing policies (Fischer 2014), or
reinforce decisions that have effectively already been taken (Cooke and Kothari 2001; Hoppe
2011), may be easier for authorities to adopt. We hypothesize that:
H10: Where the output of a participatory process diverges from the status quo, participatory
outputs are less likely to be taken up in political decisions.
This study builds on a database of 307 participatory environmental decision-making processes
(see Newig et al. 2013), which covers a variety of more- and less-participatory public decision-
making procedures (from administrative decision-making to highly collaborative processes) and
environmental issues (e.g. water management, waste facility siting, land use planning). The data
were generated via a case study meta-analysis (case survey method) (Larsson 1993; Newig and
Fritsch 2009b) of published case studies. The case survey is a meta-analytical method that
involves the conversion of rich, qualitative information from narrative case studies into
quantitative data, representing a numeric interpretation of the case study texts. The method is
particularly suited to aggregating data and synthesizing emergent finding in fields where
empirical evidence is mainly scattered among numerous single or small-N case studies, as is the
case in public participation research.
We define a case as an instance of public decision-making, which can vary in the degree
to which it is participatory in the previously defined sense, but is aimed at reaching a collectively
binding decision (‘participatory output’). Our case survey involved the following steps (see
Figure 2):
1) Case identification and selection: Faced with the broad variety of terms used to describe and
analyze participatory processes in the literature, we decided against relying on a single search
string, but rather opted for a number of combinations of search terms in several iterations.
We conducted a thorough search of several library catalogues and scientific databases
considering studies published up until 2014 in English, French, German, or Spanish language,
Sources searched include: BASE; Google Books; Google Scholar; GVK+; Science Direct; SciVerse Hub; Scopus;
SpringerLink; SSRN; Web of Science; Wiley Interscience.
Edward Challies et. al
in publicly available outlets including books, peer-reviewed journals, edited volumes, theses,
working papers, and various forms of grey literature. Geographically, for reasons of
comparability, cases were limited to Western democratic countries i.e. from Europe, North
America, and Australia and New Zealand. In this way, we identified more than 2,200 single
cases, described in over 3,300 texts. We continued to search until saturation was reached and
no new cases were being discovered, at which point we assumed that we had covered a
comprehensive set of relevant, publicly available cases. Identified texts were screened for
suitability, and those containing insufficient information for analysis were eliminated,
resulting in a database of 639 ‘codeable’ cases. We then randomly sampled 307 cases for full
2) Coding scheme development: On the basis of our conceptual understanding of participatory
decision-making processes, we developed an analytical coding scheme [ANONYMOUS]
containing 256 quantitative variables to measure the process, context, outputs and outcomes
of decision-making. Measures mostly employed 5-point scales from 0..4.
3) Case coding: Each case was independently read and coded by three trained coders. In addition
to the coding of actual variables, coders specified the reliability of the information
underpinning their coding decision for each variable (on a 3-point scale, 1 = enough
information for an informed guess, 3 = explicit, detailed and reliable information). After initial
independent coding, data for each case were collated and coders met to discuss the case in
order to address coding mistakes and explore divergent interpretations. However, in
accordance with the case survey method, and to allow for different interpretations of the
case material, coders were not required to reach consensus on codings (Kumar et al. 1993).
Despite this explicit provision for divergent codings, inter-coder reliability, assessed through
G(q,k) (Putka et al. 2008), was 0.77, and interrater agreement (rWG, James et al. 1984) was
0.73, indicating high validity of the data overall. The final data set was generated by averaging
the three coders’ scores, weighted with the respective information reliability scores (see
Bruggen et al. 2002).
Given the aim of this analysis was to explore the fate of outputs of participatory processes, we
further limited our data set to those cases that displayed a minimum degree of participation. We
applied a minimum threshold value of 1 (with a maximum of 4) for each dimension of
participation discussed above: communicative intensity; participant influence; and access to the
process. This yielded 183 cases, and after excluding a further 24 cases due to lack of information
From Collaboration to Policymaking
on politically binding decisions, we arrived at 159 participatory cases. Finally, a further 16 cases
were excluded for lack of a participatory output, which left us with a final sample of 143 cases
for analysis.
Figure 2: Case identification, screening and sampling process
We operationalized the dependent variable that is, the uptake of a participatory output in a
subsequent political decision by assessing the similarity between the former and the latter (10
cases did not lead to a political output). We measure this in a binary way, with 1 indicating full
adoption of the participatory output into the political output, and 0 everything below (e.g.
adoption of an altered proposal, or outright rejection). We rely on this rather crude, binary
measure, sacrificing some information and detail about cases, as case study records usually do
not focus on this step of participatory governance, and hence, provide very little information on
this step of implementation. While this may not yield highly nuanced insights, it will provide a
robust, conservative estimate of the fate of these participatory decisions.
Edward Challies et. al
For the independent variables, we drew on the conceptualization provided in the SCAPE
codebook (Newig et al. 2013) or aggregates thereof (e.g. through Principal Component Analyses
(PCA)). Detailed variable descriptions can be found in table 1. For the analysis, we applied
descriptive statistics, as well as regression analyses (logistic regression).
Table 1: Explanatory factors for the uptake of a participatory output
Participation culture
Degree to which participation and cooperation were
accepted as appropriate means to resolve social and
political conflicts and make public decisions, at the scale
of the DMP (56), scale: [0..4].
Social capital
Resulting PCA 1, trust in government (51), trust in
governmental actors (52), trust among stakeholders
(53), shared values and norms among stakeholders (56),
Cronbach’s Alpha=0.75
Binary variable (derived from 50)
1: County-level or below
0: Above county level
Binary variable (79):
1: Existence of a NIMBY situation
0: Absence of a NIMBY situation
PCA2: Dialogue (231), face-to-face communication (169),
consultation (230), information provision by authorities
(229), Cronbach’s Alpha=0.9
Degree to which the participants (excluding the CA)
actually developed and determined the output (232),
scale: [0..4]
Degree to which participant selection was designed in a
controlled way (161), scale: [0..4], with 0 being
completely unrestricted
PCA3: structured information elicitation (165) and
aggregation (166), usage of methods for knowledge
integration (170), Cronbach’s Alpha=0.87
Degree to which the responsible authority was
committed to (maintaining) the DMP (184), scale [0..4]
Divergence from
Degree to which the output diverges from the status-
quo. Calculated by taking the absolute value of the
degree to which the environmental output aimed at a
change (for the better or the worse) of environmental
conditions in terms of natural resource protection (261),
human health (260), and conservation (259), scale
[0..4], Cronbach’s Alpha=0.87
NOTE: Numbers in brackets indicate the original variable numbers from the SCAPE coding
scheme (Newig et al. 2013).
From Collaboration to Policymaking
As depicted in figure 3, we traced the fate of outputs from 143 participatory/collaborative
processes (A). 133 of the 143 processes that produced participatory outputs were then followed
by politically binding decisions (B). Of these, in 93 cases the political decision completely adopted
the participatory output, whereas in 39 cases, the political decision was quite independent of
the participatory output, i.e. they aligned only partially or not at all (C)
. Of the 93, 69 were
implemented (at least to some degree), whereas 24 were not (at the respective time of writing)
(D). In total, eight decisions were challenged in court, but none was substantially changed or
repealed (see figure 3).
Figure 3: The 'fate' of participatory outputs
Note: The numbers reflect the decision-making processes in our sample.
In one case, data was not sufficient to assess the match.
Edward Challies et. al
When do participatory outputs get taken up in politically binding decisions?
On the basis of the factors identified in section 2 above, we executed a regression analysis, and
compared several models. Table 2 presents the estimates for these models on the similarity
between participatory and political output.
The role of the participatory process: Model (1) includes only those variables that describe
the qualities of the participatory process. It shows particularly pronounced effects for participant
influence and knowledge integration. This indicates that processes that allow participants
particular influence in shaping the output, as well as those that have a particular focus on
knowledge elicitation, aggregation and integration, have a high likelihood of delivering outputs
that will be adopted in a political decision. These effects are robust throughout our analysis as
the subsequent models, including further controls, show. On the 0.1 level, communicative
intensity also appears significant. Interestingly, the sign is not in the assumed direction, but
rather suggests that processes with higher communicative intensity generate outputs that are
less likely to be taken up in political decisions (and vice versa). This effect is robust, when
controlling for additional factors in models (2) to (4).
The role of commitment by the responsible authority: In model (2) we introduce authority
commitment, which describes the commitment of the responsible authority (i.e. the authority
that has legal responsibility for the issue and is therefore responsible for the DMP) towards
maintaining the process. Indeed, we observe that this factor is significant. However, in
subsequent models, which control for further contextual variables, the direction of the effect
remains positive, but its significance drops below conventional thresholds.
The role of context: After considering the main process factors, model (3) introduces the
contextual factors into the analysis. This model reveals a significant negative effect for NIMBY
situations. This means that participatory outputs generated in a NIMBY context have a lower
likelihood of being adopted as, or taken up in, political decisions. Social capital, while also
displaying the hypothesized positive sign, is only significant at the lower 0.1 level. The signs of
participatory culture and local level are negative, but effects are not significant. The explanatory
power of the model increased considerably, as overall model parameters show an improved
model fit, as the AIC dropped from 161.086 to 155.377 and McFadden’s R-square increased up to
From Collaboration to Policymaking
Table 2: Model estimates for logistic regression models
Dependent variable:
Similarity Participatory Output Political Output
NIMBY situation
Communicative Intensity
Participant influence
Knowledge Integration
Authority Commitment
Output Divergence
Log Likelihood
McFadden R2
Note: *p<0.1; **p<0.05; ***p<0.01; Standard errors in parentheses.
The role of output characteristics: Finally, model (4) includes all factors previously identified as
well as the characteristics of the participatory output
. However, the output’s divergence from
the status-quo, while displaying the assumed negative sign, does not show a significant effect. In
For this final model, we also ran standard diagnostic tests, to assess the validity of the model. Tests for
multicollinearity, linearity, heteroscedasticity, the influence of outliers, and for overdispersion did not yield
problematic results.
Edward Challies et. al
this, final model, the effects of knowledge integration, communicative intensity, participant
influence and NIMBY situations remained robust. Model parameters again show an improved
model fit with the AIC being 152.761 and McFadden’s R-square rising to 0.29 indicating an
acceptable model fit.
For model (4), we also calculated the odds ratios for all predictors that is, the likelihood
of a change in the dependent variable occurring with a unit of change in the independent
variable. Figure 4 displays these, together with 95%-confidence intervals. The figure supports the
model’s results in that Knowledge, Participant influence, Communication, and NIMBY situations
do not cross the value of 1 (meaning that the odds for the dependent variable to occur are the
same as for the null model, i.e. without any intervention), indicating a clear effect. The model
suggests that the odds of a participatory output being taken up in a political decision are two
times higher if knowledge elicitation and integration increases by one unit, and 2.66 times higher
for every unit change in participant influence. Communication intensity has a log-odds of 0.46,
while changes from a non-NIMBY (0) to a NIMBY situation (1) results in a change in odds of 0.27
for the participatory output being adopted in a political output i.e. it becomes almost four-
times less likely.
Figure 4: Odds ratios (OR) of independent variables (model 4)
Note: Dots mark the odds ratios derived from model 4, bars indicate 95%-confidence intervals.
Part. Culture
Local Level
Output Deviance
Social Capital
Authority Commitment
Power Delegation
0.1 0.2 0.3 0.4 0.6 0.8 1 2 3 4 5 6 7 8
Odds Ratios
From Collaboration to Policymaking
Discussion and conclusions
This study provides an overview of the ‘fate’ of outputs from a large number of participatory
decision-making processes in environmental governance. In our sample, most participatory or
collaborative governance processes aiming to produce a politically binding decision, or to be
taken up in a political decision, did in fact succeed in this. Quite clearly, the degree of knowledge
integration built into a participatory process serves as a predictor for whether or not a resultant
outcome is taken up in a subsequent politically binding decision. This suggests that those
processes, where participants are genuinely included as co-creators of knowledge, will also more
likely feed into a political program than those where participants do not have this active role.
The positive effect of power delegation to participants could be interpreted as a further
indication of this.
The communicative intensity shows a significant negative effect on the uptake of
participatory outputs in political decisions. Seen in the context of the analysis conducted here,
the results suggest that communication, controlling for its role in knowledge elicitation and
integration and in executing participants influence, does have a negative effect. This can be seen
as an indication that high levels of communicative intensity (by itself), beyond its actual
instrumental value for other variables, may mean that processes are occupied with a lot of
discussion and negotiation, but not so much with consequential decision making.
Overall, our analysis suggests that particularly those traits characterizing the
instrumental value of participation (such as its value for knowledge generation and direct
decision) facilitate the uptake of decisions; while on the other hand, variables that capture input
legitimacy (e.g. access), empowerment and societal context do not show the same effect. This
suggests that overall, participation is especially impactful when used as a tool.
However, we do find an interesting significant effect where the environmental issue at
hand is characterized by a NIMBY situation, that is, where particular interests have to be weighed
against wider benefits. In such contexts, political decision-makers were less likely to adopt the
recommendations developed through a participatory process to address these situations. We
can, based on the present analysis, only speculate as to the reasons for these findings. Further
analyses may specify the characteristics of these cases and their outputs in more detail. But these
insights also highlight that NIMBY situations pose particularly challenging problem settings for
environmental and participatory decision-making, where decision-makers are faced with strong
tensions between individual and wider societal interests and the difficulty to balance these. This
Edward Challies et. al
has especially important implications for the democratic legitimacy of decisions and the question
of which interests are more highly valued in these situations.
This study has provided some early findings on the fate of the outputs generated in
participatory governance processes. It relied on an analysis of 143 participatory processes and
their outputs, coded through a case survey meta-analysis. It is our hope that the analysis has
contributed to a foundation for future research to inform a deeper understanding of the
processes at play in collaborative and participatory environmental decision-making processes.
More attention needs to be paid to the context and procedural features that shape the fate of a
participatory output and the wider role of democratic innovations within the policy process. For
example, building on this study, research might ash what precisely are the factors that make
NIMBY situations so particular in that the chances of a participatory output being adopted are
so much lower? How exactly are the tensions between particular and collective interests
negotiated and accommodated? What happens further along in the implementation process,
and are participatory decisions more likely to lead to changes in environmental quality on the
These are just a few seemingly important questions, given the current popularity of
collaborative and participatory environmental governance in Europe, North America, and
beyond. Major pieces of environmental legislation, such as the European Water Framework
Directive, and the Floods Directive attach considerable importance to public participation,
prescribing the involvement of non-state actors and stakeholders in environmental planning and
decision-making. Insights into the fate of the outputs of these processes are therefore relevant,
first, from an instrumental perspective, to understand how the ‘instrumental claim of
participation’ (i.e. that participation enhances the environmental quality of political decisions)
translates on the ground; and, second, from a democratic perspective, tracking the embedding
of participatory processes in the wider political process and its implications for the democratic
legitimacy of decisions.
Earlier versions of this paper were presented at the International Workshop “The Consequences
of Democratic Innovations”, Louvain-La-Neuve 9-10 September 2019, the European Consortium
of Political Research (ECPR) General Conference, Hamburg, 22-25 August 2018, and the 75th
Annual Midwest Political Science Association (MPSA) Conference, Chicago, 6-9 April 2017. We
thank the participants of these events for their valuable comments and feedback.
From Collaboration to Policymaking
List of References
Barbier, E.B. (2011). Transaction costs and the transition to environmentally sustainable
development. Environmental Innovation and Societal Transitions, 1(1), 58-69.
Beierle, T.C. & Cayford, J. (2002). Democracy in Practice: Public Participation in Environmental
Decisions. Washington, D.C.: Resources for the Future.
Bingham, L.B. (2010). Collaborative Governance. In M. Bevir (Ed.), The SAGE Handbook of
Governance (pp. 386-401). London: Sage.
Birnbaum, S. (2016). Environmental Co-Governance, Legitimacy, and the Quest for Compliance:
When and Why Is Stakeholder Participation Desirable? Journal of Environmental Policy &
Planning 18(3):30623.
Busenberg, G.J. (2000). Resources, Political Support, and Citizen Participation in Environmental
Policy: A Reexamination of Conventional Wisdom. Society & Natural Resources, 13(6), 579-
Coglianese, C. (1997). Assessing Consensus: The Promise and Performance of Negotiated
Rulemaking. Duke Law Journal 46(6):12551350.
Cooke, B. & Kothari, U. (2001). Participation: The New Tyranny? London: Zed Books.
Dahl, R.A. (1994). A Democratic Dilemma: System Effectiveness versus Citizen Participation.
Political Science Quarterly 109(1):2334.
Emerson, K. & Nabatchi, T. (2015). Collaborative Governance Regimes. Washington, D.C.:
Georgetown University Press.
Ferraro, P.J. (2009). Counterfactual thinking and impact evaluation in environmental policy. In
M. Birnbaum & P. Mickwitz (Eds.), Environmental Program and Policy Evaluation:
Addressing methodological challenges (Vol. 122, pp. 75-84). Chichester: Wiley.
Fischer, F. (1993). Citizen Participation and the Democratization of Policy Expertise: From
Theoretical Inquiry to Practical Cases. Policy Sciences 26:16587.
Fischer, M. (2014). Coalition Structures and Policy Change in a Consensus Democracy. Policy
Studies Journal, 42(3), 344-366.
Font, J., Pasadas del Amo, S., & Smith, G. (2016). Tracing the Impact of Proposals from
Participatory Processes: Methodological Challenges and Substantive Lessons. Journal of
Public Deliberation, 12(1), Article 3.
Font, J., Smith, G., Galais, C., & Alarcon, P. (2018). Cherry-picking participation: Explaining the fate
of proposals from participatory processes. European Journal of Political Research, 57(3),
Edward Challies et. al
Fung, A. (2006). Varieties of Participation in Complex Governance. Public Administration Review
66(Special Issue): 6675.
Hoppe, R. (2011). Institutional constraints and practical problems in deliberative and
participatory policy making. Policy & Politics, 39(2), 163-186.
Ingram, H. (2011). Beyond universal remedies for good water governance: A political and
contextual approach. In A. Garrido & H. Ingram (Eds.), Water for Food in a Changing World
(pp. 241-261). Oxon: Routledge.
Jager, N.W., Newig, J., Challies, E., & Kochskämper, E. (2020) Pathways to implementation:
Evidence on how participation in environmental governance leads to good environmental
outcomes, Journal of Public Administration Research and Theory 30(3), 383-399.
Klijn, E.-H., Edelenbos, J., & Steijn, B. (2010). Trust in Governance Networks: Its Impacts on
Outcomes. Administration & Society, 42(2), 193-221.
Klijn, E.-H., Van Buuren, A., & Edelenbos, J. (2012). The Impact of Governance: A normative and
empirical discussion. In D. Levi-Faur (Ed.), The Oxford Handbook of Governance (pp. 294-
308). Oxford: Oxford University Press.
Kochskämper, E., Jager, N.W., Newig, J., & Challies, E. (2018). Impact of participation on
sustainable water management planning: Comparative analysis of eight cases. In E.
Kochskämper, E. Challies, N. W. Jager, & J. Newig (Eds.), Participation for Effective
Environmental Governance: Evidence from European Water Framework Directive
Implementation (pp. 117-148). London: Routledge.
Kochskämper, E., Newig, J., Challies, E., & Jager, N.W. (2016). Participation for Effective
Environmental Governance? A Comparative Study of European Water Policy
Implementation in Germany, Spain and the United Kingdom. Journal of Environmental
Management 181:737748.
Koontz, T.M. (2006). Collaboration for sustainability? A framework for analyzing government
impacts in collaborative-environmental management. Sustainability: Science, Practice and
Policy, 2(1), 15-24.
Koontz, T.M. & Thomas, C.W. (2006). What Do We Know and Need to Know about the
Environmental Outcomes of Collaborative Management? Public Administration Review
Koontz, T.M., & Moore Johnson, E. (2004). One size does not fit all: Matching breadth of
stakeholder participation to watershed group accomplishments. Policy Sciences, 37(2),
From Collaboration to Policymaking
Larsson, R. (1993). Case Survey Methodology: Quantitative Analysis of Patterns Across Case
Studies. Academy of Management Journal 36(6):151546.
Leach, W.D. & Pelkey, N.W. (2001). Making Watershed Partnerships Work: A Review of the
Empirical Literature. Journal of Water Resources Planning and Management
(November/December): 37885.
Newig, J., Adzersen, A., Challies, E., Fritsch, O. & Jager, N. (2013). Comparative Analysis of Public
Environmental Decision-Making Processes − a Variable-Based Analytical Scheme. Lüneburg.
Newig, J., Challies, E., Jager, N.W., Kochskämper, E., & Adzersen, A. (2018). The Environmental
Performance of Participatory and Collaborative Governance: A Framework of Causal
Mechanisms. Policy Studies Journal, 46(2), 269-297.
Newig, J., & Fritsch, O. (2009a). Environmental governance: participatory, multi-level and
effective? Environmental Policy and Governance, 19(3), 197-214.
Newig, J. & Fritsch, O. (2009b). The Case Survey Method and Applications in Political Science.
Retrieved 27 August 2019 (
Newig, J. & Kvarda, E. (2012). Participation in Environmental Governance: Legitimate and
Effective? In K. Hogl, E. Kvarda, R. Nordbeck, & M. Pregernig (eds) Environmental
Governance. The Challenge of Legitimacy and Effectiveness, pp. 29-45.Cheltenham: Edward
Newig, J., Schulz, D. & Jager, N.W. (2016). Disentangling Puzzles of Spatial Scales and
Participation in Environmental GovernanceThe Case of Governance Re-Scaling Through
the European Water Framework Directive. Environmental Management 58(6): 9981014.
Newman, J., Barnes, M., Sullivan, H., & Knops, A. (2004). Public Participation and Collaborative
Governance. Journal of Social Policy 33(2): 20323.
Oh, Y. & Bush, C.B. (2014). Exploring the Role of Dynamic Social Capital in Collaborative
Governance. Administration & Society 48(2): 21636.
Ostrom, E. (1990). Governing the Commons. The Evolution of Institutions for Collective Action,
Political Economy of Institutions and Decisions. Cambridge: Cambridge University Press.
Schively, C. (2007). Understanding the NIMBY and LULU Phenomena: Reassessing Our
Knowledge Base and Informing Future Research. Journal of Planning Literature 21(3): 255-
Ulibarri, N. (2015). Tracing Process to Performance of Collaborative Governance: A Comparative
Case Study of Federal Hydropower Licensing. Policy Studies Journal 43(2): 283308.
... 10. The focus of this article is on implementation and not decision making by the local authority (see Newig et al. 2017) because 'decision' proved to be an ambiguous category in the Spanish local context (e.g., what counts as a definite decision?) and it was thus difficult to garner reliable information. ...
... 23. Newig et al (2017) also show that most contextual variables do not have an effect. Only one variable specific to environmental policies (Nimby situation) is significant in their analysis. ...
Full-text available
What happens to the proposals generated by participatory processes? One of the key aspects of participatory processes that has been the subject of rare systematic analysis and comparison is the fate of their outputs: their policy proposals. Which specific factors explain whether these proposals are accepted, rejected or transformed by public authorities? In this article contextual and proposal-related factors are identified that are likely to affect the prospect of proposals being implemented. The explanatory power of these factors are tested through multilevel analysis on a diverse set of 571 policy proposals. The findings offer evidence that both contextual and proposal-related variables are important. The design of participatory processes affects the degree of implementation, with participatory budgeting and higher quality processes being particularly effective. Most significant for explaining outcomes are proposal-level, economic and political factors: a proposal's cost, the extent to which it challenges existing policy and the degree of support it has within the municipality all strongly affect the chance of implementation.
Full-text available
There is much enthusiasm among scholars and public administrators for participatory and collaborative modes of governance as a means to tackle contemporary environmental problems. Participatory and collaborative approaches are expected to both enhance the environmental standard of the outputs of decision-making processes and improve the implementation of these outputs. In this article, we draw on a database of 307 coded published cases of public environmental decision-making to identify key pathways via which participation fosters effective environmental governance. We develop a conceptual model of the hypothesized relationship between participation, environmental outputs, and implementation, mediated by intermediate (social) outcomes such as social learning or trust building. Testing these assumptions through structural equation modeling and exploratory factor analysis, we find a generally positive effect of participation on the environmental standard of governance outputs, in particular where communication intensity is high and where participants are delegated decision-making power. Moreover, we identify two latent variables – convergence of stakeholder perspectives and stakeholder capacity building-to mediate this relationship. Our findings point to a need for treating complex and multifaceted phenomena such as participation in a nuanced manner, and to pay attention to how particular mechanisms work to foster a range of social outcomes and to secure more environmentally effective outputs and their implementation.
Full-text available
What happens to the proposals generated by participatory processes? One of the key aspects of participatory processes that has been the subject of rare systematic analysis and comparison is the fate of their outputs: their policy proposals. Which specific factors explain whether these proposals are accepted, rejected or transformed by public authorities? In this article contextual and proposal-related factors are identified that are likely to affect the prospect of proposals being implemented. The explanatory power of these factors are tested through multilevel analysis on a diverse set of 571 policy proposals. The findings offer evidence that both contextual and proposal-related variables are important. The design of participatory processes affects the degree of implementation, with participatory budgeting and higher quality processes being particularly effective. Most significant for explaining outcomes are proposal-level, economic and political factors: a proposal's cost, the extent to which it challenges existing policy and the degree of support it has within the municipality all strongly affect the chance of implementation.
Full-text available
Many have advocated for collaborative governance and the participation of citizens and stakeholders on the basis that it can improve the environmental outcomes of public decision making, as compared to traditional, top-down decision making. Others, however, point to the potential negative effects of participation and collaboration on environmental outcomes. This article draws on several literatures to identify five clusters of causal mechanisms describing the relationship between participation and environmental outcomes. We distinguish (i) mechanisms that describe how participation impacts on the environmental standard of outputs, from (ii) mechanisms relating to the implementation of outputs. Three mechanism clusters focus on the role of representation of environmental concerns, participants' environmental knowledge, and dialogical interaction in decision making. Two further clusters elaborate on the role of acceptance, conflict resolution, and collaborative networks for the implementation of decisions. In addition to the mechanisms, linking independent with dependent variables, we identify the conditions under which participation may lead to better (or worse) environmental outcomes. This helps to resolve apparent contradictions in the literature. We conclude by outlining avenues for research that builds on this framework for analysis.
Full-text available
This article attempts to shed new light on prevailing puzzles of spatial scales in multi-level, participatory governance as regards the democratic legitimacy and environmental effectiveness of governance systems. We focus on the governance re-scaling by the European Water Framework Directive, which introduced new governance scales (mandated river basin management) and demands consultation of citizens and encourages ‘active involvement’ of stakeholders. This allows to examine whether and how re-scaling through deliberate governance interventions impacts on democratic legitimacy and effective environmental policy delivery. To guide the enquiry, this article organizes existing—partly contradictory—claims on the relation of scale, democratic legitimacy, and environmental effectiveness into three clusters of mechanisms, integrating insights from multi-level governance, social-ecological systems, and public participation. We empirically examine Water Framework Directive implementation in a comparative case study of multi-level systems in the light of the suggested mechanisms. We compare two planning areas in Germany: North Rhine Westphalia and Lower Saxony. Findings suggest that the Water Framework Directive did have some impact on institutionalizing hydrological scales and participation. Local participation appears generally both more effective and legitimate than on higher levels, pointing to the need for yet more tailored multi-level governance approaches, depending on whether environmental knowledge or advocacy is sought. We find mixed results regarding the potential of participation to bridge spatial ‘misfits’ between ecological and administrative scales of governance, depending on the historical institutionalization of governance on ecological scales. Polycentricity, finally, appeared somewhat favorable in effectiveness terms with some distinct differences regarding polycentricity in planning vs. polycentricity in implementation.
Full-text available
This article evaluates the impact of governance in decision making, policy making, implementation, and service delivery in political sciences and public administration. It describes how governance can make an impact through management action and suggests that the impacts or results of governance can be found in the areas of process, content, and legitimacy. The article argues that network-management activities are crucial for governance to have an impact and explains that, although governance networks are to a degree self-regulatory, they sometimes need a push in the right direction by management activities.
The role of the public in US policy making has shifted substantially during the past several decades. This shift is particularly evident in environmental policy, where collaboration among multiple stakeholders is on the rise. Much of the literature on collaborative environmental management emphasizes the need for widespread community involvement, especially from private citizens. Many proponents of collaboration have argued that broad inclusion can lead to better environmental solutions while also establishing legitimacy, building social capital, and overcoming conflicts. Yet such broad inclusion may be costly in terms of time, energy, and resources, and it may not yield the desired results. Thus, a key question is how the breadth of public involvement is linked to collaborative group accomplishments. This study, using watershed groups in Ohio, demonstrates several links between group membership and results. Groups with a broader array of participants tend to excel in watershed plan creation, identifying/prioritizing issues, and group development and maintenance. In addition, groups comprised of a relatively balanced mix of governmental and non-governmental participants are more likely to list planning/research and group development and maintenance results than are groups comprised primarily of non-governmental participants. In contrast, groups with a narrower membership and groups that are composed primarily of non-governmental participants may focus more on pressuring government for policy change.
Effectiveness of participation in environmental governance is a proliferating assertion in literature that is also reflected in European legislation, such as the European Water Framework Directive (WFD). The Directive mandates participatory river basin management planning across the EU aiming at the delivery of better policy outputs and enhanced implementation. Yet, the impact of this planning mode in WFD implementation remains unclear, though the first planning phase was completed in 2009 and the first implementation cycle by the end of 2015. Notwithstanding the expanding body of literature on WFD implementation, a rather scattered single case study approach seems to predominate. This paper reports on implementation of the WFD in three case studies from Germany, Spain and the United Kingdom, reflecting three substantially different approaches to participatory river basin management planning, on the basis of a comparative case study design. We ask if and how participation improved the environmental standard of outputs and the quality of implementation. We found an increasing quality of outputs with increasing intensity of local participation. Further, social outcomes such as learning occurred within dialogical settings, whereas empowerment and network building emerged also in the case characterized mainly by one-way information. Finally, one important finding deviant from the literature is that stakeholder acceptance seems to be more related to processes than to outputs.
Our understanding of participatory processes is increasing rapidly. However, one area that has received sparse attention is the impact of the proposals from participatory processes on the policy and practice of public administrations. Which proposals are converted into actual policy and practice; which are modified or simply ignored? The field lacks a systematic understanding of the fate of proposals. This paper reflects on the methodological strategy adopted by the Cherry-picking project to analyze the fate of proposals from participatory processes in Spanish municipalities. The innovative project studied the impact of 611 proposals from 39 participatory processes across 25 municipalities. The paper not only describes and discusses the methodological challenges faced by the project, but also presents preliminary findings and a review of the substantive lessons learned through the design and fieldwork process.
Whether the goal is building a local park or developing disaster response models, collaborative governance is changing the way public agencies at the local, regional, and national levels are working with each other and with key partners in the nonprofit and private sectors. While the academic literature has spawned numerous case studies and context- or policy-specific models for collaboration, the growth of these innovative collaborative governance systems has outpaced the scholarship needed to define it. Collaborative Governance Regimes breaks new conceptual and practical ground by presenting an integrative framework for working across boundaries to solve shared problems, a typology for understanding variations among collaborative governance regimes, and an approach for assessing both process and productivity performance. This book draws on diverse literatures and uses rich case illustrations to inform scholars and practitioners about collaborative governance regimes and to provide guidance for designing, managing, and studying such endeavors in the future.