Content uploaded by Adam Edwards
Author content
All content in this area was uploaded by Adam Edwards on Jul 26, 2016
Content may be subject to copyright.
http://euc.sagepub.com/
Criminology
European Journal of
http://euc.sagepub.com/content/10/3/260
The online version of this article can be found at:
DOI: 10.1177/1477370813483386
2013 10: 260European Journal of Criminology
Adam Edwards, Gordon Hughes and Nicholas Lord
Urban security in Europe: Translating a concept in public criminology
Published by:
http://www.sagepublications.com
On behalf of:
European Society of Criminology
can be found at:European Journal of CriminologyAdditional services and information for
http://euc.sagepub.com/cgi/alertsEmail Alerts:
http://euc.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://euc.sagepub.com/content/10/3/260.refs.htmlCitations:
What is This?
- Apr 23, 2013Version of Record >> at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
European Journal of Criminology
10(3) 260 –283
© The Author(s) 2013
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1477370813483386
euc.sagepub.com
Urban security in Europe:
Translating a concept in public
criminology
Adam Edwards, Gordon Hughes and
Nicholas Lord
Cardiff University, UK
Abstract
A key challenge for public criminology is the translation between concepts employed in
policy discourse and those used by social scientists. Given that concepts constitute social
problems and they can have multiple meanings for policy-makers and social scientists, then
deliberation about what they signify matters in understanding how these actors can talk
to, rather than past, one another in framing policy discourse about crime and revealing
alternative policy agendas. This challenge is accentuated in the comparative context of
European criminology, which is characterized by competing tendencies to generalize about
problems of ‘Freedom, Security and Justice’ and to recognize the variegated problems and
cultures of control across Europe. In this context, the presumption of universality can
mistranslate concepts of crime and control by obscuring contextual insight, while the
presumption of particularity can inhibit cross-cultural dialogue and deliberation. The paper
explores this challenge in relation to the concept of ‘urban security’, which is prevalent in
the policy discourse on social crime prevention, particularly in Central and Southern Europe.
To establish the provenance, prevalence and significance of this concept, the paper discusses
findings from a policy Delphi that structured deliberation about the meaning of urban security
among criminologists sampled from the European Society of Criminology and policy-makers
sampled from the European Crime Prevention Network. It concludes with reflections on the
value of deliberative methods, such as the policy Delphi, for the cross-cultural validation of
criminological constructs in comparative research.
Keywords
Comparative research, policy Delphi, politics of translation, public criminology, urban
security
Corresponding author:
Adam Edwards, Cardiff School of Social Sciences, Cardiff University, Cardiff CF10 3WT, UK.
Email: EdwardsA2@cardiff.ac.uk
483386EUC10310.1177/1477370813483386European Journal of CriminologyEdwards et al.
2013
Article
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 261
Introduction
A significant current of thought in contemporary social science is the call for a ‘public
sociology’ (Burawoy, 2005) in which social scientists seek to intervene in public life,
using their particular knowledge and skills to inform and influence public discourse
about social problems. In some countries this mission is also being driven by academic
appraisal and performance management strategies requiring social scientists to demon-
strate the ‘public engagement and impact’ of their work. However, although criminology
has always had a very strong applied tradition, geared to an intense research relationship
with government, commercial and not-for-profit users of social science, it is argued that
‘public criminology’ is confronted with exceptional problems given the party political
and mass media interest in issues of crime control (Loader and Sparks, 2010). In addition
to arguments over the inherently political qualities of criminological research (Hillyard
et al., 2004), electoral and media interest generates a ‘hot’ policy environment that inhib-
its the communicative rationality of social science; insofar as they adopt a scientific
vocation, criminologists’ communication of their research findings is filtered, if not dis-
torted, by the formative intentions of public policy-makers to advance manifestos, culti-
vate electoral support, deliver policy successes within electoral cycles, manage pressure
group and mass media criticism, and so on.1 In response to this, there is an increasing
interest in deliberative methods that can structure dialogue between criminologists and
policy-makers around experience of and expertise about what is, or could be, known
about particular problems rather than around a priori political commitments (Edwards
and Sheptycki, 2009). Deliberative methods offer the prospect of dialogue that is insu-
lated from the more immediate, ad hominem, pressures on the policy process and conse-
quently a more defensible ‘construct validation’ of criminological problems.
Such deliberation presumes, however, the translation of these problems between differ-
ent kinds of social actors in ways that facilitate rather than debilitate dialogue. Opportunities
for mistranslation abound given that the conceptual language of public policy and social
science constitutes social problems in ways that signify different things to different actors,
who can consequently (and often deliberately) talk at, or past, rather than with one another.
More instrumentally, floating signifiers are particularly prevalent in public policy discourse
as competing interests seek to interest, enrol and mobilize support for their version of ‘the
problem’, so defined (see Callon, 1986). An exemplar of this is the policy discourse around
the problem of ‘transnational organized crime’, which has been criticized as a political,
arbitrary construct of post-Cold War foreign policy that obviates much of the social scien-
tific understanding of how serious crimes are organized. Yet the official investment in this
concept is such that the criminological community has been obliged to engage with it in
order to maintain the interest of the policy community (Edwards and Gill, 2002; Edwards
and Levi, 2008). As such, communication between social scientists and policy-makers (as
one significant ‘public’ for social science) is laden with competing interests and dilemmas
that need to be negotiated if dialogue is to continue. For, insofar as social scientists concede
too much to the terms of debate as set by policy-makers, they inhibit their ability to influ-
ence and inform public discourse – to be public criminologists. To not engage with policy-
makers’ terms and concerns, however, runs the risk of disinteresting them and, again,
inhibiting the capacity to influence and inform.
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
262 European Journal of Criminology 10(3)
To this end, the very terms of debate between academic and policy communities need
to be established as a necessary precursor to constructive dialogue. This is especially the
case in the cross-cultural comparative context of European criminology, which is charac-
terized by competing tendencies to generalize about problems of ‘Freedom, Security and
Justice’ and to recognize the variegated problems and cultures of control across Europe
(Smith, 2004). In this context, the presumption of universality can mistranslate concepts
of crime and control by obscuring contextual insight and ignoring the multiple significa-
tions that general concepts of crime and control can have (Edwards and Hughes, 2005,
2012). In cross-cultural research, mistranslation may also occur as a consequence of
treating concepts as idiographic (originating in a particular social context and only per-
taining to that context) when they may share common referents with alternative concepts
developed in other social contexts. If there are no common referents, there can be no
argument (Bhaskar, 1979). So, in precluding the identification of common referents, idi-
ographic accounts can inhibit cross-cultural dialogue and learning (Karstedt, 2012).
The problems of translation entailed in public criminology and encountered specifi-
cally in the European context are explored in this paper through reference to ‘urban
security’, a concept that has attracted a considerable following, particularly in Central
and Southern Europe and primarily among policy-makers, but which has limited recog-
nition in other European regions and in the social science community. Having estab-
lished the provenance and prevalence of this concept, this paper discusses findings from
a particular kind of deliberative method, a ‘policy Delphi’, which asked a panel of aca-
demics, sampled from the European Society of Criminology (ESC) and a panel of
national policy-makers, sampled from the European Crime Prevention Network
(EUCPN), about the significance of this concept and thus its utility in organizing cross-
national deliberation about crime and insecurity.
The provenance and prevalence of ‘urban security’ in
Europe
The concept of urban security is most obviously associated with the European Forum for
Urban Security (EFUS), established in Barcelona in 1987 on the initiative of Gilbert
Bonnemaison, former French socialist mayor of Epinay-sur-Seine.2 As argued in its two
manifestos, the Naples ‘Cities’ Manifesto’, agreed in 2000 (EFUS, 2000), and the more
recent Aubervilliers and Saint Denis Manifesto on ‘Security, Democracy and Cities’,
agreed in December 2012 (EFUS, 2012), the Forum exists to promote a particular under-
standing of crime and its prevention, one associated with principles of social justice,
interventions driven by social and economic policy as much as by criminal justice and
risk management, and foregrounding the role of municipal authorities as much as police
and criminal justice agencies. As such, the concept is a product of an explicitly political
standpoint within public policy discourse on crime prevention in Europe, one that has
garnered support particularly from Central and Southern Europe.3 The Aubervilliers and
Saint Denis Manifesto reaffirms this commitment to ‘social prevention policies’, particu-
larly ‘[a]t a time when Europe and the world are going through an economic crisis that
may jeopardize the social and cultural heritage of the twentieth century’ and given
that ‘Europe is experiencing imbalances and disparities, in particular an outburst of
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 263
unemployment, which has plunged European citizens into a state of anxiety, weakening
the social fabric and trust in the future . . . In each of its localities, the crisis threatens
social cohesion and solidarity, making selfishness and individualism emerge’ (EFUS,
2012: 1–2).
The preoccupation specifically with urban security is further justified by EFUS on the
grounds that ‘European and national institutions now recognise cities as essential part-
ners. Being the closest to the citizens, they combine competencies in solidarity, preven-
tion and sanction with expertise in the management of everyday problems’ (2012: 4).
This modern idea of cities as the engines of civilization and progressive action as well as
the fulcrum of social problems is reinforced by the concluding claims of the Aubervilliers
and Saint Denis Manifesto that ‘[c]ities advocate a Europe that is open to the world,
respecting regulations and laws, and taking full advantage of the diversity of their popu-
lations’ and ‘express their desire to make security a public good, based on the respect of
fundamental rights’ (2012: 4). However, in an interesting twist to an otherwise overtly
political statement, the Manifesto calls for policies to be premised on the ‘latest technical
and scientific knowledge’ and for cities ‘to find new ways to ensure that their policies are
defined and guided by both qualitative and quantitative data, and not on prejudice or
ideological stances’, with a commitment to ‘systematically assessing their prevention
actions, in order to increase efficiency and therefore bring prevention to a new stage of
professionalization’ (2012: 3).
Then again, the Manifesto qualifies this professionalization by identifying a need to
address the alienation of citizens from elite policy actors, in particular from the institu-
tions of the European Union, because ‘Europe does not elicit a strong sense of belong-
ing from its citizens’ (2012: 2). It is argued that ‘[s]ecurity policies should be designed
and constructed around the individual and collective needs of citizens, and not accord-
ing to public institutions . . . participation is a cross-cutting principle of action, ena-
bling civil society to be involved in all stages of design, implementation and evaluation’
(2012: 2).
In summary, EFUS has used the concept of urban security to reframe problems of
crime and violence as problems of social justice, not just criminal justice, which are
concentrated in cities but often have their origins elsewhere, for example in forms of
financial and organized crime, but which can and ought to be prevented through social
and economic policy interventions by partnerships of municipal authorities that are
driven both by scientific insight and by popular democratic will. As such, the concept
of urban security has come to express a number of key tensions, if not contradictions,
in European thinking about crime and violence – between the importance of preven-
tion and sanctioning as policy priorities established by ‘active citizens’ as well as
scientific and political elites. Consequently, and like many concepts coined in policy
discourse, urban security acts as a floating signifier with a multiplicity of referents,
reflecting the ability of policy advocates to capture the concept for their particular
interests.4 More generically, it can be argued that this politics of translation structures
dialogue between policy actors and is therefore a concern for social scientists inter-
ested in the public engagement and impact of their work; what does the adoption of a
concept signal, what commitments can it imply and what thinking and action can it
obviate?
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
264 European Journal of Criminology 10(3)
A possible response to deliberately flexible, easily manipulated, concepts is to reject
them, precisely for their imprecision. Why not simply frame policy discourse around
concepts that are more familiar and more prevalent in the social scientific and policy
communities? An exemplar of this argument is the reassertion of ‘crime prevention’ as a
sufficiently precise concept for structuring dialogue between these communities (Pease,
1994). This is certainly the case in policy discourse at the level of the European Union
and the current Stockholm Programme for creating an Area of Freedom, Security and
Justice. The Stockholm Programme entails a commitment for the EUCPN to establish an
Observatory for Crime Prevention, providing another focus for the professionalization of
this policy area in Europe (European Commission, 2010: 4.3.2). In this policy network,
national crime prevention strategies published by the EUCPN reveal only limited use of
the concept of urban security,5 and this concept is also much less prevalent in the
European-based English language criminology journals, including the journal of the
ESC, the European Journal of Criminology.6
If crime prevention provides a more familiar and precise signifier for policy dis-
course in Europe, it is liable, however, to alienate a significant public for public crimi-
nology, in particular those defining their work in terms of ‘urban security’ (see the
contributions by Frevel, Tominc et al., Racasens et al., Virta, and Xenakis and Cheliotis
in this special issue) or analogous concepts of ‘community safety’ (Gilling et al., this
issue) and ‘integral security’ (Devroe, this issue), as a means of locating crime preven-
tion as a problem of social and economic policy as well as of criminal justice and risk
management. There is a politics of translation underpinning the coining and use of
these competing concepts that is further clarified through reference to the manifestos
of those who would take policy in an opposite direction, advocating ‘crime science’
deliberately as a means of obviating the role of social and economic policy in the pre-
vention of crime, delimiting it instead to the pursuit of pragmatic, situational, interven-
tions (Clarke, 2004).
In establishing what these concepts do in privileging certain problems and policy
responses while subordinating or obviating others, there is nonetheless a danger of
depending on documentary sources alone, particularly the tendentious manifestos and
mission statements of policy discourse. Textual analysis can promote a false universal-
ity, where it is presumed that concepts such as ‘crime prevention’ signify the same
things in different social groups and in different cultures of control. Conversely, this
method can also promote a false particularity insofar as actual variegated cultures of
control, such as the predilection of some Central and Southern European policy actors
for ‘urban security’, are imputed from textual sources (see Stenson, 1998). In both
these instances of mistranslation, the opportunity for debate, dialogue and learning
across diverse social contexts can be lost and otherwise delimited. To better validate
criminological constructs and support the kind of dialogue implied by public criminol-
ogy, more deliberative methods are needed that can establish whether there are com-
mon referents among different publics or else to clarify how dialogue is lost in
translation. The remainder of the paper considers the contribution that one particular
kind of deliberative method, the ‘policy Delphi’ (Turoff, 1970; Ziglio, 1996), can make
to public criminology in Europe and how this method was used to clarify the signifi-
cance of urban security.
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 265
The significance of ‘urban security’ in Europe
The policy Delphi method has its origins in attempts to identify areas of consensus and
disagreement in forecasting public policy problems and possible responses where other
methods, such as experiments, were deemed impossible or inappropriate.7 It is argued,
however, that this method is also of relevance for arguments over the most appropri-
ate methods for comparative research in cross-cultural contexts such as European
criminology.8 The essence of the method is that key informants with established experi-
ence of and expertise about the policy problem in question are recruited onto panels;
dialogue between panellists is structured by questionnaires that individual panellists
complete on their own and return to the coordinators of the panel. Coordinators summa-
rize and report the responses of individual respondents back to all members of the panel
and issue a further questionnaire (Q2) inviting individual panellists to concur or disagree
both with arguments arising out of the initial questionnaire (Q1) and with the interpreta-
tion placed on these in the coordinator’s reports. Through further iterations of question-
naire–report–questionnaire (Q2 . . . Qn), the subjective accounts of panellists are
transformed into the objective opinion of the panel, whether this objective opinion
reveals a high consensus of agreement or disagreement. This iterative method of delib-
eration across various questionnaires or ‘rounds’ facilities both respondent and construct
validation of the problems in question (Ziglio, 1996). As such, the method is particularly
apposite for establishing common referents in arguments across different cultural con-
texts that might otherwise be lost in translation and through other research methods that
do not admit collective respondent and construct validation of criminological problems.
Design of the ‘Urbis policy Delphi’
To further investigate the significance of urban security among social scientists and pol-
icy-makers, three panels of key informants were recruited to a policy Delphi entitled ‘the
Urbis policy Delphi’ after the broader project funded by the European Union (EU) of
which it is a part.9 One panel was sampled from academic criminologists in the ESC and
two panels of respondents were sampled from the European-wide policy networks dis-
cussed above, the EFUS and the EUCPN. At the time of writing, the ESC and EUCPN
panels had both completed, providing opportunities for a full comparison of the signifi-
cance of urban security for social scientists and for policy-makers on these panels, and it
is the results from these two panels that are considered here.
This Urbis policy Delphi consisted of three rounds of deliberation (Q1, Q2 and Q3),
in which respondents were able to anonymously validate key issues and constructs from
previous rounds. The ESC panel consisted of 15 experts from 15 different European
countries, who were sampled from the European Society of Criminology on the basis of
their reputation for conducting research, variously on crime prevention policies in North,
South, West and East European countries. There was no attrition to this panel as all 15
panellists completed all three rounds of the Delphi process. The EUCPN panel had some
attrition over the three rounds, with 15 responses in round one, 13 responses in round two
and 9 responses in round three. Members of this panel were elicited from those who
responded to a census sample of EU member state representatives on the EUCPN. The
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
266 European Journal of Criminology 10(3)
breakdown of responses by country and by round can be seen in Table 1. A mixture of
responses from countries across Europe was evident in both panels. Respondents dis-
cussing seven countries (Belgium, the Republic of Ireland, Germany, Italy, the
Netherlands, Slovenia and Spain) featured in both panels. Each round of the Delphi was
opened up to the full EUCPN membership (27 countries). For this reason, some respond-
ents did not participate in the first round but did in subsequent rounds. Although the
sample on both panels included respondents from all European regions and provides
grounds for ‘moderatum generalization’ about usage of the concept of urban security,
Table 1. Urbis policy Delphi respondents by country, panel and questionnaire round
completed.
Country ESC panel EUCPN panel
Q1 Q2 Q3 Q1 Q2 Q3
Austria Yes
Belgium Yes Yes Yes Yes Yes Yes
Bulgaria Yes Yes Yes
Cyprus Yes Yes Yes
Czech Republic Yes Yes Yes
Denmark Yes Yes Yes
Ireland Yes Yes Yes Yes
Estonia Yes
England & Wales (UK) Yes Yes Yes
Finland Yes Yes Yes
France Yes Yes Yes
Germany Yes Yes Yes Yes Yes Yes
Greece Yes Yes Yes
Hungary Yes
Italy Yes Yes Yes Yes Yes Yes
Latvia Yes
Lithuania Yes
Luxembourg Yes Yes
Netherlands Yes
(with Belgium)
Yes
(with Belgium)
Yes
(with Belgium)
Yes Yes
Malta Yes
Norway Yes Yes Yes
Poland
Portugal Yes Yes Yes
Romania Yes Yes Yes
Scotland Yes Yes Yes
Slovenia Yes Yes Yes Yes
Spain Yes Yes Yes Yes
Sweden Yes Yes Yes
Turkey Yes Yes Yes
TOTAL 15 15 15 15 13 9
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 267
they are not intended to be representative samples of all social scientists with an interest
in the prevention of crime and insecurity in Europe or of all official conceptions of this
problem among EU member states or strategic partners of the EU. Rather, the sampling
was purposive in recruiting respondents with reputed expertise about crime prevention
but from different kinds of social group, in this instance social scientists and national
government policy-makers, in order to establish any common referents for arguments
about the prevention of crime and violence across Europe and the particular significance
of urban security within these arguments.
A particular advantage of the Delphi method, however, is that it enables collective
respondent validation of concepts through the iterative rounds of questionnaire–report–
questionnaire. In this way it demonstrates whether it is possible for a group of expert
informants to reach a consensus about the significance of concepts and the related con-
struction of social problems. Insofar as any consensus is identified, it is plausible to
argue that common referents exist and that, for our particular purposes here, these pro-
vide a means of translating concepts across cultures of control. This iterative respondent
and construct validation has particular benefits for the translation of concepts across
different literal as well as conceptual language communities. Within the same literal
language community, for example English in the British context, there is still a need for
conceptual translation between the concepts that different subjects, for example policy-
makers, practitioners and social scientists, may have about the ‘same’ social problem. As
such, social science is confronted with the ‘double hermeneutic’ of researchers placing
an interpretation on the interpretations of their respondents (Sayer, 2000: 17). Hence the
importance and challenge of public criminology in facilitating dialogue between these
multiple interpretations. In cross-national comparative contexts such as European crimi-
nology, however, a further layer of interpretative complexity exists because translation
occurs across literal, as well as conceptual, language communities.10
Key findings of the Urbis policy Delphi
The questionnaires put to members of each of these panels covered a breadth of issues
arising out of the kinds of problems and preventive approaches signified by the concept
of urban security, the multiplicity of actors given responsibility for prevention, and the
different kinds of expertise these actors possess or ought to possess in undertaking this
responsibility.
The first round (Q1) entailed the use of a qualitative, semi-structured, questionnaire
that asked panellists the following questions about the regions they had knowledge of:
1. What can ‘managing urban security’ mean?
2. What are the current challenges for managing urban security in your region?
3. What are the potential challenges for managing urban security in your region in
the coming decade?
4. Who is currently responsible for managing urban security?
5. Who ought to be responsible for managing urban security?
6. What expertise and training currently equips these authorities to respond to these
problems?
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
268 European Journal of Criminology 10(3)
7. What expertise ought to be entailed in this response?
8. How might this expertise be best developed in educational and training
programmes?
Given the particular focus of this paper on the problematization of urban security, the
discussion here is limited to questions 1–3, where panellists’ responses distinguished
between: (i) the problems of urban security; and (ii) different approaches to managing
these problems. Between them, the ESC and EUCPN experts identified a total of 25 dif-
ferent problems and 15 approaches, as summarized in Table 2.
Significantly, a number of respondents (from Northern Europe) observed that the con-
cept of urban security currently had limited recognition in the countries they were famil-
iar with, where concepts such as ‘community safety’ (the UK and the Republic of Ireland)
Table 2. Cross-panel identified problems and approaches in Q1.
Problems Approaches
1. Incivility and anti-social behaviour
2. Drug trafficking
3. Property crime (burglary, theft, robbery)
4. Criminal damage (vandalism, graffiti)
5. Fraud
6. Violence against the person (including
domestic violence)
7. Alcohol and drug misuse
8. Firearms-related crime
9. Environmental degradation
(e.g. illegal waste disposal, pollution)
10. Knife-related crime
11. Criminal gangs and organized crime
12. Human trafficking
13. Prostitution, illicit sexual services
14. Corporate crime, including corruption
15. Health and safety in the workplace
16. Corruption of public administration
17. State police violence
18. Terrorism
19. Tax evasion
20. Climate change and natural disasters
(flooding, extreme weather)
21. Protection of critical infrastructure
(water and food security, transport and
communications systems, energy grids)
22. Immigration and social cohesion
23. Mass demonstrations and civil unrest
associated with austerity
24. Social exclusion and youth unemployment
25. Degradation of governing capacity through
public expenditure
1. Enforcing the criminal law
2. Reducing social segregation and
promoting social cohesion
3. Repressing incivility
4. Increased use of imprisonment and
correctional facilities
5. Use of CCTV surveillance
6. Reassuring citizens about their security
and about their fear of crime
7. Reducing the opportunities for criminal
victimization
8. Reducing social inequalities in
household income, access to education,
employment, healthcare and housing
9. Preventing the onset of offending
behaviour and incivility
10. Punitive sentencing policies
11. Requiring citizens to take responsibility
for their own security and equipping
them with the capacity and resources to
meet this responsibility
12. Restorative justice interventions with
perpetrators and victims of criminal
offences
13. Celebrating social diversity and
promoting the rights of minority groups
14. Enhancing the democratic scrutiny and
oversight of security strategies
15. Promoting greater health and safety in
the workplace
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 269
and ‘integral security’ (Belgium and the Netherlands) were preferred signifiers of pre-
vention. Even so, there was a reticence among many panellists simply to use the concept
of ‘crime prevention’ because many were anxious to broaden the debate to include
underlying or ‘generative’ causes of crime and insecurity, in particular the social exclu-
sion and unemployment of young people, intergenerational tensions over resources and
other challenges to social cohesion, including the rapid and mass migration of popula-
tions into and around Europe. This suggested there were common referents behind these
different signifiers, the validation of which became a key focus of the second round of
the policy Delphi. Q1 produced narrative responses that were discursive and very rich in
detail, varying between 1000 and 7000 words in length. To structure dialogue between
panellists, we found it necessary, as coordinators, to place our own interpretation on the
key themes arising out of these accounts in order to derive the 25 problems and 15
approaches. Of course, this resulted in a reduction in the complexity and subtlety of the
original accounts, which could have been retained through the adoption of a more ethno-
graphic commentary. In defence of this method, however, and in contrast to cross-sec-
tional research designs,11 the iterative qualities of the Delphi method enabled panellists
to validate the interpretation we placed on their accounts. A more fundamental point is
that such reduction is unavoidable if the goal is to ascertain common referents and to
enable cross-cultural dialogue – to better capture, in this instance, the ‘Europeanness’ of
the problems and approaches signified by ‘urban security’.12
The second round (Q2) presented panellists with a structured questionnaire asking
respondents to consider the range of problems and approaches that had been identified
by the panels in Q1 and to rank them according to those they considered to be the five
highest and the five lowest priorities. Requiring panellists to rank policy priorities in this
way served the purpose of clarifying whether they agreed there was a core policy agenda
for more expansive concepts of crime prevention, irrespective of whether these were
coded as ‘urban security’, ‘community safety’ or ‘integral security’.
Both the ESC and EUCPN panels identified seven problems as ‘high priority prob-
lems’ with varying levels of consensus (see Table 3). High consensus was defined as
Table 3. Problems prioritized in Q2 of the Urbis policy Delphi by panel.
Q2 ESC panel EUCPN panel
High consensus
>75 – 100 percent
• Violence against the person
(including domestic violence)
Moderate consensus
>50 – <75 percent
• Social exclusion and youth
unemployment
• Incivilities and anti-social behaviour
• Property crime
• Violence against the person
(including domestic violence)
• Social exclusion and youth
unemployment
• Incivilities and anti-social
behaviour
• Property crime
Low consensus
>25 – <50 percent
• Alcohol and drug misuse
• Immigration and social cohesion
• Criminal gangs and organized crime
• Alcohol and drug misuse
• Immigration and social
cohesion
• Criminal gangs and organized
crime
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
270 European Journal of Criminology 10(3)
more than or equal to 75 percent of the panel identifying a problem as a top five priority,
moderate consensus as more than or equal to 50 percent but less than 75 percent of the
panel identifying a problem, and low consensus as more than or equal to 25 percent but
less than 50 percent of the panel. What is immediately striking about this ranking is the
degree of cross-panel consensus; both panels identified a core agenda of problems of
personal and property crime, incivilities and social exclusion, particularly of young peo-
ple from gainful employment, with a significant minority of panellists also identifying
problems of alcohol and drug misuse, immigration and social cohesion, and criminal
gangs and organized crime.
The ranking of approaches to problems of urban security in Q2 revealed some inter-
esting areas of inter-panel consensus and disagreement (see Table 4). There was a high
consensus across the two panels about the need to reduce social segregation and promote
social cohesion. Thereafter, however, the ESC panel placed a greater emphasis on reducing
social inequalities whereas the EUCPN panel privileged the reduction of opportunities for
Table 4. Approaches prioritized in Q2 of the Urbis policy Delphi by panel.
Q2 ESC panel EUCPN panel
High consensus
>75 – 100 percent
• Reducing social segregation and
promoting social cohesion
• Reducing social inequalities
in household income, access
to education, employment,
healthcare and housing
• Reducing social segregation and
promoting social cohesion
• Reducing the opportunities for
criminal victimization
Moderate consensus
>50 – <75 percent
• Reassuring citizens about their
security and about their fear of
crime
• Preventing the onset of
offending behaviour and
incivility
• Requiring citizens to take
responsibility for their own
security and equipping them
with the capacity and resources
to meet this responsibility
• Restorative justice
interventions with perpetrators
and victims of criminal offences
• Reducing social inequalities
in household income, access
to education, employment,
healthcare and housing
Low consensus
>25 – <50 percent
• Enhancing the democratic
scrutiny and oversight of
security strategies
• Reducing the opportunities for
criminal victimization
• Restorative justice
interventions with perpetrators
and victims of criminal offences
• Requiring citizens to take
responsibility for their own
security and equipping them
with the capacity and resources
to meet this responsibility
• Preventing the onset of
offending behaviour and
incivility
• Use of CCTV surveillance
• Reassuring citizens about their
security and about their fear of
crime
• Enforcing the criminal law
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 271
criminal victimization, reflecting a key argument in contemporary criminology about the
place of ‘dispositional’ and ‘situational’ approaches to crime prevention in public policy
(Clarke, 2004). Another key finding was the prioritization of CCTV surveillance and
criminal law enforcement by the EUCPN panel and the absence of these approaches in
the priorities identified by the ESC panel, reflecting the scepticism of much academic
criminology about the benefits of surveillance and enforcement. The EUCPN panel also
accorded a greater priority to the responsibility of citizens for their own security and to
the role of restorative interventions with perpetrators and victims of criminal offences.
To further explore and corroborate these alternative policy agendas, we used Q3 to con-
flate these approaches to prevention into four categories that are familiar to both academic
and policy-oriented criminology in Europe and that are often counterpoised to one another
in public discourse on crime and insecurity: criminal justice; restorative justice; social jus-
tice; and risk management. A problem with this discourse, however, is a tendency to over-
generalize the applicability of one or other of these approaches for preventing ‘crime’ per
se (for example, Clarke, 2004; EFUS, 2012) or to infer the eclipse of one approach (for
example, social justice) by others (for example, criminal justice and risk management) in
grand narratives about social control (for example, Garland, 2001; Wacquant, 2009).
Consequently, Q3 asked panellists to prioritize these approaches in relation to the specific
problems they had prioritized in Q2. This decision also reflects our interest in the call for a
more ‘problem-oriented’ approach to prevention in which deliberation proceeds from con-
crete problems to institutional responses rather than the more familiar reversal of this chain
of reasoning, which takes the institution (for example, the police, the prison, the courts) as
the starting point for appropriate policy responses (see Goldstein, 1990). In turn, we are
interested in how reorienting public discourse about crime and insecurity around problems
rather than institutions affects the dialogue between social science and other publics; spe-
cifically, whether it is possible to reach a consensus both about priority problems and about
the most appropriate approaches to their prevention.
Given these interests, Q3 asked panellists to identify how severe and frequent they
perceive the problems they had prioritized in Q2 to be. This follows recent work on the
concept of the ‘seriousness’ of crime that differentiates crime problems in terms of their
harmful effects, defined, in turn, in terms of their perceived severity and frequency
(Greenfield and Paoli, 2012). Panellists were asked to consider each of the problems they
had identified as current priorities, along with problems they had forecast as emerging
problems over the next decade,13 according to a seven-point Likert scale (very severe/
frequent; severe/frequent; moderately severe/frequent; uncertain; moderately innocuous/
infrequent; innocuous/infrequent; very innocuous/infrequent). Consensus among panel-
lists about the severity and frequency of these problems was ascertained through the use
of two scales: ‘mean panel response’ and ‘consensus of panel’.14 The key findings are
visualized in Figures 1 and 2, where the further out from the centre of the ‘radar diagram’
a marker is for a particular problem, the more severe and frequent the panel thought this
problem to be, and the larger the size of the marker, the greater the consensus among
panellists about the severity and frequency of a problem.
These metrics and this visualization reveal some interesting patterns of judgements
about the harms of priority problems for urban security, both within and between the
ESC and EUCPN panels. Most striking is the similarity in the overall pattern of
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
274 European Journal of Criminology 10(3)
judgements between both panels, signalling the possibilities for reaching agreement
between academic criminologists and policy-makers. Both panels corroborated violence
against the person, property crime and social exclusion and youth unemployment as
policy priorities. We suggest this pattern of inter-panel agreement after three iterative
rounds of deliberation provides strong evidence in support of the existence of common
referents among academics and policy-makers sampled from around Europe about the
problems for crime prevention and their connection to issues of social and economic
policy.
These visualizations were also used to compare and contrast inter-panel consensus
and disagreement about the appropriateness of criminal justice, risk management, restor-
ative justice and social justice policy approaches to these problems. Space prohibits a full
exposition of the revealing commonalities and divergences in the judgements of ESC
and EUCPN panellists in this regard and so discussion is restricted to some of the more
striking patterns of consensus and agreement and what, in conclusion, these tell us about
the possibilities of translating concepts of crime prevention in Europe.
Figures 3 and 4 visualize the pattern of judgements about criminal justice as an appro-
priate response to the problems prioritized by the ESC and EUCPN panels.15 Again, most
striking is the common pattern of judgements, with both panels identifying a strong
consensus in favour of criminal justice responses to problems of violence against the
person, property crime and organized crime but not for problems of social exclusion. A
clear divergence of opinion between the two panels, however, is the greater strength of
consensus among the EUCPN panellists for these responses than among ESC panellists,
arguably reflecting a greater scepticism among academic criminology about the efficacy
of criminal justice policy responses for reducing the harms associated with these kinds of
problems. Two further striking differences between the panels are over the use of crimi-
nal justice responses to incivilities or ‘anti-social behaviour’ (ASB) and alcohol and drug
misuse, with EUCPN panellists expressing a strong agreement in favour and the ESC
panel expressing uncertainty. Again, we take this pattern of judgements as evidence of
common referents around which a constructive dialogue between academic criminolo-
gists and policy-makers in Europe can be organized.
Figures 5 and 6 visualize the pattern of judgements about the appropriateness of risk
management responses to these problems, in particular measures to reduce the situational
opportunities for their commission. Here there is an interesting divergence in the pattern
between the two panels, with ESC panellists expressing a mostly high consensus of opin-
ion in favour of using this approach, particularly in relation to property crimes, but less
enthusiasm for this approach than their counterparts on the EUCPN panel, where there
was again a mostly high consensus but with higher levels of agreement (except for inci-
vilities/ASB, where agreement in the ESC panel was marginally stronger) about the
applicability of this approach to all problems including issues of immigration and social
cohesion.
The enthusiasm of the EUCPN panellists for alternatives to criminal justice responses
was also borne out by the strength of their agreement in favour of social justice responses
(those focusing on the reduction of social inequalities as a means of reducing crime and
insecurity) to all of the problems other than organized crime and the degradation of gov-
erning capacity. These judgements were also shared by ESC panellists, signalling a
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 279
strong consensus among academic criminologists and policy-makers about the irrele-
vance of social inequalities for understanding and acting against organized criminality. A
remarkable difference of judgement between the panels about the issue of organized
criminality, however, was the relevance of restorative justice responses, which ESC pan-
ellists were uncertain about but which EUCPN panellists were strongly in favour of.
Conclusion: Deliberation and the translation of urban
security
Of course, these patterns of judgement within and between the two panels invite more
forensic discussion of the appropriateness of policy approaches to different kinds of
problem than space permits here. Nonetheless, their illustration serves our purposes in
this paper in considering how the challenges of translating concepts of crime prevention
in Europe, through more expansive concepts such as urban security, might be addressed
with the use of deliberative methodologies such as the policy Delphi. This method ena-
bles the validation of constructs, within both academic and policy discourse, by different
criminological actors, such as social scientists and national policy-makers. In addition to
translating the meaning of these constructs between different social groups and across
different cultures of control, the method provides a means of organizing deliberation and
structuring dialogue. In this paper we have sought to demonstrate some aspects of the
dialogue about urban security in Europe, in particular the advantages of organizing delib-
eration around specific problems rather than particular institutions. This reorientation of
the public discourse about crime and insecurity around specific problems rather than
established forms of institutional ‘capture’ opens up a major challenge and opportunity
for rethinking policy approaches. For example, findings from Q3 of the Urbis policy
Delphi provoke further debate about the relative priority that ought to be accorded to
criminal justice, risk management and social justice approaches to organized crime and
what a restorative approach to this problem could entail. They also provoke further
debate over ‘anti-social behaviour’ as a problem primarily for criminal justice, risk man-
agement or social justice and, again, what contribution restorative policies could make to
this problem. Further research employing the Delphi method could focus further on the
particular problems that have been prioritized, seeking the informed opinion of other
kinds of social groups. Even so, this initial use of the Delphi method suggests it is pos-
sible to structure deliberation about criminological problems among key informants
across different cultures of control in time to inform and influence public policy debates.16
This particular method remains, to our knowledge, under-used across social science
and particularly within comparative criminology. There are good reasons for this because
the Delphi method is labour intensive and, when used across diverse cultural and linguis-
tic contexts, generates significant challenges of conceptual and literal translation. The
method also generates significant challenges for defining and sampling expert panellists
and reducing the attrition of panel members across the different rounds of deliberation.
Even so, we have illustrated the potential of this deliberative method for addressing these
challenges and developing a programme of empirical research capable of providing con-
struct validation of the consensus and conflict that prevails among key informants and
therefore how hegemonic are particular representations of criminological problems.
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
280 European Journal of Criminology 10(3)
In relation to the wider politics of translation constituting ‘Freedom, Security and
Justice’ in Europe, findings from the Urbis policy Delphi raise some important political
and normative questions for further deliberation. They reveal a high degree of cross-
panel consensus highlighting the persistence, rather than eclipse, of social policy inter-
ventions focused on the generative causes of crime and insecurity and to the role of cities
as engines of progressive action as well as the fulcrum of social problems. The pairing of
urban security and social justice is not, in this sense, an oxymoron. Rather, this politics
of translation is ‘an unfinished adventure’ (Bauman, 2004) with progressive potential
despite widely recognized tendencies toward punitive populism.
Funding
The paper draws on research funded by the European Union’s Leonardo da Vinci Lifelong Learning
Programme for ‘project Urbis’ (Reference: 518620-LLP-1-2011-1-IT-LEONARDO-LMP, see
http://www.urbisproject.eu/index.php/en/), which the authors gratefully acknowledge.
Notes
1. For an exemplary case study of this, see the reflections of academic participants in the
UK Home Office’s Crime Reduction Programme, 1999–2002 (2004, Volume 4, Issue 3, of
Criminology and Criminal Justice, London, Sage).
2. See http://efus.eu/en/about-us/about-efus/public/1450/ (accessed 7 March 2013).
3. Of the current 300 local authority members of EFUS, less than 5 percent were from outside
these regions (see http://efus.eu/en/our-network/, accessed 7 March 2013). EFUS currently has
seven national forums for urban security, in Belgium, France, Germany, Italy, Luxembourg,
Portugal and Spain.
4. For a discussion of this phenomenon in relation to the analogous concept of ‘community
safety’ in anglophone policy discourse on crime prevention, see Edwards and Hughes (2009).
5. Of the 17 national crime prevention strategies published on the EUCPN website, only 2 (Italy
and Hungary) made any reference to urban security (see http://www.eucpn.org/strategies/
index.asp, accessed 7 March 2013).
6. Six established English-language criminology journals based in Europe were searched,
at the time of writing this paper, for references to ‘urban security’ over the past 10 years
(or for as long as they had been in publication). This revealed limited usage of the concept:
European Journal of Criminology (3 citations); European Journal of Crime, Criminal Law
and Criminal Justice (4); Crime, Law and Social Change (2); British Journal of Criminology
(4); Criminology and Criminal Justice (2); and Crime Prevention and Community Safety (2).
7. The Delphi method had its origins in the US RAND Corporation’s contribution to US defence
strategy in the Cold War, given the absence of opportunities for gaining experimental cer-
tainty about the outcomes of a nuclear missile exchange (Gordon and Helmer, 1964). In the
less apocalyptic world of contemporary criminology, argument continues over the possibility
and desirability of experimental knowledge about ‘what works’ in the prevention of crime
(Sherman, 2008; Tilley, 2009; Hope, 2009; Young, 2011).
8. For a more developed account of this argument, see Edwards et al. (2013).
9. This research forms part of a broader project, ‘project Urbis’, funded by the European Union’s
Leonardo da Vinci Lifelong Learning Programme to recognize the ‘state of the art’ in manag-
ing urban security in Europe; to identify any need for the further professionalization of this
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 281
role, specifically through higher educational qualifications; to design a higher educational pro-
gramme of teaching and learning about managing urban security; and to pilot this programme
among current and prospective urban security managers (http://www.urbisproject.eu/index.
php/en/, accessed 7 March 2013). The Urbis policy Delphi forms part of Work Package 3 of
this project, on the ‘state of the art’ in thinking about urban security, the problems signified by
this concept, the policy responses it implies, who ought to be responsible for these responses
and what kinds of experience and expertise are needed to fulfil this responsibility. All of which
presumes, as discussed above, greater clarity about the provenance, prevalence and signifi-
cance of the concept.
10. Respondents on the ESC and EUCPN panels, who were sampled from elite transnational
policy networks for this research, were sufficiently comfortable with the use of English as
the sole medium of communication. It must be acknowledged, however, that further research
employing literal translation of questionnaires and reports in other languages may well improve
response rates, particularly for comparative research at the local level. A particular advantage of
iterative communication in the Delphi method is that this facilitates the validation of concepts
translated between the first languages of respondents and of the Delphi coordinators.
11. For example, the European Crime and Safety Survey, see http://www.unicri.it/services/
library_documentation/publications/icvs/ (accessed 7 March 2013).
12. Notwithstanding the subtle, contextualized, insights afforded by the kind of ethnographic
immersion favoured by some comparative researchers in European criminology (Nelken,
2010), it is less clear how such intensive research strategies can facilitate a more extensive
dialogue among social scientists, policy-makers and other informants and in keeping with the
shorter time horizons of the policy process, which is surely central to a practically adequate
form of public criminology in Europe.
13. Both panels identified the degradation of governing capacity (affected by the sovereign debt
crises and acute pressures on public expenditure in many European states over the next dec-
ade). In addition, the ESC panel identified the protection of critical infrastructure (including
energy, food and water supplies) as a key emerging issue while the EUCPN panel identi-
fied the growth of urban populations, particularly as a consequence of migration from poorer
regions into wealthier cities, as an emerging priority.
14. Individual panellists’ responses were collated and the mean response calculated. These were
plotted on a scale from 1 to 7 that corresponded to the Likert scales. This is visualized in Key 1
of Figures 1 and 2 and by the position of the markers either towards the outer rim of the ‘radar
diagrams’ or towards their centre; the further out from the centre the marker, the more severe/
frequent the problem in the collective judgement of the panel (where point 7, at the centre of
the diagram, corresponds to ‘very innocuous/infrequent’ and point 1 indicates very severe/
frequent). Of course, mean panel response scores can be skewed by the extreme judgements
of a minority of panellists, particularly in small panels, so, to determine the strength of the
consensus within the panel, a second layer of analysis examined whether panellists’ responses
were concentrated or dispersed around the ‘mean panel response’. In turn, this concentration
of panellists’ responses around the mean may be ‘high’ (75 percent or more of the panel),
‘moderate’ (more than or equal to half but less than three-quarters of the panel), ‘low’ (more
than or equal to a quarter but less than a half of the panel) or ‘questionable’ (less than a quar-
ter of the panel). For example, in Figure 1 (ESC panel), the severity of violence against the
person (including domestic violence) had a mean score of 1.93 (indicating ‘severe’) and all 15
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
282 European Journal of Criminology 10(3)
panellists (100 percent) chose either ‘highly severe’, ‘severe’ or ‘moderately severe’, therefore
indicating a high concentration of opinion. The strength of consensus is visualized in Key 2 in
Figures 1 and 2 and by the markers in the diagrams, where the larger the marker, the stronger
the consensus among panellists.
15. As with severity and frequency, a seven-point Likert scale was used (from strongly agree to
strongly disagree) in tandem with calculations of ‘mean panel response’ and ‘consensus of
panel’ (see note 14).
16. For example, completion of the three rounds of the ESC and EUCPN panels took 11 months,
which is well within the policy cycle of elected governments and local authorities in Europe.
References
Bauman Z (2004) Europe: An Unfinished Adventure. Cambridge: Polity.
Bhaskar R (1979) The Possibility of Naturalism. Hassocks: Harvester.
Burawoy M (2005) For public sociology. American Sociological Review 70 (1): 4–28.
Callon M (1986) Some elements of a sociology of translation. In: Law J (ed.) Power, Action and
Belief. London: Routledge & Kegan Paul.
Clarke RV (2004) Technology, criminology and crime science. European Journal on Criminal
Policy Research 10(1): 55–63.
Edwards A and Gill P (2002) The politics of ‘transnational organized crime’: Discourse, reflexiv-
ity and the narration of ‘threat’. British Journal of Politics and International Relations 4(2):
245–270.
Edwards A and Hughes G (2005) Comparing the governance of safety in Europe: A geo-historical
approach. Theoretical Criminology 9(3): 345–363.
Edwards A and Hughes G (2009) The preventive turn and the promotion of safer communities
in England and Wales: Political inventiveness and governmental instabilities. In: Crawford
A (ed.) Crime Prevention Policies in Comparative Perspective. Cullompton, Devon: Willan.
Edwards A and Hughes G (2012) Public safety regimes: Negotiated orders and political analysis
in criminology. Criminology and Criminal Justice 12(4): 433–458.
Edwards A and Levi M (2008) Researching the organisation of serious crimes. Criminal Justice
and Criminology 8(4): 363–388.
Edwards A and Sheptycki J (2009) Third wave criminology: Guns, crime and social order.
Criminology and Criminal Justice Studies 9(3): 379–397.
Edwards A, Hughes G and Lord N (2013) Crime prevention and public safety in Europe: Challenges
for comparative criminology. In: Body-Gendrot S, Levy R and Hough M (eds) The European
Handbook of Criminology: London: Routledge.
EFUS (European Forum for Urban Security) (2000) Safety and Democracy: The Cities’ Manifesto,
Naples. Paris: EFUS.
EFUS (European Forum for Urban Security) (2012) Security, Democracy and Cities: The
Aubervilliers and Saint-Denis Manifesto. Paris: EFUS.
European Commission (2010) The Stockholm Programme – An Open and Secure Europe Serving
and Protecting Citizens. Official Journal of the European Union, 2010/C 115/01.
Garland D (2001) The Culture of Control. Oxford: Oxford University Press.
Goldstein H (1990) Problem-oriented Policing. New York: McGraw-Hill.
Gordon TJ and Helmer O (1964) Report on a Long-Range Forecasting Study. Santa Monica, CA:
RAND Corporation.
Greenfield VA and Paoli L (2012) If supply-oriented drug policy is broken, can harm reduction
help fix it? Melding disciplines and methods to advance international drug-control policy.
International Journal of Drug Policy 23(1): 6–15.
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
Edwards et al. 283
Hillyard P, Sim J, Tombs S and Whyte D (2004) Leaving a ‘stain upon the silence’: Contemporary
criminology and the politics of dissent. British Journal of Criminology 44(3): 369–390.
Hope T (2009) The illusion of control: A response to Professor Sherman. Criminology and
Criminal Justice 9(2): 125–134.
Karstedt S (2012) Comparing justice and crime across cultures. In: Gadd D, Karstedt S and
Messner SF (eds) The Sage Handbook of Criminological Research Methods. London: Sage.
Loader I and Sparks R (2010) Public Criminology? London: Routledge.
Nelken D (2010) Comparative Criminal Justice: Making Sense of Difference. London: Sage.
Pease K (1994) Crime prevention. In: Magurie M, Morgan R and Reiner R (eds) The Oxford
Handbook of Criminology. Oxford: Oxford University Press.
Sayer A (2000) Realism and Social Science. London: Sage.
Sherman L (2008) Evidence and liberty: The promise of experimental criminology. Criminology
and Criminal Justice 9(1): 5–28.
Smith DJ (2004) Editorial: Criminology and the wider Europe. European Journal of Criminology
1(1): 5–15.
Stenson K (1998) Beyond histories of the present. Economy and Society 27(4): 333–352.
Tilley N (2009) Sherman vs Sherman: Realism versus rhetoric. Criminology and Criminal Justice
9(2): 135–144.
Turoff M (1970) The design of a policy Delphi. Technological Forecasting and Social Change
2(2): 149–171.
Wacquant L (2009) Punishing the Poor: The Neoliberal Government of Social Insecurity.
Michigan: Duke University Press.
Young J (2011) The Criminological Imagination. Cambridge: Polity Press.
Ziglio E (1996) The Delphi method and its contribution to decision-making. In: Adler M and
Ziglio E (eds) Gazing into the Oracle: The Delphi Method and Its Application to Social Policy
and Public Health. London: Sage.
at Cardiff University on September 8, 2013euc.sagepub.comDownloaded from
