Deliberative mapping: appraising options for addressing 'the kidney gap'

Full text

Deliberative Mapping Final Report Draft - not for quotation
1
Deliberative Mapping:
Appraising Options
for Addressing
‘the Kidney Gap’
Draft Final Report
June 2003

Deliberative Mapping Final Report Draft - not for quotation
2
The Deliberative Mapping Project Team
The multi-disciplinary research team for Deliberative Mapping is based at SPRU (University of
Sussex), ESRU (University College London) and the Policy Studies Institute (PSI). The Deliberative
Mapping Project Team members include:
Jacquie Burgess, ESRU, Department of Geography, University College London, WC1H 0AP,
j.burgess@geog.ucl.ac.uk
Gail Davies, ESRU, Department of Geography, University College London, WC1H 0AP,
g.davies@geog.ucl.ac.uk
Malcolm Eames, Policy Studies Institute, 100 Park Village East, London, NW1 3SR,
m.eames@psi.org.uk
Sue Mayer, SPRU, Sussex University, Brighton, BN1 9RF
Kristina Staley, SPRU, Sussex University, Brighton, BN1 9RF
Andy Stirling, SPRU, Sussex University, Brighton, BN1 9RF, a.c.stirling@sussex.ac.uk
Suzanne Williamson, ESRU, Department of Geography, University College London, WC1H
0AP, s.williamson@geog.ucl.ac.uk
The Deliberative Mapping project was funded by the Wellcome Trust under a programme to develop
innovative methods for public engagement in the biosciences.
For further information
For further information about the Deliberative Mapping project, please contact one of the following
team members:
Jacquie Burgess, Department of Geography, University College London, WC1H 0AP,
j.burgess@geog.ucl.ac.uk
Gail Davies, Department of Geography, University College London, WC1H 0AP,
g.davies@geog.ucl.ac.uk
Andy Stirling, SPRU, Sussex University, Brighton, BN1 9RF, a.c.stirling@sussex.ac.uk
Acknowledgements
There are many people to thank for their assistance in the development of this project. Firstly, we are
indebted to the Wellcome Trust for their generous support. We are especially grateful to our Project
Advisory Committee who have provided support for the research team throughout all stages of project,
and to our evaluators for agreeing to contribute to this final period of research. Individual team
members have been assisted in many ways in their work by colleagues in their individual institutions,
as well as through many conversations with others outside. We would also like to thank everyone at
PSI and Senate House for their practical support in running the Citizen Panels and Workshops. And
finally, of course, the project would not have been possible without the contributions of time, energy
and expertise from both specialists and citizens. We are very grateful to everyone who gave so
generously of these.

Deliberative Mapping Final Report Draft - not for quotation
3
Table of Contents
Executive Summary ______________________________________________________________ 6
1 Introduction to Deliberative Mapping_____________________________________________ 9
1.1 Introduction _______________________________________________________________ 9
1.2 Choice of Case Study: Options for Addressing the ‘Kidney Gap’_____________________ 11
1.3 Engagement in Science and Technology Assessment: A New Paradigm for Governance? 14
1.4 Analysing Stakeholder and Public Engagement__________________________________ 17
1.5 Deliberation and Dialogue in Health Policy Contexts ______________________________ 25
1.6 Conclusions______________________________________________________________ 29
2 Deliberative Mapping: a Methodology ___________________________________________ 31
2.1 Introduction ______________________________________________________________ 31
2.2 Evaluating Deliberative Participatory Processes _________________________________ 31
2.3 Developing Deliberative Mapping _____________________________________________ 34
2.4 The Deliberative Mapping Methodology ________________________________________ 38
2.5 The Deliberative Mapping Process____________________________________________ 39
2.6 Methods of Analysis _______________________________________________________ 46
2.7 Conclusions______________________________________________________________ 48
3 Options for Addressing the ‘Kidney Gap’ ________________________________________ 50
3.1 Introduction ______________________________________________________________ 50
3.2 Scope of Process and Definition of Options _____________________________________ 50
3.3 Specialist Engagement with the Options _______________________________________ 56
3.4 Facilitating Citizen Engagement with the Options ________________________________ 63
3.5 Citizen Engagement with Core Options ________________________________________ 65
3.6 Citizen Selection of and Engagement with Discretionary Options ____________________ 80
3.7 Conclusions______________________________________________________________ 84
4 Developing Criteria for Option Appraisal ________________________________________ 86
4.1 Introduction ______________________________________________________________ 86
4.2 Defining Criteria __________________________________________________________ 86
4.3 Weighting Criteria _________________________________________________________ 92
4.4 Discussion by Criteria Grouping ______________________________________________ 95
4.5 Engagement between Citizens and Specialists _________________________________ 105
4.6 Conclusions_____________________________________________________________ 107

Deliberative Mapping Final Report Draft - not for quotation
4
5 Appraising Option Performance_______________________________________________ 109
5.1 Introduction _____________________________________________________________ 109
5.2 The Appraisal Process ____________________________________________________ 110
5.3 Appraisal by Option_______________________________________________________ 114
5.4 Appraisal by Criteria ______________________________________________________ 125
5.5 Diversity and Uncertainty in Option Scoring ____________________________________ 128
5.6 Conclusions_____________________________________________________________ 136
6 Mapping Option Performance_________________________________________________ 138
6.1 Introduction _____________________________________________________________ 138
6.2 The Overall Picture _______________________________________________________ 139
6.3 Final Rankings for Citizens’ Panels __________________________________________ 141
6.4 Final Rankings by the Specialists ____________________________________________ 145
6.5 Discussion of Final Option Performance_______________________________________ 148
6.6 Patterns of Consensus and Diversity in Option Performance ______________________ 155
6.7 Conclusions_____________________________________________________________ 164
7 Process Evaluation _________________________________________________________ 165
7.1 Introduction _____________________________________________________________ 165
7.2 Evaluation Process and Results _____________________________________________ 165
7.3 Critical Reflections _______________________________________________________ 176
7.4 Pursuing Project Aims_____________________________________________________ 183
7.5 Conclusions_____________________________________________________________ 190
Bibliography___________________________________________________________________ 191
Appendix A: Membership of the Project Advisory Committee (PAC) ____________________ 197
Appendix B: Specialist criteria tables ______________________________________________ 199
Appendix C: Citizen scoring analysis _____________________________________________ 217
Appendix C.1: Option performance by criteria grouping________________________________ 217
Appendix C.2: Shifts in option performance following workshop _________________________ 219
Appendix D: Specialist scoring analysis ___________________________________________ 221
Appendix D.1 Individual specialist scores by criteria grouping ___________________________ 221
Appendix D.2 Aggregate specialist scores by criteria grouping __________________________ 229
Appendix D.3 Analysis of uncertainties in specialist scoring _____________________________ 230

Deliberative Mapping Final Report Draft - not for quotation
5
Technical Annexes
The following separate technical annexes are available on <www.deliberative-mapping.org>
Annex A: Specialist Briefing on Multi-Criteria Mapping Method
Annex B: Citizen Recruitment Information
Annex B1: Recruitment Briefing
Annex B2: Recruitment Questionnaire
Annex B3: Panel meetings timetable
Annex B4: Citizen Panel membership contract
Annex C: Citizens’ Panel Process Protocol
Annex D: Citizen Information on Options
Annex D1: Presentation on Kidney Gap
Annex D2: Citizen options Booklet
Annex D3: Citizen Questions about Options
Annex D4: Citizen Questions for workshop
Annex E: Citizen Criteria Tables
Annex F: Citizen Scoring Poster
Annex G: Joint Workshop Information
Annex G1: Citizen Information on Specialists for Joint Workshop
Annex G2: Structure of Joint Workshop
Annex G3: Joint citizen and Specialist Workshop booklets
Annex G4: Joint workshop information booklet for specialists
Annex H: Citizen Panels Reports
Annex I: Deliberative Mapping Interim Report

Deliberative Mapping Final Report Draft - not for quotation
6
Executive Summary
Background
Deliberative Mapping is a two-year project (2001-3) funded by the Wellcome Trust under a programme to
develop innovative methods for public engagement in the biosciences. Current developments in the
biosciences hold out hope for the effective treatment of many intractable illnesses, but never without controversy
on economic, social, political or ethical grounds, challenging decision-makers as never before. New governance
strategies are urgently required to deal effectively with the uncertainties posed by the introduction of new
technologies into complex, plural and unequal societies. Done well, deliberative and inclusive engagement
processes can widen the social base for the appraisal of policy options and help legitimate the basis of future
policy making.
Deliberative Mapping is an appraisal process which integrates expert and citizen assessments through a
transparent and auditable framework. Emphasising diversity, Deliberative Mapping enhances social learning to
support more robust, democratic and accountable decision-making.
The specific aims of the Deliberative Mapping project were to:
• examine how far scientific, expert-driven risk assessment techniques can be reconciled with
deliberative approaches to public engagement;
• develop and test the DM process through a full-scale public engagement exercise. A range of
specialists and citizens’ panels assessed alternative policy options for addressing a complex
medical issue.
The medical issue selected for the study was how best to reduce or close ‘the ‘kidney gap’. This draft final report
outlines the outcomes from the application of the Deliberative Mapping process to this medical issue, as well as
reflections on the development of the Deliberative Mapping methodology. The final report will be submitted to
the Wellcome Trust in August 2003, allowing the opportunity for further evaluation of both the Deliberative
Mapping process and outcomes.
Focus
The medical issue selected for the study was how best to reduce or close ‘the ‘kidney gap’. As Chapter 1
explains there is a growing disparity between the number of donor organs available and the number of patients
awaiting transplants. The gap will continue to grow as demand for organ transplants increases in the future.
Options for reducing or eliminating the kidney gap range from sophisticated bio-technological innovations such as
stem cell based therapies and xenotransplantation, to changes in organ donation practices and improved health
education. All options are characterised by scientific and technical uncertainties and/or raise social, economic,
cultural or ethical difficulties. In this context, there is a particular value in approaches to public engagement,
which provide for effective integration and mutual learning between different specialists, stakeholders and citizen
perspectives.
The Deliberative Mapping Process
The Deliberative Mapping approach integrates a number of methodologies, which are fully described in Chapter
2, including open-ended scoping interviews with specialists and stakeholders, facilitated group deliberation in a
series of citizens’ panels, a joint citizen/specialist workshop and a specialists meeting and structured
quantitative appraisal using a scoring and weighting framework based on paper-based MCM in the citizens’
process; and sophisticated MCM software in the specialist process. These methodologies were integrated in an
iterative and interactive structure, subject to stakeholder oversight, by a Programme Advisory Committee.
Thirty-four citizens from Camden, North London were recruited to include a wide range of ages, occupations,
ethnic backgrounds, and family circumstances. 17 specialists were recruited to span a wide range of
perspectives, disciplines and institutional affiliations, covering transplant policy, healthcare policy, medical
research, industry, ethical and other stakeholder perspectives. Citizens and specialists followed the same basic
framework for appraisal.
To ensure comparability between the different perspectives of citizens, specialists and stakeholders, six ‘core
options’ were identified by the research team for appraisal by all participants. These core options included:
improved transplant services, altruistic living donation, presumed consent, xenotransplantation, embryonic stem

Deliberative Mapping Final Report Draft - not for quotation
7
cells, and encouraging healthier living. Four ‘prompted options’ could also be appraised if participants wished
This common set of options are each assigned a score to reflect performance under freely defined criteria.
These are then weighted to reflect their importance. This allows determination of a set of ranks, indicating the
overall performance of each option. The result is a ‘map’ of the way performance varies under different
perspectives, which is further interpreted through qualitative analysis of transcripts which record the
deliberations of individual specialists, and the citizens’ panels.
The Overall Performance of the Core Options
Despite many differences in perspective, there is a remarkable degree of consistency between the appraisals
made by the four citizens’ panels, and the 17 specialists.
• Of the six core options, two perform markedly better overall: improved transplant services,
encouraging healthier living
• Two further options are generally ranked highly, but slightly lower or with more
qualifications than the two ‘lead options’: presumed consent, altruistic living donation
• Finally, the two ‘technology-based’ options perform markedly worse than the others overall:
xenotransplantation, embryonic stem cells
• The picture for the four discretionary options is, for various reasons, more ambiguous: adult
stem cells, improved kidney machine technology, rewarded giving, accepting death
Quantitative outputs in Chapter 6 of the final report illustrate the extent of convergence between the citizens’
panels and the various groupings of specialists, as well as indicating the areas of difference. Qualitative analysis
of the deliberations in Chapters 3, 4, 5 and 6 shows that reasonings both specialists and citizens to reach their
judgements. Both panels used many different kinds of ‘public reasons’ rather than purely personal preferences
about the options. All took their responsibilities as representatives of wider society in the Deliberative Mapping
project very seriously.
The Nature and Importance of the Appraisal Criteria
The criteria used to appraise the options were defined, selected and weighted by the participants themselves.
The four citizens’ panels developed 14 distinct criteria. The seventeen specialists produced 111 criteria. Three of
these were introduced as issues of principle, under which certain options were ruled out of appraisal. Based on
these criteria, specialists also identified a number of additional options for appraisal detailed in Chapter 3. These
refined or introduced new features not present in the core options.
Eleven broad categories of criteria were thought important. The relative importance of the criteria groups to
citizens and specialists is revealed through analysis of criteria weighting and analysis of the discussions between
the citizens. Many areas of agreement emerged from this analysis, as well as some differences in terms of
interpretation and priority between citizens and specialists, which are explored in Chapter 4.
Priorities for both citizens and specialists include
• Feasibility: Includes issues of scientific, technical, legal, institutional and political viability. Asks how
well an option will work in practice and the timeliness with which it will become practically available.
• Ethical acceptability: Includes moral questions about consent (donor and family), coercion (of donors),
animal welfare, nature/culture boundaries, other socio-political issues and general notions of the ‘good
society’.
• Economic: Includes the cost of the option in broad monetary terms. Variously accounted for at the level
of the NHS, extended to include the research system or others on whom explicit costs fall, or
encompassing hidden costs for society as a whole.
Other priorities for the citizens’ panels include
• Public safety: Addresses implications for the safety of non-patients. Includes issues such as infection
risks or other unintended or unanticipated public health impacts and health effects of surgery on living
donors.
• Wider Benefits: Includes benefits that may be gained from an option that are wider than organ donation
itself. For example, information about more healthy lifestyles will protect against other illnesses;
scientific research may produce results that are more widely applicable.

Deliberative Mapping Final Report Draft - not for quotation
8
Other priorities for specialists include
• Patient outcomes: Includes issues of medical success and patient quality of life, sometimes based on
established clinical quality of life scales and sometimes more broadly defined, including ‘quality of
dying’.
• Capacity: Addresses the contribution made to increasing the number of organs available (or reducing
the need) for transplant. It excludes wider issues of success.
Other criteria related to equity, information and transparency, socio-political motivation and other social impacts
were generally lower priority issues for both specialists and citizens, but are discussed fully in chapters 4 and 5.
Evaluation and Reflections
The DM project was guided by an explicit set of evaluative criteria at all stages of its design, implementation and
analysis. Chapter 7 includes detailed critical reflections on the lessons learned by the research team. These
reflections will be augmented in the light of further comments at the dissemination workshop. Evaluative
comments by citizen and specialist participants so far have been strongly positive.
• Citizens took pride in having completed this process. The panellists expressed a sense of empowerment
and strong feelings of ownership over the results of the appraisal. Having sufficient information provided to
engage with the issues; working through a structured decision-making process supported by facilitators; and
meeting specialists were all highly valued learning experiences. However, the citizens had mixed opinions
about the potential impacts of the DM project on policy.
• Specialists appreciated the novelty, scope and potential of DM. Specialist learning was social rather than
technical. They were all genuinely surprised by the quality of the citizens’ deliberations, their willingness to
engage with, and challenge specialist views. Some thought the process protracted; others were concerned
about definitions of ‘expertise’ in this context; a few were worried about potential issue framing problems for
the citizens’ panels. Specialists welcomed the project and thought it offered a worthwhile public engagement
strategy for policy makers.
The deliberative mapping approach combines inclusiveness and openness to divergent perspectives, specificity
and robustness in its policy implications, transparency and auditability for third parties, and efficiency and added
value for sponsoring policy institutions. One particular observation concerns the way in which the convergence in
option rankings is underlain by a rich diversity in the detailed perspectives. Taking the integration of quantitative
and qualitative assessment seriously is important. Purely discursive analysis would likely have been dominated
by a picture of contrasts and tensions. By the same token, quantitative analysis alone would have missed
important nuances of difference.
However, there are also important constraints and limitations to be acknowledged. One key issue that arises in
any appraisal process – whether deliberative or analytic – concerns the degree to which outcomes may be
subject to invisible contingencies. The DM process was designed both to minimise and provide a check on such
factors. However, this does not preclude the possibility that certain aspects of the findings may be artefacts of the
process. For instance, there is some evidence that inter-personal encounters between specialists and citizens at
the Joint Workshop may have affected appraisals for certain options. If a specialist was felt to be friendly, open
and approachable, some of the citizens responded more positively to ‘their’ option in subsequent discussion. It
also seems that the process of weighting may actually be conducted by specialists in a rather different fashion
than is assumed in the theoretical framework for decision analysis. Reflecting other PTA research findings, there
are also questions around strategic behaviour in the assessment of the options – a factor addressed by the high
auditability of the MCM procedure. Finally, although it may be seen as efficient in relation to the added value of
the outputs, Deliberative Mapping is quite complex, time consuming and expensive. It needs strong project
management and high quality facilitation. This places significant demands on sponsors, practitioners and
participants alike. Such investments of time, effort and resources are not possible or appropriate in every context
and should not be undertaken lightly
Conclusions
This process has identified a number of specific and concrete policy implications for closing the ‘kidney gap’. In
particular, these are strongly favourable for organisational options improved transplant services and
encouraging healthier living. There are correspondingly negative implications for technology-based options like
xenotransplantation and embryonic stem cells.
In conclusion, Deliberative Mapping offers a practical means to inform technical policy decisions in a robust and
accountable fashion. It shows that quantitative and qualitative appraisal techniques and individual and group-
based methods can work together effectively as part of a deliberative and inclusive process. By balancing a
variety of specialist, stakeholder and citizen perspectives, DM may help to foster more co-operative policy
dialogues.

Deliberative Mapping Final Report Draft - not for quotation
9
1 Introduction to Deliberative Mapping
‘Democratic citizenship in a modern society depends, among other things, on the ability of
citizens to comprehend, criticise and use scientific ideas and claims. … The applications of
science raise, or feed into, complex ethical and social questions, which government and industry
must handle in ways which command public confidence. At present many of these questions
arise in the biosciences (e.g. the issues raised by cloning, genetic testing, gene patents, assisted
reproduction and xenotransplantation)’.
Science and Society, House of Lords Environment Select Committee, 3rd Report, 2000.
1.1 Introduction
Current developments in the biosciences raise the possibility of new treatments and cures for
diseases, bringing with them unparalleled improvements in human health. However, these same
advances in the understanding of genetics and ability to manipulate DNA also raise profound social,
political and ethical concerns. Controversies over biomedical research are, of course, nothing new.
Long running debates over the safety of vaccination and the use of animals in vivisection both may be
traced back to the 18th century while more recent concerns include the regulation and safety of drugs
following the thalidomide tragedy, and religious opposition to research using human embryos and IVF.
The state of the UK’s health services has been one of the dominant political issues of the last decade.
Such biomedical and institutional pre-occupations are embedded in a context of rapid political, social
and cultural changes characterised by a pervasive sense of risk (Beck 1992), recently defined as an
endemic problem of ‘radical uncertainty’ (Pellizzoni 2003). The new governance of science and
technology must deal with the entanglement of knowledge and values, where decision-paths defy
linear or stepwise progression, and where the outcomes of decisions will almost certainly lead to
unforeseen (and probably unwelcome) consequences in the future.
Globalisation, especially in relation to the spatial reach of private sector corporations, new
communications media and the increased capacity of civil society to mobilise in the face of de-
regulation are creating a political culture which demands greater transparency and accountability.
Moreover, in the wake of Chernobyl, BSE and genetically modified food, citizens are increasingly
sceptical about the sufficiency of scientific expertise and distrustful of the way science is used in
government policy making (Hinchliffe 2001; Wynne, Marris, & Simmons 2001). As a result there is a
growing climate of concern and unwillingness to accept technological developments thought to
threaten deeply held values or to pose potential risks to human health and the natural world. In part,
this is because established ‘scientific’ risk assessment techniques have proven incapable of dealing
effectively with intractable uncertainties and fail to accommodate the full range of public concerns.
One response to the declining authority of government and science has been to seek to increase the
legitimacy and democratic accountability of decision-making through new public engagement
processes. Citizens’ juries and consensus conferences promote deliberation between experts and
stakeholders and the inclusion of a wider range of social groups in technology assessment, for
example (Joss & Bellucci 2002). Deliberative approaches require social relations based on trust and
mutual respect, for the process of weighing evidence through argument to r