Gene Rowe

• PhD
• Managing Director at Gene Rowe Evaluations

This chapter delves into the methodology of collecting expert judgement in the AI and the Future of Skills project. It provides an overview of the project's journey in refining its methodology and discusses associated challenges and considerations. The chapter begins by exploring the different methods of expert knowledge elicitation based on the re...

Although the individual has been the focus of most research into judgment and decision-making (JDM), important decisions in the real world are often made collectively rather than individually, a tendency that has increased in recent times with the opportunities for easy information exchange through the Internet. From this perspective, JDM research...

An expert knowledge elicitation was conducted to address three tasks identified by EFSA's Animal Health and Welfare (AHAW) Panel: categorize outdoor farm types of pigs in EU MS according to the risk of African Swine Fever (ASF) introduction into these farms and the risk of ASF spread from these farms, rank biosecurity measures according to their po...

This final report summarises the objectives, activities and delivery, evaluation, and any future recommendations for each of the four trainings delivered under this contract. Sections are organised in chronological order consistent with the order in which trainings were delivered: Section 3 details the Sheffield course, Section 4 Semi‐formal, Secti...

Groups provide several benefits over individuals for judgment and decision making, but they suffer from problems too. Structured-group techniques, like Delphi, use strictly controlled information exchange between individuals to retain positive aspects of group interaction, while ameliorating negative. These methods regularly use ‘nominal’ groups th...

Public participation is ubiquitous in many contemporary democratic societies – used for many purposes, and in many contexts, with particular and growing relevance for policy-making on science and technology issues. However, there is a dearth of evidence as to its qualities and benefits. We contend that the implementation and interpretation of parti...

Europe recognises the need for technological innovation along with the importance of bridging the gap between science and society. The European Commission has developed a strategy to foster public engagement and a sustained two-way dialogue between science and civil society, and has set up a framework for Responsible Research and Innovation. The EU...

A systematic review relevant to the following research questions was conducted 1) the extent to which different theoretical frameworks have been applied to food risk/benefit communication and 2) the impact such food risk/benefit communication interventions have had on related risk/benefit attitudes and behaviours. Fifty four papers were identified....

Food allergy is an increasing problem worldwide. Allergy to peanuts is a particular concern, given that this is rarely outgrown and may be associated with life-threatening anaphylaxis. However, it is unclear what factors are responsible for a perceived increase in prevalence rates. One matter on which scientists agree, however, is that exposure to...

We strongly concur with most of Morgan’s points regarding expert knowledge elicitation (EKE) for policy making (1); however, we contest the somewhat idealistic tone of Morgan’s paper, which we believe counters the goal of better policy making otherwise promoted. Furthermore, we disagree on two specific issues raised by Morgan that can be seen as lo...

Good policy making should be based on available scientific knowledge. Sometimes this knowledge is well established through research, but often scientists must simply express their judgment, and this is particularly so in risk scenarios that are characterized by high levels of uncertainty. Usually in such cases, the opinions of several experts will...

Objectives: Ascertaining the quality of life (QoL) in people with dementia is important for evaluating service outcomes and cost-effectiveness. This paper identifies QoL measures for people with dementia and assesses their properties. Method: A systematic narrative review identified articles using dementia QoL measures. Electronic databases sear...

The authors explore the development of the Patients’ Expectations Questionnaire (PEQ) and examination of psychometric characteristics it encompasses by reviewing surveys of primary care and hospital outpatients before and after their clinic visit. Three scales were developed for Pre-visit Ideal and Realistic expectations, and Post-visit Experiences...

Background: Peanuts are one of the main food allergens, occasionally responsible for life-threatening reactions. Thus, many studies have tried to fi nd a connection between peanut allergy prevalence and processes in the peanut chain that may contribute to the peanut allergenicity. To inform this discussion, this paper outlines experiences in peanu...

The present paper aims to review and discuss potential and existing risks of GM crops to the environment, in comparison with organic and conventional agriculture. The review of over 30 relevant papers on the environmental effects of modern agriculture allowed us to define five main sources of risks, namely: 1) Pollution by synthetic fertilizers and...

There is a growing call for greater public involvement in establishing science and technology policy, in line with democratic ideals. A variety of public participation procedures exist that aim to consult and involve the public, ranging from the public hearing to the consensus conference. Unfortunately, a general lack of empirical consideration of...

In what ways are experts involved in policy development, and with what results? This paper attempts to answer these questions through a structured review of the academic literature, focusing on the identification of ‘methodologies’ of expert involvement, and on analying the subsequent policy impact of those exercises. Coding was applied to 103 arti...

Objectives: To investigate patients' experiences of health services, and how these related to what they had expected to receive, and satisfaction with their care. Design: Surveys of patients before and after their consultations in general practice and hospital outpatients departments. Setting: Greater London and Essex Participants: In total,...

In a realistic social context, people are confronted with both positive and negative information, yet research on this topic is relatively scarce. We present 2 studies examining the role of initial attitudes on the impact of one-sided vs. balanced positive and negative information on attitudes toward food production methods. The first experiment de...

Following the BSE crisis, the importance of reassuring European consumers and involving them in debate about food safety management issues is generally recognised. Indeed, consulting stakeholders is one of the European Food Safety Authority’s (EFSA) main duties, as stated in its founding regulation (art. 42, whereas 56), where EFSA must have “effec...

There is recognition of the importance of measuring patients' experiences, expectations and satisfaction. To assess the literature on the concept and measurement of patients' expectations for health care, and to develop and test a measure of patients' expectations, using adult patients in community, general practice and hospital outpatient departme...

The strategic development of novel nanotechnologies will be determined by their public acceptance, which in turn may be influenced by public perceptions of the risks and benefits associated with the specific applications. At the present time, public opinions towards nanotechnologies remain largely inchoate, although this is likely to change with in...

Developing policy in the agrifood area is an inexact process, usually relying upon effective integration of opinions from multiple experts from different disciplines, organisational types, and regions/countries. Delphi would appear to have the potential to overcome some of the typical limitations related to soliciting expert opinion and identifying...

The issue of consumer acceptance of food technologies, and their applications, needs to be addressed early in technology development. However, whether extensive assessment of consumer acceptance is necessary for all food-related technologies a priori is uncertain. A review of studies of seven foodrelated technologies associated with different level...

Objectives: To assess whether preferences for patients with angina changed at 18-month follow-up using the Patient Preferences Questionnaire for Angina treatment (PPQA). Background: Evidence suggests that patients want information about treatment options. Reliable measurement of patient preferences for treatment is important for empowering patients...

This paper examines the Delphi technique critically to determine whether it succeeds in alleviating the 'process loss' typical of interacting groups. After briefly reviewing the technique, we go on to consider problems with Delphi from two perspectives. First, we examine methodological and technical difficulties, and the problems these have brought...

We review and integrate the extant knowledge on group-based forecasting, paying particular attention to the papers included in this special issue of the International Journal of Forecasting. We focus on the relative merits of different methods of aggregating individual forecasts, the advantages of heterogeneity in group memberships, the impact of o...

There are considerable uncertainties regarding how stakeholder engagement should be enacted. The lack of clear guidelines on good practice is arguably a consequence of an absence of evaluations on the effectiveness of past engagement exercises. Here we describe the evaluation of one engagement event concerning food risk management. This event invol...

This book tells the story of an unprecedented experiment in public participation: the government-sponsored debate on the possible commercialization of 'GM' crops in the UK. Giving a unique and systematic account of the debate process, this revealing volume sets it within its political and intellectual contexts, and examines the practical implicatio...

Early identification of emerging food risks will protect human and animal health, the environment and economy. A two-round Delphi survey identified international experts’ views regarding knowledge gaps associated with the identification and mitigation of emerging food risks, and barriers to emerging risk identification and prevention. The results s...

To use semi-structured interviews to ascertain patterns in patients' expectations of health care and the extent to which these expectations were met or not. In health policy it is important to evaluate health services from varying perspectives including consumers'. One concept of emerging importance in this regard is that of patient expectations. W...

As a consequence of recent food safety incidents, consumer trust in European food safety management has diminished. A risk governance framework that formally institutes stakeholder (including consumer) consultation and dialogue through a transparent and accountable process has been proposed, with due emphasis on risk communication. This paper deliv...

The extent of social acceptance of carbon capture and storage (CCS) is likely to significantly influence the sustainable development of CO2 storage projects. Acceptance of CCS by the key stakeholders (policy makers, the general public, the media and the local community), linked to specific projects, as well as how the technology is communicated abo...

To develop a psychometrically valid Patient Preferences Questionnaire for Angina treatment (PPQA). Seven general practices across England in 2007. Convenience sample of 383 patients with diagnosed angina. Postal self-administered questionnaire survey using the full-length PPQA. This comprised 54 items about the three main treatment modalities for a...

We have reached a point where it is difficult to improve food allergy risk management without an agreement on levels of acceptable risk. This paper presents and discusses the perspectives of the different stakeholders (allergic consumers, health professionals, public authorities and the food industry) on acceptable risk in food allergy. Understandi...

One trend in public engagement concerns involving the public in research priority setting. In this study members of the public were asked to select which of four potential projects (about food-related topics, presented by scientists) ought to be funded. The aim of the study was twofold: to trial and evaluate a method of engaging with the public abo...

Evidence of a decline in public trust associated with food risk governance over recent years has called into question the appropriateness of the current dominant risk analysis framework. Within the EU-funded SAFE FOODS project a novel risk analysis framework has been developed that attempts to address potential shortcomings by increasing stakeholde...

Rates of peanut allergy in the Western world have increased over the last 30 years, although it is unclear why. While eating behaviours are likely to be connected to allergy prevalence, the precise relevant factors are uncertain. This study aimed to investigate dietary differences and changes in dietary habits in peanut consumption (in apparent and...

A recent structured review of the qualitative risk perception literature (Hawkes and Rowe 2008) found that research has tended to focus on a narrow range of samples, using a limited range of techniques, looking at immediate perceptions (rather than changes over time), and focusing upon a limited range of hazards. As such, it may be argued that the...

There is a need for early identification of emerging food safety issues in order to prevent them from developing into health risks. In this paper, various existing methods and procedures which can be used for early identification of safety issues are reviewed, including the monitoring of the occurrence of specific hazards within the food supply, or...

In European countries, there has been growing consumer distrust regarding the motives of food safety regulators and other actors in the food chain, partly as a result of recent food safety incidents. If consumer confidence in food safety is to be improved, a systematic understanding of what consumers perceive to be best practice in risk management...

This chapter reviews extant research on the quality of expert judgement of probability and risk. Both types of judgement are of great relevance to the concerns of this book, because expert judgement under uncertainty is a key component in the making of decisions with scant information.

Het effect van eenzijdige versus tweezijdige informatievoorziening en bestaande attitudes op post-attitudes is onderzocht. Met name is gekeken naar de mogelijke asymmetrische invloed van positieve en negatieve informatie op post-attitudes. Daarnaast is gekeken hoe bestaande attitudes dit effect beïnvloeden. Eenzijdige informatie beïnvloedde attitud...

Access to cardiac treatments has been documented to vary with patients' age. It is unknown whether these variations reflect patients' treatment preferences. We aimed to investigate patients' preferences for cardiology treatments and develop a Patients' Preferences Questionnaire. Semi-structured interviews with primary care patients with diagnosed a...

Over recent years, many policy-makers and academics have come to the view that involving the public in policy setting and decision-making (or “public engagement”) is desirable. The theorized benefits of engagement (over traditional approaches) include the attainment of more satisfactory and easier decisions, greater trust in decision-makers, and th...

In February 2007, avian influenza arrived in the UK. Following evidence of illness at a turkey farm in Suffolk, around 160,000 birds were slaughtered and poultry movement controls were imposed. Given past food crises (e.g. BSE), it was clearly important to predict UK public response: for example, was mass panic imminent, or would the public respond...

Peanuts are extensively cultivated around the world, providing a foodstuff that is both cheap to produce and nutritious. However, allergy to peanuts is of growing global concern, particularly given the severity of peanut-allergic reactions, which can include anaphylaxis and death. Consequently, it is important to understand the factors related to t...

The issue of how risk is ‘perceived’ is one of significant research interest and immense practical importance. In spite of this wide interest, however, it is probably fair to say that most emerging ‘risk’ crises – whether related to natural or technological phenomena – come as a surprise to researchers and to society as a whole. Prediction of human...

To measure preferences for angina treatments among patients admitted from accident and emergency with acute coronary syndrome. Evidence suggests variability in treatment allocations amongst certain socio-demographic groups (e.g. related to age and sex), although it is unclear whether this reflects patient choice, as research on patients' treatment...

Recent emphasis within policy circles has been on transparent communication with consumers about food risk management decisions and practices. As a consequence, it is important to develop best practice regarding communication with the public about how food risks are managed. In the current study, the provision of information about regulatory enforc...

This chapter introduces the laddering technique. This is an approach to self-report data elicitation and data recording that can be used in interviewing to impose a systematic framework upon questioning, and in analysis to allow complex themes across answers to be represented. The technique incorporates the sorting procedures described in chapter 1...

In this paper, we address the issue of food risk management as practised in Western Europe. We begin by considering how food risks are managed, and then go on to consider how well they are managed. There are multiple answers to the ‘how well’ question, which are related to the varied perspectives of the different key stakeholders – from the food ri...

In developing and implementing appropriate food risk management strategies, it is important to understand how consumers evaluate the quality of food risk management practices. The aim of this study is to model the underlying psychological factors influencing consumer evaluations of food risk management quality using structural equation modeling tec...

Crises in food risk management have often been attributed to disconnection between the stakeholders involved, as when consumers do not believe risk communications or dispute risk managers' priorities. Before this problem of disconnection can be resolved, however, there is a need to clarify the nature and extent of perceptual discrepancies between s...

An important direction in recent thinking about public understanding of science and technology is embodied in the international trend within many democratic countries towards the promotion of citizen engagement. These developments entail the participation and deliberative involvement by lay publics in planning, decision-making and policy-making sit...

A research group of four judgement and decision-making experts was convened to examine the literature on risky human behaviour, including driving, and to make recommendations on what format, structure and content is likely to have the biggest impact on road safety attitudes and behaviour in the specific group of convicted traffic offenders. The ove...

When we rely on the judgment of experts to help produce our forecasts, the key issues are how to get the appropriate information from our consultants and how to get a forecast if we are using multiple experts. Gene Rowe describes the Delphi method, tells what it offers to the forecaster, and explains what the pitfalls are in its implementation. Cop...

The public debate about GM crops that took place in Britain in 2002–03 was an unprecedented experiment for the UK in public engagement. The Government, which sponsored the exercise, made a pledge: to take into account lay perspectives, as well as the views of its expert advisers, when making a decision about the possible commercialisation of this c...

In designing and implementing appropriate food risk management strategies, it is important to examine how key stakeholders perceive both the practice and effectiveness of food risk management. The objective of this study is to identify similarities and differences in perceptions of, and attitudes to, food risk management practices held by consumers...

Consumer perceptions of food hazards and how the associated risks are managed are likely to be an important determinant of consumer confidence in food safety. While there is a body of research that examines public perceptions of various types of food hazards, less attention has been directed to understanding how the public perceives food risk manag...

Involving the public in policy setting, or public engagement, is becoming increasingly popular in many democratic societies. Among the various ways of enacting engagement, the use of the Internet to either collect opinions or conduct public debate is seen as holding out great promise. However, the benefits of using the Internet over other communica...

Numerous studies have analysed consumer perceptions of the risks associated with different food hazards. However, little research has considered how consumers perceive food risk management. If consumers' perceptions of what constitutes effective food risk management practices differ from those of experts, then consumer confidence in the risk analys...

It is well established that random assignment between experimental treatment and control arms is the gold standard in clinical trials to minimise differences between the groups being compared and safeguard against bias. There is, however, a fear that such random allocation may not accord with patients' preferences for the intervention or treatment,...

In the realm of risk management, and policy-making more generally, “public engagement” is often advocated as an antidote to pathologies associated with traditional methods of policy-making, and associated deficit-model-driven communication strategies. The actual benefits of public engagement are, however, difficult to establish without thorough eva...

A current popular theme in medicine concerns whether and how patients should be involved in treatment choice. Assuming patient involvement is desirable, how should one go about eliciting preferences? A variety of quantitative and qualitative methods exist that may be used for this purpose, one of which is the repertory grid method. This method invo...

This study investigates individual opinion change and judgmental accuracy in Delphi-like groups. Results reveal that the accuracy of judgmental probability forecasts increases over Delphi rounds (in terms of proportion correct and appropriateness of confidence) when statistical summaries or written rationales are provided from other members of an i...

This article takes as its case study the "GM Nation?" public debate, a major participation process on the commercialization of agricultural biotechnology, which occurred in Britain during the summer of 2003. We investigate possible self-selection biases in over 36,000 open questionnaire responses on the risks and benefits of genetically modified cr...

Imprecise definition of key terms in the "public participation" domain have hindered the conduct of good research and militated against the development and implementation of effective participation practices. In this article, we define key concepts in the domain: public communication, public consultation, and public participation. These concepts ar...

The need to engage citizens in policy development is increasingly recognised as necessary to develop and maintain public confidence in public institutions and decision-making processes. However, public trust in public participation may actually decrease if such efforts are not evaluated in terms of: how they are conducted; the transparency of the p...

The concept of "public participation" is currently one of great interest to researchers and policy makers. In response to a perceived need for greater public involvement in decision making and policy formation processes on the part of both policymakers and the general public, a variety of novel mechanisms have been developed, such as the consensus...

Abstract The concept of public participation is one of growing interest in the UK and elsewhere, with a commensurate growth in mechanisms to enable this. The merits of participation, however, are difficult to ascertain, as there are relatively few cases in which the effectiveness of participation exercises have been studied in a structured (as oppo...

To elicit patients' preferences for the treatment of angina. Angina patients were interviewed in order to elicit their personal reasons underlying preferences for various treatment options. Interviews followed a general repertory grid technique, in which seven treatment options were presented to patients in triads. Treatments considered ranged from...

This study investigates individual opinion change and judgmental accuracy in Delphi-like groups. Results reveal that the accuracy of judgmental probability forecasts increases over Delphi rounds (in terms of proportion correct and appropriateness of confidence) when statistical summaries or written rationales are provided from other members of an i...

It has become almost an accepted fact that experts perceive or judge risks in a different manner to laypersons. This apparent finding has stemmed from the pioneering work of Slovic and colleagues (e.g., Slovic, Fischhoff, and Lichtenstein, 1985), who have suggested that experts perceive risks in terms of statistical fatalities, whereas laypersons i...

A recent study by Lusk suggests that consumers might voluntarily pay more for a genetically modified (GM) food than a non-GM equivalent if made aware of the possible health benefits. However, other research indicates that the acceptability of novel hazards is affected by a variety of factors, in addition to benefits, and that making agricultural bi...

There is currently a trend in democratic societies towards greater involvement of the public in the policy setting of official bodies. Numerous mechanisms have been developed to enable such involvement, ranging from traditional forms, such as the public meeting, to more novel forms, such as the consensus conference. This paper considers the use of...

EDITOR–Kennedy comments that a mature culture will settle on sharing power and responsibility, on a subtle negotiation between professional and patient about what each wants and what each can deliver.1 But how will clinicians and health policy makers react to patients who want the least effective treatment, which may also be …

This article investigates how accurately experts (underwriters) and lay persons (university students) judge the risks posed by life-threatening events Only one prior study (Slovic, Fischhoff, & Lichtenstein, 1985) has previously investigated the veracity of expert versus lay judgments of the magnitude of risk. In that study, a heterogeneous groupin...

This article evaluates the nine empirical studies that have been conducted on expert versus lay judgments of risk. Contrary to received wisdom, this study finds that there is little empirical evidence for the propositions (1) that experts judge risk differently from members of the public or (2) that experts are more veridical in their risk assessme...

Expert opinion is often necessary in forecasting tasks because of a lack of appropriate or available information for using statistical procedures. But how does one get the best forecast from experts? One solution is to use a structured group technique, such as Delphi, for eliciting and combining expert judgments. In using the Delphi technique, one...

There are a variety of practical and ethical reasons for policy making bodies to involve lay people in decision making on issues in which the public has a stake. Political theorists and ethicists discuss concepts such as democracy, procedural justice, and human rights, in providing the moral basis for involvement; but in a practical and expedient s...

The 'psychometric approach' to the study of public perceptions of hazards has largely been limited to describing and exploring patterns of perception using cross-sectional survey data. This article discusses a number of methodological issues that need to be considered prior to employing this approach in studies monitoring changes in perceptions, th...

Public understanding of risks is likely to be informed by the media. We report a cross-national study looking at how newspapers in Sweden and the United Kingdom characterize a variety of risks, focusing on two months around the 10th anniversary of the Chernobyl accident. Approximately four times as many reports about risks were found in Sweden as i...