Reflections on uncertainty communication: decision-relevant information

Abstract

Successful emergency management decision-making during natural hazard events is fundamentally dependent upon individual and team situation awareness (SA, i.e., how selection, interpretation, and understanding of available information defines the problem and identifies solutions) whilst operating under high time and risk pressures. The development and evolution of SA, and response effectiveness during a crisis, depends upon information and advice from external experts. This advice is characterised by stochastic (system variability) and epistemic (lack of knowledge) uncertainty, constraining decision-making and blocking or delaying action. How this uncertainty is communicated, and managed, varies throughout the phases of emergency management. Through this 'Insight' paper, we review how people cope with uncertainty, individual and team factors that affect uncertainty communication, and inter-agency methods to enhance communication. We propose communicators move from a one-way dissemination of advice, towards two-way and participatory approaches that identify decision-relevant uncertainty and data information needs pre-event, identify what communication efforts should focus on during a crisis, and thus enhance situation awareness and data sharing.

Full text

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Reflections on the communication of
uncertainty: developing decision-
relevant information
Emma E Hudson-Doyle
Joint Centre for Disaster Research,
Massey University,
Wellington, NZ
e.e.hudson-doyle@massey.ac.nz
Douglas Paton
College of Health and Human Sciences,
Charles Darwin University,
Perth, Australia
douglas.paton@cdu.edu.au
David Johnston
Joint Centre for Disaster Research,
Massey University, Wellington, NZ
GNS Science, Avalon, NZ
David.Johnston@gns.cri.nz
ABSTRACT
Successful emergency management decision-making during natural hazard events is fundamentally dependent
upon individual and team situation awareness (SA, i.e., how selection, interpretation, and understanding of
available information defines the problem and identifies solutions) whilst operating under high time and risk
pressures. The development and evolution of SA, and response effectiveness during a crisis, depends upon
information and advice from external experts. This advice is characterised by stochastic (system variability) and
epistemic (lack of knowledge) uncertainty, constraining decision-making and blocking or delaying action. How
this uncertainty is communicated, and managed, varies throughout the phases of emergency management.
Through this ‘Insight’ paper, we review how people cope with uncertainty, individual and team factors that affect
uncertainty communication, and inter-agency methods to enhance communication. We propose communicators
move from a one-way dissemination of advice, towards two-way and participatory approaches that identify
decision-relevant uncertainty and data information needs pre-event, identify what communication efforts should
focus on during a crisis, and thus enhance situation awareness and data sharing.
Keywords
Uncertainty, decision-relevance, science advice, participatory, co-production.
1. INTRODUCTION
Throughout the response and recovery to a natural hazard event, it is vital that the multiple agencies
involved in the response have effective communication processes and protocols to enhance the sharing of data
and information, and to respond and recover effectively. We will argue that effective communication within event
is fundamentally dependent upon successful pre-event activities that enhance individual and team understanding
of each other’s needs, responsibilities, and roles, such that communicated data and information can meet the needs
of decision makers in time and space (Borodzicz and van Haperen 2002, Cannon-Bowers and Bell 1997, Crego
and Harris 2002, Endsley 1997, Martin et al. 1997, Owen et al. 2013, Paton, Smith and Violanti 2000, Pliske et
al. 2001). However, the presence of uncertainty can mediate and limit this process, both in terms of the technical
or scientific uncertainty inherent to the data, risk assessment, or models, and due to uncertainty about the
communication environment. In New Zealand, a number of significant recent natural hazard events have
highlighted the need to identify how best to communicate scientific and technical uncertainty to enhance
stakeholder and individual decision making, before, during, and after a crisis (e.g. Becker et al. 2015, Jolly and

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Cronin 2014, Wein et al. 2016). This includes identifying how best to communicate scientific, technical, data, and
information uncertainty (e.g. in model representation, model and parameter choice), and how to manage the
uncertainty in the communication environment (e.g., missing, ambiguous, contradictory information arriving in
rapidly evolving situations). To achieve this, we first need to understand how a range of stakeholders react to,
conceptualise, and cope with uncertainty, as well as their decision uncertainty needs... This includes information
and decision management systems being designed to accommodate how different stakeholder groups have
different information needs and interpret information in diverse ways while functioning in the context of a given
event. In this ‘Insight’ paper, we reflect upon key issues from the recent literature on uncertainty, as a basis for
outlining our recommendation for a move towards more participatory approaches to the identification and
communication of data uncertainty. We first outline the nature of uncertainty (section 1.1) by briefly reviewing
the wide range of ‘uncertainties’ that exist in a natural hazard multi-agency response, ranging from uncertainties
in behaviours through to the many layers of uncertainty in the information. This is followed in section 2 with a
brief review of how people cope with uncertainty via different decision-making processes (section 2), and the role
of mental models in those processes (section 2.1), summarizing how the development of effective shared mental
models can improve situational awareness and reduce communication uncertainties. We also highlight how an
individual’s mental model affects their interpretation of scientific communications, and how their ‘model of
science’ thus impacts their understanding of scientific uncertainty. We then briefly discuss the different languages
used to describe uncertainty (section 3), and processes that can be adopted to help identify and categorise
uncertainty as well as reduce the ‘linguistic uncertainty’ inherent to the language we use to describe our data.
Finally, in section 4, we review additional lessons from a recent systematic literature review which focused on the
particularly challenging issue of communicating the technical uncertainty inherent to numerical and risk
modelling (Doyle et al. 2018). This review identified several key thematic areas in the literature that provide
lessons for effective uncertainty communication. We reflect here on three of these themes: the role of different
epistemologies and disciplinary cultures (section 4.1), the role of trust (section 4.2), and the role of participatory
approaches pre-event to enhance communication (section 4.3). We focus on these due to the important role they
play in enhancing inter-, and intra-, team situational awareness, and the role they play in improving the
understanding of scientist’s and decision-maker’s needs, capabilities and resources, all of which contribute to
more effective data sharing. In section 5, we conclude that such participatory approaches can help identify
decision-relevant information, and thus contribute to the development of effective communication protocols ahead
of events. Developing such competencies pre-event thus enhances multi-agency response communications in-
event (see Doyle and Paton 2017, Owen et al. 2013). Communicators must move from a one-way dissemination
of advice and uncertainty, towards these two-way and participatory approaches that seek to identify the decision-
relevant information and uncertainties, and seeks to focus communication efforts upon those needs. This is
particularly important in contexts where information needs and uses evolve over time.
1.1 The nature of uncertainty: in actions, in information.
The effective management of complex emergencies is an activity that transcends the capability of any
one organization or agency. This makes inter-agency communication an important component of an effective
response. Complicating factors in this context include a need to understand how the nature, quality and utility of
inter-agency communication is affected by agencies having different roles. Thus information and decision needs,
and the fact that all agencies need to manage evolving events. This is further complicated by event characteristics
and implications which change over time in an overarching climate of uncertainty. The latter point introduces a
need to understand uncertainty and its implications. This itself is a challenging activity as uncertainty is not a
homogenous construct.
Considering uncertainty at the individual level, Kuhlthau (1993, as cited in Sonnenwald and Pierce 2000,
p. 463) consider uncertainty to be a cognitive state that causes anxiety and stress. While this is an accurate
description of the experience for many in everyday life, it is less applicable for emergency management
professionals. Rather than imposing constraints on thinking and action, when experienced by trained personnel,
anxiety and stress can actually facilitate and empower action (Flin 1996, Paton and Flin 1999, Paton and McClure
2013) and act as a motivational force (Smithson, 2008). For crisis decision makers, recognition of the latter leads
to a need to focus more on situational uncertainty. In this context, Lipshitz and Strauss (1997, p. 150) define
uncertainty as “a sense of doubt that blocks or delays action”, that can be “classified according to the issue (i.e.,
what the decision maker is uncertain about) and source (i.e., what causes this uncertainty).” Sources include a)
“incomplete information”, b) “inadequate understanding”, and c) “undifferentiated alternatives”. Schmitt & Klein
(1996, as cited in Klein 1998) add to this list and discuss how sources of uncertainty can emanate from 1) missing
information; 2) unreliable information; 3) ambiguous or conflicting information; or 4) complex information. These
uncertainties can occur on the level of the data, the level of knowledge, and people’s level of understanding (Klein
1998) (see review in Doyle et al. 2018).

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Even when having motivating qualities, situational uncertainty introduces delays in action for several
reasons, including decision makers being unable to a) judge if the situation is typical; b) prioritize relevant cues
to inform the decision; c) form expectancies; d) develop a vision for plausible goals; and e) being uncertain
regarding which action to take. Each of these contributes in different ways to discomfort, fear and doubt (Klein
1998, p. 278). If these exceed the capacity (training, experience etc.) of the decision maker, decisions may not be
made, or if made, may not be enacted in timely and effective ways (Paton and Flin 1999). However, given that
these are inevitable consequences of working on mass emergency and disaster settings, it becomes important to
understand how such conditions can affect performance and to use the knowledge of those disaster-setting
characteristics to inform the development of capability. The use of simulations that develop both situational
understanding and stress and coping capability are especially valuable in this context (Paton and Flin, 1999). It is
also important to appreciate that inter-agency work contributes additional kinds and sources of uncertainties (that
vary from one stakeholder to another), and thus how uncertainty needs to be managed within the inter-agency
context. For example, uncertainty affects both the scientific advisors in their decisions regarding communication,
as well as the decisions made by the emergency managers that depend upon the advice they receive from scientific
advisors. Hence, it becomes important to appreciate the cascading properties of uncertainty as these can be
magnified through sequential and iterative information search, evaluation and decision processes. A significant
contribution to this derives from the fact that uncertainty in the source (and interpretation) of information varies
between these groups. For instance, advisors deal with uncertainty in their assessments of hazard processes and
predictions of future activity, whereas emergency managers face uncertainty in applying data with implicit
uncertainty to complex response management settings (e.g., regarding where to deploy resources, and knowing
that decisions change their future options as resources cannot readily be committed to other locations) (Paton and
McClure 2013). Uncertainty related to communication and information is thus both a source of uncertainty itself
as to what to communicate, as well as having sources of uncertainty within it, and being a cause of future
uncertainty (as reviewed in Doyle et al. 2018). Uncertainties can also arise between advisors and managers from
differences due to: a) discipline and training (including the degree to which processes such as cross training has
been part of inter-agency team development), and b) how they relate to their roles and responsibilities (e.g., those
from bureaucratic agencies tend to become more rigid in their adherence to rules and procedures when crises
occur). These differences, and their different frames of reference, their different information management and
processing approaches, and the application of different mental models of their situation (discussed further in
section 2.1), all act to create further uncertainties that must be understood and managed (Paton and McClure
2013). Science advice about natural hazards is often subject to many levels of uncertainty, due to the natural
stochastic uncertainty (the variability of the system), and the epistemic uncertainty (lack of knowledge) (van
Asselt 2000, Patt and Dessai 2005). Uncertainty has not always been given the prominence in conceptualising
information and decision management (both within agencies/discipline and between them) it deserves. The value
of elevating its importance has being reinforced by, for example, Becker et al. (2015) and Wein et al. (2016)
calling for the understanding of the diverse, evolving and specific decision-making needs of scientific information
that emerged in information users after the Christchurch earthquake sequence. This paper seeks to redress this
imbalance. It follows from the above discussion that to assist in effective decision making under uncertainty,
communicators should first understand “the various dimensions of uncertainty [to help] in identifying,
articulating, and prioritising critical uncertainties, which is a crucial step to more adequate acknowledgment and
treatment of uncertainty in decision support endeavours” (Walker et al. 2003, p. 5). To encompass these issues,
this paper adopts the term “uncertainty communication” to describe its area of interest.
2 COPING WITH UNCERTAINTY: DECISION MAKING, MENTAL MODELS AND INTER-AGENCY COMMUNICATION
As discussed by Patt (2009) a particularly challenging issue for uncertainty communications is the wide
range of uncertainty decision making models that exist, including economic, psychological and political models.
It is beyond our scope to review all these here, but focusing on the psychological literature (see reviews in Doyle,
McClure, Paton, et al. 2014, Doyle and Paton 2017), the theory of two “parallel processing systems” (Chaiken
and Trope 1999, Epstein 1994, Sloman 1996, Slovic et al. 2004) describe the decision making processes that occur
at an individual level. Type 1 is an affective process involving rapid, unconscious, action-oriented processing,
where people interpret risk as an emotional state or feeling (e.g., fear, dread, anxiety; Doyle, McClure, Paton, et
al. 2014, Epstein 1994, Loewenstein et al. 2001, Slovic et al. 2004, Smithson 2008). This can be “intervened by
distinctive higher order” Type 2 processes (Evans and Stanovich 2013a), or analytical processing systems
(Epstein 1994), which utilise hypothetical thinking, and more deliberate computational cognitive processes by
using rules to analyse the data and justify actions. Decisions made in a crisis can involve a complex balancing act
between these two processes, influenced by the degree of uncertainty or threat and an individual’s roles,
responsibilities and training (Doyle, McClure, Paton, et al. 2014, Evans and Stanovich 2013a, b, Keren 2013,

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Keren and Schul 2009, Kruglanski and Gigerenzer 2011, Loewenstein et al. 2001, Osman 2013, Thompson 2013).
Emergency response conditions are characterised by ill-structured problems; uncertain dynamic
environments; shifting, ill-defined, contradictory or competing goals; action/feedback loops; time stress; high
stakes; multiple players; and influences from organizational goals and norms (Crichton and Flin 2001, Doyle and
Johnston 2011, Klein 2008, Zsambok 1997). These are not ‘ideal’ for Type 2 processing (Flin 1996, p. 141–142,
Saaty 2008). Instead ‘naturalistic’ decision making processes have been identified as being able to go some way
towards facilitating functional decision making in such dynamic settings (Crego and Spinks 1997, Crichton and
Flin 2002, Pascual and Henderson 1997). These include 1) recognition primed and intuition led action; 2) action
based on procedures; 3) analytical comparison of options; and 4) creative designing of a novel course of action.
A decision maker can move through this spectrum (ordered in terms of decreasing pressure and increasing time),
depending on the task and specific demands that present at a particular stage in events characterized by evolving
conditions (Doyle and Johnston 2011, Martin et al. 1997, p. 283), as well as a responder’s strategic analytic or
tactical coordinating response level and decision needs (Paton et al. 1998, 1999, Paton and Flin 1999).
Because they inevitably involve inter-agency and inter-disciplinary collaboration, these processes
depend fundamentally upon individual and team Situation Awareness (SA; Endsley 1997, Martin et al. 1997).
Situational awareness encompasses 1) the perception of the problem elements in time and space; 2) the
comprehension of the current situation (in relation to agency/team goals) and 3) the projection of the future status.
The development of effective SA, and the projection of future status, is particularly important when considering
uncertainty, as uncertainty often evolves and grows in to the future, especially in emergency management
situations. Thus, the soliciting of advice and opinions of experts is often a vital part of these decision processes.
This may be enhanced by the development of communication protocols and techniques to identify decision
information needs ahead of time, with a particular focus on identifying which uncertainties will have the greatest
impact upon those decisions (discussed further in section 4.3).
Lipshitz & Strauss (1997, p. 160) found that decision makers “Reduce, Acknowledge or Suppress
uncertainty depending on its nature or quality”. However, this assumes a ‘rational’ or analytical decision maker,
and does not incorporate the uncertainties introduced by individual interpretative processes, biases and
interactions, or the role of more implicit or experiential modes of thinking (Epstein 1994). As discussed by Eiser
et al. (2012), there is a need to move beyond ‘rational choice’ models in natural hazards, and consider how people’s
interpretations of risk reflect their experience, feelings and bias. This is supported by the research of Doyle et al.
(2014) and McClure et al. (2014), who discussed how people’s interpretations of forecast likelihood statements
were found to be not ‘rational’ as they are influenced by prior experience and knowledge of phenomena as well
as cognitive biases. This line of inquiry makes it pertinent to consider how to capture these socio-psychological
processes and how they inform understanding of uncertainty communication and its functional implications. One
way of doing so is via the construct of mental models.
2.1 Mental models
Effective inter-agency and intra-team communication depends upon a good shared mental model,
particularly when high levels of uncertainty about evolving events exist (Doyle et al. 2018, Paton and McClure
2013). A mental model describes how someone understands an issue, any dependencies, and causal beliefs
(Bostrom et al. 2008), and how they impose meaning on uncertain and unpredictable events to make decisions
(Paton and McClure 2013). They can include a mental ‘map’ of a response operating environment (Flin 1996,
Paton and Jackson 2002, Rogalski and Samurçay 1993), with such maps encompassing the needs, responsibilities,
roles, dependencies and demands of other members of the response. They can also encompass an individual’s
prioritisation of different information and weighting of hazard attributes upon their management of risk; including
socio-political and economic criteria and demands (Paton and McClure 2013). An effective shared mental model
reduces the uncertainty about what to communicate, as a decision maker’s needs are anticipated and understood,
through a shared understanding of the task at hand (Lipshitz et al. 2001, Orasanu 1994, Pollock et al. 2003, Salas
et al. 1994). What this entails is depicted in Figures 1 and 2.
As illustrated in Figure 1(a), a wide range of mental models can exist, including those about the hazard,
the communication network, and an individual’s or organisation’s responsibilities and needs, and so on. These
mental models affect the interpretation of information, which can also be mediated by several other cognitive,
social, psychological, environmental, and experiential factors (such as those illustrated in Figure 2). In the
absence of effective cross training or team development, the situation depicted in Figure 1 (b) prevails. People
function in adjacent roles, but each brings their own content (from Figure 1(a)) to the role. Hence there is no
coherence or inter-agency/disciplinary functioning. This poor shared mental model can result in uncertainties not
just in what to communicate, but also how and who to communicate too, and often requires explicit requests for
information in response (Crichton and Flin 2002, Klein 1997, Paton and Jackson 2002), The goal of activities
such as cross training or team building is to create the relationships depicted in Figure 1(c), where each agency
representative has their own role to play, but (as depicted by the overlap), they do so in a more coherent way and

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
one that creates a “whole is greater than the sum of its parts” approach to functional response. This has several
practical implications. A good shared mental model enhances levels of understanding, creating an environment
for more effective data and information sharing, and improving situational awareness by increasing understanding
of each other needs in both time and space (Endsley 1997, Martin et al. 1997).
Figure 1: a) Examples of the various mental models within an individual’s over-arching or super-ordinate mental
model during a volcanic crisis. b) A poor shared mental model between individuals. C) A good shared mental model
(Doyle and Paton 2017).
Figure 2: The factors that affect the interpretation of forecasts, and the influences on resultant decision
making (Doyle, McClure, Paton, et al. 2014).
Effective communication occurs when an individual continues to use their ‘expert’ mental models in
complement with the over-arching or superordinate mental model. They then integrate their individual mental
models of their role, how they relate to others within their organization, and how their organisation fits within the
wider response (Doyle and Paton 2017, Owen et al. 2013). This illustrates how stakeholders do not need to have
identical understanding and roles. Rather they need to have shared understanding of, for example, how their
respective knowledge, experience and skills complement others in ways that ensure that the emergent shared

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
experience of an event increases the likelihood of their acting in coherent and complementary ways. A “team”
whose structure reflects Figure 1(b) would function less effectively than a team whose functioning reflected a
Figure 1(c) structure. Understanding this notion of coherence has further implications.
An additional level of uncertainty can arise in inter-agency communications through the mental models
gulf (Morgan et al. 2001). This describes a gap between “what experts know and the plan they develop, versus
what key public know and prefer” (Heath et al. 2009, p. 129), see also Doyle et al. (2014, Doyle and Paton 2017).
However, if response teams (including advisors) have developed an effective shared mental model (via training
and other techniques ahead of time; see section 4.3), then communication in an event can move from resource
intensive explicit requests for information (Crichton and Flin 2002, Klein 1997), to implicit supply of advice as
they anticipate information that others need (Kowalski-Trakofler et al. 2003, Lipshitz et al. 2001, Paton and Flin
1999). This also allows decision makers to focus on task management and reduces uncertainty around missing or
requested information. As reviewed in Doyle et al. (2015) the quality of these shared mental models can be
improved through both training and effective team-based simulations (Borodzicz and van Haperen 2002, Cannon-
Bowers and Bell 1997, Crego and Spinks 1997, Paton, Smith and Violanti 2000, Pliske et al. 2001), as well as
from the analysis of past responses. An implicit communication style dominates in effective teams (Kowalski-
Trakofler et al. 2003, Lipshitz et al. 2001, Paton and Flin 1999, Paton and Jackson 2002), and in a science advice
situation ensures information is useful, useable and used (Aitsi-Selmi, Blanchard, et al. 2016, Rovins et al. 2014).
Advisors must recognise the specific needs of decision makers prior to an event and establish procedures to
provide that information within the event’s organisational structure (Doyle et al. 2015, Doyle and Paton 2017,
Owen et al. 2013). This is particularly important in distributed decision making scenarios (Paton and Jackson
2002, Rogalski 1999, Rogalski and Samurçay 1993).
As reviewed in Doyle et al. (2018), when communicating information that includes uncertainty, an
individual’s model of the world and science affects their perceptions of the scientific uncertainty because
individuals interpret information based upon their ‘science model’ (Maxim and Mansier 2014), as illustrated in
Figure 2. For example, Budescu et al. (2012) found that interpretations of uncertain verbal statements varied
depending upon ideologies and beliefs in climate change, while Tak et al. (2015) concluded that in the absence of
any textual explanation of an uncertainty range, people will apply their own internal model of the uncertainty
distribution. People are not just influenced by the framing of a message, but also by their prior expectation of the
message (Rabinovich and Morton 2012). This also depends on whether they assume a classical model of science
(which considers science to be the ‘search for truth’), or a Kuhnian model of science (which considers ‘science as
debate’) (Kuhn 1962). In the latter, actions are less likely to be undeterred by uncertainty, and uncertainty may
actually increase motivations (Rabinovich and Morton 2012). People with a Kuhnian model of science are
perceived by Rabinovich & Morton (2012) to trust a message more if it includes uncertainty as it matches their
expectations. However, a classicist would distrust an uncertainty message as they search for absolutes. It is thus
important to assess and adapt communications depending upon audience beliefs (see section 4.1 and Doyle et al.
2018). Similarly we can seek to understand how individuals adapt and create opportunities to function in uncertain
and dynamic conditions, and how they rationalise their experience through different frames of reference and their
social models, via the fields of social constructionism and symbolic interactionism (Burr 1995, Falkheimer and
Heide 2006, Griffin 2006), where “crisis communication is understood and analyzed as a sense-making process
(e.g. Weick, 1979) where reality is negotiated and constructed in cultural contexts and situations, rather than
distributed from a sender to a recipient” (Falkheimer and Heide 2006, p. 180)
3 THE LANGUAGE OF UNCERTAINTY: TYPOLOGIES AND LINGUISTIC UNCERTAINTY
Klein (1998, p. 277) highlight that when uncertainty is discussed the language used is often muddled,
being discussed in terms of “risks, probabilities, confidence, ambiguity, inconsistency, instability, confusions, and
complexity”, including uncertainty about ‘future states, the nature of the situation, the consequences of action,
and preferences’. Doyle et al. (2018) highlight the wide range of schemes that exist in the literature to define and
classify uncertainties (Bammer et al. 2008, Bjerga et al. 2012, Blind and Refsgaard 2007, Daipha 2012, Grubler
et al. 2015, Handmer 2008, Janssen et al. 2005, Kloprogge et al. 2011, Kwakkel et al. 2010, van der Sluijs et al.
2011, Smithson et al. 2008, Stirling 2010, Walker et al. 2003). For example, Walker et al. (2003, p. 8) developed
a typology for uncertainty management in model-based decision support, which considers three overarching
categories: First, 1) the location of the uncertainty encompassing: a) the context of the model, b) the model
uncertainty, c) input variables uncertainty, d) parameter uncertainty, and e) accumulated model outcome
uncertainty. Next, 2) The level of uncertainty, which considers where the uncertainty sits along a scale from
determinism, statistical uncertainty, scenario uncertainty, recognised ignorance, indeterminacy, through to total
ignorance. Finally, 3) the nature of the uncertainty: whether it is epistemic and due to knowledge imperfection,
or whether it is a variability uncertainty (or ontological uncertainty) due to behavioural variability (micro), social
variability (micro and macro) and natural randomness.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
A key element of the relationship-development processes depicted in Figure 2 is the opportunity for
stakeholders to familiarize themselves with sources of uncertainty from others, and to take steps to explore
strategies to reduce it or accommodate it in their own planning and operations. For example, if communicators
clearly identify and define all types of uncertainty in technical science (Adler and Hirsch Hadorn 2014, Aven and
Renn 2015), they can ‘avoid misinterpretations of uncertainty characterizations’ (Adler and Hirsch Hadorn 2014,
p. 668) and provide advice that is more rigorous, robust and ‘democratically accountable' (Stirling 2010, p. 1029).
As reviewed in Doyle et al (2018), this is particularly important for model uncertainties as larger ‘deep’
uncertainties can form due to the interdependences between model assumptions and relationships (Walker et al.
2003). However, to communicate all uncertainties present can overwhelm a message and decision, as providing
as much advice as possible can hinder the decision process due to cognitive overload and an overuse of available
resources (Eppler and Mengis 2004, Omodei et al. 2005, Quarantelli 1997, Schraagen and van de Ven 2011).
Thus, it is important to classify and identify the uncertainties present, such that those most relevant and important
to the decisions can be identified and be the focus of communication efforts, and such that stakeholders can
accommodate this in their own planning and operations.
Such typology and classification schemes are fundamentally dependent upon a shared understanding of
the definitions and words to reduce linguistic uncertainty (Elith et al. 2002, Griethe and Schumann 2005, Grubler
et al. 2015, Kloprogge et al. 2007, Leyk et al. 2005, Moss 2011, Van Steenbergen et al. 2012). Shared classification
schemes and prioritization of uncertainties can be developed through participatory and engagement techniques
ahead of time, discussed in section 4.3. Any standardisation of the language and methods used to represent and
communicate uncertainty, must remember organizational, disciplinary, context, and individual differences in
understanding that will affect the appropriate terminology to use (Briggs et al. 2012, Budescu et al. 2009, Han
2013, Mastrandrea and Field 2010, Moss and Schneider 2000, Patt and Dessai 2005). This is discussed further in
section 4.1.
Beyond the appropriate classification schemes for uncertainty, and the linguistic uncertainty regarding
the classification terms between disciplines, issues also arise regarding the perceptions of the language used to
communicate uncertainty, such as probabilities, both at an individual and discipline level (Doyle, McClure,
Johnston, et al. 2014, Paton and McClure 2013). For example, it has become increasingly popular to use
probability statements in volcanic crisis communications (Doyle, McClure, Johnston, et al. 2014), which involve
knowledge of both the dynamical phenomena and the uncertainties involved (Sparks 2003). These probabilistic
statements, whether in numeric or linguistic formats, can be misinterpreted due to their framing (e.g., discipline
or decision making goals etc.), directionality (increasing or decreasing, etc.) and probabilistic format (numbers or
words, time frames used, etc.). This can bias people’s understanding, thereby affecting people’s action choices
(Barclay et al. 2008, Budescu et al. 2009, Cronin 2008, Doyle, McClure, Johnston, et al. 2014, Doyle, McClure,
Paton, et al. 2014, Haynes et al. 2008, Honda and Yamagishi 2006, Joslyn et al. 2009, Karelitz and Budescu 2004,
Lipkus 2010, McGuire et al. 2009, Solana et al. 2008, Teigen and Brun 1999). For example, Doyle et al. (2014),
found that by simply changing a forecast statement from a “chance of an eruption in the next 10 years” to “within
the next 10 years” significantly affected participants’ perceptions of when in that time window an event was likely
to occur. In addition, Doyle et al. (2011a) found that scientists and non-scientists differ in translations of verbal
likelihood phrases into numerical equivalents, supporting the development of ‘translation’ schemes that consider
the instinctive interpretations of the target audience, a practice now adopted by NZ’s geological monitoring
agency, GeoNet (Doyle and Potter 2015).
4 FURTHER LESSONS FROM THE LITERATURE
Beyond the nature, decision-making and language of uncertainty, further lessons can be drawn from the
literature regarding how different perspectives of uncertainty affect inter-agency science communication. Through
a recent systematic meta-synthesis literature review of 111 publications across a wide range of disciplines
including psychology, policy, communication, law, climate change, health, geosciences, meteorology, risk
analysis, and environmental management, Doyle et al. (2018) identified a number of themes fundamental to
effective communication of model related uncertainty. This systematic review followed similar approaches to
Johnson et al. (2014) which involved the 1) identification of key search terms based on questions related to
communicating model uncertainty, 2) peer review of those search terms, 3) a SCOPUS database search using
those terms, which returned 1,131 documents, 4) reading the abstracts of these documents and including and
excluding them depending upon their relevance and scope, resulting in 85 documents, 5) inclusion of additional
secondary documents of relevance found through the lead author’s existing Mendeley database, PsychInfo, and
other articles recommended by expert colleagues. This resulted in the final 111 documents, as fully described in
Doyle et al. (2018). Unlike a traditional systematic literature review, rather than describe the characteristics and
distribution of the literature (as of e.g., Connolly et al. 2012), the focus was rather to identify the key themes and
emerging lessons in the literature similar to that of a meta-synthesis (as described by Cronin et al. 2008), where
the literature is analysed thematically in a manner similar to qualitative data (Braun and Clarke 2006).

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Through this process, the following thematic areas were identified in the 111 documents: a) the need for
clear typologies to identify and communicate uncertainties, b) the need for effective engagement with users to
identify which uncertainties to focus analysis and communication resources upon, c) how to manage challenging
uncertainties such as ensembles, confidence, bias, consensus and dissensus, d) methods for communicating
specific uncertainties, and e) the lack of evaluation of many techniques and approaches currently in use. We herein
focus on the epistemology, trust, and participatory approach themes, and reflect upon these lessons for uncertainty
communication. We focus on these themes here due to the fundamentally important role they play in enhancing
inter-, and intra-, team situational awareness and the development of a mutual understanding of scientist’s and
decision maker’s needs, capabilities, and resources. Which are important competencies for effective multi-agency
response (as discussed in section 2; see Doyle and Paton 2017, Owen et al. 2013). In particular, pre-event
participatory approaches to the generation of science data and communication protocols can improve the
effectiveness of data sharing and multi-agency team situation awareness, as it can lay the basis for implicit
communication styles where the scientists “recognize and understand the needs of the decision makers, [and] their
timelines and thresholds” (Doyle and Paton 2017, p. 8), while encouraging shared ownership of uncertainty (as
discussed in section 4.3).
4.1 Different perspectives of uncertainty: epistemologies, disciplines, and thematic contexts.
As discussed in Section 2.1, the different ‘science mental models’ that people hold will affect their
interpretation of and action related to uncertainty (Kuhn 1962). These different science models closely relate to
the different epistemic cultures (the existence of diverse ways of knowing derived from unique disciplinary
histories, interests, and goals) present in different disciplines, thematic contexts, and organisations (as reviewed
in Doyle et al. 2018). As described by Knorr Cetina (2013) epistemic culture describes the factors that capture the
interiorized processes of knowledge creation, such as what is ‘objective’ and a ‘true’ representation of the world.
It describes the set of “practices, arrangements and mechanisms bound together by necessity, affinity and
historical coincidence which, in a given area of professional expertise, make up how we know what we know”
(ibid, p. 4). It is important to incorporate, acknowledge, and account for different epistemic cultural differences
represented by the different disciplines and professions involved in a multi-agency uncertainty classification,
communication, and shared management scheme (Doyle et al. 2018, Murphy et al. 2011). Thus, the different
uncertainty-decision making tolerances can be accounted for (Demeritt et al. 2007). For example, scientists will
often aim to reduce epistemic uncertainty, while engineers often accept uncertainty as being core to innovation,
invention, and engineering solutions (Murphy et al. 2011). Often how this uncertainty is communicated relates to
the different ethical standards across disciplines such as science, engineering, law, journalism, etc. (Austin et al.
2015). To address how to manage and communicate uncertainty, many disciplines (e.g. in climate change,
meteorology, and volcanology: Gill et al. 2008, IAVCEI Subcomittee for Crisis Protocols et al. 1999, Mastrandrea
and Field 2010, Moss and Schneider 2000) have established guidelines that advocate for the clear and transparent
communication of uncertainty, a documentation of all processes related to uncertainty, and the use of formalised
probabilistic terms and frameworks for assessment and communication. However, these approaches often do not
account for the differences in how scientists and non-scientists construe the information they develop and
disseminate, and how they process information (Chaiken and Trope 1999, Epstein 1994, Sloman 1996, Slovic et
al. 2004). See the above discussion in section 2 on affective and analytical processes. In addition, they do not
consider how non-scientist, emergency managers are being influenced by their awareness of the social, economic,
and political dimensions of their decision making (e.g., the implicit need to manage response decisions with
economic or political views that are imposed upon emergency managers; see review in Paton and McClure
(2013)). Experimental research has also identified how these epistemological and organizational differences
influence both the perception and response to uncertain information (Budescu et al. 2012, Doyle, McClure,
Johnston, et al. 2014, Doyle, McClure, Paton, et al. 2014, Maxim and Mansier 2014, Rabinovich and Morton
2012, Tak et al. 2015). For example, in an analysis of actions taken in response to a hypothetical probabilistic
volcanic eruption forecast, Doyle, McClure, Paton, et al. (2014) found that scientists tended to query the scientific
advice and express dissatisfaction about the quality and lack of information as reasons for not evacuating. They
found that non-scientists tended to express a desire to wait for more information, while still stating their intention
to start the process of preparing for evacuation. This illustrates how the scientists were reflecting their core
business of data collection, while the non-scientists were reflecting their role as recipients of information and as
passive planners about what could happen. In addition, even though information was very limited, more non-
scientists chose evacuation compared to scientists, focusing on actions they would take, and acknowledging or
suppressing the uncertainty in order to make their decisions (Lipshitz and Strauss 1997). Meanwhile scientists
focused more on the information and the quality of that information, choosing to reduce the uncertainty in the
source (information) before proceeding.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
These findings highlight that a system of shared uncertainty management between scientists and decision
makers both during emergency planning and analysis processes, during mitigation decisions, and in response and
recovery must acknowledge and account for different discipline and organisational language and epistemological
differences. In particular, Grubler et al. (2015) has found that epistemic differences can fundamentally affect the
initial problem formulation, due to linguistic uncertainty and a lack of common language. In addition, different
systems of scientific enquiry, ranging from analytic, empirical, synthetic, or conflictual models lead to a different
emphasis in this initial problem formulation. Thus, a shared uncertainty communication and management scheme
should adopt a pluralistic approach which acknowledges these differences, and then provides a basis for a more
equal partnership between social and natural science (Doyle et al. 2018, Stirling 2010). Developing such a
pluralistic approach depends upon an effective engagement and participatory approach between scientists and
decision makers ahead of time, discussed further in section 4.3. Before discussing the concept of participatory
approaches further, we first discuss the role of trust, due to its fundamental role in the maintenance and
development of the relationships required for effective engagement, particularly when operating in circumstances
where uncertainty prevails.
4.2 Trust
Effective multiple agency communication in a crisis is heavily dependent upon trust (see reviews in
Doyle and Paton 2017, McIvor and Paton 2007, Paton et al. 2008). In environments of high uncertainty the quality
of interpersonal trust is essential for collective action (Garnett and Kouzmin 2007, Pollock et al. 2003, Siegrist
and Cvetkovich 2000). Trust is most important under conditions of high uncertainty, and can be further challenged
when recipients in situations of high pressure, low time, and limited resources, are totally reliant on advisors with
whom they may rarely interact with under normal circumstances (Doyle and Paton 2017, Haynes et al. 2007). As
reviewed by Doyle and Paton (2017), it is reassuring to know that some research has shown that the
communication of uncertainty can enhance the credibility and trustworthiness of the information provider
(Johnson and Slovic 1995, 1998, Miles and Frewer 2003, Smithson 1999, Wiedemann et al. 2008), making the
provider seem more ‘honest’ (Johnson 2003). However, other research has found that communicating uncertainty
decreases people’s trust in, and credibility of, the provider (Johnson and Slovic 1995, 1998, Miles and Frewer
2003, Smithson 1999, Wiedemann et al. 2008), enabling people to justify inaction or their own agenda, or to
perceive higher or lower risks than exist. This effectively extends the discussion of epistemic culture to
stakeholders outside the scientific arena. This decline in trust can reflect a lack of understanding of how the ways
of knowing are derived from a unique disciplinary history, interests, and goals, and how that affects the
communication process and can result in conflict and distrust due to recipients failing to appreciate that a source
has different priorities and interests and capabilities. Several other factors affect whether uncertainty decreases or
increases trust, including the context, the relationship between provider and receiver, and past experiences (see
reviews in Doyle, McClure, Paton, et al. 2014, Doyle and Paton 2017).
Organisational cultures also influence trust. As reviewed by Doyle and Paton (2017, p. 315), scientists
usually work in flatter more organic organizational cultures in which information flow is common, which makes
it easier for them to share information and build trust. However, decision makers from government departments
and emergency service agencies experience higher levels of hierarchical relationships and are predisposed towards
maintaining their own agency-based independence (Dietz et al. 2010, Dirks and Ferrin 2001). Rivalry can thus
emerge between organizations, preventing effective information sharing (Iannella and Henricksen 2007, Marcus
et al. 2006, Marincioni 2007, Militello et al. 2007, Owen et al. 2013, Smircich and Smircich 2012, Waugh and
Streib 2006). This further impacts the development of trust required for effective collaboration between these
culturally diverse agencies (Banai and Reisel 1999, Mcknight et al. 1998, Siegrist and Cvetkovich 2000), and thus
effective information sharing (Kapucu 2006, Mohr and Spekman 1994).
Agencies who only come together for the first time to collaborate in a response setting may lack trust
(Dirks and Ferrin 2001). Functional collaboration, and shared experience designed to facilitate team building and
shared understanding of the respective goals, needs and contributions of each stakeholder, helps to develop trust.
Such pre-response activities, including training, planning and risk mitigation collaborations, and development of
pre-event communication protocols and uncertainty management schemes, contribute to the development of
shared mental models and enhanced functional relationships (see also section 4.3). In addition, if pre-event
training is not possible, trust can be developed in-event through the concept of swift trust (Curnin et al. 2015,
Faraj and Xiao 2006, Goodman and Goodman 1976, Hyllengren et al. 2011, Meyerson et al. 1996). As reviewed
in Doyle and Paton (2017), swift trust develops when team members have been assigned roles that align with the
response issues and the temporary work group’s needs (Curnin et al. 2015). It is also more likely to emerge if
members know that there is a high likelihood of future collaboration (in incident reviews, simulations), and if they
identify that success relates to the super-ordinate management as much as it does to how they contribute their
personal expertise (Kramer 1999).

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
4.3 Participatory, engagement, training, and other collaboration techniques
Throughout this review we have identified that a number of pre-event techniques for training and relationship
building can enhance response by improving individual and team situational awareness, shared mental models of
roles, responsibilities and needs, developing shared understanding of epistemological differences and perceptions
of uncertainty, developing trust, and helping to build a mutual understanding of uncertainty management via
communication protocols and the development of a common language (see reviews in Doyle and Johnston 2011,
Paton and Jackson 2002). According to Kozlowski (1998, p. 120–122), team training should be considered as a
sequence or series of developmental experiences that are carried out across a series of different environments, to
build “knowledge and skills in an appropriate sequence across skill levels, content and target levels”. Adopting a
suite of training activities increases opportunities for developing an understanding of the technical issues involved
and the multi-agency context in which they occur (Blickensderfer et al. 1998, Borodzicz and van Haperen 2002),
and we suggest that they also provide opportunities to develop shared understanding of the relevant uncertainties,
in both decisions and information (Doyle et al. 2018).
As reviewed in Doyle and Paton (2017), several training methods have been identified that can enhance
naturalistic decision-making (e.g., Cannon-Bowers and Bell 1997), enhance decision skills (e.g., Pliske et al.
2001), train effective teams (e.g., Salas et al. 1997), and develop effective critical incident and team-based
simulations (e.g., Crego and Spinks 1997, Flin 1996). These include cross training, positional rotation, scenario
planning, collaborative exercises and simulations, shared exercise writing tasks, and ‘train the trainer’ type tasks,
in addition to workshops, seminars and specific knowledge sharing activities (Bloom and Menefee 2014, Keough
and Shanahan 2008, Marks et al. 2002, Moats et al. 2008, Paton et al. 2015, Schaafstal et al. 2001, Volpe et al.
1996). For all of these the goal is not just knowledge and skill development but to also address “how the disaster
context influences performance and well-being” (Paton, Smith and Johnston 2000, p. 176). For example, scenario
planning and story boarding creates multiple scenarios of “different futures” that are “credible and yet uncertain”
(Keough and Shanahan 2008), in ways that accommodate the perspectives of multiple agencies. Thus response
scenarios are developed that more accurately reconcile the needs, goals, and expectations of diverse agencies
(Bloom and Menefee 2014, Moats et al. 2008, Paton et al. 2015). Exercises provide opportunities to practice
communications, contingent planning, and enhance team mental models, as well as providing opportunities for
scientific agencies to rehearse strategies to convey and include uncertainty (Doyle et al. 2011b, Doyle, McClure,
Johnston, et al. 2014, Doyle, McClure, Paton, et al. 2014).
Based on the issues identified herein we consider that tools such as scenario planning can also be used as
engagement techniques pre-event to help identify the science, information, data, and uncertainty response needs
pre-event, and to facilitate communication of uncertainty and science for preparedness activities. For example,
shared data uncertainty management approaches could draw on the methods for integrating stakeholder
perspectives described by Scolobig and Lilliestam (2016). These include the multi-criteria analysis decision
support method (which aims to identify the preferred solution), the plural rationality approaches (which seek
compromise rather than consensus on problem formulation), or the scenario construction approaches (which aim
to illustrate the impacts of different solutions).
We note that Scolobig and Lilliestam (2016) state that no approach is “better” than another (p. 1), rather “they
are suited for different problems and research aims”, and the choice of approach determines the type and depth of
stakeholder engagement. Further approaches to developing shared uncertainty management and advice
communication protocols can also be identified in the participatory, co-production, knowledge exchange, and
engagement literature (e.g., Beven and Alcock 2012, Clark et al. 2016, Linnerooth-Bayer et al. 2016, Page et al.
2016, Reyers et al. 2015, Scolobig et al. 2015, Scolobig and Pelling 2016, Wall et al. 2017). As reviewed in Doyle
et al 2018, a participatory approach to developing communication protocols involves “two-way communication
and the relationship between scientists and policy-makers” (Patt 2009, p. 231), that prioritises giving “decision-
makers enough information to know when they need to invest the time and resources to take part in a participatory
process, and when they do not” (Patt 2009, p. 246). Adopting a participatory approach can help make science
information more credible and legitimate (Patt and Dessai 2005), by moving away from a style where
communications are formatted to meet a specific model of stakeholder decision making, which vary considerably
(Patt 2009), and towards an approach where the decision makers themselves identify which science and
uncertainties are important for their needs. Using tools such as scenario planning in this process can further help
decision makers identify with scientists what decisions will be impacted by the uncertainties present, and through
this identify the decision relevant information and uncertainty that communication efforts should focus upon
(Scolobig and Lilliestam 2016).
Participatory approaches to identifying user-science uncertainty priorities, and a shared scheme for the
communication and management of uncertainty are also advocated for by Faulkner et al. (2007), Janssen et al.
(2005), and Beven et al. (2015). For example, Faulkner et al. (2007) outline a process to develop a classification
of uncertainty in flood risk management, which focuses on a ‘translational discourse’ to communicate and manage
uncertainty which “incorporates a conversation that maximizes the facilitation of the decision-making process”
(ibid, p. 698). Through this, they move away from the traditional ‘one-way’ communication of uncertainty, where

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
users and stakeholders have no ‘ownership’ of uncertainty. They also include ‘purposeful translations’ of the
information to meet the decision-making needs (see also Faulkner et al. 2014). Such an approach thus prioritises
the needs of the stakeholders in both the analysis and communication of scientific uncertainty (Fischhoff and
Davis 2014), and thus depends upon a partnership model between scientists and users to help identify those
decision-relevant needs.
Such communication approaches are thus closely related to the scientific co-production of knowledge (Clark
et al. 2016, Page et al. 2016, Reyers et al. 2015, Scolobig and Lilliestam 2016, Scolobig and Pelling 2016, Wall
et al. 2017). As described by Page et al. (2016) co-production of knowledge is a collaborative process between
multiple stakeholders, including researchers and decision-makers, and incorporates three phases: co-design of the
problem framing and the research design, co-development of the knowledge and the operationalization of research
methods, and co-dissemination of findings. Such co-production is “important to help ensure knowledge is
credible, salient and legitimate” (p. 86). However, it is not without its challenges. As outlined by Scolobig and
Pelling (2016), effective science-policy co-production is a “long process” (p. S22) and new science and new policy
developed in parallel does not automatically result in effective co-production. They outline how the “insulation
of science from the institutional context” results in problematic separation of knowledge production and its use
(p. S22). Alternatively, a shared uncertainty management scheme could draw from established knowledge
exchange approaches. For example, Beven and Alcock (2012) outline an approach by the UK Natural
Environment Research Council to bring academics and practitioners together to define guidelines for the
incorporation of risk and uncertainty in assessments.
These various approaches to assessing decision makers needs for science generation and communication can
be considered ‘pluralistic’ in the sense that they recognise the conditional nature of knowledge and different
priorities and perspectives (Demeritt et al. 2007, Morgan et al. 2001, Murphy et al. 2011), and “enhances
stakeholder deliberation by respecting legitimate differences in values and worldviews” (Linnerooth-Bayer et al.
2016, p. S69). As stated by Sword-Daniels et al. (2018, p. 298) “community-based disaster risk management and
other participatory approaches provide mechanisms by which to incorporate this plurality of perspectives into the
co-production of knowledge” (see also Williams and Dunn 2003; Cronin et al. 2004; Gaillard 2006; Cadag and
Gaillard 2014). They highlight that joint fact-finding techniques can be employed to help groups create a shared
vision and inform collective decision-making, even in situations with high degrees of uncertainty (Karl et al. 2007,
Schenk 2016).
This approach to uncertain problems is closely related to that of post-normal science (Funtowicz and Ravetz
1993, 1999, Janssen et al. 2005, Maxim and Mansier 2014, van der Sluijs et al. 2011) which acknowledges that
‘facts are uncertain, values are in dispute, the stakes are high and decisions urgent’ (Funtowicz and Ravetz 1993,
p. 744). As reviewed in Doyle et al. (2018), such an approach is reflective, and moves beyond just the quantitative
tools inherent to uncertainty analysis (e.g. sensitivity analysis or Monte-Carlo type simulations) to include
technical, methodological, epistemological and societal dimensions of uncertainty (van der Sluijs et al. 2011) and
the value-ladenness of assumptions in a risk or model assessment (Kloprogge et al. 2011). A post-normal science
approach recognizes that risks are interpreted and managed subjectively, proposing problem-solving frameworks
that account for this plurality of perspectives (see also Krauss et al. 2012), and can encompass the ‘social history
of uncertainty’ to solicit social science expertise in communications (Moss 2011, Patt 2007).
A pluralistic participatory approach acknowledges that people are not being irrational when they respond
differently to information and uncertainty, but rather they are influenced by different individual, social, and
cultural values (see also Eiser et al. 2012 and above discussion on mental models). A plural approach to
communication provides a basis for more equal partnership (Stirling 2010). It requires a high level of transparency
where what is communicated (in level of detail and level of quantification) depends on the needs of the decision-
maker (Loucks 2002). As discussed by Aitsi-Selmi, Murray, et al. (2016), current methods for risk assessment
and management are often variable and non-standardised which results in a “lack of transparency of understanding
of uncertainty” (p. 11). This hampers both the communication to decision makers and their use of scientific
outputs. They highlight that two-way coproduction of knowledge and participatory approaches are needed to
enhance the use and application of science. We note however, as reviewed in Doyle et al. (2018), that for
engagement and participatory approaches to work, it is vital that a code of practice and professional guidelines
are developed for engagement, which considers funding, leadership and ethical standards (Beven et al. 2015,
Faulkner et al. 2007, Janssen et al. 2005) which can vary significantly between different disciplines (Austin et al.
2015).

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
5 CONCLUSIONS
As models become more advanced, and complexity increases, the number of uncertainties present will also
increase (Maslin 2013). For example, as highlighted by Klein (1998, p. 279):
- “Sometimes it is tempting to believe that we can use information technology to eliminate certain types
of uncertainty. For example, an intelligent system could screen all messages to detect inconsistencies and
weed these out. This dream is unrealistic. The next generation of computers will not eliminate uncertainty
caused by inconsistencies.”
There is thus a need to move towards approaches that do not just consider the suppression and reduction of
uncertainty, but also acknowledge and incorporate uncertainty. This necessitates an increase in uncertainty
tolerance (Han 2013) amongst both decision makers and advisers, and a move away from the 1949 deficit model
of risk communication (see Daipha 2012, Markon and Lemyre 2013).
Doyle et al. (2014) highlights that contrary to the accepted anecdotal practice of not wanting to overwhelm
decision makers with uncertainty information, on the assumption that the uncertainties will damage the trust and
credibility of the decision maker, decision makers are often actually very comfortable with uncertainty (Doyle,
McClure, Paton, et al. 2014, Johnson 2003, Johnson and Slovic 1995, 1998, Miles and Frewer 2003, Smithson
1999, Wiedemann et al. 2008). As stated by one emergency management participant in research by Hudson-Doyle
et al. (2014) “uncertainty is endemic in crises, you just have to go with the best available information at the time”.
This relates to the Sword-Daniels (2018) concept of “embodied uncertainty”, which considers uncertainty as both
a conscious and subconscious lack of certainty. Such uncertainty is “differentially internalised, depending on past
experiences, social identities, beliefs, values, institutional structures, resources available, and social norms” (p.
296). By broadening this concept of uncertainty, Sword-Daniels advocate for a shift in thinking towards
“accepting (rather than reducing) uncertainty” (p. 296). As discussed by Klein (1998, p. 279), “skilled decision
makers appear to know when to wait and when to act. Most important, they accept the need to act despite
uncertainty.”
Through this Insight paper, we have reviewed several factors and key issues relating to uncertainty and how
that affects communication and data or information sharing throughout natural hazard events. We reviewed in
section 1.1 the wide range of uncertainties that exist in a response, from uncertainty in actions through to the
layers of uncertainties in data and models. We then outlined the literature on how people cope with uncertainty,
and how their mental models of the data, their communication network with other individuals, and their needs and
responsibilities, impact communications under uncertain situations (section 2) as well as how such mental models
and epistemologies affect their interpretation of information and judgment of uncertainty in science advice
(sections 2.1 and 4.1), and the role of trust in that process (section 4.2). We discussed the languages used and how
typologies can be used to identify and categorise uncertainty in science and data, and how it can reduce the
linguistic uncertainty (section 3). Finally, this was followed by a discussion of the importance of participatory
approaches (section 4.3) to enhance effective communication of science advice by providing opportunities to
identify decision-relevant information and uncertainty needs ahead of an event, such that in-event the data and
information shared can be targeted to meet those needs. This also helps identify which of the many uncertainties
in the data and models communicators should focus their communication efforts upon, which is of particular
importance in high-pressure, time-limited, response environments.
We have highlighted the very multi-faceted nature of uncertainty, and propose uncertainty communication
moves away from a one-way advisor led identification, analysis, and communication of uncertainties, towards
these participatory type approaches that develop shared uncertainty management schemes. The currently accepted
approach of a one-way ‘science-push’ communication of data, advice, and uncertainties, outlined by a number of
organisational guidelines, may ensure advisers are honest, open and transparent. However, it may also result in
the communication of information that is not useful, is unusable, and ultimately unused. It can result in decision
makers needing to make time and resource intensive explicit requests for information in a crisis to address their
decision needs (Kowalski-Trakofler et al. 2003, Lipshitz et al. 2001, Paton and Flin 1999), and can also result in
information overload which can overwhelm the decision making process (Eppler and Mengis 2004, Omodei et al.
2005, Quarantelli 1997, Schraagen and van de Ven 2011).
Thus, as depicted in Figure 3, we propose that communicators, scientists, and advisers must develop coherent
inter-agency/stakeholder relationships that include identifying the decision-relevant uncertainties, and related
information and data ahead of a crisis event via participatory engagement with decision makers to ensure that
communication efforts are focused upon the most relevant information required in the short time, high-pressure
situations characteristic of natural hazards. In the absence of this activity, the scenario depicted in Figure (3a) will
prevail. Requests for information will be ad hoc and based on agency/personnel reactions to events. Effective,
functional inter-agency/stakeholder relationship development on the other hand, lends itself to the development
of coherent shared models of relationships that have clarified respective information needs in ways that
accommodate uncertainties. This affords opportunities to develop the kinds of relationships depicted in Figure
3(b) which creates ways to proactively respond to complex events. Through this, the implicit supply of information

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
(and communication of only the decision makers’ perceived relevant uncertainties) will enhance both the situation
awareness and decision making of the response team. Pre-event two-way communication between decision-
makers and advisers can help develop a mutual understanding as to the relevant uncertainties that need to be
assessed and communicated for these decision needs (Blind and Refsgaard 2007). By communicating useful,
useable, and used information (Aitsi-Selmi, Blanchard, et al. 2016, Rovins et al. 2014), we ensure that our advice
abides by one of the core ethical principles of communication outlined by O’Neil (2002), that of “audience
relevance” (see also Keohane et al. 2014). In addition, such an approach helps to ensure research is ‘socially
responsible’ (Daedlow et al. 2016) in terms of societal goals and values, where the “transparent information and
involvement of stakeholders during the research process can mitigate uncertainties and risks and is a morally
responsible action” (p. 4). Such a partnership approach to identifying information needs also advocates for an
approach that considers decision makers throughout the entire science generation process, right from the initial
problem formulation, and not the traditional dissemination of science after results are identified. Thus, if during
the reduction and readiness phases of emergency management, advisors adopt a participatory approach with
decision-makers to identify, categorise and prioritise the uncertainties in their data, then their communication and
data provision capabilities will be enhanced during the higher pressure response and recovery efforts.
Finally, we conclude this ‘insight’ review by highlighting that for all uncertainty communications used in any
one-way or two-way process it is vital and imperative that they utilise communication tools, techniques,
languages, and images that have been evaluated or empirically tested wherever possible (Bostrom et al. 2015,
Briggs et al. 2012, Mastrandrea and Field 2010, Moss 2011). Unfortunately, as discussed by Doyle et al. (2018),
this is not always the case (Benke et al. 2011, Bonneau et al. 2014, Bostrom et al. 2008, Boukhelifa and Duke
2009, Brus and Svobodova 2012, Deitrick and Wentz 2015, Hope and Hunter 2007, Tak et al. 2015), and to do
otherwise may result in interpretations and actions that are significantly different to the intended and desired
communication.
Figure 3: The importance of shared mental models and shared uncertainty management and
communication schemes. a) When understanding of needs is weak, communication is dominated by
disseminated information and requires time and resource intensive explicit requests from the decision
makers for their needs to be met. b) When understanding of needs is good, and decision information
needs are identified ahead of time, communication is dominated by implicit supply of tailored useful
information and the decision makers only have to make minimal explicit requests.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
ACKNOWLEDGMENTS
EEHD was supported by a funding from EQC and GNS Science 2014–2016, and funding from the National
Science Challenges: Resilience to Nature’s Challenges Kia manawaroa – Ngā Ākina o Te Ao Tūroa 2016-2019.
We thank many colleagues for fruitful discussions that helped shape the thinking behind questions throughout the
design and research in this review, these include: Richard Smith, Matt Gerstenberger, Nick Horspool, Mark
Stirling, Sally Potter, John McClure, Sara McBride, Julia Becker, Jacqueline Dohaney, Wendy Saunders, Elspeth
Tilley, and Mary Anne Thompson, as well as many members of the eSocSci Communication Research and Natural
Hazards Network, NZ.
REFERENCES
Adler, C. E. and Hirsch Hadorn, G. (2014) ‘The IPCC and treatment of uncertainties: topics and sources of
dissensus’, Wiley Interdisciplinary Reviews: Climate Change, 5(5), pp. 663–676. doi: 10.1002/wcc.297.
Aitsi-Selmi, A. et al. (2016) ‘Reflections on a Science and Technology Agenda for 21st Century Disaster Risk
Reduction: Based on the Scientific Content of the 2016 UNISDR Science and Technology Conference on the
Implementation of the Sendai Framework for Disaster Risk Reduction 2015–20’, International Journal of
Disaster Risk Science. Beijing Normal University Press, 7(1), pp. 1–29. doi: 10.1007/s13753-016-0081-x.
Aitsi-Selmi, A., Blanchard, K. and Murray, V. (2016) ‘Ensuring science is useful, usable and used in global
disaster risk reduction and sustainable development: a view through the Sendai framework lens’, Palgrave
Communications. Nature Publishing Group, 2(May), p. 16016. doi: 10.1057/palcomms.2016.16.
van Asselt, M. B. A. (2000) Perspectives on uncertainty and risk: the PRIMA approach to decision support.
Dordrecht, The Netherlands: Kluwer Academic Publishers.
Austin, J. et al. (2015) ‘The Ethics of Communicating Scientific Uncertainty’, Environmental Law Reporter:
News & Analysis, 45(2), p. 10105.
Aven, T. and Renn, O. (2015) ‘An Evaluation of the Treatment of Risk and Uncertainties in the IPCC Reports on
Climate Change’, Risk Analysis, 35(4), pp. 701–712. doi: 10.1111/risa.12298.
Bammer, G., Smithson, M. and Goolabri Group (2008) ‘The nature of uncertainty’, in Uncertainty and Risk:
Multidisciplinary Perspectives. London, UK, UK: Earthscan, pp. 289–304.
Banai, M. and Reisel, W. D. (1999) ‘Would you trust your foreign manager? An empirical investigation’, The
International Journal of Human Resource Management. Taylor & Francis , 10(3), pp. 477–487. doi:
10.1080/095851999340431.
Barclay, J. et al. (2008) ‘Framing volcanic risk communication within disaster risk reduction: finding ways for
the social and physical sciences to work together’, Geological Society, London, Special Publications, 305(1),
pp. 163–177. doi: 10.1144/SP305.14.
Becker, J. S. et al. (2015) ‘Aftershock Communication during the Canterbury Earthquakes , New Zealand :
Implications for response and recovery in the built ...’, 2015 NZSEE Conference, Paper Numb.
Benke, K. K., Pettit, C. J. and Lowell, K. E. (2011) ‘Visualisation of spatial uncertainty in hydrological
modelling’, Journal of Spatial Science, 56(1), pp. 73–88. doi: 10.1080/14498596.2011.567412.
Beven, K. et al. (2015) ‘Communicating uncertainty in flood inundation mapping: a case study’, International
Journal of River Basin Management, 13(3), pp. 285–295. doi: 10.1080/15715124.2014.917318.
Beven, K. J. and Alcock, R. E. (2012) ‘Modelling everything everywhere: A new approach to decision-making
for water management under uncertainty’, Freshwater Biology, 57(SUPPL. 1), pp. 124–132. doi:
10.1111/j.1365-2427.2011.02592.x.
Bjerga, T., Aven, T. and Zio, E. (2012) ‘An application of a new framework for model (output) uncertainty
analysis in risk assessment’, in 11th International Probabilistic Safety Assessment and Management
Conference and the Annual European Safety and Reliability Conference 2012, PSAM11 ESREL 2012, pp.
6814–6822. Available at: http://www.scopus.com/inward/record.url?eid=2-s2.0-
84873183187&partnerID=40&md5=daba1c7e6a6fcff1216fbdafdc65aaee.
Blickensderfer, E., Cannon-Bowers, J. A. and Salas, E. (1998) ‘Cross-Training and Team Performance’, in
Cannon-Bowers, J. A. and Salas, E. (eds) Making Decisions under stress: implications for training and
simulation. Washington, D.C., USA: American Psychological Association, pp. 299–311. doi: 10.1037/10278-
011.
Blind, M. W. and Refsgaard, J. C. (2007) ‘Operationalising uncertainty in data and models for integrated water
resources management’, Water Science & Technology, 56(9), p. 1. doi: 10.2166/wst.2007.593.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Bloom, M. and Menefee, M. K. (2014) ‘Scenario Planning Contingency Planning’, Public Productivity &
Management Review, 17(3), pp. 223–230.
Bonneau, G. P. et al. (2014) ‘Overview and state-of-the-art of uncertainty visualization’, Mathematics and
Visualization, 37, pp. 3–27. doi: 10.1007/978-1-4471-6497-5_1.
Borodzicz, E. and van Haperen, K. (2002) ‘Individual and Group Learning in Crisis Simulations’, Journal of
Contingencies and Crisis Management, 10(3), pp. 139–147. doi: 10.1111/1468-5973.00190.
Bostrom, A. et al. (2015) ‘Methods for Communicating the Complexity and Uncertainty of Oil Spill Response
Actions and Tradeoffs’, Human and Ecological Risk Assessment: An International Journal, 21(3), pp. 631–
645. doi: 10.1080/10807039.2014.947867.
Bostrom, A., Anselin, L. and Farris, J. (2008) ‘Visualizing Seismic Risk and Uncertainty’, Annals of the New York
Academy of Sciences, 1128(1), pp. 29–40. doi: 10.1196/annals.1399.005.
Boukhelifa, N. and Duke, D. J. (2009) ‘Uncertainty visualization - Why might it fail?’, in Proceedings of the 27th
international conference extended abstracts on Human factors in computing systems - CHI EA ’09. New York,
New York, USA: ACM Press, p. 4051. doi: 10.1145/1520340.1520616.
Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative research in Psychology,
(May 2012), pp. 37–41. doi: 10.1191/1478088706qp063oa.
Briggs, A. H. et al. (2012) ‘Model Parameter Estimation and Uncertainty: A Report of the ISPOR-SMDM
Modeling Good Research Practices Task Force-6’, Value in Health. Elsevier Inc., 15(6), pp. 835–842. doi:
10.1016/j.jval.2012.04.014.
Brus, J. and Svobodova, J. (2012) ‘The spatial expression of an uncertainty for the quality assessment of the
digital elevation models’, in 12th International Multidisciplinary Scientific GeoConference and EXPO -
Modern Management of Mine Producing, Geology and Environmental Protection, SGEM 2012, pp. 417–
424. Available at: http://www.scopus.com/inward/record.url?eid=2-s2.0-
84890653401&partnerID=40&md5=dde42a6220dbcdc52bbca65d010d021b.
Budescu, D. V., Por, H.-H. and Broomell, S. B. (2012) ‘Effective communication of uncertainty in the IPCC
reports’, Climatic Change, 113(2), pp. 181–200. doi: 10.1007/s10584-011-0330-3.
Budescu, D. V, Broomell, S. and Por, H.-H. (2009) ‘Improving Communication of Uncertainty in the Reports of
the Intergovernmental Panel on Climate Change’, Psychological Science, 20(3), pp. 299–308. doi:
10.1111/j.1467-9280.2009.02284.x.
Burr, V. (1995) An introduction to social constructionism. Abingdon, UK: Routledge. Available at:
https://books.google.co.nz/books/about/An_Introduction_to_Social_Constructionis.html?id=7du5229Pc4EC
&redir_esc=y (Accessed: 9 June 2018).
Cadag, J. R. and Gaillard, J. C. (2014) ‘Integrating people’s capacities in disaster risk reduction through
participatory mapping’, in Lopez-Carresi, A. et al. (eds) Disaster Management: International lessons in risk
reduction, response and recovery. New York: Routledge, pp. 269–286.
Cannon-Bowers, J. A. and Bell, H. E. (1997) ‘Training Decision Makers for Complex environments: Implications
of the Naturalistic Decision Making Perspective’, in Zsambok, C. E. and Klein, G. (eds) Naturalistic Decision
Making. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 99–110.
Chaiken, S. and Trope, Y. (1999) Dual process theories in social psychology. New York: Guilford Press.
Clark, W. C. et al. (2016) ‘Crafting usable knowledge for sustainable development’, Proceedings of the National
Academy of Sciences, 113(17), pp. 4570–4578. doi: 10.1073/pnas.1601266113.
Connolly, T. M. et al. (2012) ‘A systematic literature review of empirical evidence on computer games and serious
games’, Computers & Education. Elsevier Ltd, 59(2), pp. 661–686. doi: 10.1016/j.compedu.2012.03.004.
Crego, J. and Harris, C. (2002) ‘Training Decision-Making by Team based simulation’, in Flin, R. and Arbuthnot,
K. (eds) Incident Command: Tales from the hot seat. Aldershot, England: Ashgate Publishing Limited, pp.
258–269.
Crego, J. and Spinks, T. (1997) ‘Critical Incident management simulation’, in Flin, R. et al. (eds) Decision making
under stress: emerging themes and applications. Aldershot, England: Ashgate Publishing Limited, pp. 85–94.
Crichton, M. and Flin, R. (2001) ‘Training for emergency management: tactical decision games.’, Journal of
hazardous materials, 88(2–3), pp. 255–66.
Crichton, M. and Flin, R. (2002) ‘Command Decision Making’, in Flin, R. and Arbuthnot, K. (eds) Incident
Command: Tales from the hot seat. Aldershot, England: Ashgate Publishing Limited, pp. 201–238.
Cronin, P., Ryan, F. and Coughlan, M. (2008) ‘Undertaking a literature review: a step-by-step approach’, British

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Journal of Nursing, 17(1), pp. 38–43. doi: 10.12968/bjon.2008.17.1.28059.
Cronin, S. J. et al. (2004) ‘Participatory methods of incorporating scientific with traditional knowledge for
volcanic hazard management on Ambae Island, Vanuatu’, Bulletin of Volcanology, 66(7), pp. 652–668. doi:
10.1007/s00445-004-0347-9.
Cronin, S. J. (2008) ‘The Auckland Volcano Scientific Advisory Group during Exercise Ruaumoko: observations
and recommendations Data collection’, in Civil Defence Emergency Management: Exercise Ruaumoko.
Auckland: Auckland Regional Council.
Curnin, S. et al. (2015) ‘A theoretical framework for negotiating the path of emergency management multi-agency
coordination.’, Applied ergonomics. Elsevier Ltd, 47, pp. 300–7. doi: 10.1016/j.apergo.2014.10.014.
Daedlow, K. et al. (2016) ‘Socially responsible research processes for sustainability transformation: an integrated
assessment framework’, Current Opinion in Environmental Sustainability, 23, pp. 1–11. doi:
10.1016/j.cosust.2016.09.004.
Daipha, P. (2012) ‘Weathering Risk: Uncertainty, Weather Forecasting, and Expertise’, Sociology Compass, 6(1),
pp. 15–25. doi: 10.1111/j.1751-9020.2011.00437.x.
Deitrick, S. and Wentz, E. a. (2015) ‘Developing Implicit Uncertainty Visualization Methods Motivated by
Theories in Decision Science’, Annals of the Association of American Geographers, 105(3), pp. 531–551. doi:
10.1080/00045608.2015.1012635.
Demeritt, D. et al. (2007) ‘Ensemble predictions and perceptions of risk, uncertainty, and error in flood
forecasting’, Environmental Hazards, 7(2), pp. 115–127. doi: 10.1016/j.envhaz.2007.05.001.
Dietz, G., Gillespie, N. and Chao, G. T. (2010) ‘Unravelling the complexities of trust and culture’, in Saunders,
M. N. K. et al. (eds) Organizational Trust. Cambridge, UK: Cambridge University Press, pp. 3–41. doi:
10.1017/CBO9780511763106.002.
Dirks, K. and Ferrin, D. (2001) ‘No Title’, Organization Science, 12(4), pp. 450–467.
Doyle, E. E. H. et al. (2011a) ‘The communication of uncertain scientific advice during natural hazard events’,
New Zealand Journal of Psychology, 40(4).
Doyle, E. E. H. et al. (2011b) ‘The Communication of Uncertain Scientific Advice During Natural Hazard
Events’, New Zealand Journal of Psychology, 40(4), pp. 39–50.
Doyle, E. E. H., McClure, J., Johnston, D. M., et al. (2014) ‘Communicating Likelihoods and Probabilities in
Forecasts of Volcanic Eruptions’, Journal of Volcanology and Geothermal Research. Elsevier B.V., 272, pp.
1–15. doi: 10.1016/j.jvolgeores.2013.12.006.
Doyle, E. E. H., McClure, J., Paton, D., et al. (2014) ‘Uncertainty and decision making: Volcanic crisis scenarios’,
International Journal of Disaster Risk Reduction. Elsevier, 10, pp. 75–101. doi: 10.1016/j.ijdrr.2014.07.006.
Doyle, E. E. H. and Johnston, D. M. (2011) ‘Science advice for critical decision-making’, in Paton, D. and
Violanti, J. (eds) Working in high risk environments: developing sustained resilience. Springfield, Illinois,
USA: Charles C Thomas, pp. 69–92.
Doyle, E. E. H., Johnston, D. M. and Smith, R. (2018) ‘Communicating model uncertainty for natural hazards: a
systematic review’, International Journal of Disaster Risk Reduction, Under Revi.
Doyle, E. E. H. and Paton, D. (2017) ‘Decision-Making : Preventing Miscommunication and Creating Shared
Meaning Between Stakeholders’, Advances in Volcanology. doi: 10.1007/11157.
Doyle, E. E. H., Paton, D. and Johnston, D. M. (2015) ‘Enhancing scientific response in a crisis : evidence- based
approaches from emergency management in New Zealand’, Journal of Applied Volcanology, 4(1), p. 1. doi:
10.1186/s13617-014-0020-8.
Doyle, E. E. H. and Potter, S. H. (2015) ‘Methodology for the development of a probability translation table for
GeoNet’, GNS Science Report, 67(January), p. 18. Available at:
http://www.gns.cri.nz/Home/Products/Publications/Science-Reports.
Eiser, J. R. et al. (2012) ‘Risk interpretation and action: A conceptual framework for responses to natural hazards’,
International Journal of Disaster Risk Reduction. Elsevier, 1, pp. 5–16. doi: 10.1016/j.ijdrr.2012.05.002.
Elith, J., Burgman, M. A. and Regan, H. M. (2002) ‘Mapping epistemic uncertainties and vague concepts in
predictions of species distribution’, Ecological Modelling, 157(2–3), pp. 313–329. doi: 10.1016/S0304-
3800(02)00202-8.
Endsley, M. R. (1997) ‘The role of situation awareness in Naturalistic Decision Making’, in Zsambok, C. E. and
Klein, G. (eds) Naturalistic Decision Making. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 269–284.
Eppler, M. J. and Mengis, J. (2004) ‘The concept of information overload: A review of literature from organization

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
science, accounting, marketing, MIS, and related disciplines’, Information Society, 20(5), pp. 325–344. doi:
10.1080/01972240490507974.
Epstein, S. (1994) ‘Integration of the cognitive and the psychodynamic unconscious.’, The American Psychologist,
49(8), pp. 709–24.
Evans, J. S. B. T. and Stanovich, K. E. (2013a) ‘Dual-Process Theories of Higher Cognition: Advancing the
Debate’, Perspectives on Psychological Science, 8(3), pp. 223–241. doi: 10.1177/1745691612460685.
Evans, J. S. B. T. and Stanovich, K. E. (2013b) ‘Theory and Metatheory in the Study of Dual Processing: Reply
to Comments’, Perspectives on Psychological Science, 8(3), pp. 263–271. doi: 10.1177/1745691613483774.
Falkheimer, J. and Heide, M. (2006) ‘Multicultural Crisis Communication: Towards a Social Constructionist
Perspective.’, Journal of Contingencies & Crisis Management. Wiley/Blackwell (10.1111), 14(4), pp. 180–
189. doi: 10.1111/j.1468-5973.2006.00494.x.
Faraj, S. and Xiao, Y. (2006) ‘Coordination in Fast-Response Organizations’, Management Science. INFORMS
, 52(8), pp. 1155–1169. doi: 10.1287/mnsc.1060.0526.
Faulkner, H. et al. (2007) ‘Developing a Translational Discourse to Communicate Uncertainty in Flood Risk
between Science and the Practitioner’, AMBIO: A Journal of the Human Environment, 36(8), pp. 692–704.
doi: 10.1579/0044-7447(2007)36[692:DATDTC]2.0.CO;2.
Faulkner, H., Alexander, M. and Leedal, D. (2014) ‘Translating Uncertainty in Flood Risk Science’, in Applied
Uncertainty Analysis for Flood Risk Management. London, UK: IMPERIAL COLLEGE PRESS, pp. 627–
662. doi: 10.1142/9781848162716_0024.
Fischhoff, B. and Davis, a. L. (2014) ‘Communicating scientific uncertainty’, Proceedings of the National
Academy of Sciences, 111(Supplement_4), pp. 13664–13671. doi: 10.1073/pnas.1317504111.
Flin (1996) Sitting in the hot seat: Leaders and Teams for Critical Incident Management. Chichester, England:
John Wiley & Sons, Ltd.
Funtowicz, S. O. and Ravetz, J. R. (1993) ‘Science for the post-normal age’, Futures, pp. 739–755. doi:
10.1016/0016-3287(93)90022-L.
Funtowicz, S. O. and Ravetz, J. R. (1999) ‘Post-Normal Science- an insight now maturing’, Futures, 31, pp. 641–
646.
Gaillard, J. C. (2006) ‘Traditional societies in the face of natural hazards: the 1991 Mt.Pinatubo eruption and the
Aetas of the Philippines’, International Journal of Mass Emergencies and Disasters, 24(1), pp. 5–43.
Garnett, J. L. and Kouzmin, A. (2007) ‘Communicating throughout Katrina: Competing and Complementary
Conceptual Lenses on Crisis Communication.’, Public Administration Review: Administrative Failure in the
Wake of Katrina, 67(1), pp. 171–188. Available at:
http://www3.interscience.wiley.com/journal/118485150/abstract.
Gill, J. et al. (2008) World Meteorological Organization Guidelines on Communicating Forecast Uncertainty,
WMO/TD No. 1422, World Merteorological Organization. Edited by H. Kootval. (WMO/TD).
Goodman, R. A. and Goodman, L. P. (1976) ‘Some Management Issues in Temporary Systems: A Study of
Professional Development and Manpower-The Theater Case’, Source: Administrative Science Quarterly,
21(3), pp. 494–501. Available at: http://www.jstor.org/stable/2391857 (Accessed: 9 June 2018).
Griethe, H. and Schumann, H. (2005) ‘Visualizing Uncertainty for Improved Decision Making’, in Proceedings
of the 4th International Conference on Business Informatics Research, BIR 2005, pp. 23–34.
Griffin, E. (2006) A first look at communication theory. 6th edn. New York, NY, US: McGraw-Hill. Available at:
http://psycnet.apa.org/record/2005-12493-000 (Accessed: 9 June 2018).
Grubler, A., Ermoliev, Y. and Kryazhimskiy, A. (2015) ‘Coping with uncertainties-examples of modeling
approaches at IIASA’, Technological Forecasting and Social Change. Elsevier Inc., 98, pp. 213–222. doi:
10.1016/j.techfore.2015.06.004.
Han, P. K. J. (2013) ‘Conceptual, Methodological, and Ethical Problems in Communicating Uncertainty in
Clinical Evidence’, Medical Care Research and Review, 70(1 Suppl), p. 14S–36S. doi:
10.1177/1077558712459361.
Handmer (2008) ‘Emergency Management thrives on uncertainty’, in Bammer, G. and Smithson, M. (eds)
Uncertainty and Risk: Multidisciplinary Perspectives. London, UK: Earthscan, pp. 231–243. Available at:
http://i2s.anu.edu.au/i2s-publications/uncertainty-book.
Haynes, K., Barclay, J. and Pidgeon, N. (2007) ‘The issue of trust and its influence on risk communication during
a volcanic crisis’, Bulletin of Volcanology, 70(5), pp. 605–621. doi: 10.1007/s00445-007-0156-z.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Haynes, K., Barclay, J. and Pidgeon, N. (2008) ‘Whose reality counts? Factors affecting the perception of volcanic
risk’, Journal of Volcanology and Geothermal Research, 172(3–4), pp. 259–272. doi:
10.1016/j.jvolgeores.2007.12.012.
Heath, R., Lee, J. and Ni, L. (2009) ‘Crisis and Risk Approaches to Emergency Management Planning and
Communication: The Role of Similarity and Sensitivity’, Journal of Public Relations Research, 21(2), pp.
123–141. doi: 10.1080/10627260802557415.
Honda, H. and Yamagishi, K. (2006) ‘Directional Verbal Probabilities’, Experimental Psychology -formerly
Zeitschrift für Experimentelle Psychologie, 53(3), pp. 161–170. doi: 10.1027/1618-3169.53.3.161.
Hope, S. and Hunter, G. J. (2007) ‘Testing the Effects of Thematic Uncertainty on Spatial Decision-making’,
Cartography and Geographic Information Science, 34(February 2015), pp. 199–214. doi:
10.1559/152304007781697884.
Hudson-Doyle, E. E., Paton, D. and Johnston, D. M. (2014) ‘Emergency Management decision-making in the
laboratory: the case of uncertain volcanic advice’, in IRDR Conference 2014: Integrated Disaster Risk
Science: A tool for sustainability; Beijing, China, June 7-9, 2014. Beijing, China: IRDR International, p. 18.
Available at: http://www.irdrinternational.org/wp-content/uploads/2013/02/IRDR-Conference-2014-
Abstract-Book.pdf.
Hyllengren, P. et al. (2011) ‘Swift trust in leaders in temporary military groups’, Team Performance Management:
An International Journal, 17(7/8), pp. 354–368. doi: 10.1108/13527591111182625.
Iannella, R. and Henricksen, K. (2007) ‘Proceedings of the 4th international ISCRAM conference’, in Van de
Walle, B., Burghardt, P., and Nieuwenhuis, C. (eds) Managing information in the disaster coordination
centre : lessons and opportunities, pp. 1–11.
IAVCEI Subcomittee for Crisis Protocols, IAVCEI and Newhall, C. (1999) ‘Professional conduct of scientists
during volcanic crises’, Bulletin of Volcanology, 60(5), pp. 323–334.
Janssen, P. H. M. et al. (2005) ‘A guidance for assessing and communicating uncertainties’, Water Science and
Technology, 52(6), pp. 125–131.
Johnson, B. B. (2003) ‘Further notes on public response to uncertainty in risks and science.’, Risk analysis : an
official publication of the Society for Risk Analysis, 23(4), pp. 781–9. Available at:
http://www.ncbi.nlm.nih.gov/pubmed/12926570.
Johnson, B. B. and Slovic, P. (1995) ‘Presenting uncertainty in health risk assessment: initial studies of its effects
on risk perception and trust’, Risk Analysis, 15(4), pp. 485–494.
Johnson, B. B. and Slovic, P. (1998) ‘Lay views on uncertainty in environmental health risk assessment’, Journal
of Risk Research\, 1(4), pp. 261–279.
Johnson, V. A. et al. (2014) ‘Evaluations of disaster education programs for children: A methodological review’,
International Journal of Disaster Risk Reduction, 9, pp. 107–123. doi: 10.1016/j.ijdrr.2014.04.001.
Jolly, A. D. and Cronin, S. J. (2014) ‘From eruption to end-user ; bridging the science – management interface
during the 2012 Te Maari eruption , Tongariro Volcano , New Zealand’, Journal of Volcanology and
Geothermal Research. Elsevier B.V., 286(August), p. 183. doi: 10.1016/j.jvolgeores.2014.08.025.
Joslyn, S. L. et al. (2009) ‘The effects of wording on the understanding and use of uncertainty information in a
threshold forecasting decision’, Applied Cognitive Psychology, 23(1), pp. 55–72. doi: 10.1002/acp.1449.
Kapucu, N. (2006) ‘Interagency Communication Networks During Emergencies’, The American Review of Public
Administration. Sage PublicationsSage CA: Thousand Oaks, CA, 36(2), pp. 207–225. doi:
10.1177/0275074005280605.
Karelitz, T. M. and Budescu, D. V (2004) ‘You say “probable” and I say “likely”: improving interpersonal
communication with verbal probability phrases.’, Journal of experimental psychology: Applied, 10(1), pp. 25–
41. doi: 10.1037/1076-898X.10.1.25.
Karl, H. A., Susskind, L. E. and Wallace, K. H. (2007) ‘A dialogue not a diatribe effective intergration of science
and policy through join fact finding’, Environment: Science and Policy for Sustainable Development, 49(1),
pp. 20–34.
Keohane, R. O., Lane, M. and Oppenheimer, M. (2014) ‘The ethics of scientific communication under
uncertainty’, Politics, Philosophy & Economics, 13(4), pp. 343–368. doi: 10.1177/1470594X14538570.
Keough, S. M. and Shanahan, K. J. (2008) ‘Scenario Planning: Toward a More Complete Model for Practice’,
Advances in Developing Human Resources, 10(2), pp. 166–178. doi: 10.1177/1523422307313311.
Keren, G. (2013) ‘A Tale of Two Systems: A Scientific Advance or a Theoretical Stone Soup? Commentary on

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Evans & Stanovich (2013)’, Perspectives on Psychological Science, 8(3), pp. 257–262. doi:
10.1177/1745691613483474.
Keren, G. and Schul, Y. (2009) ‘Two Is Not Always Better Than One: A Critical Evaluation of Two-System
Theories’, Perspectives on Psychological Science, 4(6), pp. 533–550. doi: 10.1111/j.1745-6924.2009.01164.x.
Klein (1998) Sources of Power: How people make decisions. 2nd printi. Cambridge, Massachusetts; London, UK:
The MIT Press.
Klein, G. (1997) ‘The current status of the naturalistic decision making framework’, in Flin, R. et al. (eds) Decision
making under stress: emerging themes and applications. Aldershot, England: Ashgate Publishing Limited, pp.
11–28.
Klein, G. (2008) ‘Naturalistic decision making.’, Human factors, 50(3), pp. 456–60. doi:
10.1518/001872008X288385.
Kloprogge, P., van der Sluijs, J. P. and Petersen, A. C. (2011) ‘A method for the analysis of assumptions in model-
based environmental assessments’, Environmental Modelling & Software. Elsevier Ltd, 26(3), pp. 289–301.
doi: 10.1016/j.envsoft.2009.06.009.
Kloprogge, P., van der Sluijs, J. and Wardekker, A. (2007) Uncertainty communication: Issues and good practice.
v.2. Report NWS-E-2007-199. Utrecht, The Netherlands. Available at: www.mnp.nl/guidance.
Knorr Cetina, K. (2013) ‘Interdisciplinary Science Reviews Culture in global knowledge societies: knowledge
cultures and epistemic cultures’. doi: 10.1179/030801807X163571org/10.1179/030801807X163571.
Kowalski-Trakofler, K. M., Vaught, C. and Scharf, T. (2003) ‘Judgment and decision making under stress: an
overview for emergency managers’, International Journal of Emergency Management, 1(3), pp. 278–289. doi:
10.1504/IJEM.2003.003297.
Kozlowski, S. W. J. (1998) ‘Training and Developing Adaptive Teams ’:, in Cannon-Bowers, J. a. and Salas, E.
(eds) Making Decisions under stress: implications for training and simulation. Washington, D.C., USA:
American Psychological Association, pp. 115–153.
Kramer, R. M. (1999) ‘TRUST AND DISTRUST IN ORGANIZATIONS: Emerging Perspectives, Enduring
Questions’, Annu. Rev. Psychol, 50, pp. 569–98. Available at:
https://www.annualreviews.org/doi/pdf/10.1146/annurev.psych.50.1.569 (Accessed: 9 June 2018).
Krauss, W., Schäfer, M. S. and von Storch, H. (2012) ‘Introduction: Post-Normal Climate Science’, Nature and
Culture, 7(2), pp. 121–132. doi: 10.3167/nc.2012.070201.
Kruglanski, A. W. and Gigerenzer, G. (2011) ‘Intuitive and deliberate judgments are based on common
principles.’, Psychological Review, 118(1), pp. 97–109. doi: 10.1037/a0020762.
Kuhn, T. S. (1962) The structure of scientific revolutions. Chicago, USA, USA: University of Chicago Press
Chicago, IL.
Kwakkel, J. H., Walker, W. E. and Marchau, V. A. W. J. (2010) ‘Classifying and communicating uncertainties in
model-based policy analysis’, International Journal of Technology, Policy and Management, 10(4), p. 299.
doi: 10.1504/IJTPM.2010.036918.
Leyk, S., Boesch, R. and Weibel, R. (2005) ‘A Conceptual Framework for Uncertainty Investigation i n Map-
based Land Cover Change Modelling’, Transactions in GIS, 9(3), pp. 291–322. doi: 10.1111/j.1467-
9671.2005.00220.x.
Linnerooth-Bayer, J. A. et al. (2016) ‘Expert engagement in participatory processes: translating stakeholder
discourses into policy options’, Natural Hazards. Springer Netherlands, 81(1), pp. 69–88. doi:
10.1007/s11069-015-1805-8.
Lipkus, I. M. (2010) ‘Numeric, verbal, and visual formats of conveying health risks: suggested best practices and
future recommendations.’, Medical decision making : an international journal of the Society for Medical
Decision Making, 27(5), pp. 696–713. doi: 10.1177/0272989X07307271.
Lipshitz, R. et al. (2001) ‘Focus Article: Taking stock of naturalistic decision making’, Journal of Behavioral
Decision Making, 14(5), pp. 331–352. doi: 10.1002/bdm.381.
Lipshitz, R. and Strauss, O. (1997) ‘Coping with Uncertainty: A Naturalistic Decision-Making Analysis’,
Organizational Behavior and Human Decision Processes, 69(2), pp. 149–163. doi: 10.1006/obhd.1997.2679.
Loewenstein, G. F. et al. (2001) ‘Risk as feelings.’, Psychological Bulletin, 127(2), pp. 267–286. doi:
10.1037//0033-2909.127.2.267.
Loucks, D. P. (2002) ‘Quantifying and communicating model uncertainty for decisionmaking in the everglades’,
in Risk-Based Decision Making in Water Resources X, Proceedings of the tenth Conference. Cambridge:

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
American Society of Civil Engineers, Virginia, USA, pp. 40–58. Available at:
http://www.scopus.com/inward/record.url?eid=2-s2.0-
0242365401&partnerID=40&md5=9e36c7bc5771db6115da39d7248ab9d6.
Marcus, L. J., Dorn, B. C. and Henderson, J. M. (2006) ‘Meta-Leadership and National Emergency Preparedness:
A Model to Build Government Connectivity’, BIOSECURITY AND BIOTERRORISM: BIODEFENSE
STRATEGY, PRACTICE, AND SCIENCE, 4(2). Available at:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.6326&rep=rep1&type=pdf (Accessed: 9 June
2018).
Marincioni, F. (2007) ‘Information technologies and the sharing of disaster knowledge: The critical role of
professional culture’, Disasters, 31(4), pp. 459–476. doi: 10.1111/j.1467-7717.2007.01019.x.
Markon, M.-P. L. and Lemyre, L. (2013) ‘Public Reactions to Risk Messages Communicating Different Sources
of Uncertainty: An Experimental Test’, Human and Ecological Risk Assessment: An International Journal,
19(4), pp. 1102–1126. doi: 10.1080/10807039.2012.702015.
Marks, M. a. et al. (2002) ‘The impact of cross-training on team effectiveness.’, Journal of Applied Psychology,
87(1), pp. 3–13. doi: 10.1037//0021-9010.87.1.3.
Martin, L., Flin, R. and Skriver, J. (1997) ‘Emergency decision making - A wider decision framework?’, in Flin,
R. et al. (eds) Decision making under stress: emerging themes and applications. Aldershot, England: Ashgate
Publishing Limited, pp. 280–290.
Maslin, M. (2013) ‘Cascading uncertainty in climate change models and its implications for policy’, The
Geographical Journal, 179(3), pp. 264–271. doi: 10.1111/j.1475-4959.2012.00494.x.
Mastrandrea, M. and Field, C. (2010) Guidance note for lead authors of the IPCC fifth assessment report on
consistent treatment of uncertainties, IPCC Cross-working group meeting on consisten treatment of
uncertainties.
Maxim, L. and Mansier, P. (2014) ‘How is Scientific Credibility Affected by Communicating Uncertainty? The
Case of Endocrine Disrupter Effects on Male Fertility’, Human and Ecological Risk Assessment: An
International Journal, 20(1), pp. 201–223. doi: 10.1080/10807039.2012.719387.
McClure, J., H. Doyle, E. E. and Velluppillai, J. M. (2014) ‘A tale of two cities: Judgments about earthquake and
aftershock probabilities across time windows’, International Journal of Disaster Risk Reduction. Elsevier, in
press, pp. 1–12. doi: 10.1016/j.ijdrr.2014.11.010.
McGuire, W. J. J. et al. (2009) ‘Improving communication during volcanic crises on small, vulnerable islands’,
Journal of Volcanology and Geothermal Research. Elsevier B.V., 183(1–2), pp. 63–75. doi:
10.1016/j.jvolgeores.2009.02.019.
McIvor, D. and Paton, D. (2007) ‘Preparing for natural hazards: normative and attitudinal influences’, Disaster
Prevention and Management, 16(1), pp. 79–88. doi: 10.1108/09653560710729839.
Mcknight, H., Cummings, L. L. and Chervany, N. L. (1998) ‘Initial trust formation in new organizational
relationships’, Academy of Management REview, 23(3). Available at:
https://msu.edu/~mcknig26/InitialTrustAMR.pdf (Accessed: 9 June 2018).
Meyerson, D., Weick, K. E. and Kramer, R. M. (1996) ‘Swift trust and temporary groups’, in Kramer, R. E. and
Tyler, T. R. (eds) Trust in organizations: Frontiers of theory and research. Thousand Oaks, CA, US: Sage
Publications, Inc, pp. 166–195. Available at: http://psycnet.apa.org/psycinfo/1996-97618-006.
Miles, S. and Frewer, L. J. (2003) ‘Public perception of scientific uncertainty in relation to food hazards’, Journal
of Risk Research, 6(3), pp. 267–283. doi: 10.1080/1366987032000088883.
Militello, L. G. et al. (2007) ‘Information flow during crisis management : challenges to coordination in the
emergency operations center’, Cogn Tech Work, 9, pp. 25–31. doi: 10.1007/s10111-006-0059-3.
Moats, J. B., Chermack, T. J. and Dooley, L. M. (2008) ‘Using Scenarios to Develop Crisis Managers:
Applications of Scenario Planning and Scenario-Based Training’, Advances in Developing Human Resources,
10(3), pp. 397–424. doi: 10.1177/1523422308316456.
Mohr, J. and Spekman, R. (1994) ‘Characteristics of Partnership Success: Partnership Attributes, Communication
Behavior, and Conflict Resolution Techniques’, Source: Strategic Management Journal Strategic
Management Journal, 15(15), pp. 135–152. Available at: http://www.jstor.org/stable/2486868 (Accessed: 9
June 2018).
Morgan, M. G. et al. (2001) Risk communication: A mental models Approach. London: Cambridge University
Press.
Moss, R. H. (2011) ‘Reducing doubt about uncertainty: Guidance for IPCC’s third assessment’, Climatic Change,

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
108(4), pp. 641–658. doi: 10.1007/s10584-011-0182-x.
Moss, R. H. and Schneider, S. . (2000) ‘Uncertainties in the IPCC TAR : Recommendations to lead authors for
more consistent assessment and reporting .’, in Pachauri, R., Taniguchi, T., and Tanaka, K. (eds) IPCC
Supporting Material, Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the
IPCC, pp. 33–51. Available at:
http://stephenschneider.stanford.edu/Publications/PDF_Papers/UncertaintiesGuidanceFinal2.pdf.
Murphy, C., Gardoni, P. and Harris, C. E. (2011) ‘Classification and Moral Evaluation of Uncertainties in
Engineering Modeling’, Science and Engineering Ethics, 17(3), pp. 553–570. doi: 10.1007/s11948-010-9242-
2.
O’Neil, O. (2002) Autonomy and Trust in Bioethics. Cambridge, U.K., U.K.: Cambridge University Press.
Omodei, M. M. et al. (2005) ‘“More is better?”: A bias toward overuse of resources in Naturalistic decision
making settings’, in Montgomery, H., Lipshitz, R., and Brehmer, B. (eds) How professionals make decisions.
Mahwah, NJ: Lawrence Erlbaum Associates, pp. 29–41.
Orasanu, J. M. (1994) ‘Shared mental models and crew decision making’, in Johnston, N., McDonald, N., and
Fuller, R. (eds) Aviation Psychology in Practice. Aldershot, England, pp. 255–285.
Osman, M. (2013) ‘A Case Study: Dual-Process Theories of Higher Cognition--Commentary on Evans &
Stanovich (2013)’, Perspectives on Psychological Science, 8(3), pp. 248–252. doi:
10.1177/1745691613483475.
Owen, C., Bearman, C. and Brooks, B. (2013) ‘Developing a research framework for complex multi–team
coordination in emergency management’, International Journal Of Emergency Management, 9(1), pp. 1–17.
doi: 10.1504/IJEM.2013.054098.
Page, G. G. et al. (2016) ‘Co-designing transformation research: lessons learned from research on deliberate
practices for transformation’, Current Opinion in Environmental Sustainability. Elsevier B.V., 20(October),
pp. 86–92. doi: 10.1016/j.cosust.2016.09.001.
Pascual, R. and Henderson, S. (1997) ‘Evidence of Naturalistic Decision Making in Military Command and
Control’, in Zsambok, C. E. and Klein, G. (eds) Naturalistic Decision Making. Mahwah, NJ: Lawrence
Erlbaum Associates, pp. 217–226.
Paton, D. et al. (1999) ‘Managing Natural Hazard Consequences: Planning for information management and
decision making’, Journal of the American Society of Professional Emergency Planners, 6, pp. 37–47.
Paton, D. et al. (2008) ‘Risk perception and volcanic hazard mitigation: Individual and social perspectives’,
Journal of Volcanology and Geothermal Research, 172(3–4), pp. 179–188. doi:
10.1016/j.jvolgeores.2007.12.026.
Paton, D. et al. (2015) ‘Developing a comprehensive model of hazard preparedness: Lessons from the
Christchurch earthquake’, International Journal of Disaster Risk Reduction. Elsevier, 14, pp. 37–45. doi:
10.1016/j.ijdrr.2014.11.011.
Paton, D. and Flin, R. (1999) ‘Disaster stress: an emergency management perspective’, Disaster Prevention and
Management, 8(4), pp. 261–267. doi: 10.1108/09653569910283897.
Paton, D. and Jackson, D. (2002) ‘Developing disaster management capability: an assessment centre approach’,
Disaster Prevention and Management, 11(2), pp. 115–122. doi: 10.1108/09653560210426795.
Paton, D., Johnston, D. M. and Houghton, B. F. (1998) ‘Organisational response to a volcanic eruption’, Disaster
Prevention and Management, 7(1), pp. 5–13. doi: 10.1108/09653569810206226.
Paton, D. and McClure, J. (2013) Preparing for Disaster: Building Household and Community Capacity.
Springfield, Illinois, USA: Charles C Thomas Pub Ltd.
Paton, D., Smith, L. A. and Johnston, D. M. (2000) ‘Volcanic hazards: Risk perception and preparedness’, 29(2),
pp. 86–91.
Paton, D., Smith, L. A. and Violanti, J. (2000) ‘Disaster response: risk, vulnerability and resilience’, Disaster
Prevention and Management, 9(3), pp. 173–180. doi: 10.1108/09653560010335068.
Patt, A. (2007) ‘Assessing model-based and conflict-based uncertainty’, Global Environmental Change, 17(1),
pp. 37–46. doi: 10.1016/j.gloenvcha.2006.10.002.
Patt, A. (2009) ‘Communicating uncertainty to policy makers’, in Baveye, P., Mysiak, J., and Laba, M. (eds)
Uncertainties in Environmental Modelling and Consequences for Policy Making. Springer Science & Busness
Media B.V., pp. 231–251.
Patt, A. and Dessai, S. (2005) ‘Communicating uncertainty: lessons learned and suggestions for climate change

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
assessment’, Comptes Rendus Geosciences, 337(4), pp. 425–441. doi: 10.1016/j.crte.2004.10.004.
Pliske, R. M., McCloskey, M. J. and Klein, G. (2001) ‘Decision skills training: facilitating learning from
experience’, in Salas, E. and Klein, G. (eds) Linking expertise and naturalistic decision making. Mahwah, NJ:
Lawrence Erlbaum Associates, pp. 37–53.
Pollock, C. et al. (2003) ‘Team Resilience’, in Paton, D., Violanti, J., and Smith, L. (eds) Promoting Capabilities
to Manage posttraumatic stress: Perspectives on resilience. Springfield, Ill: Charles C. Thomas, pp. 74–88.
Quarantelli, E. L. (1997) ‘Ten criteria for evaluating the management of community disasters.’, Disasters, 21(1),
pp. 39–56.
Rabinovich, A. and Morton, T. A. (2012) ‘Unquestioned Answers or Unanswered Questions: Beliefs About
Science Guide Responses to Uncertainty in Climate Change Risk Communication’, Risk Analysis, 32(6), pp.
992–1002. doi: 10.1111/j.1539-6924.2012.01771.x.
Reyers, B. et al. (2015) ‘Navigating complexity through knowledge coproduction: Mainstreaming ecosystem
services into disaster risk reduction’, Proceedings of the National Academy of Sciences, 112(24), pp. 7362–
7368. doi: 10.1073/pnas.1414374112.
Rogalski, J. (1999) ‘Decision making and management of dynamic risk’, Cognition, Technology & Work.
Springer, 1(4), pp. 247–256.
Rogalski, J. and Samurçay, R. (1993) ‘A method for tactical reasoning (MTR) in emergency managment: analysis
of individual acquisition and collective implementation’, in Rasmussen, B., Brehmer, B., and Leplat, J. (eds)
Distributed decision making: cognitive models for co-operative work. New York, New York, USA: Wiley,
pp. 287–298.
Rovins, J. E., Doyle, E. E. H. and Huggins, T. J. (2014) ‘2nd Integrated Research on Disaster Risk Conference -
Integrated Disaster Risk Science : A Tool for Sustainability 1’, GRF Davos Planet@RIsk, 2(5), pp. 332–336.
Saaty, T. L. (2008) ‘Decision making with the analytic hierarchy process’, International Journal of Services
Sciences. Inderscience, 1(1), pp. 83–98. Available at:
http://inderscience.metapress.com/index/02T637305V6G65N8.pdf.
Salas, E., Cannon-Bowers, J. A. and Blickensderfer, E. (1997) ‘Enhancing reciprocity be- tween training theory
and practice: Principles, guidelines, and specifications’, in Ford, J. and Associates (eds) Improving training
effectiveness in work organizations. Hillsdale, NJ: Erlbaum, pp. 291–322.
Salas, E., Stout, R. J. and Cannon-Bowers, J. A. (1994) ‘The role of shared mental models in developing shared
situational awareness’, in Gilson, R. D., Garland, D. J., and Koonce, J. M. (eds) Situational Awareness in
Complex Systems: Proceedings of a Cahfa Conference. Daytona Beach, FL: Embry-Riddle Aeronautical
University Press, pp. 298–304.
Schaafstal, A. M., Johnston, J. H. and Oser, R. L. (2001) ‘Training teams for emergency management’, Computers
in Human Behavior, 17(5–6), pp. 615–626. doi: 10.1016/S0747-5632(01)00026-7.
Schenk, T. (2016) ‘Facts for now, facts for use: satisficing and adapting in joint fact-finding’, in Joint Fact-Finding
in Urban Planning and Environmental Disputes. Abingdon, UK: Taylor & Francis Group, pp. 102–119. doi:
10.4324/9781315651842-16.
Schraagen, J. M. and van de Ven, J. (2011) ‘Human factors aspects of ICT for crisis management’, Cognition,
Technology and Work, 13(3), pp. 175–187. doi: 10.1007/s10111-011-0175-6.
Scolobig, A. et al. (2015) ‘Towards people-centred approaches for effective disaster risk management: Balancing
rhetoric with reality’, International Journal of Disaster Risk Reduction. Elsevier, 12, pp. 202–212. doi:
10.1016/j.ijdrr.2015.01.006.
Scolobig, A. and Lilliestam, J. (2016) ‘Comparing Approaches for the Integration of Stakeholder Perspectives in
Environmental Decision Making’, Resources, 5(4), p. 37. doi: 10.3390/resources5040037.
Scolobig, A. and Pelling, M. (2016) ‘The co-production of risk from a natural hazards perspective: science and
policy interaction for landslide risk management in Italy’, Natural Hazards. Springer Netherlands, 81, pp. 7–
25. doi: 10.1007/s11069-015-1702-1.
Siegrist, M. and Cvetkovich, G. (2000) ‘No Title’, Risk Analysis, 20(5), pp. 713–719.
Sloman, S. a. (1996) ‘The empirical case for two systems of reasoning.’, Psychological Bulletin, 119(1), pp. 3–
22. doi: 10.1037//0033-2909.119.1.3.
Slovic, P. et al. (2004) ‘Risk as analysis and risk as feelings: some thoughts about affect, reason, risk, and
rationality.’, Risk analysis : an official publication of the Society for Risk Analysis, 24(2), pp. 311–22. doi:
10.1111/j.0272-4332.2004.00433.x.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
van der Sluijs, J. P., Petersen, A. and Funtowicz, S. (2011) ‘Reflective approaches to uncertainty assessment and
communication’, in Lentsch, J. and Weingart, P. (eds) The Politics of Scientific Advice. Cambridge: Cambridge
University Press, pp. 259–269. doi: 10.1017/CBO9780511777141.014.
Smircich, L. and Smircich, L. (2012) ‘Concepts of Culture and Organizational Analysis’, Administrative Science
Quarterly, 28(3), pp. 339–358. doi: 10.2307/2392246.
Smithson, M. (1999) ‘Conflict Aversion: Preference for Ambiguity vs Conflict in Sources and Evidence.’,
Organizational behavior and human decision processes, 79(3), pp. 179–198. doi: 10.1006/obhd.1999.2844.
Smithson, M. (2008) ‘Psychology’s ambivalent view of uncertainty’, in Bammer, G. and Smithson, M. (eds)
Uncertainty and Risk: Multidisciplinary Perspectives. London, UK: Earthscan Publications Ltd, pp. 205–217.
Smithson, M., Bammer, G. and Goolabri Group (2008) ‘Coping and managing under uncertainty’, in Bammer, G.
and Smithson, M. (eds) Uncertainty and Risk: Multidisciplinary Perspectives. London, UK: Earthscan
Publications Ltd, pp. 321–333.
Solana, M. C., Kilburn, C. R. . R. J. and Rolandi, G. (2008) ‘Communicating eruption and hazard forecasts on
Vesuvius, Southern Italy’, Journal of Volcanology and Geothermal Research, 172(3–4), pp. 308–314. doi:
10.1016/j.jvolgeores.2007.12.027.
Sonnenwald, D. H. and Pierce, L. G. (2000) ‘Information behavior in dynamic group work contexts: interwoven
situational awareness, dense social networks and contested collaboration in command and control’,
Information Processing & Management, 36(3), pp. 461–479. doi: 10.1016/S0306-4573(99)00039-4.
Sparks, R. S. J. (2003) ‘Forecasting volcanic eruptions’, Earth and Planetary Science Letters, 210(1–2), pp. 1–
15. doi: 10.1016/S0012-821X(03)00124-9.
Van Steenbergen, N., Ronsyn, J. and Willems, P. (2012) ‘A non-parametric data-based approach for probabilistic
flood forecasting in support of uncertainty communication’, Environmental Modelling & Software. Elsevier
Ltd, 33, pp. 92–105. doi: 10.1016/j.envsoft.2012.01.013.
Stirling, A. (2010) ‘Keep it complex’, Nature, 468(7327), pp. 1029–1031. doi: 10.1038/4681029a.
Sword-Daniels, V. et al. (2018) ‘Embodied uncertainty: living with complexity and natural hazards’, Journal of
Risk Research, 21(3), pp. 290–307. doi: 10.1080/13669877.2016.1200659.
Tak, S., Toet, A. and van Erp, J. (2015) ‘Public Understanding of Visual Representations of Uncertainty in
Temperature Forecasts’, Journal of Cognitive Engineering and Decision Making, 9(3), pp. 241–262. doi:
10.1177/1555343415591275.
Teigen, K. H. and Brun, W. (1999) ‘The directionality of verbal probability expressions: effects on decisions,
predictions, and probabilistic reasoning’, Organizational Behavior and Human Decision Processes, 80(2), pp.
155–190.
Thompson, V. A. (2013) ‘Why It Matters’, Perspectives on Psychological Science, 8(3), pp. 253–256. doi:
10.1177/1745691613483476.
Volpe, C. E. et al. (1996) ‘The Impact of Cross-Training on Team Functioning: An Empirical Investigation’,
Human Factors: The Journal of the Human Factors and Ergonomics Society, 38(1), pp. 87–100. doi:
10.1518/001872096778940741.
Walker, W. et al. (2003) ‘Defining uncertainty: a conceptual basis for ucnertainty management in model-based
decision support’, Integrated Assessment, 4(1), pp. 5–17.
Wall, T. U., Meadow, A. M. and Horganic, A. (2017) ‘Developing Evaluation Indicators to Improve the Process
of Coproducing Usable Climate Science’, Weather, Climate, and Society, 9(1), pp. 95–107. doi:
10.1175/WCAS-D-16-0008.1.
Waugh, W. L. and Streib, G. (2006) ‘Collaboration and Leadership for Effective Emergency Management’, Public
Administration Review, 66(s1), pp. 131–140. doi: 10.1111/j.1540-6210.2006.00673.x.
Wein, A. et al. (2016) ‘Communicating with the Public during an Earthquake Sequence : Improving
Communication of Geoscience by Coordinating Roles’, Seismological Research Letters, 87(1), pp. 1–7. doi:
10.1785/0220150113.
Wiedemann, P., Borner, F. and Schultz, H. (2008) ‘Lessons learned: Recommendations for Communicating
conflicting evidence for risk characterization’, in Wiedemann, P. M. and Schultz, H. (eds) The role fo evidence
in Risk Characterisation: Making sense of conflicting data. Weinheim: Wilery-VCH Verlag GmbH & Co.
KGaA, pp. 205–213.
Williams, C. and Dunn, C. E. (2003) ‘GIS in participatory research: Assessing the impact of landmines on
communities in North-west Cambodia’, Transactions of Geographical Information Systems, 7, pp. 393–410.

Hudson-Doyle et al.
Reflections on uncertainty communication: decision-relevant information
Full Insight Paper – Data Issues for Situation Disaster Awareness
Proceedings of ISCRAM Asia Pacific 2018 (K. Stock and D. Bunker, eds).
Zsambok, C. (1997) ‘Naturalistic Decision Making: Where are we now?’, in Zsambok, C. E. and Klein, G. (eds)
Naturalistic Decision Making. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 3–16.