Content uploaded by Scott Everett Robinson
Author content
All content in this area was uploaded by Scott Everett Robinson on Dec 31, 2019
Content may be subject to copyright.
Threat Intensity and the Public Use of Warning
Information: A Quasi-experimental Assessment of the
New Ecology of Weather Information
Scott E. Robinson, Jason Pudlo, and Wesley Wehde
July 5, 2017
1 Introduction
Government organizations work using a variety of tools (Salamon, 2002)1. While most of
the attention in the political science and public administration literatures are on regulatory
activity and the direct provision of services, the government also provides information on a
variety of subjects. On some subjects, the government is a key and irreplaceable provider
of essential information. The government, for example, is a primary provider of information
on a variety of product safety issues. Once released, we know very little about how this
information is (or is not) shared, who seeks or shares this information, etc. These dynamics
represent the underlying ecosystem for public information. It is important for scholarship to
explore this ecosystem of information in parallel to continuing development of research on
other tools such as regulation and service provision.
Given the dense information environment of contemporary life (with people receiving
information from many sources including traditional news sources, social media platforms,
and many others), government organizations need to understand how people receive infor-
mation and then pass that information along to others. The success of a message reaching a
large audience relies on a combination of a broad initial transmission of the message and the
propagation of that information as the initial recipients share the message with others. As
people come to increasingly rely on social media as a source of information, it is likely that
the propogation component of successful message transmission will grow in importance.
This article explores the dynamics of information with an investigation of the specific
case of tornado warning information. Specifically, the analysis involves the sharing of infor-
mation about a series of dangerous storms in Oklahoma in 2016. This manuscript assess the
media through which people received information (via in-person conversation, text message,
Facebook, etc.) as well as the effect of particular disaster experiences on the propensity of
people to use various means to receive (and send) information. Of particular interest is how
members of the public receive information and pass information along to others. The results
indicate that perceived tornado experiences motivate the use of a variety of technologies –
with the strongest effects coming to newer technologies like text messaging a Facebook.
2 Literature
Understanding the role of government information requires a broad understanding of how
the public uses and spread information. Given the dense information environment of con-
temporary life (with people receiving information from many sources including traditional
news sources, social media platforms, and many others), government organizations need to
understand how people receive information and then pass that information along to oth-
ers. The success of a message reaching a large audience relies on a combination of a broad
initial transmission of the message and the propagation of that information as the initial
recipients share the message with others. As people come to increasingly rely on social me-
dia as a source of information, it is likely that the second component of successful message
1The data for this project were collected with support from the National Science Foundation under Grant
No. IIA-1301789. The analysis was conducted with support from NOAA’s Office of Weather and Air Quality
through the U.S. Weather Research Program.
1
transmission will grow in importance.
This manuscript assesses the dynamics of information released by the National Weather
Service as part of a series of dangerous storms in Oklahoma in 2016. We seek to under-
stand how members of the public receive information and pass that information along to
others. This manuscript assess the methods through which people received information (via
in-person conversation, text message, Facebook, etc.) as well as the effect of particular dis-
aster experiences on the propensity of people to use various means to receive (and send)
information. The results indicate that perceived local tornadic activity increases the use of
telephone and text messaging but has no effect on in-person conversations. Furthermore,
perceived tornadic activity increases the use of Facebook - but only during and after the
event, as expected.
2.1 Tornado and Disaster Warnings
The government is a key provider of information during weather events. For example, the
National Weather Service is responsible for both forecasting weather outlooks and issuing
warnings about hazardous weather such as tornadoes (NOAA, 2011). Once a warning is
issued its dissemination can be affected by both the source spreading the information and
the medium through which it is spread.
Past studies suggest that citizens depend upon three main sources for warning informa-
tion. First, information may reach citizens directly from government agencies though chan-
nels such as the Integrated Public Alert and Warning System (Kapucu, Arslan and Demiroz,
2010). This may also include communication from the National Weather Service to local
emergency management. Second, information is also distributed by mediating organizations
such as national and local news organizations, for-profit weather forecasting groups such as
the Weather Channel, or non-profit organizations such as the American Red Cross. This
category might also include apps and other specially designed software that relays hazard
warnings. Third, information is distributed through person-to-person interaction in social
networks (for a description of this process, see Coleman et al. (2011); Brotzge and Donner
(2013); Houston et al. (2015)). By any of these means, the basic data and warnings stem
from announcements from the National Weather Service itself FEMA (2017).
In addition to the sources of information, there are several notable variations on the
medium of communication for crisis information. In this study, we investigate three cate-
gories of traditional, new electronic, and social media to describe how information moves.
Traditional media includes phone calls and in-person conversations. Electronic media in-
cludes email and text messaging services. Finally, social media is used to describe the use of
services like Facebook, Twitter, Instagram, and so on. These categories are similar to those
used by other scholars (Stokes and Senkbeil, 2017).
Emerging scholarship shows that traditional media is still more trusted than social media
(Alexander, 2014). Yet, social media is comparable to other sources of online news and its
credibility can be influenced by specific characteristics of a message. For example, Castillo
finds features of a message on social media, such as the structure of the message or social
media user demographics, affect credibility of a message on the social media service Twitter.
Notably, the inclusion of URLs and hyperlinks to the sources and the propagation tree
increase the credibility of the message (Castillo, Mendoza and Poblete, 2011). In a disaster
2
context, research following a 2011 tsunami in Japan supported these conclusions (Li and
Sakamoto, 2015).
This manuscript focuses on the medium through which the public receives weather-related
information and with which the public shares this information with others. For this reason,
we are focusing on media with which a respondent could both send and receive information.
These sources supplement other media such as television and radio (which have been studied
more extensively, as reviewed above). We look at the social nature of disaster warnings and
the use of social networks and media in disasters.
2.2 The Social Nature of Disaster Warning
Research on tornado warnings has emphasized both the technical nature of the warning
and the social nature of reception, comprehension, and response to a warning. Technical
elements include increases in radar technology and the ability to detect and predict tornadoes
(Simmons and Sutter, 2005), proper sheltering behavior and safety (Sherman-Morris, 2009),
and evaluations of NWS warning products (Sutter and Erickson, 2010; Ripberger et al., 2014;
Lindell et al., 2015; Ash, Schumann and Bowser, 2013). For increases in technology and
improvements in warning products to be effective, there must be a corresponding response
from the public, this is where the social nature of disaster warnings becomes apparent. Mileti
and Sorenson, among others, have observed that in order for a response to happen after a
warning is issued, the source must be trusted or the message verified (1990).
Trust matters for all three sources of warning information. First, in the tornado warning
literature, a great deal of research has been done on trust in the NWS and the dual problems
of false-alarms and missed events (Brotzge, Erickson and Brooks, 2011; Ripberger et al., 2015;
Trainor et al., 2015). In these studies, false-alarms are defined as situations where a tornado
was warned, but no tornado occurred while a missed-event is when no warning is given,
but a tornado occurred. Given advances in radar and early warning technology, missed-
events are increasingly rare but the advances increase the number of false-alarms. Notably,
research suggests that citizens continue to respond to warning information from the NWS
after false-alarms but are more sensitive to missed events. Second, mediating sources, such
as local news, depend on social connections as well. Studies from tornado prone areas of
the country show that TV audiences develop a deep relationship with local meteorologists,
which enhance the credibility of the information they pass along (Sherman-Morris, 2005).
Third, interpersonal communication relies upon existing social trust to disseminate warning
information. Examples from this category include person-to-person conversations, calling
friends or family on the telephone, or, for our purposes, using social media to disseminate
information about storms. Interpersonal communication for receiving and spreading tornado
warnings highlights the social nature of disasters and the ways in which social networks and
social media converge in disaster warning.
The importance of trust in emergency messaging establishes a key point in our under-
standing of the dynamics of public information. Respondents are not passive. They selec-
tively receive and send information – here, based on trust in the source of the information.
This behavior reinforces the importance of seeing public information as involved in an ecosys-
tem of influences rather than as a purely technological or engineering problem.
3
2.3 Social networks and social media in disasters
The social ecology of public information likely depends, in part, on the policy context of the
information. In this analysis we focus on natural disaster and emergency management as
the context of the information being disseminated.
In many ways, disasters are a “wicked problem” (Rittel and Webber, 1973). It is not
just that disasters themselves cause great harm to life and property; they present a problem
that is exceedingly difficult to solve. At all points of the disaster cycle, from planning, to
response, to recovery, and mitigation, there are competing claims about the nature of the
problem and the best solution. Adding a layer of complication to an already challenging
task for emergency managers and communities, disasters are also inherently social (Drabek,
2013; Aldrich and Meyer, 2015). Yet, it is precisely the wicked and social nature of disasters
that opens up disaster warning and response to innovative applications of social networks
and social media.
Trying to address some of the disaster response difficulties in the response and recovery
phases of the World Trade Center Attacks in 2001 and Hurricane Katrina in 2005, Jaeger et
al. highlight the social and technical nature of disasters. Not only were various agencies and
groups of responders not familiar with each other’s procedures, they were also plagued by the
lack of interoperability between their equipment or technical failures. As a solution, Jaeger
et al. argue for “community response grids” that would harness technology to “build and
foster response systems that would aid communities before, during, and after an emergency,
providing channels for contacting authorities, uploading and distributing information, and
coordinating the responses of social networks” Jaeger et al. (2007). Similar observations
about the failure of collaboration by disaster responders after 2001 was made by Comfort et
al., prompting them to advocate for a more adaptive disaster management processes(Comfort
and Kapucu, 2006). In someways, these failures have helped to illuminate the larger network
surrounding disaster response in what some have called “disaster governance”. That is, at all
points of the disaster cycle, governmental agencies are surrounded by other private-sector,
civil society groups, and non-profit organizations (Tierney, 2012). Given advances in social
media over the last decade, this wider audience also includes citizen engagement through
interactive media.
As these examples show, while emergency management has historically used a top-down
approach, researchers of disaster management are increasingly looking at the social and de-
centralized nature of disasters. Wicked problems present the opportunity for increased em-
phasis on collaborative or networked governance structures for disaster management (Head
and Alford, 2015; Kapucu, Arslan and Demiroz, 2010; Robinson, 2006) and incorporating
social media into the warning and response process (Alexander, 2014). This collaboration
suggests a complex information ecosystem for the spread of public information.
The new ecology of public information now includes new social media outlets such as
Facebook or Twitter. Scholars have sought to understand how social media is used by all
actors in the disaster process and at all stages. While the impact of social media is still being
assessed in the early phases of a disaster, social media has become an especially powerful
tool in the response and recovery phases through spreading information to disaster victims,
fighting rumors, fundraising, and organizing volunteers (Bird, Ling and Haynes, 2012; Hu
and Kapucu, 2016; Gao et al., 2011). Our study helps to address this deficiency by adding
4
to a small but diverse and growing body of literature looking at social media and disaster
response.
Most Americans have a social media account of some kind. The Pew Research Center
estimates that nearly 70 percent of Americans have a Facebook account while around 20
percent have a Twitter account (Greenwood, Perrin and Duggan, 2016). Other social media
such as Instagram, a photo sharing service, or LinkedIn, a professional networking service,
rank slightly higher than Twitter but have not been shown to be popular tools during a
disaster. Keeping with previous studies, this project focuses looks at the use of social media
with special attention towards Facebook and less attention towards social media platforms
less used during disaster.
Drawing on research from public administration, disaster research, and computer sci-
ences, several notable findings emerge from the literature. First, Twitter and Facebook are
used differently and by different demographics. Facebook is widely used across demographic
groups while Twitter tends to be used by younger and higher educated Americans than Face-
book. This is true in national non-disaster settings (Greenwood, Perrin and Duggan, 2016)
and seems to hold in the context of an disaster (Bird, Ling and Haynes, 2012; Alexander,
2014). Social media platforms also seem to be used differently by both agencies dissem-
inating information and users who consume the information. This is illustrated to some
extent by differences in how fire and police departments used Facebook and Twitter after
Hurricane Sandy struck the United States in 2102. Facebook was a more interactive service
and agencies engaged in two way conversations with constituents while Twtitter tended to
be used in one-way information provision (Hughes et al., 2014)
Second, source credibility seems to affect the spread of information in social networks.
Analysis of Twitter data shows that the social media posts from news media we less likely
to be shared than posts from individuals with comparable information when controlling for
the number of “likes” a post receives (Li and Sakamoto, 2015). Yet, somewhat in contra-
diction, posts from news media are more likely to be shared in general and are perceived as
more trustworthy (Castillo, Mendoza and Poblete, 2011). In a well-known rumor fighting
campaign, emergency management in Queensland, Australia used social media to aggres-
sively correct misinformation after a series of floods (Bird, Ling and Haynes, 2012; Ehnis
and Bunker, 2012). Research has shown that the presence of urls, hashtags, and @ symbols
seem to increase the perceived credibility of posts and, consequently, the amount of shares
a post receives (Li and Sakamoto, 2015; Wukich and Steinberg, 2013).
Third and related to the second, information mavens (Westerman, Spence and Van
Der Heide, 2012) are evident in the use of social media. An innovative study showed that
social networks on social media pre-disaster contained high percentages of isolates, social me-
dia users posting data not connected with other users. However, once an emergency warning
was issued, the percentages of isolates dropped dramatically as users began sharing informa-
tion from other users about the issued warning information (Tyshchuk et al., 2012). This
suggests that as in past research on social networks, certain individuals act as information
hubs and disseminate warning information at higher rates than others.
The new ecology of public information in emergency management suggests important
propositions for this analysis. First, there is reason to believe that new technologies may have
different patterns of use than traditional technologies for people to share information. While
previous research has suggested some differences in utilization rates across technology, we will
5
assess whether the use of these technologies responds in different ways to weather threats.
Second, different people may react differently to weather related threats. We investigate
this second point by narrowing the analysis to compare the changes of information sharing
behaviors of older respondents and women to the overall average changes initially reported.
As a result, our research design will illuminate how threat intensity affects the public’s use of
different media to receive and send information – and whether the effect of threat intensity
varies by age and gender.
3 Research Design
Investigating the spread of information requires a strategy for observing individual reception
of information and the decision to pass this information along to others. We draw data from a
large survey of Oklahoma residents that took place in 2016. The survey instrument includes
basic demographic information along with specific questions related to the experience of
residents with a then recent series of storms. The questions allow us to assess individual’s
reported experiences with the storms (and the hazards they brought) along with reports of
individual reception and transmission of storm-related information. We use these data to
construct a quasi-experimental test of hypotheses related to the connection between disaster
experience and individual communication decisions (especially as related to social media and
new communication technologies).
3.1 The Survey Instrument
The survey instrument is part of a series of regularly occurring surveys of a carefully con-
structed panel of Oklahoma residents. The panel construction provides a number of advan-
tages over general random digit dialed (RDD) survey designs. First, the construction of the
panel ensures that the demographics will represent the population of Oklahoma as-a-whole.
Second, the iteration of the panel produces a higher response rate among the panel mem-
bers. The higher response rate reduces the likelihood that idiosyncratic sampling problems
related to survey timing could influence responses. Third, the panel design allows the survey
responses to connect to a large body of previous survey questions from the prior survey
batteries (though this analysis does not rely on questions from previous surveys).
The survey included several batteries of questions related to experiences with weather
and climate change and energy development questions. A large section focused on a series of
storms that passed through the state in the three months prior to the survey administration
(hopefully ensuring the recall of survey respondents about their experiences with the storm).
Each respondent was asked if they remembered enough about the particular storm in their
community to answer questions about it. If they said yes, he or she was then asked how he
or she was affected by the randomly selected storm (if at all). Respondents were allowed
to report effects including rain, winds, flooding, hail, and tornados (each reported indepen-
dently). Respondents also had the opportunity to report that they were unaffected. We
classify all people who reported that they were affected or reported either winds, flooding,
hail, or tornados as an effected respondent. For this analysis, we are considering differences
among affected respondents and exclude those who reported no effects or an inability to
6
recall.
Each respondent was then asked a series of questions about whether they received a
tornado warning and, if so, through what media. The survey followed with questions related
to whether respondents had sent or received information through a variety of media including
personal conversation, phone, e-mail, text message, Facebook, and others. We use the survey
responses related to reported disaster experience along with the questions on communication
behaviors to test the hypotheses presented at the end of the previous section.
3.2 The Weather Experience Quasi-Experiment
The general research strategy for this analysis relies on a quasi-experimental design. Tornado
and weather events are notoriously difficult to predict and affect areas randomly (within a
largely uniform geography like Oklahoma). Given that the weather events themselves are
randomly distributed within the state, the disaster experience variables are randomly as-
signed to respondents. It is important to emphasize that this randomness is only accurate
within a bounded geographic area like the state of Oklahoma. One can distinguish be-
tween the probability of tornado strikes in Oklahoma from the probability of such strikes
in Delaware - for example. However, within Oklahoma tornadoes have been remarkably
difficult to predict and there are no high or low probability areas within the state.
As a result of this quasi-experimental design, we can provide a relatively simple test
of the effects of disaster experience on communication behaviors. Any differences observed
between groups with different disaster experiences is attributable to the disaster experiences
themselves rather than (measured or unmeasured) covariates because of the random exposure
to the tornado. The effect of membership in any one of the disaster experience groups
is directly testable as a difference of means of each behavior compared to the alternative
experiential groups.
3.3 Assessment of the Quasi-experimental Design
The evidence that tornadic activity is random across the study area of the state of Oklahoma
is strong. However, there is some reason to be hesitant to discard the possibility that
unmeasured correlates related to geography (and all other correlates spatially distributed)
could interfere with any experimental inferences. This study analyzes data resulting from
two storm fronts in the spring of 2016. While the location of these fronts was random (thus,
the prior probability of exposure for each respondent was equal), the exposures resulting
from the two storms were not uniform. As a result, it is possible that the small sample
of experienced storm fronts did create a peculiar sample within the state. This problem is
similar to the possibility of randomization failure in survey or laboratory experiments (where,
despite randomization, the proportion of treatment and control samples are not equivalent
on key independent variables).
To test this possibility, we first compared the samples of respondents who received the
specific storm treatments to those who received the general spring treatment on several key
demographic variables. This test allows us to examine if remembering the storm storm and
experiencing any of its associated hazards works as a mechanism of random assignment in
7
Table 1: Randomization Checks for Survey Question Tracks
Track
Specific Storm Entire Spring
Age (in years)** 59.4 60.5
Hispanic 1.9% 1.5%
Education 52.2% 50.5%
Employment 51.2% 48.4%
Race 88.1% 88.2%
Income $70,800 $71,300
Square foot of house 2010 2060
Difference of means t-test, ** significant at .05 level.
Table 2: Randomization Checks for Reported Tornado Experience
Perception of Tornado
Yes No
Age (in years)** 58.5 60.1
Hispanic** 0.7% 2.2%
Education* 50.1% 504.8%
Employment 50.3% 52.2%
Race 87.3% 88.7%
Income $71,000 $71,400
Square foot of house 2010 2030
Difference of means t-test, ** significant at
.05 level, * significant at .10 level.
our survey. To do so, we perform a series of difference of means t-tests. Table 1 presents the
results of these tests.
Table 1 suggests that, for the given demographic characteristics, the two storm systems
referenced in our survey have generally worked as a random assignment mechanism. We find
only one statistically significant difference in the average ages of our respondents. Respon-
dents who either did not remember the storm or remembered but were not affected by the
storm are on average 1 year older than respondents who remembered and experienced the
storm. While statistically significant, this difference is small and substantively unimportant.
Therefore, we find evidence that the storm system, generally, created an as-if random assign-
ment. Next we move on to assess whether or not the tornadoes within these storm systems
work as random assignment mechanisms. To do this, we compare those who reported expe-
riencing some storm event (rain, hail, wind, or flood) but no tornado to those who reported
experiencing a tornado.
As Table 2 suggests, when comparing those who reported experiencing the storm in
some respect, those who reported experiencing a tornado are somewhat different from their
counterparts. Those who reported experiencing the tornado are slightly younger on average.
They are also less likely to be Hispanic or have received a Bachelors degree or above. Again,
8
however, these differences are substantively small. Additionally, our data suffer from a small
cell size problem with Hispanic respondents as only 4 respondents were both Hispanic and
reported experiencing a tornado. Given these findings, we argue that, in our case, the
tornado has created as-if random assignment within our data. Therefore, in the following
results section we treat the reported experience of a tornado as random treatment potentially
affecting communication patterns of respondents.
4 Results
Table 3 presents the full results for communication patterns using “traditional” communica-
tion methods as well as text messaging. Respondents were asked to identify the number of
people they contacted (Send) or contacted them (Receive) during or immediately before the
hazard. Therefore, our analysis takes place among only those respondents who both remem-
bered the storm and claimed to have experienced at least one related hazard. We then use
reported experience of a tornado as a randomized treatment which can affect communication
patterns. Thus, our comparison is between those who reported experiencing a tornado from
the storm and those who reported some other hazard (rain, wind, hail, or flooding) but not
a tornado. A few important trends emerge from this table.
Table 3: Comparing Communication Patterns on Traditional
and Electronic Media among Respondents Based on Re-
ported Tornado Experience
Yes [% (n)] Total N
Send Conversation Tornado 53.3 (244) 458
No Tornado 49.8 (370) 742
Receive
Conversation
Tornado 39.3 (172) 437
No Tornado 34.8 (250) 720
Send Phone Tornado 64.4 (320) 497
No Tornado 58.9 (478) 812
Receive Phone Tornado 54.3 (257) 473
No Tornado 47.0 (363) 773
Send Text Tornado 52.8 (249) 472
No Tornado 42.0 (314) 748
Receive Text Tornado 45.4 (211) 463
No Tornado 36.5 (269) 737
Send E-mail Tornado 6.0 (24) 334
No Tornado 8.3 (55) 662
Receive E-mail Tornado 7.0 (28) 402
No Tornado 6.3 (45) 709
Bold cells represent statistically significant differences
of proportions using a z-test at the .05 level.
First, our data suggest that reported experience of a tornado affects communication
9
through phone calls and text but does not affect whether a person talks face-to-face to
another person about the storm or its hazards. This suggests in-person communication
during storms may differ from other forms. One explanation for this may be that most
respondents were at home during the time of the storm, likely limiting the number of people
they could talk to during this period.
Finally, and more importantly, Table 3 suggests that reported experience of a tornado
reduces the proportions of respondents who report not communicating with others and in-
creases the proportions who report communicating with others. The first column on the
other hand reveals that tornado experience perception increases the proportion of respon-
dents who communicate with others through various methods – including phone and text
messaging. The effect holds for both sending and receiving information through these media.
Specifically, looking at phone calls, we see that experiencing a tornado results in an
approximately 5% increase in the likelihood an individual will send a phone call. Similarly,
experiencing a tornado results in an about 7% increase in the likelihood an individual reports
receiving a phone call. For text messages, the differences are even larger. Individuals who
report experiencing a tornado are over 10% more likely to have sent a text about the storm.
For receiving a text, tornado experience increases the likelihood of receiving a text by just
less than 9%.
These results become even clearer in Figures 1 and 2. For all three of the four methods of
communication, respondents who perceived experiencing a tornado are more likely to send
and receive any communication. In the cases of phone calls and text messages, these differ-
ences are statistically significant. Given these effect sizes, our data suggest that perceived
experience of a tornado has a stronger effect on the text messaging behavior of respondents
than other, more “traditional” methods of communication.
Figure 1: The Effect of Tornado Exposure on Traditional Media Use
10
Figure 2: The Effect of Tornado Exposure on Electronic Media Use
Figures 1 and 2 also suggest that respondents are more likely to report sending commu-
nication than receiving communication. These findings could suggest a number of processes.
They might represent respondents’ more accurate memory of their own actions rather than
others or respondents’ bias to attribute communication to themselves rather than others
as a form of social desirability bias. These results are especially pronounced for unwritten
communication methods, as opposed to texts and e-mails. Figure 1 demonstrates that re-
spondents are more likely to report initiating a phone call or conversation as opposed to
receiving one. On the other hand, Figure 2 suggests that proportions for sending and re-
ceiving texts and e-mails are quite similar. Respondents are as likely to report sending a
text/e-mail as they are receiving one.
Moving to social media, our data suggest similar trends. Table 4 displays who reported
seeing Facebook at various times by perception of tornado experience. For 3 of the 4 com-
parisons our data reveal statistically significant differences in proportions. This evidence
suggests that respondents who perceived experiencing a tornado were more likely to see rel-
evant information on Facebook, except before the storm. These results suggest experience
of a tornado does not affect communication on Facebook before the storm. This makes
sense given that the storm has not yet happened. Rather, the storm works as an impetus to
provoke information seeking on Facebook during and after the storm. This pattern suggests
that Facebook may serve better in a post-event capacity for spreading response and recovery
information than immedite warning information.
Figure 3 clarifies this relationship. The tornado reporting respondents see information
on Facebook more on average. This is especially true after the storm but also true during
the storm. However, use of Facebook is generally much higher after the storm as well. The
potential reasons for this are plethora. Facebook users might be using the website to check
on friends and family or to see damage assessments. Additionally, users may be intending
to post information about their safety and at the same time see information about others.
11
Table 4: Comparing Social Media Usage among
Respondents Based on Reported Tornado Experi-
ence
% Yes (n)
See Facebook
Before
Tornado 18.6 (106)
No Tornado 18.3 (177)
See Facebook
During
Tornado 19.7 (112)
No Tornado 14.1 (139)
See Facebook After Tornado 41.5 (236)
No Tornado 29.3 (283)
See Facebook Ever Tornado 49.9 (284)
No Tornado 40.3 (389)
Bold Cells represent statistically significant
differences of proportions using a z-test at the
.05 level.
Figure 3: The Effect of Tornado Exposure on Facebook Use
There seems to be a problem with the figure 2 file.
12
Our results suggest that tornadoes, independent of all other storm characteristics, gen-
erally increase communication activities. These effects are least prominent for face-to-face
communication or conversations. In these cirumstanes individuals have limited ability to
increase in-person communication during a storm. Generally, they are at home and are lim-
ited in their ability to travel by the storm. When looking at which media are preferred for
communication of storm information we find that phone calls are generally the most com-
mon while e-mail is the least common. When comparing sending to receiving, we find clear
differences between communication methods. During storms, respondents are more likely to
report initiating communication compared to receiving communication. This is especially
true for “traditional” methods such as face-to-face and phone calls. On the other hand,
respondents report similar rates of sending and receiving texts and e-mails. Finally, we see
that increased threat intensity does increase communication efforts, primarily through the
use of text messages and Facebook.
5 The Information Ecology of Sub-populations
Having examined the effects of threat intensity, specifically reported tornado experience,
on communication patterns generally we now examine how these patterns differ by sub-
population. In particular, we are interested in the effects of age and gender on communication
patterns in the aftermath of storms. We exclude e-mails from our sub-population analyses
because it was by far the least popular communication method among our respondents.
Only 79, less than 7.5% of the total, respondents reported sending an e-mail and even fewer,
73, reported receiving an e-mail with storm information. Examining sub-populations of
these already small samples would provide little insight into the effects of gender or age on
communication patterns through e-mail. Therefore, we examine text messaging as our sole
example of electronic media. We proceed in a fashion similar to the previous section first
examining all methods of communication by gender, generally and then specifically. We then
investigate how age plays a role in communication patterns following a severe weather event
both generally and through specific mediums.
5.1 Gender, Threat Intensity, and Disaster Communication
In Table 5, we see that generally, across media and threat intensities, women are more likely
to send and report receiving communication than their male counterparts. Holding threat
intensity constant, we find statistically significant differences between men and women in 8 of
14 cases. Interestingly 6 of these cases occur for individuals who do not report experiencing a
tornado. This suggests that a high threat event such as a tornado reduces the communication
disparity between men and women. This also becomes evident when we examine the effects of
threat intensity on the communication patterns within gender. Men who report experiencing
a tornado report higher communication levels across all methods, reaching a .05 level of
statistical significance in 4 of 7 comparisons. Women who report experiencing a tornado also
communicate at higher rates; however, only 2 of 7 comparisons reach a .05 level of statistical
significance. Table 5 demonstrates that tornado experience has a gender specific effect on
communication methods other than face-to-face communication. To better illustrate these
13
Table 5: Communication Methods by Gender and Threat Intensity
Tornado No Tornado
Men Women Men Women
Send Conversation 51.9 (96) 56.7 (148) 47.2 (135) 52.8 (235)
Receive Conversation 33.9 (60)* 44.4 (112)* 30.5 (86)* 38.0 (164)*
Send Phone 63.8 (125) 66.9 (194) 51.8 (155)* 63.5 (323)*
Receive Phone 47.6 (90)* 59.9 (166)* 42.8 (125)* 50.0 (238)*
Send Text 53.2 (100) 53.2 (148) 36.6 (102)* 45.6 (212)*
Receive Text 45.2 (85) 46.6 (125) 30.7 (86)* 40.6 (183)*
See Facebook 45.9 (107) 52.7 (176) 33.8 (122)* 44.3 (267)*
Bold Cells represent statistically significant differences of proportions using a
z-test at the .05 level within gender but between threat intensity (tornado vs.
no tornado). The * represents statistically significant differences of proportions
using a z-test at the .05 level by gender within threat intensity.
relationships, we examine in greater detail phone, text, and Facebook communication by
gender.
Figure 4: The Effect of Tornado Exposure on Phone Calls by Gender
In Figure 4, we see female respondents are more likely to initiate and report receiving
phone calls, compared to their male counterparts, regardless of threat intensity. In Table
5 we see that reported tornado experience causes approximately 13% more men to report
sending a phone call and approximately 10% more women to report receiving a phone call.
Other effects were much smaller, between 3 and 5 percent, and therefore not statistically
significant.
14
Figure 5: The Effect of Tornado Exposure on Text Messaging by Gender
As with phone calls, female respondents are more likely to share or receive information
than male respondents. However, the pattern is generally less distinct for gender. As such,
Figure 5 suggests that the increase in male text message usage in response to tornadic activity
eliminates some of the general difference in text messaging usage by gender. While male
respondents are generally less communicative than female respondents, for text messaging
sending and receiving, they are more strongly activated by intense storm threats such as a
tornado. Experiencing a tornado causes an increase of almost 20% in men sending texts and
an almost 15% increase in men reporting receiving texts.
15
Figure 6: The Effect of Tornado Exposure on Facebook Usage by Gender
Finally, we examine the gender differences of respondents in their reports of seeing storm
information on Facebook at any time in Figure 6. Females are, once again, more likely
to see storm information on Facebook than males are. However, again, males who report
experiencing a tornado see storm information on Facebook at higher rates, almost 15%, than
males who did not report experiencing a tornado thus reducing the gender gap. Tornado
experience also increases the proportion of women of see storm information on Facebook by
about 12%.
Overall, gender plays an important role in communication in the aftermath of severe
storms. While women communicate more on average, men are more likely to be prompted
to communicate by a more intense threat such as a tornado. However, these conclusions
16
Table 6: Communication Methods by Age and Threat Intensity
Tornado No Tornado
<45 >64 <45 >64
Send Conversation 65.4 (51)* 45.0 (72)* 62.8 (76)* 45.1 (129)*
Receive Conversation 48.7 (37) 34.6 (53) 55.5 (65)* 27.8 (78)*
Send Phone 65.9 (54) 65.7 (113) 58.6 (75) 61.2 (205)
Receive Phone 50.6 (40) 60.4 (102) 47.1 (57) 49.8 (160)
Send Text 75.3 (67)* 33.3 (52)* 63.9 (85)* 26.8 (75)*
Receive Text 58.8 (50)* 33.5 (53)* 55.9 (71)* 25.4 (71)*
See Facebook 77.3 (75)* 28.8 (59)* 60.9 (92)* 27.6 (110)*
Bold Cells represent statistically significant differences of proportions
using a z-test at the .05 level within age group but between threat intensity.
The * represents statistically significant differences of proportions using a
z-test at the .05 level by age group within threat intensity.
apply primarily to text messaging and Facebook communication. The effects of a tornado
on male communication patterns are quite strong, often between 10% and 20%, resulting
in communication rates more similar to their female counterparts. Having demonstrated
differences in communication by gender, we now turn our attention to effect of age on com-
munication in response to severe weather. Table 6 provides an overview of the contingent
effect of tornado experience and age on communication patterns.
5.2 Age, Threat Intensity, and Disaster Communication
Unlike the simple pattern for gender in which female respondents communicate more fre-
quently for every method, the pattern for age groups is more complicated. Specifically,
younger respondents communicate more through each method except phone. Older respon-
dents, that is those reporting an age of 65 and older, are more likely to report sending and
receiving phone calls about the storm than younger (under 45) respondents though not sta-
tistically significantly2. Table 6 suggests communication patterns are highly contingent on
age as 9 of 14 possible age comparisons reach a .05 level of statistical significance. On the
other hand, only 2 of 14 possible comparisons holding age constant but varying threat in-
tensity result in statistically significant results. The effect of age on communication is stark
in many cases as in Figures 7 and 8.
2We use these cut offs for a number of reasons. First, our respondents skew much older on average. While
we might prefer to use an even younger group for comparison, this results in too few respondents. Therefore,
we use 45 as our cutoff, in so doing our comparison group accounts for 17% of respondents. We use 65 as
our upper cutoff as this is a policy relevant age- when individuals are eligible for Medicare. This includes
the top 36% of our respondents by age.
17
Figure 7: The Effect of Tornado Exposure on Conversation by Age
Younger respondents are approximately 20% more likely to report initiating a conversa-
tion about the storm than older respondents, regardless of tornado experience. For those
who did not experience a tornado, younger respondents report receiving a conversation at
over twice the rate of older respondents. Interestingly, tornado experience actually depresses
or lowers the proportion of younger respondents who report receiving a conversation, though
not statistically significantly.
Figure 8: The Effect of Tornado Exposure on Text Messaging by Age
18
The relationship for text messaging is even stronger as shown in Figure 8. For example,
among those who experienced a tornado, 40% more younger respondents report sending a
text than older respondents. Regardless of tornado experience, younger respondents report
using text messages at rates over twice that of their older counterparts. The general re-
lationship between age and communication is apparent. However, to better illustrate the
contingent relationship between age and threat intensity we examine in greater detail phone
and Facebook communication.
Figure 9: The Effect of Tornado Exposure on Phone Calls by Age
Unlike with conversation, Figure9 demonstrates older participants are actually slightly
more likely to share or report receiving storm information through phone calls than their
younger counterparts. Additionally, we see that about 10% more older individuals report
receiving a phone call if they have experienced a tornado as compared to older individuals
who did not report experiencing a tornado. The data suggest that older individuals may
be receiving more phone calls , possibly from friends and family checking in on them, when
they experience a tornado as compared to when they do not.
19
Figure 10: The Effect of Tornado Exposure on Facebook Usage by Age
Strikingly, Figure 10 suggests younger respondents are much more likely to see storm
information on Facebook than older respondents. Regardless of tornado experience, younger
respondents report seeing information on Facebook at a rate over twice that of older respon-
dents. Additionally, younger respondents who report experiencing a tornado are also more
active in seeing information on Facebook than younger respondents who did not report expe-
riencing a tornado. According to our data, experiencing a tornado results in approximately
16% more younger respondents seeing storm information on Facebook.
Examining sub-populations communication patterns has demonstrated potential implica-
tions for targeting of storm warnings and storm information messages. Across all methods,
females are more likely to communicate, send and receive, than their male counterparts,
20
regardless of threat intensity. Threat intensity more strongly affects male communication
patterns, especially for electronic and social media but also making phone calls. Younger
respondents, in this case under the age of 45, generally communicate more than their older,
65 and older, counterparts, except in the case of phone usage. Older respondents are more
likely to report sharing or receiving storm information through phone calls than their younger
counterparts. Threat intensity is related with older individuals receiving more phone calls
and younger individuals seeing information more on Facebook. These patterns suggest the
digital divide affects disaster communication and cannot be entirely overcome by increased
in-person communication for older individuals.
6 Conclusion
The storms of the spring of 2016 served as a useful opportunity to learn more about how
people receive and use information about risks and hazards. People from a wide variety of
backgrounds living in a wide variety of circumstances faced a similar threat. This study takes
advantage of the quasi-random exposure of the residents of Oklahoma to test propositions
about the effect of the tornado warning on people’s use and distribution of storm information,
including warnings.
The results suggest that the storm threat did not affect the tendency of respondents
to engage in conversation (sending or receiving such in-person communication). This non-
effect is interesting in two respects. First, much of the conventional wisdom in disaster
research suggests that people rely on in-person communication to spread disaster information
– especially preparedness information. In the event warning process, the percentage reporting
conversations are lower than most of the other forms of communication and unaffected by
the storm warning. The communication the storm directly inspires seems to channel through
the other forms of communication suggesting that these other forms of communication have
a particularly important role in the storm warning process. Second, increased reliance on
other forms of communication may change the nature of storm warning information. Previous
evidence has suggested that personal conversations are the most trusted form of information.
Does a transition to other media reduce trust in the messages that increasingly rely on these
other channels? These questions warrant future investigation.
Our research design allowed us to look for asymmetric effects on the propensity to send
and receive messages via various technologies, though we found no such asymmetries. Where
the tornado warnings spurred additional communication, it did so to both the sending and
the receiving side of the communication. The effect sizes were of similar magnitudes – even
where the base probabilities were different. It may require a different approach to researching
this question to reveal any asymmetries of this kind due to the parallelism in the question
wording format.
The most interesting differences were in the effect sizes of the warning instigated changes
in phone, text, and Facebook messaging. The increased phone activity were 5.5% and 7.3%
for sending and receiving phone calls respectively. The probability increase for sending text
messages was 10.8% while receiving was 8.9%. Similarly, the proportion seeing information
on Facebook increased 9.6%. The effect of the tornado warning was larger on the newer forms
of communication technology. A deeper assessment of the effect on people of different gender
21
and ages reveals a combination of mitigating and reinforcing forces. This may indicate where
the elasticity in communication lies. Where we can change communication behavior is in
the new forms of communication - while patterns of traditional communication by in-person
conversation is inelastic (and phone, less elastic than new technologies).
While these findings are an interesting early assessment of the effect of threat intensity
on the public’s use of communication to spread warning information, there are several lim-
itations of the design that limit generalizability. As is often the case with disaster-oriented
research, this project only assessed behaviors in a specific geographic area (Oklahoma) and
in regards to a specific hazard (a tornadic storm system). The generalizability to other pop-
ulations and other hazards is an open question. The experimental design provides strong
confidence in the internal validity of the effect of the tornado’s effect on communication
patterns but the external validity to related issues like hurricanes and associated warnings
is an open question.
With these limitations in mind, this has proven to be an interesting initial exploration of
the public’s use of communication media in regards to disaster warnings. There are several
natural next steps. First, while there seems to be a great deal of communication within the
population during and after the storm, the communication intensity is not evenly distributed.
This assessment treated communication as a binary of whether people communicated or
not. This conceals a great deal of variation in the frequency of communication - with
some people making many Facebook posts and others only one. Learning who posts most
often could provide useful insight into the nature of the communication network for tornado
warnings. Second, not all respondents were at home when the warnings were issued. We
should investigate any differences in information seeking or sharing behaviors among those
at work compared to those at home (the latter being a large majority of the respondents).
Finally, this research is based on recalled reception of the tornado warning. It may be the
case that this recall is faulty. We could incorporate data on the geographic area of the
actual warnings to compare to the recalled tornado warnings. The distribution of the errors
is important for the proper assessment of the costs and benefits of different warning strategies
and the threshold to declare a warning (or the geographic reach of the warning).
After this first step, we are excited to explore the complex behaviors of the public as they
learn of disasters - and tell others.
References
Aldrich, Daniel P. and Michelle A. Meyer. 2015. “Social Capital and Community Resilience.”
American Behavioral Scientist 59(2):254–269.
Alexander, David E. 2014. “Social Media in Disaster Risk Reduction and Crisis Manage-
ment.” Science and Engineering Ethics 20(3):717–733.
Ash, Kevin D., Ronald L. Schumann and Gregg C. Bowser. 2013. “Tornado Warning Trade-
Offs: Evaluating Choices for Visually Communicating Risk.” Weather, Climate, and So-
ciety 6(1):104–118.
Bird, Deanne, Megan Ling and Katharine Haynes. 2012. “Flooding Facebook - the use of so-
22
cial media during the Queensland and Victorian floods.” Australian Journal of Emergency
Management, The 27(1):27.
Brotzge, J., S. Erickson and H. Brooks. 2011. “A 5-yr Climatology of Tornado False Alarms.”
Weather and Forecasting 26(4):534–544.
Brotzge, J. and W. Donner. 2013. “The Tornado Warning Process: A Review of Current
Research, Challenges, and Opportunities.” Bulletin of the American Meteorological Society
94(11):1715–1733.
Castillo, Carlos, Marcelo Mendoza and Barbara Poblete. 2011. Information Credibility on
Twitter. In Proceedings of the 20th International Conference on World Wide Web. ACM
pp. 675–684.
Coleman, Timothy A., Kevin R. Knupp, James Spann, J. B. Elliott and Brian E. Peters.
2011. “The History (and Future) of Tornado Warning Dissemination in the United States.”
Bulletin of the American Meteorological Society 92(5):567–582.
Comfort, Louise K. and Naim Kapucu. 2006. “Inter-organizational coordination in extreme
events: The World Trade Center attacks, September 11, 2001.” Natural Hazards 39(2):309–
327.
Drabek, Thomas E. 2013. The Human Side of Disaster, Second Edition. 2nd edition ed.
Boca Raton, FL: CRC Press.
Ehnis, Christian and Deborah Bunker. 2012. “Social media in disaster response : Queensland
Police Service - public engagement during the 2011 floods.” ACIS 2012 : Location, location,
location : Proceedings of the 23rd Australasian Conference on Information Systems 2012
pp. 1–10.
FEMA. 2017. “Integrated Public Alert & Warning System |FEMA.gov.”.
URL: https://www.fema.gov/integrated-public-alert-warning-system
Gao, Huiji, Xufei Wang, Geoffrey Barbier and Huan Liu. 2011. Promoting Coordination for
Disaster Relief – From Crowdsourcing to Coordination. In Social Computing, Behavioral-
Cultural Modeling and Prediction. Springer, Berlin, Heidelberg pp. 197–204.
Greenwood, Shannon, Andrew Perrin and Maeve Duggan. 2016. Social Media Update 2016.
Technical report Pew Research Center: Internet, Science & Tech.
Head, Brian W. and John Alford. 2015. “Wicked problems: Implications for public policy
and management.” Administration & Society 47(6):711–739.
Houston, J. Brian, Joshua Hawthorne, Mildred F. Perreault, Eun Hae Park, Marlo Gold-
stein Hode, Michael R. Halliwell, Sarah E. Turner McGowen, Rachel Davis, Shivani Vaid,
Jonathan A. McElderry and Stanford A. Griffith. 2015. “Social media and disasters: a
functional framework for social media use in disaster planning, response, and research.”
Disasters 39(1):1–22.
23
Hu, Qian and Naim Kapucu. 2016. “Information Communication Technology Utilization for
Effective Emergency Management Networks.” Public Management Review 18(3):323–348.
Hughes, Amanda L., Lise A. A. St. Denis, Leysia Palen and Kenneth M. Anderson. 2014. On-
line Public Communications by Police & Fire Services During the 2012 Hurricane Sandy.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New
York, NY, USA: ACM pp. 1505–1514.
Jaeger, Paul T., Ben Shneiderman, Kenneth R. Fleischmann, Jennifer Preece, Yan Qu and
Philip Fei Wu. 2007. “Community response grids: E-government, social networks, and
effective emergency management.” Telecommunications Policy 31(10-11):592–604.
Kapucu, Naim, Tolga Arslan and Fatih Demiroz. 2010. “Collaborative emergency manage-
ment and national emergency management network.” Disaster Prevention and Manage-
ment: An International Journal 19(4):452–468.
Li, Huaye and Yasuaki Sakamoto. 2015. “Re-Tweet Count Matters: Social Influences on
Sharing of Disaster-Related Tweets.” Journal of Homeland Security and Emergency Man-
agement 12(3):737–761.
Lindell, Michael K., Shih-Kai Huang, Hung-Lung Wei and Charles D. Samuelson. 2015.
“Perceptions and expected immediate reactions to tornado warning polygons.” Natural
Hazards 80(1):683–707.
Mileti, Dennis S. and John H. Sorensen. 1990. Communications of Emergency Public Warn-
ing: A Social Science Perspective and State-of-the-Art Assessment. Technical report Fed-
eral Emergency Management Agency Washington, D.C.: .
NOAA. 2011. NOAA’s National Weather Service Strategic Plan: Building a Weather-Ready
Nation. Technical report.
Ripberger, Joseph T., Carol L. Silva, Hank C. Jenkins-Smith, Deven E. Carlson, Mark James
and Kerry G. Herron. 2015. “False Alarms and Missed Events: The Impact and Origins
of Perceived Inaccuracy in Tornado Warning Systems.” Risk Analysis 35(1):44–56.
Ripberger, Joseph T., Carol L. Silva, Hank C. Jenkins-Smith and Mark James. 2014. “The
Influence of Consequence-Based Messages on Public Responses to Tornado Warnings.”
Bulletin of the American Meteorological Society .
Rittel, Horst W. J. and Melvin M. Webber. 1973. “Dilemmas in a general theory of planning.”
Policy Sciences 4(2):155–169.
Robinson, Scott E. 2006. “A Decade of Treating Networks Seriously.” Policy Studies Journal
34(4):589–598.
Salamon, Lester M. 2002. The Tolls of Government: A Guide to the New Governance. New
York, NY, USA: Oxford University Press.
24
Sherman-Morris, Kathleen. 2005. “Tornadoes, television and trust—A closer look at
the influence of the local weathercaster during severe weather.” Environmental Hazards
6(4):201–210.
Sherman-Morris, Kathleen. 2009. “Tornado warning dissemination and response at a uni-
versity campus.” Natural Hazards 52(3):623–638.
Simmons, Kevin M. and Daniel Sutter. 2005. “WSR-88D Radar, Tornado Warnings, and
Tornado Casualties.” Weather and Forecasting 20(3):301–310.
Stokes, Courtney and Jason C. Senkbeil. 2017. “Facebook and Twitter, communication and
shelter, and the 2011 Tuscaloosa tornado.” Disasters 41(1):194–208.
Sutter, Daniel and Somer Erickson. 2010. “The Time Cost of Tornado Warnings and the
Savings with Storm-Based Warnings.” Weather, Climate, and Society 2(2):103–112.
Tierney, Kathleen. 2012. “Disaster Governance: Social, Political, and Economic Dimen-
sions.” Annual Review of Environment and Resources 37(1):341–363.
Trainor, Joseph E., Danielle Nagele, Brenda Philips and Brittany Scott. 2015. “Tornadoes,
Social Science, and the False Alarm Effect.” Weather, Climate, and Society 7(4):333–352.
Tyshchuk, Y., C. Hui, M. Grabowski and W. A. Wallace. 2012. Social Media and Warning
Response Impacts in Extreme Events: Results from a Naturally Occurring Experiment.
In 2012 45th Hawaii International Conference on System Sciences. pp. 818–827.
Westerman, David, Patric R. Spence and Brandon Van Der Heide. 2012. “A social network
as information: The effect of system generated reports of connectedness on credibility on
Twitter.” Computers in Human Behavior 28(1):199–206.
Wukich, Clayton and Alan Steinberg. 2013. “Nonprofit and Public Sector Participation in
Self-Organizing Information Networks: Twitter Hashtag and Trending Topic Use During
Disasters.” Risk, Hazards & Crisis in Public Policy 4(2):83–109.
25
