ArticlePDF Available

The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016


Abstract and Figures

This study examines the agenda-setting power of fake news and fact-checkers who fight them through a computational look at the online mediascape from 2014 to 2016. Although our study confirms that content from fake news websites is increasing, these sites do not exert excessive power. Instead, fake news has an intricately entwined relationship with online partisan media, both responding and setting its issue agenda. In 2016, partisan media appeared to be especially susceptible to the agendas of fake news, perhaps due to the election. Emerging news media are also responsive to the agendas of fake news, but to a lesser degree. Fake news coverage itself is diverging and becoming more autonomous topically. While fact-checkers are autonomous in their selection of issues to cover, they were not influential in determining the agenda of news media overall, and their influence appears to be declining, illustrating the difficulties fact-checkers face in disseminating their corrections.
Content may be subject to copyright.
new media & society
1 –22
© The Author(s) 2017
Reprints and permissions:
DOI: 10.1177/1461444817712086
The agenda-setting power of
fake news: A big data analysis
of the online media landscape
from 2014 to 2016
Chris J Vargo
University of Colorado Boulder, USA
Lei Guo and Michelle A Amazeen
Boston University, USA
This study examines the agenda-setting power of fake news and fact-checkers who
fight them through a computational look at the online mediascape from 2014 to 2016.
Although our study confirms that content from fake news websites is increasing, these
sites do not exert excessive power. Instead, fake news has an intricately entwined
relationship with online partisan media, both responding and setting its issue agenda.
In 2016, partisan media appeared to be especially susceptible to the agendas of fake
news, perhaps due to the election. Emerging news media are also responsive to the
agendas of fake news, but to a lesser degree. Fake news coverage itself is diverging and
becoming more autonomous topically. While fact-checkers are autonomous in their
selection of issues to cover, they were not influential in determining the agenda of news
media overall, and their influence appears to be declining, illustrating the difficulties fact-
checkers face in disseminating their corrections.
Big data, computational social science, fact-checking, fake news, intermedia agenda
setting, journalism, misinformation, network agenda setting, partisanship
Corresponding author:
Chris J Vargo, College of Media, Communication and Information, University of Colorado Boulder,
1511 University Ave, Boulder, CO 80309-0478, USA.
712086NMS0010.1177/1461444817712086new media & societyVargo et al.
2 new media & society 00(0)
In late 2016, as the US election day approached, “fake news” gained growing public
interest. In November and December, more people Googled the phrase than the com-
bined previous 15 months (Google Trends, 2017). Now, fake news can be “produced
purposefully by teenagers in the Balkans or entrepreneurs in the United States seeking to
make money from advertising …” (Maheshwari, 2016). Hundreds of websites have
popped up around the Internet that appear credible at face value but are fake in nature
(Silverman, 2016).
Journalists have little ability to proactively fight fake news. Even worse, partisan
media can be susceptible to its influence (e.g. Collins, 2016). Other news organizations
fight fake news. The BBC, for instance, has announced a commitment to debunk fake
news that is shared widely on social media (Jackson, 2017). Fact-checking organiza-
tions have become another bulwark against fake news with PolitiFact,,
ABC News, the Associated Press, and Snopes all fighting it on Facebook (Isaac, 2016).
However, this reactive desire to thwart fake news has required traditional media to
divert resources—in the form of time and attention—to fighting it (Easton, 2016).
What is worse, by being forced to respond to fake news journalists may be affording
fake news websites with the ability to push topics, issues, and even attributes into the
public agenda.
What we know about fake news so far is predominantly based on anecdotal evidence.
Empirical research is sparse as to the greater effects fake news has had on journalistic prac-
tices in different media outlets. Has fake news disrupted the ways real news report? Does
fake news have the ability to shift journalistic attention—especially those from partisan
media—to and from issues? Likewise, while a number of fact-checking organizations are
dedicated to publicizing and correcting factual errors (Amazeen, 2013), little is known
about the extent to which they quell fake news and influence other media coverage. Do
fact-checking activities attract attention from the greater journalism community?
To answer these questions, this article will leverage intermedia agenda-setting theory
and the Network Agenda-Setting (NAS) model to assess the relationship fake news, fact-
checkers, and online news media—particularly partisan media—have with each other.
Based on the Global Database of Events, Language, and Tone (GDELT; Leetaru and
Schrodt, 2013), this study takes a computational approach to investigate the role fake
news has in the online new media landscape from 2014 to 2016.
Intermedia NAS
Agenda-setting theory originally examined what topics trend in the news and how that
affects the opinions of audiences (McCombs, 2014). The first level of agenda setting
asserts that the frequency in which news media mention and cover objects (e.g. issues
and public figures) largely dictates what objects audiences think are important to society.
This is not to say that audiences blindly believe the news. Instead, the news media sets
the public salience for objects or attributes. When substantial news coverage is dedicated
to an issue (e.g. economy), people consider the economy an important issue—even
though audiences may have diverging opinions about the issue (e.g. how to fix the econ-
omy). This nuance is critical when considering the agenda-setting power of fake news:
even if some audience members are aware that fake news is fake, the mere rise in cover-
age (fake or real) could result in an agenda-setting effect.
Vargo et al. 3
The agenda-setting effect is not limited to news and audiences. As an extension of the
original theory, intermedia agenda setting focuses on the interaction between different
media outlets in setting each other’s news agenda. Early studies suggested that elite media
influenced smaller news organizations (Reese and Danielian, 1989). This occurs partly
because journalists validate work by looking at their peers, especially colleagues at estab-
lished, elite news media (McCombs, 2014). The literature has shown that the New York
Times and Washington Post often set the agenda of newspapers, television, and radio.
However, more recent studies show that emerging media (e.g. political blogs and online
partisan news websites) are now more powerful in setting the agenda of other media outlets
(Vargo and Guo, 2017; Meraz, 2011). These new findings uncover the possibility that fake
news may also influence the news coverage of other media outlets. Drawing upon the theo-
retical framework of intermedia agenda setting, this study seeks to examine how fake news,
fact-checking websites, and other online media organizations interplay with each other.
Our investigation approaches agenda-setting effects through a more nuanced perspec-
tive: the NAS model, another theoretical advancement of the original theory. The NAS
model asserts that the agenda of media is both implicit and explicit (Vargo et al., 2014).
Traditional agenda-setting solely measures the explicit mentions of issues and attributes
in stories. The NAS model measures the contextual relationships that issues share with
each other. For example, an agenda for a news organization is not just how it covers one
issue for a certain time. Rather, how often issues are mentioned together during the same
news period measures the relationships between different news items. The NAS model
further proposes that the salience of these network relationships for issues can be trans-
ferred from the news media to the audience’s mind (Guo and McCombs, 2016). According
to the NAS model, if the US news media recurrently cover the country’s energy and its
foreign relations problems together, audiences will also consider the two issues intercon-
nected. The NAS model suggests that the news media can construct the public’s perceived
importance of interconnections among issues, as well as the popularity of such issues.
A number of empirical studies have been conducted to test the model in various socio-
cultural contexts (see Guo and McCombs, 2016). In addition, the initial NAS model has
been extended to examine network agenda building (e.g. Neil et al., 2016) and network
intermedia agenda setting (e.g. Vargo and Guo, 2017). Collectively, this research demon-
strates that the network relationships among different news items and messages can be
transferred between varied stakeholder agendas: from media to public, from different
interest groups to media, as well as from media to media. This study seeks to further
contribute to the NAS model by systematically assessing the network intermedia agenda-
setting impact of fake news on other media outlets. For this analysis, we focus on differ-
ent media outlets’ network issue agenda, that is, how different media organizations
associate various issues to portray the social reality and how those issue networks trans-
fer between different media agendas. The following section reviews the limited existing
research on fake news, fact-checkers, and the online media landscape to explore the
potential direction (i.e. who follows whom) of the NAS effects.
Fake news, online media, and fact-checkers
The definition of fake news has been evolving as more knowledge about it has accrued.
At its broadest, it has been identified as “news stories that have no factual basis but are
4 new media & society 00(0)
presented as news” (Allcott and Gentzkow, 2017: 5). Others narrow the definition to
include only “completely false information that was created for financial gain”
(Silverman, 2017) and which resembles credible journalism in order to maximize atten-
tion (Hunt, 2016, but also see Mustafaraj and Metaxas, 2017). Fake news is different
from state- or industry-sponsored disinformation campaigns for political purposes and is
also separate from bad reporting and ideologically driven news that is uncongenial to
one’s views (Silverman, 2017). At its core, “fake news necessitates assumptions about
some kind of authentic or legitimate set of news practices” (Baym, 2005: 261). When
readers believe that a website is journalistic in nature, they can be exploited and per-
suaded to believe untrue things. Fake news evolves from a long line of satire used to hold
politicians and media accountable (Painter and Hodges, 2010). While fake news has
always been present, recent research suggests that it is now more popular than ever
(Dewey, 2016; Silverman, 2016). What has emerged in recent years are websites dedi-
cated solely to propagating fake news. Unlike Painter and Hodges’ (2010) notion of press
accountability, these fake news websites are financially motivated (Dewey, 2016) and
generally fabricate information to stir controversy (Maheshwari, 2016). Content on these
sites is sensationalized in intentional ways to drive up the volume of clicks and shares
(Mustafaraj and Metaxas, 2017; Silverman and Alexander, 2016). In a meta-analysis,
researchers have found over 100 websites that regularly publish false information and
remain active today (Shao et al., 2016).
It is unknown whether fake news can set the agendas of other news media. Given that
many journalists often pay attention to fake news (IFCN, 2016; Jackson, 2017), in part
to address it factually and because fake news is rising in popularity, it stands to reason
that fake news websites may possess an agenda-setting power of their own. That is, they
may have the ability to affect the popularity of issues simply by introducing misinforma-
tion that journalists must address. A network analysis has found that the sites targeted
with the most inbound hyperlinks from fake news networks were mainstream media,
social networking sites, and Wikipedia. Few of the targeted sites linked back to the fake
news sites (Albright, 2016). However, given the lack of empirical evidence so far, we
address the subject by posing a research question:
RQ1. Will the network issue agenda of fake news predict the overall news media’s
agenda online?
Partisan media and fake news
The relationship between fake news and partisan media is worthy of particular attention.
Instead of valuing balance, fairness, and objectivity, partisan media often frame stories in a
way to advance certain political agendas (Levendusky, 2013b) driven by in-group or tribal
identification (Harford, 2017; Roberts, 2017). Traditional partisan media include cable
news (e.g. Fox News) and talk radio (e.g. the Rush Limbaugh Show). The Internet has
contributed to the proliferation of new forms of partisan media: partisan websites and blogs
such as Drudge Report and Daily Kos. With the emergence and popularity of social media
services, partisan news coverage is more popular than ever before (Weeks and Holbert,
2013). Moreover, these social media networks facilitate the spread of misinformation via
Vargo et al. 5
automated, anonymous accounts which target users already engaged in conversation on a
particular topic (Mustafaraj and Metaxas, 2017).
When it comes to the interaction between partisan media and fake news, anecdotal evi-
dence suggests that in an extremely polarized political environment, partisan media tend to
enable the propagation of fake news. Driven by motivation to attack the opposing party
during the 2016 US presidential campaign, fake news site RealTrueNews fictional report-
ing of Hillary Clinton’s leaked speeches to Wall Street banks was soon picked up by Fox
News. The popularity of the fiction being reported as if it were factual was then extensively
reported by conservative-leaning websites such as The Daily Beast (Collins, 2016). As this
case illustrates, fake news sites can set the issue agenda of partisan media of both sides.
Academic research on partisan media’s interplay with fake news is rare. Rojecki and
Meraz’s (2016) analysis of actors responsible for transmitting factitious information
blends (FIBs), a new form of misinformation, sheds some light. Based on the 2004 US
presidential election, the study qualitatively analyzed Google search results of two FIBs
from a variety of media sources. It found that regarding a political controversy against
the Democratic candidate John Kerry, conservative websites and blogs became central
gatekeepers in the release and spread of the misinformation. By contrast, both liberal and
conservative sites contributed to the growth in covering and propagating an FIB about
the Republican candidate George W. Bush. The authors thus concluded that partisan
media facilitate the viral spread of partisan misinformation. Considering both anecdotal
and empirical evidence, though limited, it seems logical to expect that partisan media
follow the agenda of fake news, more so than other types of media outlets:
H1. When compared to the reverse relationship, the agenda of fake news websites will
be more likely to predict the network issue agenda of partisan media.
H2. The agenda of fake news websites will be more likely to predict the network issue
agenda of partisan media than other types of media outlets.
Revealing that only conservative websites were reactive in both cases, Rojecki and
Meraz’s (2016) study also indicates a potentially stronger connection between fake news
and conservative-oriented partisan media. Relevantly, a recent survey demonstrates that
self-reporting Republicans (84%) were significantly more likely to believe fake news
headlines than were Democrats (71%; Silverman and Singer-Vine, 2016). Moreover,
during the 2016 election, fake news stories favoring Trump were shared on Facebook
over three times more often than were fake stories about Hillary Clinton (Allcott and
Gentzkow, 2017). It stands to reason that if fake news is more prevalent and widely
believed among conservatives, it is also more likely to set the agendas of the media that
cater to these audiences:
H3. The network issue agenda of fake news websites will be more likely to predict the
agenda of conservative than liberal partisan media.
No research has examined the intermedia agenda-setting relationship between fake
news and other, nonpartisan media. Right-wing media have been shown to set the elec-
tion coverage agenda of mainstream media (Benkler et al., 2017). However, it is not clear
6 new media & society 00(0)
to what degree fake news stories were involved in this coverage. Thus, it remains a ques-
tion whether traditional, elite media such as the New York Times and Washington Post,
news agencies, or other nonpartisan, emerging news sites such as CNET and Gawker
respond to fake news. Therefore, we ask
RQ2. Will the network issue agenda of fake news predict other, nonpartisan media’s
Aside from news media, the practice of fact-checking has drawn interest as a means
to counter the impact of fake news. This study also examines the role these fact-checking
organizations have played in this evolving online media landscape.
Fact-checking aspires to the normative standard first espoused by Lippmann (2009
[1922]) of making the unintelligible facts known to the masses so as to foster informed
decision making. Lippmann’s position has theoretical grounding in the social responsi-
bility of the press paradigm, formalized by Siebert et al. (1956). These perspectives
formed the basis for the modern fact-checking model of journalism, which emerged out
of the concern that traditional reporting had ceased to hold political figures accountable
for the accuracy of their claims (Graves and Konieczna, 2015). What distinguishes con-
temporary fact-checkers such as PolitiFact and from traditional journalis-
tic conventions are their focus on determining and drawing attention to whether a claim
is factually accurate rather than eliminating errors or falsehoods in reporting (Amazeen,
2013; Graves and Glaisyer, 2012). Specifically, fact-checkers decide what to check based
on whether or not a statement is factually verifiable. In the context of this analysis, it
would be reasonable to expect that
H4. When compared with the reverse relationship, the network issue agenda of fact-
checking websites will be more likely to follow the agenda of fake news websites.
Ultimately, the objectives of fact-checkers are threefold: informing the public,
improving political rhetoric, and influencing other journalists (Amazeen, 2013; Graves
and Glaisyer, 2012). To achieve these goals, fact-checkers are highly reliant on other
news organizations to increase the spread and impact of their reporting through the media
ecosystem (Amazeen, 2013; Graves and Konieczna, 2015). Although fact-checkers are
frequently cited by other journalists (Amazeen, 2013), what is unknown is whether fact-
checkers have the ability to alter the agendas of news media coverage. While journalists
cite fact-checking, they may do so in a reactive nature, for instance, only when they need
to refute fake news. Conversely, it could be fact-checking organizations that give light to
fake news. If this were the case, fact-checkers themselves could possess significant
agenda-setting power among media.
When it comes to the predictive power of fact-checkers and partisanship, research
shows that compared with conservative media, liberal media were more attentive to fact-
checking activities (Graves and Glaisyer, 2012). In addition, liberals have been found to
Vargo et al. 7
be more receptive to fact-checking than conservatives (Barthel et al., 2016). Although
conversations around fact-checking tend to be politically polarized, some of it is apoliti-
cal. Indeed, a network analysis study revealed the fact-checker that was the most influ-
ential in terms of total links from other sites was the general interest site, Snopes. The
three, national fact-checkers,, PolitiFact, and the Washington Posts Fact
Checker, have all engaged in highly political conversations yet still receive relatively
high attention from centrist outlets (Graves and Glaisyer, 2012). It then stands to reason
that the agenda-setting power of fact-checking websites may still be high for nonpartisan
media outlets. Given the lack of literature on the agenda-setting power of fact-checkers,
we chose to pose a research question:
RQ3. Will the network issue agenda of fact-checking websites predict the overall
news media agendas of various types of media online?
Fact-checkers may influence the agenda of partisan media like discussed above or
follow partisan media’s coverage by checking their statements. The manner in which
fact-checkers select claims to evaluate has been a point of contention for critics, with
some claiming they use biased selection methods that are driven by partisanship
(Amazeen, 2013; Davis, 2012). Compared with other media, partisan media are more
likely to frame news in a way to advance certain political agendas (Levendusky, 2013a)
and, therefore, more likely to contain statements that are rooted in verifiable facts that
could be misleading—and rife for selection by fact-checkers. However, no empirical
research has followed the attention of fact-checking coverage at the level of media type
(e.g. partisan, emerging, and traditional). As an initial analysis, this study broadly
attempts to assess the degrees to which fact-checkers follow different types of media:
RQ4. Will the network issue agenda of fact-checking websites follow the overall news
media agendas of various types of media online?
This article uses GDELT’s Global Knowledge Graph (GKG) as its data source (Leetaru,
2012a, 2015a). On a daily basis, GDELT monitors news globally and employs a com-
puter-assisted content analysis that identifies people, locations, themes, emotions, narra-
tives, and events (Leetaru, 2015).1 The dataset has given researchers the ability to
computationally analyze news content of all sorts: real, fake, and fact-checking oriented
(Abbar et al., 2015; Vargo and Guo, 2017).
Coding for issues
GDELT offers themes that represent core topics of discussion.2 Themes cover a broad
range of issues, topics, and attributes,3 many of which are similar to those studied in
agenda-setting studies (e.g. “Econ_Bankruptcy,” “Econ_Cost of living,” “Military_
Cooperation,” and “Refugees.”). With almost 300 themes, GDELT offers a higher level
hierarchy to arrange the data in a way that makes it comparable to other agenda-setting
8 new media & society 00(0)
research. This analysis of intermedia agenda setting relied on Vargo and Guo’s (2017)
theme categorization and sorted the data by themes that have been thought to broadly
encompass major issues in US news coverage (Neuman et al., 2014). The themes are
composed of 16 issues: taxes, unemployment, economy, international relations, border
issues, health care, public order, civil liberties, environment, education, domestic politics,
poverty, disaster, religion, infrastructure, and media and Internet.4
News media types
Fake news. GDELT ingests all news-like content from online sources including Google
News. It does not have any type of quality control system, so as a result, it—like many
other media sources—contains content from fake news websites. Shao et al. (2016) cre-
ated and maintain a meta-analysis of fake news websites for use with their service Hoaxy,
which tracks fake news media from nine different sources such as US News and World
Report, CBS News, and Snopes Field Guide.5 For this study, we included a fake news
website in the analysis if it was identified by more than one of the nine sources. In all, 96
fake news websites were searched for in the GDELT database. In total, 60 fake news
websites and 171,365 stories from fake news websites were found in the GDELT data
from 2014 to 2016.6
Fact-checking websites. The selection of fact-checkers for this study derived from multi-
ple sources. The Duke Reporter’s Lab at Duke University maintains a list which includes
fact-checkers that (1) examine all political parties and ideological sides, (2) examine
discrete claims and reach conclusions, (3) track political promises, (4) are transparent
about sources and methods, (5) disclose their funders and affiliations, and/or (6) are pri-
marily driven by a mission of news and information (Adair and Stencel, 2016). Other
fact-checkers are signatories of the Poynter Institute’s International Fact-Checking Net-
work (IFCN), a forum for fact-checkers that reviews statements by public figures, major
institutions, and other widely circulated claims that are of interest to society (About the
International Fact-checking Network, n.d.). Members of the IFCN commit to the follow-
ing principles: (1) nonpartisanship and fairness, (2) transparency of sources, (3) transpar-
ency of funding and organization, (4) transparency of methods, and (5) open and honest
corrections (International Fact-checking Network Fact-checkers’ Code of Principles,
n.d.). Fact-checkers that do not derive from these two sources yet abide by the spirit of
these general principles were also considered for inclusion. Of these websites, only those
fact-checking organizations that were solely dedicated to fact-checking and available in
the GDELT dataset were included: Climate Feedback,, Gossip Cop,
Health News Review, PolitiFact, Snopes, and Overall, GDELT con-
tained 13,036 stories for the seven sources from 2014 to 2016.
News media websites. The study also drew upon Vargo and Guo (2017) for online news
media categorization. The study identified the top 2760 US news media websites in
GDELT. Their analysis further sorted media into five different categories: (1) elite media
(i.e. the New York Times and the Washington Post), (2) news agencies, (3) traditional
media, (4) online partisan media, and (5) emerging media (i.e. nonpartisan and online
Vargo et al. 9
media). This study updated the list of online partisan media for 2016 by including the
National Review, Vox,, CounterPunch, Veterans Today, and Truthdig.
com in the analysis.7
To address hypotheses in this study, we further updated the existing list of partisan
media by delineating between liberal and conservative slant. In total, two coders coded
the 70 partisan media sites as liberal or conservative. Coders studied each site, search-
ing the Internet for claims from credible news organizations or media watchdogs
asserting that a given site was indeed partisan or to see whether a site self-identified as
partisan. After an initial review and discussion of each site, the coders agreed that 62
= 1) of the media sites were partisan. In total, 31 sites were liberal and 31 were con-
servative in nature. Across the 3 years, there were 594,634 stories from liberal sources
and 625,295 articles from conservative outlets. In total, three sources were not deemed
to be news media, and five media outlets were nonpartisan in nature and as such were
added to the “emerging media” category. The emerging category comprised 14,120,889
stories from 767 outlets.
Overall, nine media groups were considered in the analysis: (1) fake news websites, (2)
fact-checking websites, (3) online partisan media, (3a) liberal media, (3b) conservative
media, (4) elite media (n = 2, stories = 549,009), (5) news agencies (n = 2, stories = 539,841),
(6) traditional media (n = 1911, stories = 25,719,311) and (7) all news media (i.e. groups
3–6). A graph depicting the rising number of fake news and fact-checking articles can be
seen in Figure 1.
Computer-assisted NAS analysis
GDELT CKG data can be downloaded freely from GDELT’s website in tab-separated
values format. The data are structured by news events. Each event is a row of data. News
events are defined as collection of news stories from various sources that contain the same
set of themes. Using Python, each row of data was scanned to see whether each event
contained any media or issues of interest in this study. When an event contained a media
source that was identified in one of the media types above, the themes in that event were
inspected to see any matched the 16 issue constructs previously noted. If an event matched
a media source and themes that corresponded with multiple issues, all possible unordered
pairs of issues were identified and considered as ties (Wasserman and Faust, 1994). For
example, if an article mentioned economy, border issues, and civil liberties, it was deter-
mined that the article had three ties: (1) economy and border issues, (2) economy and civil
liberties, and (3) border issues and civil liberties. All ties were then summed by day and
media type. Each tie’s corresponding weight (i.e. strength) was a summation of the num-
ber of stories that mentioned that issue pair (e.g. economy and foreign policy).
Eigenvector centrality is a measure of influence in a network (Ruhnau, 2000). A
higher centrality score refers to a greater number of connections between a node (an
issue in the analysis here) and all the other nodes in the network. The more ties an issue
has with other issues, the higher centrality value the issue has, and the more centrally it
is located in the resulting networks. Eigenvector centrality was the key unit of analysis.
As such, it was calculated for each media type. For instance, fake news websites had 16
centrality scores, one for each issue. This score relates to how central each issue was in
10 new media & society 00(0)
its coverage. This was done for each day in the 3-year sample, and the data were treated
as time series.
Time series modeling
The centrality scores (issue centrality score × media type × day) were treated as a time
series. This analysis was performed for each year of data. Granger causality models were
constructed for each issue and for each media type. Time series X is said to “Granger
cause” another time series Y if regressing for Y in terms of past values of both X and Y
results in a better model for Y than regressing only on past values of Y. Running F-tests
provided values of significance in which Granger causality could be determined. All
tests were run at five lags, that is, 1-day lag, 2-day lag, 3-day lag, 4-day lag, and 5-day
lag, respectively. While ordinary least squares (OLS) models can assign best fit for one
lag, including lags of multiple days allows the research at present to address different
types of agenda-setting relationships that differing stories can have.
A significant fake news effect was operationalized in this article as having a signifi-
cant effect for at least 9 of the 16 issues studied here. This threshold gives us the power
to say that the majority of an entire media agenda was explained by a given relationship.
In theory, any one issue with a significant test means a significant effect was observed.
However, given the large number of causality tests performed here, any one test comes
with a significant probability of type I and type II errors. Second, the sheer number of
Figure 1. The number of fake news stories and fact-checking articles by month (2014–2016).
Vargo et al. 11
issue to issue tests from one media type to another is “big data” itself. Aside from major-
ity rule, this analysis looks at patterns across years. If a media relationship fell below the
majority threshold, but the same issue was significantly predicted across all 3 years, an
effort was made to call out this relationship in this article. However, one-off relationships
that fell below the majority thresholds were not addressed here due to the large number
and potential spuriousness of the result.
Table 1 provides a summary of the Granger causality tests run for this analysis. The
number of significant tests are displayed for each relationship of interest. Table 2 shows
the detailed Granger causality results for one of the most important relationships inves-
tigated in this study: the ability of fake news websites to predict the agenda of all news
Fake news and online media
RQ1 asked whether the network issue agenda of fake news would predict the overall news
media’s agenda online. The results showed that the fake news websites Granger caused
the agenda of all news media as a group in terms of seven issues in 2014 and 2015 and
four issues in 2016 for at least one lag (see Table 1). Given that we consider nine issues as
a cutoff for significant NAS effect, it appeared that fake news did not set the overall online
media agenda for the 3 years examined; instead, its NAS power turned out to be shrinking
over time. However, our results did suggest fake news was successful in transferring the
centrality salience of certain issues to the news media agenda (see Table 2). For example,
in each of the 3 years, whenever fake news fabricated stories about international relations,
the news media would react in <5 days.
With respect to the relationship between fake news and partisan media (H1), the
results showed that the fake news websites Granger caused the agenda of all online par-
tisan media in terms of 8 issues in 2014 and 2015 and 12 issues in 2016 for at least one
lag. Reversely, online partisan media predicted the network agenda of fake news in terms
of 13 issues in 2014, 13 issues in 2015, and 9 issues in 2016 for at least one lag. As such,
we concluded that partisan media were more likely to predict—rather than follow—the
agenda of fake news in 2014 and 2015. In 2016, the relationship between the two media
groups was reciprocal, but fake news was indeed more likely to influence the agenda of
partisan media compared with the reverse relationship. The two media groups responded
to each other in covering issues of environment, unemployment, economy, international
relations, civil liberties, and religion. Fake news also unidirectionally predicted the issue
agenda of partisan media in reporting infrastructure, disaster, border issues, domestic
politics, health care, and public order. Notably, many of these issues were under heated
debate in the 2016 US presidential campaign season. Given the evidence, H1 was sup-
ported with respect to the online media landscape in 2016 but not in the previous 2 years.
H2 hypothesized that fake news would be more likely to predict the network agenda
of partisan media than other types of media. The results showed that the fake news web-
sites significantly set the network agenda of liberal-oriented partisan media (n = 9) and
12 new media & society 00(0)
emerging media (n = 9) in 2014; elite media (n = 9), liberal-oriented partisan media
(n = 10), and conservative-oriented partisan media (n = 11) in 2015, and liberal-oriented
partisan media (n = 9), all online partisan media (n = 12), and emerging media in 2016
(n = 9). The pattern did show that fake news influenced the agenda of various partisan
Table 1. Number of significant Granger causality tests, by year.
Independent variable Dependent variable 2014 2015 2016
Fake NYT/WaPo 8 9 6
Fake News agencies 6 4 2
Fake Traditional 7 8 7
Fake Liberal 9 10 9
Fake Conservative 4 11 7
Fake All online partisan 8 8 12
Fake Emerging 9 7 9
Fake Fact-checking 6 3 5
Fake All fact-based media 7 7 4
All online partisan Fake 13 11 9
Conservative Fake 10 9 11
Fact-checking Fake 4 5 3
NYT/WaPo Fake 9 10 8
News agencies Fake 11 10 7
Traditional Fake 11 12 10
Emerging Fake 10 12 10
All fact-based media Fake 12 13 11
Liberal Fake 13 11 8
Fact-checking NYT/WaPo 5 4 4
Fact-checking News agencies 3 3 4
Fact-checking Traditional 7 2 3
Fact-checking Liberal 4 5 5
Fact-checking Conservative 3 4 4
Fact-checking All online partisan 4 4 4
Fact-checking Emerging 10 6 5
Fact-checking Fake 4 5 3
Fact-checking All fact-based media 6 4 2
All online partisan Fact-checking 6 8 6
Conservative Fact-checking 8 6 6
NYT/WaPo Fact-checking 5 6 4
News agencies Fact-checking 6 0 4
Traditional Fact-checking 8 4 5
Fake Fact-checking 6 3 5
Emerging Fact-checking 8 6 6
All fact-based media Fact-checking 7 6 7
Liberal Fact-checking 7 8 4
Number of significant Granger causality tests, maximum number possible is 16, one for each issue.
Vargo et al. 13
Table 2. Granger test of causality of fake news on all news media’s network issue agendas.
Lag Tax Unemployment Economy International
Environment Education Domestic
Poverty Disaster Religion Infrastructure Media and
1. F(1, 351) 2.32 1.36 4.27* 13.56** 14.33** 4.76* 2.74 0.54 0.14 2.78 2.03 0.23 0.05 3.51 4.23* 0.36
2. F(2, 352) 1.69 0.96 3.45* 4.97** 7.98** 3.24* 1.75 0.34 0.18 1.86 1.42 0.05 2.63 1.42 1.81 0.3
3. F(3, 353) 1.37 1.31 3.78* 3.39* 4.96** 1.48 0.97 0.18 3.15* 2.21 0.91 0.4 1.79 1.52 0.99 0.56
4. F(4, 344) 1.75 1.17 3.42** 2.72* 4.65** 0.92 0.75 0.63 2.28 2.24 1.41 0.41 1.21 2.91* 1.09 0.57
5. F(5, 345) 1.3 0.95 3.29** 2.84* 2.98* 0.84 0.78 0.63 2.39* 1.72 1.25 0.4 1.12 2.09 1.06 0.46
1. F(1, 361) 1.29 0.35 8.57** 12.57** 1.01 2.22 0.01 6.19* 0.69 8.82** 0.78 9.05** 2.69 7.30** 2.13 0.38
2. F(2, 352) 2.49 0.46 5.67** 5.41** 0.34 6.05** 1.53 2.23 1.33 3.69* 0.25 5.09** 1.44 5.27** 0.59 0.39
3. F(3, 353) 1.86 0.76 4.24** 5.95** 0.51 3.80* 1.9 1.03 1.96 3.46* 0.24 2.83* 1.17 3.46* 0.31 0.91
4. F(4, 354) 1.38 0.58 2.50* 4.85** 0.84 2.97* 1.46 0.84 1.3 2.68* 0.21 2.44* 1.16 2.65* 0.29 0.78
5. F(5, 345) 1.03 0.62 1.72 3.01* 0.79 2.39* 1.24 2.46* 1.13 2.30* 0.62 2.35* 0.98 1.71 0.35 1.13
1. F(1, 361) 0.01 0.19 0.45 6.72** 0.62 2.27 1.78 10.79** 2.53 0.01 0.74 1.87 11.39** 2 0.78 0.98
2. F(2, 352) 0.1 0.65 0.18 3.91* 0.51 1.22 1.49 7.00** 1.33 0.44 0.54 0.88 7.68** 3.03* 0.58 0.42
3. F(3, 353) 0.64 1.52 1.37 3.25* 1.01 0.73 1.04 4.12** 1.12 0.33 1.49 0.59 4.96** 2.72* 0.35 0.28
4. F(4, 354) 0.7 1.14 0.95 2.22 0.85 0.6 0.92 2.76* 0.67 0.61 1.28 0.74 3.67** 2.83* 0.29 1.61
5. F(5, 355) 0.49 0.87 1.18 1.63 0.82 0.57 0.71 2.26* 1.17 0.73 1.01 0.81 2.73* 2.42* 1.36 1.21
All lag times are in days (e.g. 2 = 2 days).
*p < 0.05; **p < 0.01.
14 new media & society 00(0)
media in each year, but it also demonstrated that some other media outlets, especially
emerging media, were also reactive to the fake news coverage. Therefore, H2 was sup-
ported but to a limited extent.
Contrary to what we expected, the results also revealed that fake news was more likely
to influence the issue agenda of liberal media than their conservative counterparts. Thus,
H3 was rejected. However, fake news appeared to be significantly influenced by both
liberal and conservative media. Remarkably, in 2016, conservative media were found to
Granger cause the network issue agenda of fake news in terms of 11 issues for at least one
lag. The same significant agenda-setting effect was not found for liberal media. Taken all
together, the results seem to suggest that conservative media transferred the issue salience
to the fake news websites, which then affected what issues liberal media decided to report.
In answering RQ2, fake news was also powerful in influencing the network issue
agenda of emerging and elite media in certain years, as mentioned above. When consid-
ering the reverse relationship, the fake news websites were found to be reactive to all
types of media. In particular, fake news significantly followed the network issue agenda
of emerging media and traditional media throughout 2014–2016.
The role of fact-checkers in the online mediascape
As Table 1 illustrates, fact-checking websites appeared to be largely autonomous from
other online media agendas. Surprisingly, a significant NAS effect was found in only one
Granger causality test considering all possible relationships: in 2014, fact-checking web-
sites Granger caused the issue agenda of emerging news websites in terms of 10 issues
for at least one lag. Furthermore, the connection between fact-checkers and other media
outlets appeared to be diminishing over time. While the findings provided little evidence
to support our hypothesis, a brief discussion of results related to it and our research ques-
tions are offered below.
In testing H4, our results showed that the fact-checking organizations did not necessar-
ily follow the issue agenda of fake news. In 2015, they were even found to be more likely
to transfer the issue salience to fake news: fact-checkers and fake news were reciprocal in
reporting border issues and civil liberties. Moreover, the fake news websites also followed
the fact-checking websites for producing stories about the environment, unemployment,
and public order. Ironically, fact-checkers, originally with the intention to correct fake
news, may provide ideas for fake news to “cover” under some circumstances.
To answer RQ3, the results showed that, again, the fact-checking websites did not
predict the overall news media agenda online. Granger causality tests indicated a declin-
ing influence over the issue agenda of all fact-based media from a high of six in 2014, to
four in 2015, declining to two in 2016. The two issues that fact-checkers were able to
transfer the salience of to the overall online media agenda in 2016 were disaster and
international relations. Finally, in addressing RQ4, fact-checkers did predict the network
issue agenda of emerging media in 2014 but not in the following years.
Overall online mediascape 2014–2016
Overall, Table 3 provides a look at how influential and autonomous each media’s news
agendas were from 2014 to 2016 when considering the entire media landscape. This
Vargo et al. 15
table presents weighted indegree and outdegree measures, with lower indegree scores
denoting more agenda autonomy and higher outdegree scores denoting more agenda
influence. Put another way, indegree measures the ability for all other media to explain
that media, and outdegree measures the ability that media has to control other media
Thankfully, fake news media do not appear to be gaining agenda-setting power across
the entire mediascape (from 64 in 2014 down to 61 in 2016). This finding stands despite
the increased attention partisan media gave to fake news in 2016. This suggests that just
as partisan media tuned in more to fake news, other nonpartisan media began to tune out.
However, fake news does appear to be diverging from the entire mediascape. Their agen-
das are becoming more autonomous (from a score of 97 in 2014 down to 77). This is
particularly worrisome given fake news’ relatively stable ability to influence the entire
mediascape. Taken together, while fake news is approximately as powerful as it was in
2014, it appears to be more topically independent than in 2014.
Fact-checkers appear to be the most autonomous group studied here. Their agendas
are least explainable by the mediascape. However, this autonomy seems to come at a
price. The news mediascape as a whole seems to be paying less attention to them. In
particular, the outdegree scores here suggest that fact-checking websites had approxi-
mately half (34 compared to 61) of the influence that fake news did in 2016.
Fake news spreads on social media and is perhaps more popular than ever (Dewey, 2016;
Silverman, 2016). Previous research has shown that American adults are susceptible to
fake news headlines (Silverman and Singer-Vine, 2016). Thus, fake news distorts break-
ing news (Hermann et al., 2016) and may even disrupt global politics (Frenkel, 2016). Our
Table 3. Weighted indegree and outdegree scores by media type.
Media type IndegreeaOutdegreea
2014 2015 2016 2014 2015 2016
All fact-based media 88 72 66 119 104 104
All online partisan 95 89 95 103 93 82
Conservative 95 103 86 98 82 84
Emerging 103 89 96 106 97 104
Fact-checking 61 47 47 46 37 34
Fake 93 93 77 64 67 61
Liberal 103 87 84 99 83 76
NYT/WaPo 97 92 95 103 70 80
News agencies 97 55 68 81 69 69
Traditional 96 73 77 109 98 97
Indegree scores can be thought of as the degree to which other media predicted that media type. Outde-
gree can be thought of as the degree to which that media predicted other media types.
All scores here are weighted by number of significant issue relationships (e.g. Granger causality tests)
16 new media & society 00(0)
study confirms that content generated from fake news sites is on the rise (see Figure 1) and
furthers our understanding of fake news by assessing its ability to “push” or “drive” the
popularity of issues in the broader online media ecosystem. Such a rapid rise in fake news
content generation is problematic. It remains unclear just why fake news websites now
generate more content than ever. Have recent advances in algorithmic content creation
allowed fake news to automate news stories?8 This question begs for further research.
By studying an exhaustive collection of media types across 3 years, a picture of fake
news—and the fact-checkers who fight them—has emerged. Fake news did not appear
to control the agenda of the whole media landscape from 2014 to 2016. If anything, the
NAS power of fake news across all media seems to be steady or slightly declining.
This is particularly heartening news for the journalism industry and consumers alike.
However, when considering the entire mediascape, the agendas of fake news websites
appear to be diverging and becoming more autonomous. This finding is concerning
and suggests that fake news has more freedom than ever. Across all 3 years, fake news
was able to set the agenda for the key issue of international relations. Moreover, for
2 years, it set the agenda on the issues of the economy and religion. Further study is
warranted to examine why these types of fake news stories were so successful in their
agenda-setting ability.
Our analysis here also reveals that partisan media are intricately entwined with fake
news. On one hand, fake news is particularly responsive to the agendas of partisan media
across many issues. For all 3 years studied here, fake news seemed to take cues from the
partisan media when it came to stories that mentioned the economy, education, environ-
ment, international relations, religion, taxes, and unemployment. If fake news is to be
better understood, fought, and ultimately stopped—further study of these issues and
what makes them so tempting will ultimately allow journalists to know when the condi-
tions for partisan fake news skimming is ripe.
On the other hand, in 2016, our data suggest that partisan media were far more respon-
sive to the agendas of fake news than in years past. Fake news had the ability to control
the popularity for a shocking 12 of the 16 issues studied here. This increased responsive-
ness could be due to the fact that partisan media were more driven by their motivation to
attack the candidates in 2016. When looking at all 3 years studied, partisan media seemed
attentive to fake news coverage of border issues, international relations, and religion.
While the data presented here cannot offer clear reasons as to why these issues are con-
sistently brought from fake news to the partisan media agenda, we believe that these
issues paint an intuitive picture. Partisan media are known largely for their controversial
stances on these topics. It could be that partisan media use fake news to not only support
their claims but also use the increased Internet “buzz” around these issues as an excuse
to continue the discussion with their own fake reportage. As Amazeen (2014) has
observed, “Partisan media can be considered a guerilla marketing approach in pursuit of
building an agenda that makes a particular ideology seem commonsensical” (p. 287).
Thus, further study of the relationship between partisan media and fake media is war-
ranted. Furthermore, in light of recent findings showing the success of partisan media in
influencing the news coverage of other media outlets, including mainstream media
(Vargo and Guo, 2017; Meraz, 2011), future research should test this potential two-step
flow: fake news partisan media all online media.
Vargo et al. 17
Emerging media, which is also online-only, appears to be responsive to the agendas
of fake news, as well. The online nature of emerging media may make it more attentive
to all online information, including fake news. Taken all together, online partisan and
nonpartisan media were closely intertwined with fake news websites, producing an
extremely complicated and uncertain online mediascape.
Fact-checkers appeared to be largely autonomous (see Table 1). Their decisions on
what issues to cover did not appear to be dictated by fake news or any other type of news.
On one hand, this evidence serves to undermine the accusations of critics who claim fact-
checkers display a partisan bias in claim selection (Amazeen, 2013; Davis, 2012). On the
other hand, this lack of consistent bias in attention suggests that fact-checking websites
are not aggressively refuting certain media. Indeed, due largely to resource-driven con-
straints, early fact-checking conventions were to correct a claim and move on. Even
when patterns of deception were evident, fact-checkers saw “little point in repeating
ourselves … we didn’t run a new story every time” (the same inaccurate claim emerged;
Amazeen, 2012: 43). Fact-checkers have only begun to pivot toward an “ongoing story
structure” that facilitates connecting fact-checks to claims that persist (Amazeen, 2013:
27). The previously mentioned initiative by Facebook to algorithmically include fact-
checkers as third-party network agents is one example of this necessary convention.
The results of this study also suggest that fact-checkers were not influential in predict-
ing the agenda of news media overall. This is consistent with other research indicating
that corrections do not spread as widely as misinformation (Friggeri et al., 2014; Zollo
et al., 2015). It also illustrates the difficulties fact-checkers face in achieving their goal
of influencing other journalists (Amazeen, 2013; Graves and Konieczna, 2015). However,
this analysis only collected data from independent fact-checkers unaffiliated with media
organizations. Had the study included news sites that offered fact-checking in addition to
other reporting (e.g. the Washington Post’s Fact Checker), the results may have differed.
Nonetheless, this is a networked problem that must be solved collectively, particularly as
new evidence suggests the spread of misinformation is facilitated by nonhuman “bots”
that give the illusion a topic is popular (Flam, 2017).
In all, this analysis shows that fake news can influence the issue agendas of partisan
and emerging news media coverage. While the influence that fake news had on partisan
media grew in 2016, the overall influence of fake news on the entire mediascape appears
unchanged. This suggests that just as partisan media adopted fake news agendas, other
media began to resist. Further research should study the increasing link between partisan
media and fake news. One possible explanation for this finding is that partisan media
have begun to leverage fake news to bolster and selectively support their own agendas.
If this is indeed the case, it is yet another cause for concern regarding the effects of par-
tisan media on a healthy society.
It is important to reiterate: the agenda-setting effect does not mean media coverage
was littered with the same factual errors. In many cases, news media likely adopted fake
news agendas to refute claims. A notable limitation of this study is that it stops short of
measuring the agenda-setting power of specific false claims that fake news generates, a
direction future research should consider pursuing.
Still, the power of altering issue salience is not one to be taken lightly. As Donald Trump
himself once said in his book The Art of the Deal (Trump and Schwartz, 2009, p. 57):
18 new media & society 00(0)
I’m not saying that [journalists] necessarily like me. Sometimes they write positively,
and sometimes they write negatively. But from a pure business point of view, the benefits
of being written about have far outweighed the drawbacks. It’s really quite simple. If I
take a full-page ad in the New York Times to publicize a project, it might cost $40,000,
and in any case, people tend to be skeptical about advertising. But if the New York Times
writes even a moderately positive one-column story about one of my deals, it doesn’t
cost me anything, and it’s worth a lot more than $40,000…. the point is that we got a lot
of attention, and that alone creates value.
The author(s) received no financial support for the research, authorship, and/or publication of this
1. Global Database of Events, Language, and Tone (GDELT) indexes news stories from
Associated Press, United Press International, Washington Post, the New York Times, and all
national and international news from Google News with the exception of sports, entertain-
ment, and strictly economic news.
2. GDELT researchers create themes by training a computer system. The system recognizes
keywords in text that are associated with a theme. Leetaru (2012b) lays out the structure of the
underlying algorithms. To validate a theme, the manual review of randomly selected articles
is conducted to ensure external validity. GDELT maintains that theme detection is accurate
and on par with leading computer-assisted classification systems (Leetaru, 2012b).
3. To establish a theme, a machine learning algorithm is trained to recognize keywords in text
that is associated with that theme. Leetaru (2012b) outlines the process, but in general, a
theme is trained through feeding the system articles identified by humans. Humans review
randomly selected articles to verify the final algorithm’s accuracy. The results show that
GDELT’s CKG system performs well (Leetaru, 2012b).
4. Vargo and Guo (2017) tasked human coders to match GDELT themes to the 16 issue con-
structs. Coders agreed on 271 of the 285 theme assignments (α = 0.841).
5. The full list of sources, as well as the fake news websites themselves, is hosted by Indiana
University here:
6. An important distinction here is that we are not asserting that each and every story in our
database of fake news websites is fake. Instead, we are looking more broadly at the agendas
of websites that are known to create fake news. Previous studies have looked at the individual
influence that specific fake news articles have on society. Here, we take a look at the entire
body of work that a fake news website creates. This allows us to see whether fake news agen-
das have broader effects on the journalism industry.
7. In the previous list, partisan allegations on Wikipedia served as a basis for paritisanship
(Vargo and Guo, 2017). At the time of this analysis, some pages had clearer Wikipedia (or
other sources) mentions denoting or accusing of partisan behavior.
8. Ultimately, this study did not identify who the actors behind these fake news websites are.
While previous research has suggested that most fake news is monetarily driven (Maheshwari,
2016), the specific methods by which stories are created is largely unknown. Is fake news
computerized and algorithmic? While our study did not tackle this question, our research
shows that fake news appears to be more autonomous than ever. This means that fake news is
not simply amplifying agendas of existing media but creating their own original agendas.
Vargo et al. 19
Abbar S, An J, Kwak H, et al. (2015) Consumers and suppliers: attention asymmetries. A case
study of Al Jazeera’s news coverage and comments. Computational Journalism Symposium,
2 October. New York: ACM.
About the International Fact-checking Network (n.d.) The poynter institute. Available at: http://
Adair B and Stencel M (2016) How we identify fact-checkers. Duke Reporter’s Lab, 26 June.
Available at:
Albright J (2016) The #Election2016 micro-propaganda machine. Medium, 18 November.
Available at:
Allcott H and Gentzkow M (2017) Social media and fake news in the 2016 election. Journal of
Economic Perspectives 31(2): 211–236.
Amazeen MA (2012) Blind spots: examining political advertising misinformation and how U.S.
news media hold political actors accountable. Dissertation Abstracts International 74(2).
Available at:
Amazeen MA (2013, October). Making a difference: a critical assessment of fact-checking in 2012.
New America Foundation Media Policy Initiative Research Paper. Available at: https://www.
Amazeen M (2014) Voter disdain: twenty-first century trends in political advertising. In D.
Coombs and B. Batchelor (eds) We Are What We Sell: How Advertising Shapes American
Life… And Always Has. Santa Barbara, CA: Prager, pp. 282–301.
Barthel M, Gottfried J and Lu K (2016) Trump, Clinton supporters differ on how media should
cover controversial statements. Pew Research Center, 17 October. Available at: http://www.
Baym G (2005) The daily show: discursive integration and the reinvention of political journalism.
Political Communication 22(3): 259–276.
Benkler Y, Faris R, Roberts H, et al. (2017) Study: Breitbart-led right-wing media ecosystem
altered broader media agenda. Columbia Journalism Review, 3 March. Available at: http://
Collins B (2016) This “conservative news site” trended on Facebook, showed up on Fox
News—and duped the world. 18 October. Available at:
Davis K (2012) Study: politifact rates GOP as biggest liar. The Center for Media and Public
Affairs, 21 September. Available at:
Dewey C (2016) Facebook fake-news writer: ‘I think Donald Trump is in the White House because
of me.’ The Washington Post. Available at:
Easton L (2016) The fight against fake news. AP Newswire, 15 December. Available at: https://
Flam F (2017) Fighting fake news with science. Bloomberg View, 27 March. Available at: https://
Frenkel S (2016) Germany is fighting fake news on Facebook and wants Europe along for the ride.
Buzzfeed, 23 November. Available at:
20 new media & society 00(0)
Friggeri A, Adamic LA, Eckles D, et al. (2014) Rumor cascades. In: International AAAI confer-
ence on web and social media, North America, May. Available at:
Google Trends (2017) 13 January. Available at:
Graves L and Glaisyer T (2012) The fact-checking universe in spring 2012. New America
Foundation, February. Available at:
Graves L and Konieczna M (2015) Sharing the news: journalistic collaboration as field repair.
International Journal of Communication 9: 1966–1984.
Guo L and McCombs M (eds) (2016) The Power of Information Networks: New Directions for
Agenda Setting. New York and London: Routledge.
Harford T (2017) The problem with facts. Financial Times, 9 March. Available at: https://www.
Hermann P, Svrluga S and Miller ME (2016) Alleged gunman tells police he wanted to res-
cue children at DC pizza shop after hearing fictional internet accounts. The Washington
Post, 5 December. Available at:
Hunt E (2016) What is fake news? How to spot it and what you can do about it. The Guardian,
17 December. Available at:
International Fact-Checking Network (2016) An open letter to Mark Zuckerburg from the world’s
fact-checkers. The Poynter Institute, 17 November. Available at: https://www.poynter.
International Fact-checking Network Fact-checkers’ Code of Principles (n.d) The Poynter Institute.
Available at:
Isaac M (2016) Facebook mounts effort to limit tide of fake news. New York Times, 15
December. Available at:
Jackson J (2017) BBC sets up team to debunk fake news. The Guardian, 12 January. Available
Leetaru K and Schrodt PA (2013) GDELT: global data on events, location, and tone, 1979–
2012. In: ISA annual convention. Available at:
Leetaru KH (2012a) Data Mining Methods for the Content Analyst: An Introduction to the
Computational Analysis of Content. New York: Routledge.
Leetaru KH (2012b) Fulltext geocoding versus spatial metadata for large text archives: towards a
geographically enriched Wikipedia. D-lib Magazine 18(9): 5.
Leetaru KH (2015) Mining libraries: lessons learned from 20 years of massive computing on the
world’s information. Information Services & Use 35(1–2): 31–50.
Leetaru KH, Perkins T and Rewerts C (2014) Cultural computing at literature scale: encoding
the cultural knowledge of tens of billions of words of academic literature. D-lib Magazine
20(9): 8.
Levendusky M (2013a) Partisan media exposure and attitudes toward the opposition. Political
Communication 30(4): 565–581.
Levendusky M (2013b) Why do partisan media polarize viewers? American Journal of Political
Science 57(3): 611–623.
Vargo et al. 21
Lippmann W (2009 [1922]) Public Opinion. Sioux Falls, SD: NuVision Publications.
McCombs M (2014) Setting the Agenda. 2nd ed. Cambridge; Malden, MA: Polity Press.
Maheshwari S (2016) How fake news goes viral: a case study. The New York Times, 20 November.
Available at:
Meraz S (2011) Using time series analysis to measure intermedia agenda-setting influence in tra-
ditional media and political blog networks. Journalism & Mass Communication Quarterly
88(1): 176–194.
Mustafaraj E and Metaxas PT (2017) The fake news spreading plague: was it preventable? Cornell
University Library. Available at: arXiv:1703.06988[cs.SI]
Neil J, Schweickart T, Zhang T, et al. (2016) The dash for gas: examining third-level agenda-
building and fracking in the United Kingdom. Journalism Studies. Available at: http://dx.doi.
Neuman WR, Guggenheim L, Mo Jang S, et al. (2014) The dynamics of public attention: agenda-
setting theory meets big data. Journal of Communication 64(2): 193–214.
Painter C and Hodges L (2010) Mocking the news: how the daily show with Jon Stewart holds
traditional broadcast news accountable. Journal of Mass Media Ethics 25(4): 257–274.
Reese SD and Danielian LH (1989) Intermedia influence and the drug issue: converging on
cocaine. In: Shoemaker PJ (ed.) Communication Campaigns about Drugs: Government,
Media, and the Public. New York: Lawrence Erlbaum Associates, pp. 29–46.
Roberts D (2017) Donald Trump and the rise of tribal epistemology. Vox, 22 March. Available
Rojecki A and Meraz S (2016) Rumors and factitious informational blends: the role of the web in
speculative politics. New Media & Society 18(1): 25–43.
Ruhnau B (2000) Eigenvector-centrality—a node-centrality? Social Networks 22(4): 357–365.
Shao C, Ciampaglia GL, Flammini A, et al. (2016) Hoaxy: a platform for tracking online mis-
information. In: Proceedings of the 25th international conference companion on world
wide web, international world wide web conferences steering committee, April. New York:
ACM. pp. 745–750.
Siebert FS, Peterson T and Schramm W (1956) Four Theories of the Press. Urbana, IL: University
of Illinois Press.
Silverman C (2016) This analysis shows how viral fake election news stories outperformed
real news on Facebook. Buzzfeed, 16 November. Available at:
Silverman C (2017) What exactly is fake news? The Fake Newsletter, 26 February. Available at:
Silverman C and Alexander L (2016) How teens in the Balkans are duping Trump supporters with
fake news. Buzzfeed, 3 November. Available at:
Silverman C and Singer-Vine J (2016) Most Americans who see fake news believe it, new survey
says. Buzzfeed, 6 December. Available at:
Trump DJ and Schwartz T (2009) Trump: The Art of the Deal. New York: Ballantine Books.
Vargo CJ, Guo L, McCombs, M, et al. (2014) Network issue agendas on Twitter during the 2012
US presidential election. Journal of Communication 64(2): 296–316.
22 new media & society 00(0)
Vargo C and Guo L (2017) Networks, big data, and intermedia agenda-setting: an analysis of
traditional, partisan, and emerging online U.S. news. Journalism & Mass Communication
Quarterly. Preprint available online.
Wasserman S and Faust K (1994) Social Network Analysis: Methods and Applications. Cambridge:
Cambridge University Press.
Weeks BE and Holbert RL (2013) Predicting dissemination of news content in social media a focus
on reception, friending, and partisanship. Journalism & Mass Communication Quarterly
90(2): 212–232.
Zollo F, Bessi A, Del Vicario M, et al. (2015) Debunking in a world of tribes. Cornell University
Library. Available at: arXiv:1510.04267
Author biographies
Chris J Vargo (PhD, University of North Carolina at Chapel Hill) is an assistant professor of big
data and analytics at The University of Colorado Boulder. He specializes in the use of computer
science methods to investigate social media using theories from the communication and political
science disciplines. Research methods of specialization include text mining, machine learning,
computer-assisted content analysis, data forecasting, information retrieval, and network analysis.
Lei Guo is currently an assistant professor at Boston University. She earned her PhD from the
University of Texas at Austin in 2014. Her research focuses on the development of media effects
theories, emerging media technologies and democracy, and international communication. She and
Dr Maxwell McCombs proposed the third level of agenda-setting theory—the Network Agenda-
Setting model—and tested the model in various settings using computer-assisted text analysis
methods such as semantic network analysis, sentiment analysis, and data visualization.
Michelle A Amazeen (PhD, Temple University) is an assistant professor at Boston University. Her
research examines misinformation and the blurred lines between advertising, journalism, and poli-
tics. A multi-method researcher, she explores how various types of consumer and political misin-
formation affect public perceptions and the effectiveness of correction efforts. Her research on
fact-checking has been funded by the American Press Institute as is her current research on study-
ing the effects of native advertising disclosure transparency on publisher reputations.
... Classifying if a claim is either true or false based on evidence (Vargo et al., 2018;Gurrapu et al., 2022). ...
... Fact-checking has become a popular application of NLP in recent years given its impact on assisting with misinformation and a majority of the work has been with claim verification (Vargo et al., 2018;Gurrapu et al., 2022). Based on a paper published in 2016, there are 113 active fact-checking groups and 90 of which were established after 2010 (Graves and Cherubini, 2016). ...
Full-text available
Recent advances in deep learning have improved the performance of many Natural Language Processing (NLP) tasks such as translation, question-answering, and text classification. However, this improvement comes at the expense of model explainability. Black-box models make it difficult to understand the internals of a system and the process it takes to arrive at an output. Numerical (LIME, Shapley) and visualization (saliency heatmap) explainability techniques are helpful; however, they are insufficient because they require specialized knowledge. These factors led rationalization to emerge as a more accessible explainable technique in NLP. Rationalization justifies a model's output by providing a natural language explanation (rationale). Recent improvements in natural language generation have made rationalization an attractive technique because it is intuitive, human-comprehensible, and accessible to non-technical users. Since rationalization is a relatively new field, it is disorganized. As the first survey, rationalization literature in NLP from 2007 to 2022 is analyzed. This survey presents available methods, explainable evaluations, code, and datasets used across various NLP tasks that use rationalization. Further, a new subfield in Explainable AI (XAI), namely, Rational AI (RAI), is introduced to advance the current state of rationalization. A discussion on observed insights, challenges, and future directions is provided to point to promising research opportunities.
... The vast quantity of misinformation circulating in the contemporary media ecosystem has drawn increasing attention from scholars Vargo et al., 2018) with a particular focus on whether and how misperceptions can be corrected (Bode & Vraga, 2015;Flynn et al., 2017). While some research has broadly addressed misinformation and corrections (Ecker et al., 2022;Lewandowsky et al., 2017), no generalized model has attempted to theorize the antecedent conditions affecting whether individuals are likely to recognize that information is inaccurate and how they may consequently respond (Amazeen & Krishna, 2023;Walter & Murphy, 2018). ...
... For instance, for an intervention message to have any effect on an individual's misinformation-driven misperception, the individual must first be exposed to the intervention. Yet, the media are much more likely to amplify misinformation than correction messages (Guess et al., 2020;Vargo et al., 2018). Thus, the opportunity for the public to encounter The MRRM corrections to misinformation may not be as prevalent as might be suggested by the expanding literature on this subject (Graves & Amazeen, 2019). ...
Full-text available
Although research on misinformation and corrections has recently proliferated, no systematic structure has guided the examination of conditions under which misinformation is most likely to be recognized and the potential ensuing effects of recognition. The Misinformation Recognition and Response Model (MRRM) provides a framework for investigating the antecedents to and consequences of misinformation recognition. The model theorizes that how people cope with exposure to misinformation and/or intervention messages is conditioned by both dispositional and situational individual characteristics and is part of a process mediated by informational problem identification, issue motivation, and—crucially—recognition of misinformation. Whether or not recognition is activated then triggers differential cognitive coping strategies which ultimately affect consequent cognitive, affective, and behavioral outcomes. Working to explore the notion of misinformation will be more fruitful if researchers take into consideration how various perspectives fit together and form a larger picture. The MRRM offers guidance on a multi-disciplinary understanding of recognizing and responding to misinformation.
... As shown before, misinformation is a challenge for both society in general and the media. In the years leading up to the pandemic, it was possible to witness both the increase in false information circulating mainly through spaces such as social networks and the emergence of numerous initiatives specializing in its verification [10]. However, if we look closely at the period that began with the mass confinement of the population in the first months of February 2020, the target of disinformation (and also of information) is focused on one content above many others: COVID-19 [11]. ...
Full-text available
Three years after a pandemic that demonstrated the importance of reliable health information in a news agenda dominated by coronavirus disease 2019 (COVID-19), we analyze the situation of health disinformation in Spain on the basis of the verifications carried out by its main fact-checking platforms. The results show that COVID-19 shared center stage with other topics in the health area. In addition, a unique agenda is evident in each situation in the study, indicating a fact-checking strategy that is differentiated according to the media outlet and type of specialization (generalist fact-checker or one specialized in health). Vaccination, nutrition, and disease treatment emerge as the most important thematic subfields. Most health hoaxes are manufactured, i.e., created from scratch, rather than being manipulated or reconfigured from real preexisting elements. The format of text and image together predominates, and new social networks (TikTok or Telegram) have appeared as platforms for the circulation of hoaxes. This indicates that providing necessary health literacy to society and giving health issues greater presence in current fact-checking agendas are strategies for combatting disinformation, which can have serious consequences, regardless of whether there is a public health crisis such as the one experienced recently.
... This situation may determine policies, communication and the course of public debate on a certain phenomenon (Becatti et al., 2019;Schmidt et al., 2018). Indeed, social media users, depending on their world views, have access to information while ignoring dissenting information (Baronchelli, 2018;Nordio et al., 2017;Vargo et al., 2018;Vicario et al., 2019). There is a tendency to form polarized groups around narratives. ...
Full-text available
The aim of this study is to analyze the shift in the social media discourse during the COVID-19 pandemic. The sample included Turkish users on Twitter, who shared opinions about the pandemic between March 9 and October 31, 2020. The collected tweets were first classified with the Long Short-Term Memory (LSTM) architecture , which used the global vector for word representation embedding method. In addition, due to the grammatical and semantic structure of the Turkish language, we employed the Zemberek library for the text pre-processing stage. We analyzed data according to two categories: user-to-public posts and user-to-user posts. User-to-user data were investigated with effective social network analysis techniques. Empirical results showed that Twitter users posted and disseminated information mainly related to economy, politics and world topics.
... Another way in which the phenomenon has impacted newsrooms is its incidence in each outlet's news agenda. Although Vargo et al. (2018) found that, in 2016, in the United States, fake news did not have a special impact on media agendas, in Costa Rica, inquiries to journalists reflect a dilemma against which each media outlet must take a position: Let fake news leak onto the news agenda or prevent it to defend its own agenda. As seen in the results, some news outlets consider it important to investigate and deny the false content that circulates on social media and even have specialized platforms to do said task, such as the No Coma Cuento section of La Nación. ...
Full-text available
This article gives voice to two protagonists of the disinformation phenomenon: audiences and journalists. A two-objective comparative analysis is presented: on the one hand, to explore the audiences’ role in stimulating the disinformation phenomenon in Costa Rica and characterize the reception dynamics and interaction that they establish with fake news within the country; on the other hand, to give journalists a voice to compare and contrast their perceptions with their audiences’ perceptions. A qualitative methodology based on semi-structured interviews with a small and non-representative sample of Costa Rican journalists holding leadership positions at some of the country’s most important media outlets was used. Also, six discussion groups with audiences were held. This article performs a comparative analysis of what journalists and participating audiences think about the disinformation phenomenon, the term “fake news,” the responsibilities in the fight against this problem and the possible solutions. The results suggest that, according to journalists, misinformative content circulates on social media and does not come from the traditional media they represent. In contrast, audiences present an antagonistic position since they tend to associate disinformation with the traditional media. However, they also consider traditional media as a source of verification of false content circulating on social media
The proliferation of online disinformation has become of major societal concern. Because of online programmatic algorithms, brands may find their ads running on disinformation websites alongside disinformation. In this experimental study (N = 617), we investigate people’s brand-related and news-related responses in this context. Results show that when an advertised brand is displayed on the same webpage as a disinformation article, brand attitude and brand trust are negatively affected; this effect is even more pronounced when the brand is thematically congruent with the disinformation article. This suggests that brands (especially congruent ones) might be contaminated by disinformation. In addition, presenting an ad alongside a disinformation article increases the credibility of the article, as well as people’s level of agreement with the article’s content. This seems to suggest that advertising can lend validity to disinformation and amplify its (harmful) effects. These results have important and timely managerial and societal implications.
The dissemination of fabricated information is not a new phenomenon in human society. However, recent developments in social media have massively increased the creation and dissemination of this information. This study employed a scoping review method to ascertain fabricated information contexts, regulatory frameworks, and impediments in regulating the information. Google and Google Scholar search engines were used to identify documents published between 2006 and 2022 on fabricated information and regulatory frameworks. The data in these studies were subjected to thematic analysis. The study reveals that Facebook and Twitter produce a large quantity of fabricated information and that most of the information is created and disseminated from political and health contexts. Besides, the study shows that despite the available regulatory frameworks for curbing fabricated information, the problem persists. This has been attributed to diverse challenges associated with the regulation of information. A universal mechanism and regulatory framework should be enacted to regulate fabricated information effectively.
In light of concerns over the spread of so-called “fake news” on social media, organizations, and policymakers have increasingly sought to identify tools that can be used to stem the dissemination of misinformation and disinformation. Some evidence suggests that brief media literacy interventions might serve as an important means of helping social media users discern between “real” and “fake” news headlines. However, empirical research indicates that these effects tend to be relatively modest in magnitude. To that end, this study explored the degree to which epistemic self-efficacy beliefs may be able to positively “boost” media literacy interventions. Specifically, we used a series of 2 × 2 experiments to test the contention that the combinatory effects of epistemic self-efficacy and media literacy interventions will better equip users with the resources necessary to discern between disinformation and objectively produced news content. The results failed to indicate the presence of combinatory effects. We did, however, find initial evidence that epistemic self-efficacy beliefs may be importantly associated with the ability to properly classify both fake and mainstream news content.
Conference Paper
Full-text available
In 2010, a paper entitled "From Obscurity to Prominence in Minutes: Political Speech and Real-time search" won the Best Paper Prize of the WebSci'10 conference. Among its findings were the discovery and documentation of what was labeled a "Twitter bomb", an organized effort to spread misinformation about the democratic candidate Martha Coakley through anonymous Twitter accounts. In this paper, after summarizing the details of that event, we outline the recipe of how social networks are used to spread misinformation. One of the most important steps in such a recipe is the "infiltration" of a community of users who are already engaged in conversations about a topic, to use them as organic spreaders of misinformation in their extended subnetworks. Then, we take this misinformation spreading recipe and indicate how it was successfully used to spread fake news during the 2016 U.S. Presidential Election. The main differences between the scenarios are the use of Facebook instead of Twitter, and the respective motivations (in 2010: political influence; in 2016: financial benefit through online advertising). After situating these events in the broader context of exploiting the Web, we seize this opportunity to address limitations of the reach of research findings and to start a conversation about how communities of researchers can in- crease their impact on real-world societal issues.
Full-text available
In 2010, a paper entitled "From Obscurity to Prominence in Minutes: Political Speech and Real-time search" won the Best Paper Prize of the Web Science 2010 Conference. Among its findings were the discovery and documentation of what was termed a "Twitter-bomb", an organized effort to spread misinformation about the democratic candidate Martha Coakley through anonymous Twitter accounts. In this paper, after summarizing the details of that event, we outline the recipe of how social networks are used to spread misinformation. One of the most important steps in such a recipe is the "infiltration" of a community of users who are already engaged in conversations about a topic, to use them as organic spreaders of misinformation in their extended subnetworks. Then, we take this misinformation spreading recipe and indicate how it was successfully used to spread fake news during the 2016 U.S. Presidential Election. The main differences between the scenarios are the use of Facebook instead of Twitter, and the respective motivations (in 2010: political influence; in 2016: financial benefit through online advertising). After situating these events in the broader context of exploiting the Web, we seize this opportunity to address limitations of the reach of research findings and to start a conversation about how communities of researchers can increase their impact on real-world societal issues.
Full-text available
This large-scale intermedia agenda–setting analysis examines U.S. online media sources for 2015. The network agenda–setting model showed that media agendas were highly homogeneous and reciprocal. Online partisan media played a leading role in the entire media agenda. Two elite newspapers—The New York Times and The Washington Post—were found to no longer be in control of the news agenda and were more likely to follow online partisan media. This article provides evidence for a nuanced view of the network agenda–setting model; intermedia agenda–setting effects varied by media type, issue type, and time periods.
Full-text available
Massive amounts of misinformation have been observed to spread in uncontrolled fashion across social media. Examples include rumors, hoaxes, fake news, and conspiracy theories. At the same time, several journalistic organizations devote significant efforts to high-quality fact checking of online claims. The resulting information cascades contain instances of both accurate and inaccurate information, unfold over multiple time scales, and often reach audiences of considerable size. All these factors pose challenges for the study of the social dynamics of online news sharing. Here we introduce Hoaxy, a platform for the collection, detection, and analysis of online misinformation and its related fact-checking efforts. We discuss the design of the platform and present a preliminary analysis of a sample of public tweets containing both fake news and fact checking. We find that, in the aggregate, the sharing of fact-checking content typically lags that of misinformation by 10--20 hours. Moreover, fake news are dominated by very active users, while fact checking is a more grass-roots activity. With the increasing risks connected to massive online misinformation, social news observatories have the potential to help researchers, journalists, and the general public understand the dynamics of real and fake news sharing.
Following the 2016 US presidential election, many have expressed concern about the effects of false stories ("fake news"), circulated largely through social media. We discuss the economics of fake news and present new data on its consumption prior to the election. Drawing on web browsing data, archives of fact-checking websites, and results from a new online survey, we find: 1) social media was an important but not dominant source of election news, with 14 percent of Americans calling social media their "most important" source; 2) of the known false news stories that appeared in the three months before the election, those favoring Trump were shared a total of 30 million times on Facebook, while those favoring Clinton were shared 8 million times; 3) the average American adult saw on the order of one or perhaps several fake news stories in the months around the election, with just over half of those who recalled seeing them believing them; and 4) people are much more likely to believe stories that favor their preferred candidate, especially if they have ideologically segregated social media networks.
Organized journalism in every era offers examples of news sharing: cooperative practices by which rival news outlets work together to produce or distribute news. Today, this behavior is being institutionalized by certain emergent news organizations. To understand news sharing, we argue, requires attention to how these journalists seek to not only practice but repair the field of journalism. This article analyzes news sharing as a form of field repair, drawing on ethnographic studies of investigative news nonprofits and professional fact-checking groups. We argue that journalism's "high-modern" era, with its broad alignment of economic and professional goals, highlighted competitive rather than collaborative elements of newswork. As that alignment unravels, journalists are engaging in explicit news sharing in pursuit of two intertwined goals: to increase the impact of their own reporting and to build institutional resources for public-affairs journalism to be practiced more widely across the field.
Hydraulic fracturing, or fracking, is the process of extracting reserves of natural gas and oil from shale formations deep underground. This process, initially met with public support in the United Kingdom, has since become a highly contentious issue primarily debated between government, industry, and anti-fracking advocacy groups. Through the employment of three levels of agenda-building as a theoretical lens of examination (object, attribute, and network connections), this study investigated the political discourse between these stakeholder groups and national news media in the United Kingdom between 2012 and 2014. In total, 1354 unique messages were coded, with 840 information subsidies and 514 news media content analyzed. To determine the short-, mid-, and long-term effects of agenda-building, a time-lag analysis was conducted. The results suggest solid support for all three levels of agenda-building. Our findings indicate that anti-fracking advocacy groups were more successful at influencing news media content across all three time periods, in addition to supporting the growing influence of digital information subsidies as an effective tool for agenda-building strategies. The theoretical and practical implications of these findings are discussed.
With continuous advancements and an increase in user popularity, data mining technologies serve as an invaluable resource for researchers across a wide range of disciplines in the humanities and social sciences. In this comprehensive guide, author and research scientist Kalev Leetaru introduces the approaches, strategies, and methodologies of current data mining techniques, offering insights for new and experienced users alike.
Online social networks provide a rich substrate for rumor propagation. Information received via friends tends to be trusted, and online social networks allow individuals to transmit information to many friends at once. By referencing known rumors from, a popular website documenting memes and urban legends, we track the propagation of thousands of rumors appearing on Facebook. From this sample we infer the rates at which rumors from different categories and of varying truth value are uploaded and reshared. We find that rumor cascades run deeper in the social network than reshare cascades in general. We then examine the effect of individual reshares receiving a comment containing a link to a Snopes article on the evolution of the cascade. We find that receiving such a comment increases the likelihood that a reshare of a rumor will be deleted. Furthermore, large cascades are able to accumulate hundreds of Snopes comments while continuing to propagate. Finally, using a dataset of rumors copied and pasted from one status update to another, we show that rumors change over time and that different variants tend to dominate different bursts in popularity. Copyright © 2014, Association for the Advancement of Artificial Intelligence ( All rights reserved.