ArticlePDF Available

Abstract and Figures

Research suggests that fact checking corrections have only a limited impact on the spread of false rumors. However, research has not considered that fact-checking may be socially contingent, meaning there are social contexts in which truth may be more or less preferred. In particular, we argue that strong social connections between fact-checkers and rumor spreaders encourage the latter to prefer sharing accurate information, making them more likely to accept corrections. We test this argument on real corrections made on Twitter between Janurary 2012 and April, 2014. As hypothesized, we find that individuals who follow and are followed by the people who correct them are significantly more likely to accept the correction than individuals confronted by strangers. We then replicate our findings on new data drawn from November 2015 to February, 2016. These findings suggest that the underlying social structure is an important factor in the correction of misinformation.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
Download by: [] Date: 05 September 2017, At: 22:08
Political Communication
ISSN: 1058-4609 (Print) 1091-7675 (Online) Journal homepage:
Political Fact-Checking on Twitter: When Do
Corrections Have an Effect?
Drew B. Margolin, Aniko Hannak & Ingmar Weber
To cite this article: Drew B. Margolin, Aniko Hannak & Ingmar Weber (2017): Political Fact-
Checking on Twitter: When Do Corrections Have an Effect?, Political Communication, DOI:
To link to this article:
View supplementary material
Published online: 05 Sep 2017.
Submit your article to this journal
View related articles
View Crossmark data
Political Fact-Checking on Twitter: When Do
Corrections Have an Effect?
Research suggests that fact checking corrections have only a limited impact on the
spread of false rumors. However, research has not considered that fact-checking may
be socially contingent, meaning there are social contexts in which truth may be more or
less preferred. In particular, we argue that strong social connections between fact-
checkers and rumor spreaders encourage the latter to prefer sharing accurate infor-
mation, making them more likely to accept corrections. We test this argument on real
corrections made on Twitter between Janurary 2012 and April, 2014. As hypothesized,
we find that individuals who follow and are followed by the people who correct them
are significantly more likely to accept the correction than individuals confronted by
strangers. We then replicate our findings on new data drawn from November 2015 to
February, 2016. These findings suggest that the underlying social structure is an
important factor in the correction of misinformation.
Keywords accountability, fact-checking, misinformation, rumor, social networks
The dissemination and acceptance of false political rumors threatens the efficacy of
democracy (Allport & Postman, 1947; Conover, Gonçalves, Flammini, & Menczer,
2012; Gottfried, Hardy, Winneg, & Jamieson, 2013). The dissemination of fake news, in
particular, has received renewed attention in the news media after the discovery of its
proliferation during the 2016 U.S. presidential election (Giglietto, Iannelli, Rossi, &
Valeriani, 2016). Yet, unfortunately, empirical studies of rumor and misinformation sug-
gest that stemming the spread of fake news is difficult. Experimental research suggests that
when misinformation is consistent with an individuals existing worldview corrections will
have a minimal impact (Einwiller & Kamins, 2008; Garrett, Nisbet, & Lynch, 2013;
Lewandowsky, Ecker, Seifert, Schwarz, & Cook, 2012) and in some cases, cause the
individual to hold more strongly to false beliefs (Nyhan & Reifler, 2010). Recent large-
scale studies of rumor diffusion are equally discouraging. In a national survey, Garrett
(2011) finds that exposure to corrections had only a small impact on belief. Friggeri,
Adamic, Eckles, and Cheng (2014) show that rumor propagation on Facebook is slowed
Drew B. Margolin is Assistant Professor, Department of Communication, Cornell University. Aniko
Hannak is Doctoral Candidate, College of Computing and Information Science, Northeastern University.
Ingmar Weber is a Senior Scientist, Qatar Computing Institute, Hamad Bin Khalifa University.
Address correspondence to Drew Margolin Department of Communication, 472 Mann Library
Building, Cornell University, Ithaca NY 14853, USA. E-mail:
Color versions of one or more of the figures in the article can be found online at www.
Political Communication, 00:124, 2017
© 2017 Taylor & Francis Group, LLC
ISSN: 1058-4609 print / 1091-7675 online
Downloaded by [] at 22:08 05 September 2017
by the presentation of correcting information; however, they also find that many rumors
can be corrected repeatedly and still continue to diffuse through a social network. Shin,
Driscoll, and Bar (2016) find that the publication of research debunking a rumor by a fact-
checking website does not substantially alter the rate of rumor adoption in Twitter.
Analyses of large-scale information consumption patterns on social media also suggest
that some people may seek out rumors and avoid sources that might correct them (Bessi
et al., 2015; Mocanu, Rossi, Zhang, Karsai, & Quattrociocchi, 2015). While some studies
do show that corrections can have an effect (Weeks, 2015) the effects are often weak and
require special contexts.
These findings can be discouraging for those who hope that individuals will internalize and
practice deliberative norms in political communication (Mutz, 2008). According to the norms of
deliberative democracy, individuals should prefer factual information, forming their views about
policy and other public views like intuitive scientists (Tetlock, 2002). People behave like
intuitive scientists when they seek accurate explanations and understandings of true relationships
in the world. Citizens as intuitive scientists is a basic assumption of democratic governance
(Bimber, 2003). If citizens dont prefer facts to false rumor and myth, it can be difficult to justify
the aggregation of their voices as a means of guiding government policy (Sunstein, 2006).
That human beings do not consistently behave like intuitive scientists does not mean
that they never do so (Tetlock, 2002). Tetlock argues that individualspreference for
intuitive scientific behavior is socially contingent. That is, people behave like intuitive
scientists when their social contexts provide incentives for scientific behavior. These
incentives are present in some contexts and absent in others, influencing the individuals
reasoning style. Of particular relevance to the study of political misinformation is the
motivation to behave as intuitive politicians (Tetlock, 2002). People behave like intuitive
politicians when they seek to maintain a positive reputation or fulfill the social duties for
which they are accountable.
Tetlock argues that, far from being restricted to professional politicians, intuitive
political behavior is rational and widespread. Intuitive political reasoning is also not less
sophisticated. Kahan (2012) finds evidence that those who are better at sophisticated
analytical reasoning are often better at defending their own social groups point of view,
possibly enhancing their reputation or fulfilling an important role as a result.
Intuitive political reasoning is also not universal but is, instead, activated by social
circumstances. For example, as demonstrated in the famous Asch experiments (Asch,
1951,1956), in situations where there are strong pressures to conform, individuals will-
ingly adopt clearly false ideas (in that case, agreeing that a shorter line is equal to a longer
line) when surrounded by group members who already hold them. Individuals also modify
their performance on other objectivetasks based on information about who is going to
be judging them (Lerner & Tetlock, 1999). The motivations to conform stem from the
importance of maintaining a good reputation and ties to friends or social group.
Tetlocks argument raises the possibility that a key to promoting the preference for truth
over falsity is the promotion of social conditions that either encourage scientific thinking and
discourage political thinking or at least align the two, encouraging individuals to see it as part
of their expected social role to think scientifically. Yet previous work on the correction of false
rumors has not explicitly examined variation in the social context in which corrections take
place. In particular, laboratory studies emphasize asocial corrections, such as from professional
news media (e.g., Nyhan & Reifler, 2010), while computational field studies (e.g., Friggeri
et al., 2014;S
hinetal.,2016) have focused on the macro-social consequences of corrections,
such as how much they influence the diffusion of false rumors across a social network, rather
than the consequences of social context for the individual being corrected. In this article we
2 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
test for the influence of one broad set of social conditions: the social network relationship
between an individual who believes false information and the individual who provides them
with corrective truth. Our test takes advantage of two unique affordances of social media. First,
these new technologies offer the potential for a broader dissemination of facts to counter false
claims (Garrett, 2011). In todays digital media environment, individuals can draw on a wealth
of well-sourced and widely recognized information providers including government websites,
Wikipedia, and other traditional and nontraditional sources to engage in immediate fact-
checking interventions, in which one individual challenges the false claim of another through
reference to a verified source of high-quality information (Hannak et al., 2014). Second, both
the conversational dynamics and social network relationships between individuals on social
media are publicly observable. This enables us to observe a set of uniquely relevant, although
rare, events in which individuals engage in fact-checking within their conversations and
compare outcomes based on their relationship to one another.
In this study, we collect a set of fact-checking interventions, which we refer to as
snopes,that respond to political rumors on Twitter. We then distill our observations to
the subset of cases where (a) the snope is clearly a correction of a false idea; (b) the
correction was unsolicited; and (c) the individual who is being corrected (the snopee)
replies to the individual who corrected them (the snoper), allowing us to analyze their
response to the correction. We then test hypotheses about how the relationship between
snoper and snopee should influence these replies based on social network theory (Burt,
2005; Coleman, 1988) and intergroup competition theory (Tajfel & Turner, 1979). While
previous work has used similar techniques to identify false rumors and the overall
effectiveness of corrections (e.g., Friggeri et al., 2014; Shin et al., 2016), our study differs
in that it explicitly examines the influence of another variablethe social relationship
between the snoper and snopeeon this effectiveness.
Social Motivations for Intuitive Science
Thinking scientifically benefits individuals in two basic contexts: (a) where individuals
directly benefit from having accurate knowledge; and (b) where there are strategic
incentives to scientific thinking, such as to appear to be scientific in the eyes of others.
The first case is comprised of situations where an individuals beliefs inform decisions that
have direct, unmediated consequences to them, and thus accurate beliefs, constructed
through scientific thinking, are likely to lead to better personal outcomes than inaccurate
ones. For example, in the Asch study case, a person who falsely perceives the length of a
line might make mistakes in making household repairs, leading to further difficulties or
This kind of direct consequence is rare in political contexts, however, because any
single individuals beliefs and attitudes have so little influence over the actual direction of
policy (Kahan, 2012). In democracies, an individuals personal beliefs only directly bear
on government decisions in the rare case where his or her vote makes the difference in
electoral outcomes (Christakis & Fowler, 2009). Furthermore, even in these cases, an
individuals beliefs about one issue may not influence his or her vote (Enns, 2015). These
factors remove individual incentives to be accurate. As Kahan (2012) writes,
because what any ordinary individual believes about policy will not make a
difference, the collective irrationality of ideologically motivated reasoning
does not by itself create any reliable pressure or mechanism to induce
Political Fact-Checking on Twitter 3
Downloaded by [] at 22:08 05 September 2017
individuals to process information in a different, and morally and politically
superior, way. (p. 42)
For example, in our data a number of individuals corrected one anothers estimations
of the U.S. federal budget deficit. Although these disputes involved huge sums, the fact
that both U.S. fiscal policy and its outcomes are highly complex and decided through
negotiations between many politicians makes it is hard to see that an individual personally
overestimating (or underestimating) the federal deficit by (even as much as) hundreds of
billions of dollars would cause them any direct consequences.
Although the direct consequences of false political information are hard to discern,
individuals can experience negative social consequences of spreading false political
information. Individuals who subscribe to false beliefs can draw scorn or distrust from
peers and those who spread misinformation may not be trusted (Ellwardt, Steglich, &
Wittek, 2012; Shapin, 1994). For example, in our data we observe numerous instances of
people rebuking those who have shared false information. These rebukes are often
followed by attempts to save face. One individual asserted that Obamacare required U.S.
citizens to be implanted with microchips. Five minutes later, a snoper sent her a link to a refutation of the claim and said Do some homework.The snopee immedi-
ately replied to show they had done so: I just did, right before you sent that LOL It wasnt
on snopes when I first seen it. I checked.
These individual-level incentives make up only a part of peoples motivation to
participate in political conversation. An important power in political conversation is an
individuals ability to influence others (Bond et al., 2012; Eveland, Morey, & Hutchens,
2011). Individuals can promote the diffusion of both true and false beliefs within their
social communities, affecting collective outcomes.
At the collective level, however, it is not clear that truth should always be preferred.
Social groups must balance the tension between within-group and between-group
dynamics (Wilson, 2012). Within the group, accurate information should be preferred to
inform collective decision making (Sunstein, 2006). But competition with other groups
leads to different incentives. These incentives can be particularly strong in cases of
intergroup conflicts of interest, in which groups compete for control of a common
resource, like political power (Turner, 1975).
Much as there are advantages to voting or expressing preferences strategically, rather
than honestly, in electoral contests (Austen-Smith & Feddersen, 2009), groups can gain
from advancing false ideas that serve a strategic purpose, such as eliciting contributions to
a public good (Burton-Chellew, Ross-Gillespie, & West, 2010) or mobilizing aggression
(Cohen, Montoya, & Insko, 2006).
In particular, when a social cluster or group of individuals agree on myths they can
support one another to take control of a collective decision, defeating competition from
rival groups. Solidarity about an idea within the group can both strengthen commitment to
it and intimidate rivals, irrespective of its truth content (Koudenburg, Postmes, & Gordijn,
2016). To return to the Asch study example, although an individual might be harmed by
incorrectly perceiving the length of a line (for example, mistaking one foot to be 11
inches), a group of individuals that can unrelentingly agree that a foot is equal to 11 inches
might gain an advantage in negotiating with other groups over things such as appropriate
weights and measures for commercial commodities (Crease, 2011). As a result, groups
impose pressures on individuals to express their agreement with the group as a means of
demonstrating their identification with it (Kahan, 2012).
4 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
Social Structural Antecedents of Intuitive Science
The narrow set of conditions under which individuals have incentives to think scientifi-
cally about political information may appear to suggest that deliberative practice is a rare
anomaly or aberration; however, this pessimistic interpretation overlooks the social struc-
tural conditions in which political conversation takes place (Morey, Eveland, & Hutchens,
2012). In fact, social network research suggests that social-structural incentives, sometimes
referred to as the rationale of social capital (Burt, 2005), are the basis for many norma-
tively preferred behaviors. For example, although honesty is an essential condition of
effective communication in many contexts (Grice, 1975), it is largely encouraged by
structural accountability to others who can control ones reputation and future access to
information and resources (Uzzi, 1997). In fact, Shapin (1994) argues that the modern
conception of intuitive science formed via social norms of interpersonal honesty within
aristocratic social networks.
Similarly, attending to a diversity of views requires cognitive effort and often little
personal gain. Individuals engage in it, however, as a consequence of their attempt to
maintain valued social ties. That is, people who have friends who have diverse political
views, or are connected to others with diverse views, attend to diverse political informa-
tion as a result of their desire to do something else they valuetalk with friends (Eveland
et al., 2011; Lazer, Rubineau, Chetkovich, Katz, & Neblo, 2010; Song, 2015). The
disincentives for such attention are thus outweighed by other, non-deliberative incentives
that force diversity upon them. Individuals also appear more willing to tolerate disagree-
ment and amend their views in light of it when conversing with friends (Morey et al.,
In each of these cases, there exist social-structural conditions that compel adherence to
a deliberative norm even when an individual would not, absent those structural conditions,
value or adhere to that norm on their own. Similarly, social networks are likely to be a
basis for the incentives to think scientifically about political information. In particular,
both the dyadic relationship between the snoper and snopee and their relationship to their
larger social network environment will have an impact on both reputational and intergroup
First, when individuals have a dyadic relationship, reputation and accountability are
more likely to be important (Coleman, 1988; Lerner & Tetlock, 1999). A typical assump-
tion in friendship relationships is that friends are epistemic peers,meaning both assume
they are equally competent at evaluating the truth of a claim (Elga, 2011). In these cases,
refusal to accept correct information from a friend can lead to gossip suggesting the
individual is obstinate or untrustworthy (Sommerfeld, Krambeck, & Milinski, 2008).
Relationships with friends can also create safe spaces in which to reexamine ones beliefs
(Morey et al., 2012). By contrast, when such corrections are given by strangers they are
likely to bring substantially lower reputational risks. Thus, the incentive to think scienti-
fically in order to save face and demonstrate ones competence is stronger when one is
corrected by a friend as opposed to corrected by a stranger.
Individuals also have collective motivations for engaging in public conversation.
Social media has created the opportunity for individuals to participate in networked
publics where they can not only communicate information but actively build solidarity
and shared understanding in a community (Papacharissi & de Fatima Oliveira, 2012;
Varnelis, 2012). Since, as described in the previous section, the group can benefit from
spreading false information (particularly about a rival), an important question is whether
the group prefers intuitive scientists or intuitive politicians in a particular case.
Political Fact-Checking on Twitter 5
Downloaded by [] at 22:08 05 September 2017
The social relationship between the snoper and the snopee can be a signal of this
preference. Individuals learn the norms of their group via admonitions from other group
members (Cohen et al., 2006). As Kahan (2012) describes, the individual can retain or
improve their standing in the group when they show a willingness to adopt or promote
false views on its behalf. It follows that when other group members correct their false
assertion, this utility is lost and the individual has less incentive to promote the false view.
Since friends are statistically likely (although not guaranteed) to be members of the
same social group as the individual (Christakis & Fowler, 2009), a correction from a
friend is likely to imply that the group does not accept this rumor and/or does not see
strategic value in promoting it. Consistent with this argument, Garrett (2011) finds that
individuals are more likely to believe false rumors when they are e-mailed to them by
friends. By contrast, corrections from strangers are more likely to be from non-group
members or even out-group membersthose whose interests oppose the interests of the
group (Tajfel & Turner, 1979). In these cases, individuals receive no signal suggesting
that acceptance will have group benefits, and may in fact infer that acquiescence to the
correction would do their group harm. In fact, Friggeri and colleagues(2014)studyof
fact-checking interventions on Facebook, where the majority of interactions take place
over existing friendships, indicates that although fact-checks are exceedingly rare rela-
tive to sharing of false rumors, they do lead to a significant increase in the deletion of
false-rumor posts. Thus:
H1: Snopees are more likely to accept corrections from snopers who are friends than from
snopers who are strangers.
Reputation and group loyalty are not earned and performed solely within the dyadic
relationship between snoper and snopee. Conversations in social media are organized around
larger social structures that can influence their interaction (Himelboim, 2010;Maireder&
Schlogl, 2014). In this networked context, snoper and snopee are not only related dyadically
(as friends or strangers) but also through ties with shared third parties (Monge & Contractor,
2003). The extent to which a pair of individuals share these triadic relationships is often
described as the embeddedness of their relationship (Moody & White, 2003;Uzzi,1997).
Individuals within embedded relationships tend to face stronger reputational pressures
because multiple third parties can report to one another on any misdeeds (Coleman,
1988). In Twitter, in particular, only individuals who follow both the snoper and the snopee
can observe the entirety of their interaction. At the same time, the consequences of losing
face are heightened within embedded networks (Burt, 2005). When others learn of the
offense via gossip, the individual risks the loss of all of the embedded ties.
Similarly, the probability that two individuals who share many embedded ties are part
of the same social group is even greater than if they simply share an existing, dyadic
relationship (Monge & Contractor, 2003). Thus, snopees who are in the same embedded
community as their snoper should be more likely to take the snopers rebuke as a signal
that their shared collective prefers truth in this case. For these reasons we predict the
H2: The larger the shared audience between snoper and snopee, the more likely that the
snopee will accept the correction.
Another feature of social structure that may influence individualsdecisions to think
scientifically is their social role (González-Bailón, Wang, & Borge-Holthoefer, 2014). The
6 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
two-step flow model suggests most individuals consult a small number of local experts
for political knowledge (Katz, 1957), a behavior observed in Twitter and other online
forums (Choi, 2014; Himelboim, 2010; Lin, Keegan, Margolin, & Lazer, 2014). Whether
these leaders or starsin the network are more or less likely to acquiesce to snopes than
typical individuals, or followers,is difficult to predict a priori. On the one hand, leaders
have more freedom and power to challenge other individuals. On the other hand, they are
often expected to demonstrate a greater commitment to the group and uphold group norms
and values (Reicher, Haslam, & Hopkins, 2005). For example, Lin and colleagues (2013)
find that elites within political parties are more likely to deny political information that is
obvious to the general public (that their preferred candidate performed poorly in a debate).
More broadly, research indicates that the overall impact of experts on aggregate diffusion
patterns is substantially influenced by the overall social structure (Watts & Dodds, 2007).
Thus, there is not likely to be an easily observable aggregate signal of expert influence on
misinformation, particularly since corrections are so rare (Friggeri et al., 2014). Testing the
influence of potential opinion leaders thus requires examining specific cases.
In sum, while we have a clear expectation that the size of an individuals
audiencethe number of followers they havemay be an important variable in
determining their response to a correction, we do not have a specific prediction
about the direction of this relationship. We thus propose the following research
questions (RQs):
RQ1a: How does the size of the snopees audience affect the tendency to accept the
RQ1b: How does the size of the snopers audience affect the snopees tendency to accept
the correction?
Methodstudy 1
Data and Measures
Our procedure is similar to that used in Hannak et al. (2014). We first identify political
snopes: reply tweets that have a high probability of containing corrections about a political
fact. We then gather social network datafriend and follower listsof the user who sent
the reply and the user who was replied to and characterize their relationship to each other.
Next we examine the dynamics of their exchange. First, we take the subset of replies in
which the snopee responds. Next we code each case (that drew a response) to determine
whether it is a correction. Finally, we analyze the responses to these correcting snopes to
see whether the correction was accepted. These procedures are described in more detail
Identifying Snopes. We begin by selecting all tweets posted between January 2012 and
April 2014 that meet two criteria: they (a) reply to another user, and (b) contain a reference
to one of three fact-checking website domains (,, PolitiFact.
Using these sites as indicators of corrections is common practice in fact-checking
research (see Friggeri et al. [2014] and Shin et al. [2016]). We choose references to these
websites in our study because, first, we expect that a high proportion of the tweets that use
them in replies will be factual corrections (as compared with all replies to any rumor tweet
Political Fact-Checking on Twitter 7
Downloaded by [] at 22:08 05 September 2017
that links to outside sources, such as newspaper articles or Wikipedia). This enables us to
focus our more detailed, human coding efforts on relevant cases. Second, and importantly
for our study, this method provides the benefit that all such replies contain roughly
equivalent factual evidencethe standard format used on the fact-checking website.
Thus, variations in acceptance of the correction should not be influenced by systematic
differences in the absence/presence or quality of evidence provided.
The aforementioned studies that examine references to fact-checking websites focus
on the behavior of anyone exposed to the rumor or the correction; our focus is on the
specific behavior of the snopee. Thus, our method only begins with these snopes as
candidates for analyzing the dynamics of correction, and drills deeper to identify a smaller
set of cases where a fact-check is known to have occurred and where the corrected
individualthe snopeeresponds in some way.
NomenclatureSnope Triplets. To facilitate interpretation of these conversational
responses, which involve potentially confusing concepts like replies to replies,we define
our data in terms of snope triplets. Snope triplets are defined by three tweets:
Tweet A0 = the snopedtweetthe tweet sent by the snopee that drew the
snoping reply from the snoper.
Tweet B1 = the snopingtweetthe tweet sent in reply to A0 that contains
the snope.
Tweet A2 = the reply from the snopee to the snoping tweet.
Identifying Political Snopes. We filter the snopes (B1s) based on the specific URLs in the
replying tweets to identify those snopes that relate to U.S. politics. Specifically, for the
domains and we used all rumors within the subcategory
politicsas well as anything that had Obamaor Romneyin the URL. While the
latter became less relevant after the 2012 election, results from Friggeri and colleagues
(2014) indicate that rumors often have a long life. We use this criterion to ensure that any
uncategorized rumors that included either presidential candidate were available for our
analysis. For we used all tweets that contained this domain in the URL
since the site is related to politics.
Characterizing Snoper-Snopee Dyads. We obtain the accounts followed and the accounts
who follow each snoper and snopee in our data.
This allows us to characterize each
relationship in terms of the following variables:
Mutual Friends. When both the snoper and the snopee mutually follow each other,
we characterize this relationship as mutual friendsor, in common parlance,
Strangers. When neither the snoper nor the snopee follow each other, we character-
ize this relationship as strangers.
Snoper Follows Snopee. If the snoper follows the snopee but the snopee does not
reciprocate, we characterize this as snoper follows snopee only.
Snopee Follows Snoper. If the snopee follows the snoper but the snoper does not
reciprocate, we characterize this as snopee follows snoper only.
Although these relationships, observed in Twitter, are likely not as strong as face-to-
face relationships, we expect the logic of accountability for reputation and group roles to
8 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
apply, as research indicates that people are broadly aware of who their Twitter followers
are (Hofer & Aubert, 2013) and that they imagineand adjust their tweets to satisfy their
follower community when composing their tweets (Marwick & Boyd, 2011).
Snopee Follower Count. The total number of Twitter followers for the snopee.
Snoper Follower Count. The total number of Twitter followers for the snoper.
Shared Followers. Snopers and snopees can also be characterized in terms of the
number of followers they have in common. For each dyad we count the number of
shared followers as the intersection of the sets of accounts that follow both the
snoper and the snopee. This quantity measures the extent to which these individuals
share an audience.
We note here that that the Twitter application programming interface (API) does not allow us
to identify the precise timing of these friend/following relationships. That is, it is possible that
after an individual fails to accept a snope from a friend, they are unfriended and become strangers,
or that after an individual successfully snopes a stranger, they become friends. Recent studies on
unfriendingon Twitter suggest that this behavior is not particularly common. For example,
Moon (2011) reports that unfriending occurs on just over 10% of relationships over a 50-day
period. More importantly, these relationships are disproportionately newer, indicating weaker ties
that, theoretically, are more like strangerrelationships, and much less likely to occur in
mutually following relationships, a finding corroborated by Xu, Huang, Kwak, and Contractor,
2013). In other work, Garimella et al (2014) find that prior to and after an unfollowdue to a
relationship dissolution the number of messages to the unfollowed party decreases.
To check for support of our assumption, we analyzed the full conversation history
between each snoper and snopee in our dyadthat is, all of the times they @mentioned
each other before or after the snope. Analysis revealed that conversations were robust to
snoping events, as dyads that tended to converse before one snoped the other also tended
to converse after the snope for both friends and strangers and irrespective of the outcome
of the snope incident (β= .390, p< .001). This provides indirect evidence that the
relationship between the snoper and snopee did not materially change with the snope
Identifying A0-B1-A2 Triplets. After gathering social network data we have 2,044 original
snopes for which relevant social network data were available for both the snoper and the
snopee. Of these, 542 contained replies from snopees (A2s; after removing 15 cases where
individual sent the same reply twice). The fact that many snopes do not generate replies
poses the question of how these should be treated. From one point of view non-replies
might be coded as non-acceptances since the snopee provides no observable acknowl-
edgment of the validity of the correction. Treating non-replies as non-acceptances can be
problematic, however, because they outnumber replies by almost 3 to 1. This means that
any systematic pattern that explains whether or not snopees reply will be likely to
statistically dominate snopee tendencies within replies. In addition, Hannak et al (2014)
have already studied differential rates of reply to snopes and found that, consistent with the
theoretical arguments in this article, strangers are more likely to ignore (fail to reply to)
snopes than friends. We observe similar dynamics here as strangers reply to fewer (24%,
266 of the 1,102) of the snopes they received compared to mutual friends (44%, 191 of the
435 snopes). Similarly, rates of reply when the snopee follows the snoper (but is not
reciprocated) are similar to that of mutual friends (45%, 32 of the 71 snopes). By contrast,
however, when the snoper follows the snopee (but is not reciprocated), rates of reply are
quite low (13%, 53 out of 421 snopes).
Political Fact-Checking on Twitter 9
Downloaded by [] at 22:08 05 September 2017
Overall these figures suggest that for our primary contrast of interestfriends
versus strangersfriends are not likely to be affectedbydifferencesintherateof
reply. In particular, both prior work and our data indicate that friends are more
likely to respond to snopes than strangers, making an additional tendency to reply in
a more accepting way further consistent with the theory. We thus exclude non-
replies and focus only on the 542 cases where the snopee replied to the snope (A2
exists). Nonetheless, as a check of robustness on significant findings in our main
model we also run models in which non-replies are included as non-acceptances.
Of these 542 there were 91 cases where one or more of the tweets had been deleted or
was served from a protected account, or was not in English, and thus we did not code these
or include them in any analysis. This left 451 complete triplets for study.
Categorizing Snopes and Responses. As Shin and colleagues (2016) report, and consistent
with Hannak et al. (2014), many references to fact-checking are not corrections but promote
a rumor or are responses to queries about its truthfulness (i.e., indicating the snopee is
already thinking like an intuitive scientist). Our theory does not apply to these cases so they
must be removed for our analysis. We code each of these 451 triplets for whether the B1 is
a correction, and, if it is a correction, we code the exchange for whether the A2 accepts the
correction or not. Categories are described below and additional examples provided in
Tab le 1.
Non-correcting snope. A correction points out that the snopee is wrong about a fact
or spreading false information. Non-corrections took three basic forms. First, some-
times the snopee explicitly solicited facts on a topic. For example, one individual
had encountered an incriminating photo of President Barack Obama and asked,
Have you seen this and know when it was taken??Second, there were cases
where the snoping tweet referred to information that was irrelevant to or actually
supported, rather than corrected, the original snopee tweet. Also, since our algorithm
detects any replies that refer to a snoping website, as an artifact it detects all replies
to the Twitter account of snoping websites themselves, and we code these as non-
correctingsince they do not actually cite fact-checking corrections.
Accepts. The snopee explicitly acknowledges that the fact presented by the snopee is
correct or clearly accepts it as correct by, say, admitting that they were wrong.
Alternatively, the snopee accepts the legitimacy of the fact as possibly true, stating
that it is plausible or worthy of consideration. This case thus captures whether the
snope was successfulinsofar as it modified the snopees sense of what is true, or at
least of what it is acceptable to claim as true. Importantly, the snopee need not be
convinced of the validity of a larger argument or new political position; they must
simply acknowledge that the fact in the snope is accurate (see Figure 1 and Table 1
for examples).
Ambiguous or Other. This includes cases where the snopee rejects the correction
explicitly as well as other cases that were difficult to classify in epistemic terms. For
example, sometimes the snopee does not comment on the truth or falseness of the
snope fact at all but shows hostility toward the correction or snoper. This includes
statements like I dont care if its trueas well as statements that personally attack
the snopers intentions or change the subject without making a statement about the
fact itself. This also includes cases where it is too difficult to interpret what the
snopee is saying, often because they use slang or refer to entities or ideas unfamiliar
to the coder. We initially attempted to code these cases into subcategories of
10 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
rejectionof the snope and other.However, obtaining a consistent coding proto-
col was difficult because rejectionof a fact, the truth of a claim, was rare in a pure
form and hard to meaningfully distinguish from the rejectionof the social beha-
vior being corrected. For example, it is unclear if the individual in the top left corner
of Figure 1 (all I wanna say is) is rejecting the correction or simply ignoring or
moving on from it. Similarly, the tweet in the last row of Table 1 (how come your
link didnt) argues with the snoper but doesnt quite indicate rejection of the truth
of the article. Since our research question focuses on the positive outcome (accep-
tance), we did not pursue coding rejections further.
Coding was conducted by two coders (the author and a research assistant). The coders
independently evaluated a subset of tweets (about 10%) and met to discuss and recode
them until they were in agreement regarding the coding protocol. Then the remaining
snope triplets were coded independently. The coders agreed on 71% of cases across the
three mutually exclusive categories (non-correcting, accepts, ambiguous or other), yield-
ing a Cohens kappa = .58. Since this level shows only moderateagreement it was
decided that further analysis would only be conducted on the cases where both coders
Tab l e 1
Coding scheme
Category Typical Forms Example
Original (A0) tweet solicits
fact-check or further
evidence for a claim
@mention sounds too funny to be
Snope (B1) corroborates
original (A0) tweet
Snoper: @snopee even more
appalling. How this didnt make
national news in 2007 is beyond me
Replies to fact-checking
@factcheckdotorg Hey the link is
broken, may want to repost with
working link. http://url
Accepts Reply (A2) accepts snope
(B1) as true or correct
@snoper Sorry I was incorrect, but i
would argue that march of 2008,
4 years ago, gas prices comparable
with a steep decline[sic]
Reply (A2) considers (B1) is
@snoper Snopes is owned by George
Soros. However, it can be debated
that the ring says No God but
God instead
Ambiguous or
Reply (A2) rejects snope (B1) @snoper Snopes is as left wing
oriented as they come & thus their
factstend to reflect their politics
Reply (A2) is ambiguous with
respect to truth of snope
@snoper The success that Romney
had had in his life is the reason you
hate him. How come your link
didnt cover executive orders?
Political Fact-Checking on Twitter 11
Downloaded by [] at 22:08 05 September 2017
agreed with the characterization of the snope (correcting, non-correcting) and the sno-
pees acceptance of the fact. This yields 322 triplets, of which 229 (71%) are corrections
and thus suitable for analysis of whether a snopee acceptsafact.Asacheckon
robustness, we also analyze the full (451) cases from the point of view of each coders
interpretation (whether the snope was a correctionand whether it was accepted)and
obtain substantively similar results (see supplemental Appendix).
Statistical tests are conducted using logistic regression. The dependent variable is whether
or not each snopees response to the snope is an acceptance. The relationships between
snoper and snopee are treated as covariates in this analysis. We also run ordinary least
squares (OLS) regression and find substantively similar results (see supplemental
Appendix). Although the dependent variable is binary, there is now evidence that OLS
provides appropriate results in most cases (Angrist & Pischke, 2009). We use the OLS
regression to provide more easily interpretable information about the impact of our
theorized variables on the absolute level of correction acceptance, rather than just their
impact on relative odds.
Resultsstudy 1
Preliminary Analysis
As previously described, our coding system yielded 322 cases where there was complete
data and agreement about whether the snope was a correction. Of these, 93 cases were
non-corrections.Although this is a substantial loss in terms of statistical power, it did
not appear to introduce any bias, as the rate at which corrections and non-corrections come
from friends and strangers is remarkably similar, with 70.4% of snopes from friends being
Figure 1. Frequency of Acceptance by Social Relationship
12 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
corrections and 72.5% of snopes from strangers being corrections (chi-square = .60, df =1,
p= .56). Our remaining analysis focuses on the 229 corrections within this set. In these
229 cases, 135 (59%) were instances where the snopee acceptedthe snope as true.
Hypothesis Tests
Hypothesis 1. Hypothesis 1 (H1) predicted that snopees would be more willing to accept
when snopes came from friends as opposed to from strangers. We begin our analysis by
examining the impact of the dyadic relationship between snoper and snopee. Figure 1
shows a breakdown of whether the snopee accepts the snope fact according to whether the
snopee and snoper had one of two relationships: snopee and snoper each follow each other
(mutual friends), and neither follows each other (strangers). We see that for stranger-to-
stranger snopes, the percentage in which the fact is accepted is 39%, whereas it is more
than 73% for cases where snopes are between mutual friends. This trend is similar for the
other relationship types (not shown in the figure for visual clarity): 7 out of 11 (64%) of
snopees who follow their snopers accept the fact, and 23 out of 29 (79%) of snopees who
are followed by their snopers accept the fact.
Regression results highlighting these variables, shown in column 1 of Table 2,
indicate that when the snoper follows the snopee, either as a mutual friend or as an
unreciprocated follower, this significantly increases the likelihood of the snopee
accepting a snopers fact as true. Although they are somewhat reduced, these results
are robust when controlling for other hypothesized variables (Tabl e 2 ,column4),
with the odds of a mutual friend accepting a correction being 2.5 times (β= .899,
p< .05) that of a stranger accepting a correction. In terms of absolute rates, OLS
regression results indicate that strangers have a 24% chance of accepting a
correction, with mutual friends increasing the chance by 22% to 46%. This result
holds in each of our robustness checkswhen non-responses are treated as non-
acceptancesand when different codersperspectives are accounted for (see supple-
mental Appendix). We also observe that when the snoper follows the snopee
exclusively the odds of the snopee accepting the correction are 3.4 times greater
(β= 1.211, p< .05). However, these results do not hold when non-responses are
treated as non-acceptances,possibly due to the fact that these particular relation-
ships garner fewer replies (only 13% as reported earlier). It is thus difficult to rule
out the possibility of selection effects here, with snopees only replying to their
(unreciprocated) followers when they plan to agree. Nonetheless, the robust support
for mutual friends provides support for H1.
Hypothesis 2. Hypothesis 2 predicted that snopees would respond more productively
when they were embedded in a shared audience with snopers. We measure this shared
audience by counting the number of Twitter followers shared by the snoper and snopee.
Results for this variable on the acceptance of the snope fact are shown in column 2 of
Table 2. There is a statistically significant relationship (β= .449, p< .001). The larger the
number of followers in common with the snoper, the more likely the snopee will accept the
fact in the snope. These bivariate results hold in each of our robustness tests. The number
of shared followers is also significant in the full model (Table 2, column 4) (β= .265,
p< .01); however, these results are not significant in any of the robustness tests (see
supplemental Appendix). Thus, overall these results indicate partial support for H2.
Political Fact-Checking on Twitter 13
Downloaded by [] at 22:08 05 September 2017
Research Question 1. Research question (RQ) 1a asked whether the size of a snopees
audience had an influence on the snopees decision to accept the snope fact and/or to
defend their original statement. RQ1b asked whether the size of a snopers audience had
an influence on these decisions by the snopee. Column 3 of Table 2 shows a significant
bivariate relationship between the number of followers possessed by the snopee and their
likelihood of accepting the snope fact (β= .194, p< .05) and is reversed (i.e., negative) in
our robustness checks using all replies, possibly due to snopees with large followings
being unlikely to send replies. The relationship is also no longer significant when
accounting for the other hypothesized variables (column 4). Column 3 also shows no
statistically significant relationship between the snopers audience size and the likelihood
that the snopee will accept the fact.
Methodstudy 2
The results of Study 1 were limited in two important respects. First, all of the data
were collected from one particular time period that featured an electoral competition
between Barack Obama and Mitt Romney. It is possible that observed differences
areduetothekindsofrumorsandcorrections that related to those candidates,
rather than a broader principle. Second, by restricting our data collection to only
politically relevant tweets, we were not able to compare political snoping to
Tab l e 2
Study 1Do snopees accept the snoping fact as true?
Dependent Variable:
(1) (2) (3) (4)
Mutual friends 1.450*** 0.899*
(0.315) (0.415)
Snopee follows snoper 1.009 0.742
(0.661) (0.680)
Snoper follows snopee 1.639*** 1.211*
(0.480) (0.508)
Shared followers (log) 0.449*** 0.265*
(0.095) (0.131)
Snopee follower count (log) 0.194* 0.112
(0.089) (0.101)
Snoper followers (log) 0.054 0.022
(0.086) (0.098)
Constant 0.450* 0.366 1.148 1.172
(0.210) (0.199) (0.703) (0.834)
Observations 229 229 229 229
Log likelihood 141.150 141.959 152.189 137.108
Akaike information criterion 290.301 287.919 310.377 288.215
Note. Logistic regression.
*p< 0.05.*** p< 0.001.
14 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
nonpolitical snoping. In particular, while both kinds of snopes should be subject to
reputational mechanisms, intergroup competitive dynamics should be more salient
for political snopes where there is a clear conflict of interest between parties
(Turner, 1975), and thus comparing political to nonpolitical snopes should shed
light on the relative importance of these mechanisms.
We thus attempted to replicate our significant findings on a new data set. On February
3, 2016, we manually scraped the Twitter website for tweets that contained hyperlink
references to the fact-checking site Manual scraping enables quick access to
recent data but does not afford access to longer term archives like Firehose does. Since the
former was sufficient for our purposes, we selected this method.
Data going back to October 31, 2015, was obtained by repeatedly scrolling
down for more search results in the Web interface and using Chrome Developer
Tools to save the Data Object Model. We then subset these tweets to include only
those that contain our snope triplet structure in which one tweet is snoped and then
the snopee replies to this snoping tweet.
As stated earlier, we loosened our filtering criteria to include both political and
nonpolitical snopes. Snopes were categorized as political if they referred to rumors marked
as politicalon or if they contained reference to a known politician such as
Barack Obama, Hillary Clinton, or Donald Trump.
To further check for the robustness of our methods, these tweets were then
coded by workers for, rather than the original set of coders.
CrowdFlower is a crowdsourcing platform on which paying clients can submit
micro tasks, also referred to as Human Intelligence Tasks (HIT), and paid contribu-
tors/coders can sign up to work on these tasks for pay. Each task is typically a small
set of multiple-choice questions.
Our scraper returned 759 tweets with the triplet structure. Of these, 465 of the
snoping tweets were identified consistently by two coders as an unsolicited correc-
tion to a rumor. We then deployed four or five coders to code each of these 465 for
whether the snopee accepted the correction. Average agreement was 91%. Since the
coding was performed by many different individuals (23 different individuals parti-
cipated on at least one snoping case), we calculate Krippendorffsalpha=.661
(Krippendorff, 1980). We then selected only the 414 triplets where at least 75% of
the coders agreed on the category for further analysis (i.e., 3 out of 4, or 4 out of 5).
Resultsstudy 2
To examine the robustness and generalizability of our finding from Study 1 we retest H1 and
H2 using the same model specification on the Study 2 data. Results, shown in Tabl e 3 ,are
substantively similar to those found in our original data set. As in our original test of H1, the
coefficient for mutual friends is significant in all models, with mutual friends increasing the
odds of acceptance by 5.1 times over strangers in the full model comparable to that in Study 1
(column 4) (β=1.633,p< .001). Comparison of the OLS coefficients indicates that the
magnitude of the change is also similar. Mutual friends are estimated to increase the prob-
ability of acceptance by 22% in Study 1 and 31% in Study 2 (see supplemental Appendix).We
also observe a statistically significant coefficient when the snoper follows the snopee exclu-
sively. However, the robustness analysis from Study 1 suggests this is likely subject to
selection bias and we do not interpret further.
Also, as in our original test of H2, the number of common followers also showed a
significant, positive relationship to acceptance when considered on its own (Table 3,
Political Fact-Checking on Twitter 15
Downloaded by [] at 22:08 05 September 2017
Tabl e 3
Study 2Do snopees accept the snoping fact as true?
Dependent Variable:
(1) (2) (3) (4) (5) (6)
Mutual friends 1.610*** 1.633*** 1.496*** 1.388**
(0.260) (0.311) (0.319) (0.504)
Snopee follows snoper 0.759 0.675 0.602 0.597
(0.608) (0.632) (0.639) (0.641)
Snoper follows snopee 1.922*** 2.108*** 2.030*** 2.019***
(0.407) (0.441) (0.445) (0.447)
Shared followers (log) 0.152** 0.024 0.018 0.020
(0.055) (0.088) (0.089) (0.089)
Snopee follower count (log) 0.041 0.151 0.137 0.137
(0.066) (0.083) (0.084) (0.084)
Snoper followers (log) 0.003 0.030 0.019 0.023
(0.062) (0.080) (0.080) (0.081)
Political rumor 0.558* 0.639
(0.270) (0.396)
Mutual friends x political rumor 0.150
Constant 1.857*** 1.407*** 0.764 0.715 0.398 0.311
(0.190) (0.184) (0.591) (0.703) (0.727) (0.792)
Observations 414 414 414 414 414 414
Log likelihood 213.400 234.908 238.466 211.410 209.313 209.275
Akaike information criterion 434.801 473.817 482.932 436.821 434.625 436.550
Note. Logistic regression.
*p< 0.05. ** p< 0.01. *** p< 0.001.
Downloaded by [] at 22:08 05 September 2017
column 2; β= .152, p< .01), but the effect was very small, increasing the probability of
acceptance by only 3% in the OLS model. Also, this relationship was not significant in the
full model. Thus, H2, which received weak support in Study 1, does not obtain support in
Study 2, suggesting that this mechanism is not independent of that which produces H1.
Post Hoc Analyses
Attitude Consistency. One possible explanation for our findings is that snopes from friends
are more likely to be accepted because friends are more likely than strangers to correct
rumors that the snopee does not wish to be true. If this is correct, then existing social
relationships may still be important to fact-checking interventions, but not necessarily
because they encourage the snopee to think more scientifically.
Because we perform our analysis only on cases where the rumor is asserted and
exclude cases where a snope is solicited(via the expression of doubt or a request for
corroboration), we expect most rumors to be attitude consistent, and thus most corrections
to be resisted regardless of the source. Nonetheless, to explore this possibility we examine
the extent to which the rumors shared by the snopee are consistent with their attitudes.
Measuring the attitude consistency of a rumor is difficult with our data because (a) we
do not have access to attitude surveys for our subjects, and so any categorization is
speculative; and (b) inspection of specific rumors shows that many do not align clearly
with easily identified attitudes such as partisanship. For example, the third most-shared
rumor in our data is a fake photo with Megyn Kelly (formerly of Fox News) and a man
who is purported to be a Saudi prince. It is not clear what prior information available on an
individuals Twitter account would indicate whether this rumor is consistent with their
We nonetheless categorize a modest subset of our rumors where attitude consistency
is easier to define in order to get a descriptive sense of attitude constancy patterns in our
data. We examine all cases of rumors and snopes related to either partys leading
presidential candidateDonald Trump and Hillary Clinton (82 total, for which 74 had
profile information available). We then look for the declaration of a partisan identity (e.g.,
Republican) or a candidate preference (Trump supporter) on the snopees profile page.
Coding was performed independently by two of the authors (Cohens kappa = .83 after
first round of coding, with further clarification of criteria leading to 100% agreement). In
our data there are 30 snopes with reference to a rumor about Trump and, of these, 16 were
attitude consistent (meaning the rumor was negative about Trump and was sent by a
Democrat or was positive about Trump and sent by a Republican or Trump supporter), and
0 being attitude inconsistent. There were also 14 uncategorizable based on the users
profile information (i.e., they made no mentions of candidates, parties, or political loyalites
on their profile). Similarly, there are 44 snopes with reference to a rumor about Clinton
and, of these, 32 were attitude consistent and 0 were attitude inconsistent, with 12 being
uncategorizable. Although the uncategorized profiles make it difficult to infer the extent to
which all rumors are attitude consistent, the fact that we find no evidence that any rumors
are attitude inconsistent suggests that variations in attitude consistency are not likely to
explain the entirety of our results.
A second test is to consider only these snopes of attitude-consistent rumors to see if
we recover similar patterns as in our full data set. Similar to our main result, we find that
only 13% of snopes of attitude-consistent rumors from strangers are accepted compared
with 40% of snopes of attitude-consistent rumors from non-strangers (chi-square = 3,
df =1,p= .08). If we restrict the non-strangers to mutual friends the percentage
Political Fact-Checking on Twitter 17
Downloaded by [] at 22:08 05 September 2017
acceptance increases to 60% (chi-square = 6.86, df =1,p< .01). From these analyses we
conclude that while variations in the attitude consistency of rumors snoped across relation-
ship types may explain a portion of our results, they are not likely to explain them entirely.
Nonetheless, as we acknowledge in the limitations, this question requires more thorough
Comparing Reputational Versus Group-Related Motivations. As we argued in the theore-
tical section, both H1 and H2 can be justified by two distinct mechanisms: the snopees
concern for their reputation and the snopees concern for their groups ability to compete
with others. Study 2 enables us to compare the influence of reputational mechanisms,
which should apply equally to both kinds of rumors, and group competition-based
mechanisms, which should be stronger for political rumors. Specifically, retaining false
ideas and rejecting truth should be a blemish to a snopees reputation whether or not the
information is political. By contrast, the motivation to accept corrections from friends, or
to reject corrections from strangers, out of a sense of group morality (Cohen et al., 2006)
should be stronger for political rumors, particularly during an election season when groups
are competing for resources and ideological control (Kahan, 2012).
As described earlier, we define political rumors as those for which the snopes URL
includes politicsor the name of an elected official or candidate. Of our 414 cases, 317 were
political and 97 were not. We first observe that stranger-stranger snopes outnumber friend-
friend snopes by more than 2 to 1 for political corrections (205 to 78), yet are only a bit more
than half of those for nonpolitical corrections (32 to 52; χ
=31.91,df =1,p< .001). Results
also indicate that, overall, the rate of acceptance of political rumors is lower (21%) than the rate
of acceptance of non-political rumors (42%, χ
=15.53,df =1,p<.001).
Further analysis indicates that the snoper-snopee relationship and the nature of
the topic of the correction have an independent relationship to acceptance, although
not a significant interaction (see Tab le 3, column 6). Column 5 of Tab le 3 shows the
results when politicalis included as a predictor of acceptance in the main model.
Mutual friends remains significant, while the political variable also shows a sig-
nificant, negative effect (β=.558, p< .05; odds ratio = .57). Examination of the
OLS models shows that the magnitude of the coefficients for snoper-snopee relation-
ship and the nature of the correction (comparable for these dummy variables)
suggests that the reputational force of accountability is stronger. Specifically, a
political rumor correction from a friend was more likely to be accepted than an
apolitical correction from a stranger (27.6% increase for friendship 11.2% decrease
for the rumor being political = 16.4% increase in probability of acceptance). Again,
we have no evidence of a significant interaction, however, providing no evidence
that the relationship matters even more when corrections are political, as implied by
the intergroup competition explanation.
Review of Findings
Our study departs from recent work on rumors and fact-checking by examining the
behavior of the corrected person as opposed to those in the social network as a whole
(e.g., Friggeri et al., 2014; Shin et al., 2016). In this study we argued that since individuals
feel more accountable to their friends and shared community, and are more likely to share
collective interests with them, they should be more likely to respond scientifically”—
18 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
accepting facts that challenge their statementswhen they have a bond or share a
community with the person who has corrected them. The results support these claims.
Across two different data sets collected more than three years apart, we find support for the
hypothesis that when the people involved in the correction have a mutual relationship, the
correction is more likely to be accepted. Controlling for this direct relationship, however,
there was only limited evidence that the extent to which the individual responded to the
correction in front of a common community or audience had an effect. There was also little
evidence that the size of a snopers or snopees audience played a significant part in the
fact-checking response.
We also compared the responses to corrections of political rumors to corrections of
nonpolitical rumors. Results indicate that while political rumor corrections are less likely
to be accepted overall, the dynamics for both kinds of corrections are similar: corrections
from friends and followers are more likely to be accepted. In particular, a political
correction from a friend was more likely to be accepted than an apolitical correction
from a stranger.
As described in the method section, the Twitter API does not provide information on when
snoper-snopee friendships are established or disbanded. Our analysis revealed indirect
evidence that their relationships did not substantially change as the result of the snoping
incident; however, in future work it might be of interest to apply existing approaches to
estimate the date when a social link was established (Meeder et al., 2011). Controlled
experiments in which existing friends and strangers are recruited to correct one another
could also eliminate this confound.
As noted earlier, our method cannot completely rule out the possibility that friends are
correcting friends in a different manner from how strangers are correcting strangers, either
because their corrections are attitude consistent or on less controversial rumors. These
effects would still be consistent with the idea that fact-checking is socially contingent;
however, the theoretical mechanism would be somewhat different. Specifically, instead of
activating intuitive science, friends may be sensitive to one anotherslatitude of accep-
tanceon emotionally charged topics (Sherif & Hovland, 1961). While we provide some
evidence that our hypotheses hold even when controlling for attitude consistency, a proper
test would require true prior information about the snopees attitude in relation to the
rumor before the correction. These kinds of data would be more appropriately obtained in
a controlled experiment design where attitudinal pretests could be given and then correc-
tions issued from friends or strangers via experimental manipulation.
Another limitation is that we focus on the dyadic, localrelationship between the snoper
and snopee. Although we also include information about the common audience size in our
models, more globalnetwork information could also be included to quantify their closeness,
such as their path-distance to one another. Similarly, one could try and infer a Twitter users
political preference and look at whether two users are in the same political camp, regardless of
their dyadic relationship (Conover et al., 2012) This approach brings its own limitation, as it is
not clear that individuals who do not know one another would be aware of one anothers
political loyalties. Nonetheless, this is a direction worth pursuing further.
Last but not least our results are obtained using one particular online social network,
Twitter, in a specific political contextU.S. presidential politicsand so the question of
generalizability naturally arises. To address this we have embedded our work in the wider
frame of research on rumors and fact-checking. As our findings are consistent with this body
Political Fact-Checking on Twitter 19
Downloaded by [] at 22:08 05 September 2017
of work, we believe that our findings are sound and not a domain or data-source-induced
artifact. Nonetheless, future work should consider testing these ideas both in new platforms,
including, if possible, face-to-face rumor corrections, and well as new political contexts,
such as those in other countries or outside of the context of a hotly contested election.
Broader Implications
Our results suggest that, consistent with Tetlocks(2002) argument about social con-
tingency, peoples decisions to behave like intuitive scientists when engaging with
political information depends substantially on the social structural context in which
conversations take place. One implication of this is that the source of the ineffectiveness
of corrections observed in public discourse (Shin et al., 2016) may be due to the social
position of these corrections, rather than an innate tendency for people to resist facts. In
other words, corrections may appear to be ineffective because corrections from strangers
are both more common and less likely to be accepted. These tendencies will serve to paint
a bleak picture in any data set that does not account for the snoper-snopee relationship.
Moreover, many data sets (e.g., Friggeri et al., 2014) include relational data regarding
snopers, snopees, and third-party observers. According to the theory we have proposed
here, we should observe differences not only in the behavior of the snopee after being
snoped by a friend but also in the behavior of all other friends of the snoper.
This raises the question of whether the dynamics of face-to-face rumor correction are
similar to those observed on Twitter in this study. It may be that stranger-stranger
corrections are dominant on Twitter because of the unique opportunity that social media
provide for individuals to carry on conversations with complete strangers. In other
conversational settings, people spend their time talking to their friends rather than to
strangers (Eagle, Pentland, & Lazer, 2009). Thus, it may be that friend-friend corrections
are more common than stranger corrections in face-to-face settings, and thus acceptance of
facts is more common than social media and laboratory studies might lead us to conclude.
However, it may also be the case that intergroup competition motivations deter friend-
friend corrections (or encourage stranger-stranger corrections), such that friends are
reluctant to correct one another, or to even investigate one anothers claims, when they
share group-enhancing myths. Thus testing whether the dynamics we have observed on
Twitter hold in face-to-face conversations is an important area of future research.
The importance of social context in encouraging intuitive science also suggests a
theoretical shift in the study of misinformation. In typical research on misinformation
correction individuals are presented with corrections from neutralsources like research-
ers or professional media such as newspapers. Our results suggest that a key variable
missing in these studies is accountabilitya motivation to have accurate information
about a topic where there is no individual consequence to having incorrect knowledge.
This can be operationalized with friendship relationships, as in this study, or in other ways
where accountability is manipulated (see Lerner & Tetlock, 1999).
More broadly these findings suggest that, as with recent concerns over the selective
exposure to homogeneous political content (Bakshy, Messing, & Adamic, 2015;Bennett&
Iyengar, 2008; Stroud, 2010), the prevalence of political misinformation may be borne of
homophily in social networks. Friendship ties across ideological groups may be able to serve
as a bridge, wherein people accept the validity of criticism even from those with whom they
generally disagree. The fact that, in our results, the dyadic relationship between snoper and
snopee had a much stronger effect than the extent to which they were embedded in a shared
community suggests that relationships that bridge communities can have this effect.
20 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
This study tested, and found evidence for, the idea that social network relationships are
important conditions for determining whether individuals will accept corrections to poli-
tical misinformation using real-world conversations in which individuals corrected one
another. As with similar findings on the role of networks in political discussion, it is hoped
that this work will stimulate further research to better understand when and how mis-
information corrections can be made most effective.
1. Using the Twitter Firehose, access to tweet IDs provided courtesy of Elad Yom-Tov of
Microsoft Research.
2. Twitters public REST API.
The authors would like to thank Elad Yom-Tov for his generosity in supplying tweet IDs,
as well as Bruce Desmarais, Brian Keegan, David Lazer, and four anonymous reviewers
for their helpful comments on this work.
This research was supported in part by a grant from the Cornell Institute for Social
Supplemental Material
Supplemental data for this article can be accessed on the publishers website at: https://doi.
Allport, G. W., & Postman, L. (1947). The psychology of rumor. Oxford, UK: Henry Holt.
Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: an empiricists companion
(Vol. 1). Princeton, NJ.: Princeton university press Princeton.
Asch, S. E. (1951). Effects of group pressure on the modification and distortion of judgments. In
H.Geutzkow (Ed.), Groups, leadership, and men. 222236, Pittsburgh, PA: Carnegie.
Asch, S. E. (1956). Studies of independence and conformities. Psychological Monographs, General
and Applied, 70(9), 170.
Austen-Smith, D., & Feddersen, T. J. (2009). Information aggregation and communication in
committees. Philosophical Transactions of the Royal Society B: Biological Sciences,
364(1518), 763769. doi:10.1098/rstb.2008.0256
Bakshy, E., Messing, S., & Adamic, L. (2015). Exposure to ideologically diverse news and opinion
on Facebook. Science,348(6239), 11301132.
Bennett, W. L., & Iyengar, S. (2008). A new era of minimal effects? The changing foundations of
political communication. Journal of Communication,58(4), 707731. doi:10.1111/
Bessi, A., Coletto, M., Davidescu, G. A., Scala, A., Caldarelli, G., & Quattrociocchi, W. (2015).
Science vs. conspiracy: Collective narratives in the age of misinformation. PLOS ONE,10
(2), e0118093. doi:10.1371/journal.pone.0118093
Political Fact-Checking on Twitter 21
Downloaded by [] at 22:08 05 September 2017
Bimber, B. A. (2003). Information and American democracy: Technology in the evolution of
political power. New York, NY: Cambridge University Press.
Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H.
(2012). A 61-million-person experiment in social influence and political mobilization. Nature,
489(7415), 295298. doi:10.1038/nature11421
Burt, R. S. (2005). Brokerage and closure: An introduction to social capital. Oxford, UK. Oxford
University Press.
Burton-Chellew, M. N., Ross-Gillespie, A., & West, S. A. (2010). Cooperation in humans:
Competition between groups and proximate emotions. Evolution and Human Behavior,31
(2), 104108. doi:10.1016/j.evolhumbehav.2009.07.005
Choi, S. (2014). Flow, diversity, form, and influence of political talk in social-media-based public
forums. Human Communication Research,40(2), 209237. doi:10.1111/hcre.12023
Christakis, N. A., & Fowler, J. H. (2009). Connected: The surprising power of our social networks
and how they shape our lives. New York/Little Brown.
Cohen, T. R., Montoya, R. M., & Insko, C. A. (2006). Group morality and intergroup relations:
Cross-cultural and experimental evidence. Personality and Social Psychology Bulletin,32
(11), 15591572. doi:10.1177/0146167206291673
Coleman, J. S. (1988). Social capital in the creation of human-capital. American Journal of
Sociology,94, S95S120. doi:10.1086/228943
Conover, M. D., Gonçalves, B., Flammini, A., & Menczer, F. (2012). Partisan asymmetries in online
political activity. EPJ Data Science,1(1), 119. doi:10.1140/epjds6
Crease, R. P. (2011). World in the balance: The historic quest for an absolute system of measure-
ment. New York/W.W. Norton.
Eagle, N., Pentland, A. S., & Lazer, D. (2009). Inferring friendship network structure by using
mobile phone data. Proceedings of the National Academy of Sciences,106(36), 1527415278.
Einwiller, S. A., & Kamins, M. A. (2008). Rumor has it: The moderating effect of identification on
rumor impact and the effectiveness of rumor refutation. Journal of Applied Social Psychology,
38(9), 22482272. doi:10.1111/jasp.2008.38.issue-9
Elga, A. (2011). Reflection and disagreement. In A. I. Goldman & D. Whitcomb (Eds.), Social
epistemology: Essential readings (pp. 158182). Oxford, England; New York, NY: Oxford
University Press.
Ellwardt, L., Steglich, C., & Wittek, R. (2012). The co-evolution of gossip and friendship in
workplace social networks. Social Networks,34(4), 623633. doi:10.1016/j.socnet.2012.07.002
Enns, P. K. (2015). Relative policy support and coincidental representation. Perspectives on Politics,
13(04), 10531064. doi:10.1017/S1537592715002315
Eveland, W. P., Morey, A. C., & Hutchens, M. J. (2011). Beyond deliberation: New directions for the
study of informal political conversation from a communication perspective. Journal of
Communication,61(6), 10821103. doi:10.1111/j.1460-2466.2011.01598.x
Friggeri, A., Adamic, L. A., Eckles, D., & Cheng, J. (2014). Rumor cascades. In Proceedings of the
Eighth International AAAI Conference on Weblogs and Social Media (pp. 101110). Retrieved
Garrett, R. K. (2011). Troubling consequences of online political rumoring. Human Communication
Research,37(2), 255274. doi:10.1111/j.1468-2958.2010.01401.x
Garrett, R. K., Nisbet, E. C., & Lynch, E. K. (2013). Undermining the corrective effects of media-
based political fact checking? The role of contextual cues and naïve theory. Journal of
Communication,63(4), 617637. doi:10.1111/jcom.12038
Giglietto, F., Iannelli, L., Rossi, L., & Valeriani, A. (2016). Fakes, news and the election: A new
taxonomy for the study of misleading information within the hybrid media system. Retrieved
González-Bailón, S., Wang, N., & Borge-Holthoefer, J. (2014). The emergence of roles in large-scale
networks of communication. EPJ Data Science,3(1), 116. doi:10.1140/epjds/s13688-014-
22 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
Gottfried, J. A., Hardy, B. W., Winneg, K. M., & Jamieson, K. H. (2013). Did fact checking matter in
the 2012 presidential campaign?American Behavioral Scientist,57(11), 15581567.
Grice, H. P. (1975). Logic and conversation. In P. Cole and J.L. Morgan (Eds.), Syntax and
semantics, Vol. 3: Speech acts (pp. 225242). New York: Seminar Press.
Hannak, A., Margolin, D., Keegan, B., & Weber, I. (2014). Get back! You dont know me like that: The
social mediation of fact checking interventions in Twitter conversations. In Proceedings of the
Eighth International AAAI Conference on Weblogs and Social Media (pp. 187196). Retrieved
Himelboim, I. (2010). Civil society and online political discourse: The network structure of unrest-
ricted discussions. Communication Research,38(5), 634659. doi:10.1177/0093650210384853
Hofer, M., & Aubert, V. (2013). Perceived bridging and bonding social capital on Twitter:
Differentiating between followers and followees. Computers in Human Behavior,29(6),
21342142. doi:10.1016/j.chb.2013.04.038
Kahan, D. M. (2012). Ideology, motivated reasoning, and cognitive reflection: An experimental
study. Judgment and Decision Making,8, 407424.
Katz, E. (1957). The two-step flow of communication: An up-to-date report on an hypothesis. Public
Opinion Quarterly,21(1), 6178. doi:10.1086/266687
Koudenburg, N., Postmes, T., & Gordijn, E. H. (2016). Beyond content of conversation: The role of
conversational form in the emergence and regulation of social structure. Personality and Social
Psychology Review, (21) 5071. doi:10.1177/1088868315626022
Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA:
Lazer, D., Rubineau, B., Chetkovich, C., Katz, N., & Neblo, M. (2010). The coevolution of networks
and political attitudes. Political Communication,27(3), 248274. doi:10.1080/
Lerner, J. S., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological
Bulletin,125(2), 255275. doi:10.1037/0033-2909.125.2.255
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation
and its correction: Continued influence and successful debiasing. Psychological Science in the
Public Interest,13(3), 106131. doi:10.1177/1529100612451018
Lin, Y., Margolin, D., Keegan, B., & Lazer, D. (2013). Voices of Victory: A Computational Focus
Group Framework for Tracking Opinion Shift in Real Time. 22nd International World Wide
Web Conference (pp. 737748). Rio.
Lin, Y.-R., Keegan, B., Margolin, D., & Lazer, D. (2014). Rising tides or rising stars? Dynamics
ofshared attention on Twitter during media events. PLOS ONE,9(5), e94093. doi:10.1371/
Maireder, A., & Schlogl, S. (2014). 24 hours of an #outcry: The networked publics of a socio-
political debate. European Journal of Communication,29(6), 687702. doi:10.1177/
Marwick, A. E., & Boyd, D. (2011). I tweet honestly, I tweet passionately: Twitter users, context
collapse, and the imagined audience. New Media & Society,13(1), 114133. doi:10.1177/
Meeder, B., Karrer, B., Sayedi, A., Ravi, R., Borgs, C., & Chayes, J. (2011). We know who you
followed last summer: Inferring social link creation times in Twitter. In Proceedings of the 20th
International Conference on World Wide Web (pp. 517526). New York, NY: ACM.
Mocanu, D., Rossi, L., Zhang, Q., Karsai, M., & Quattrociocchi, W. (2015). Collective attention in
the age of (mis)information. Computers in Human Behavior,51, 11981204. doi:10.1016/j.
Monge, P., & Contractor, N. (2003). Theories of communication networks. New York, NY: Oxford
University Press.
Political Fact-Checking on Twitter 23
Downloaded by [] at 22:08 05 September 2017
Moody, J., & White, D. R. (2003). Structural cohesion and embeddedness: A hierarchical concept of
social groups. American Sociological Review,68, 103127. doi:10.2307/3088904
Moon, S. (2011). Analysis of Twitter unfollow: How often do people unfollow in Twitter and why?
In International Conference on Social Informatics (p. 7). Springer.
Morey, A. C., Eveland, W. P., & Hutchens, M. J. (2012). The whomatters: Types of interpersonal
relationships and avoidance of political disagreement. Political Communication,29(1), 86103.
Mutz, D. C. (2008). Is deliberative democracy a falsifiable theory? Annual Review of Political
Science,11(1), 521538. doi:10.1146/annurev.polisci.11.081306.070308
Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions.
Political Behavior,32(2), 303330. doi:10.1007/s11109-010-9112-2
Papacharissi, Z., & de Fatima Oliveira, M. (2012). Affective news and networked publics: The
rhythms of news storytelling on #Egypt. Journal of Communication,62(2), 266282.
Reicher, S., Haslam, S. A., & Hopkins, N. (2005). Social identity and the dynamics of leadership:
Leaders and followers as collaborative agents in the transformation of social reality. The
Leadership Quarterly,16(4), 547568. doi:10.1016/j.leaqua.2005.06.007
Shapin, S. (1994). A social history of truth: Civility and science in seventeenth-century England.
Sherif, M., & Hovland, C. I. (1961). Social judgment: Assimilation and contrast effects in commu-
nication and attitude change, University of Chicago.
Shin, J., Driscoll, K., & Bar, F. (2016). Political rumoring on Twitter during the 2012 U.S.
presidential election: Rumor diffusion and correction. New Media & Society,122.
Sommerfeld, R. D., Krambeck, H.-J., & Milinski, M. (2008). Multiple gossip statements and their
effect on reputation and trustworthiness. In Proceedings of the Royal Society B: Biological
Sciences,275(1650), 25292536. doi:10.1098/rspb.2008.0762
Song, H. (2015). Uncovering the structural underpinnings of political discussion networks: Evidence
from an exponential random graph model: Underpinnings of political discussion networks.
Journal of Communication,65(1), 146169. doi:10.1111/jcom.12140
Stroud, N. J. (2010). Polarization and partisan selective exposure. Journal of Communication,60
(3), 556576. doi:10.1111/jcom.2010.60.issue-3
Sunstein, C. R. (2006). Infotopia: How many minds produce knowledge. New York, NY: Oxford
University Press.
Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup conflict. The Social
Psychology of Intergroup Relations,33, 47.
Tetlock, P. E. (2002). Social functionalist frameworks for judgment and choice: Intuitive politicians,
theologians, and prosecutors. Psychological Review,109(3), 451471. doi:10.1037//
Turner, J. C. (1975). Social comparison and social identity: Some prospects for intergroup behaviour.
European Journal of Social Psychology,5(1), 134. doi:10.1002/ejsp.2420050102
Uzzi, B. (1997). Social structure and competition in interfirm networks: The paradox of embedded-
ness. Administrative Science Quarterly,42,3567. doi:10.2307/2393808
Varnelis, K. (2012). Networked publics. MA/MIT Press, Cambridge.
Watts, D. J., & Dodds, P. S. (2007). Influentials, networks, and public opinion formation. Journal of
Consumer Research,34(4), 441458. doi:10.1086/518527
Weeks, B. E. (2015). Emotions, partisanship, and misperceptions: How anger and anxiety moderate
the effect of partisan bias on susceptibility to political misinformation. Journal of
Communication,65(4), 699719. doi:10.1111/jcom.12164
Wilson, E. O. (2012). The social conquest of earth. W.W. Norton & Company. Liveright Pub. Corp,
New York.
Xu, B., Huang, Y., Kwak, H., & Contractor, N. (2013). Structures of broken ties: Exploring unfollow
behavior on Twitter. In Proceedings of the 2013 Conference on Computer Supported
Cooperative Work (pp. 871876). New York, NY: ACM. doi:10.1145/2441776.2441875
24 Drew B. Margolin et al.
Downloaded by [] at 22:08 05 September 2017
... De hecho, desde la academia son muchos los ISSN: 2660-4213 Narrativas digitales contra la desinformación. Verificación de datos y alfabetización en la sociedad... ISBN: 978-84-17600-85-3 Colección: Periodística, 105 estudios que han tratado de determinar si la actividad que realizan los verificadores de datos es verdaderamente efectiva en la lucha contra la desinformación: algunos estudios muestran su utilidad (Chung;Kim, 2021;Hameleers, 2020;Lee et al., 2022;Zhang et al., 2021) pero otros cuestionan su impacto (Margolin et al., 2018;Pérez-Curiel;Velasco-Molpeceres, 2020;Herrero-Damas, 2021). Las propias creencias e ideología de los individuos condicionan la capacidad de los fact-checkers para corregir la desinformación (Walter et al., 2020) y algunas personas comparten intencionadamente contenidos aunque se haya demostrado su falsedad (Ardèvol-Abreu et al., 2020), lo que dificulta considerablemente su tarea. ...
Full-text available
En este capítulo ofrecemos una aproximación descriptiva a la figura de los fact-chekcers o verificadores de datos, definiéndolos como personas o equipos que se encargan de comprobar la veracidad de informaciones publicadas en medios de comunicación o difundidas en Internet, con el objetivo de desmentir las noticias falsas y asegurar la fiabilidad de la información. Profundizamos en su valor como promotores de la verdad en el discurso público y garantes de la transparencia en el marco de los sistemas democráticos y realizamos un breve repaso histórico de su nacimiento, evolución y extensión a nivel mundial. Clasificamos estas organizaciones en función de su modo de financiación, su alcance geográfico y su alcance temático y describimos su proceso de trabajo, basado en la detección, análisis y difusión de informes sobre la veracidad de los contenidos. Por último, nos asomamos a los desafíos que estas entidades deben enfrentar en el ejercicio de su labor, como la falta de fuentes confiables, la limitación de tiempo para abordar cantidades cada vez mayores de desinformación o el desconocimiento y la desconfianza de los usuarios.
... Tetlock (2002) coins the term "intuitive politician" to describe the behavior of risk-averse subjects who seek to preserve their reputation by aligning themselves with socially accepted positions. "People behave like intuitive politicians when they seek to maintain a positive reputation or fulfill the social duties for which they are accountable"(Margolin et al., 2018). ...
Full-text available
work is licensed under a Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives (CC-IGO BY-NC-ND 3.0 IGO) license ( legalcode) and may be reproduced with attribution to the IDB and for any non-commercial purpose, as provided below. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC-IGO license. Abstract Previous research has extensively investigated why users spread misinformation online, while less attention has been given to the motivations behind sharing fact-checks. This paper reports a four-country survey experiment assessing the influence of c onfirmation and refutation frames on engagement with online fact-checks. Respondents randomly received semantically identical content, either affirming accurate information ("It is TRUE that p") or refuting misinformation ("It is FALSE that not p"). Despite semantic equivalence, confirmation frames elicit higher engagement rates than refutation frames. Additionally, confirmation frames reduce self-reported negative emotions related to polarization. These findings are crucial for designing policy interventions aiming to amplify fact-check exposure and reduce affective polarization, particularly in critical areas such as health-related misinformation and harmful speech. JEL classifications: D83, D91
... And people accept information that comes from a perceived trusted source [24]. Furthermore, individuals who are wrong about the information or disseminate untrue details about a particular fact respond more positively to being corrected when they share a relational bond or community connection with the corrector [40]. Those engaged in misinformation correction prefer approaches that rely on relationship-building because those approaches allow them to demonstrate politeness and express emotions [41]. ...
Full-text available
This paper investigates the relationship between scientists' communication experience and attitudes towards misinformation and their intention to correct misinformation. Specifically, the study focuses on two correction strategies: source-based correction and relational approaches. Source-based approaches combatting misinformation prioritize sharing accurate information from trustworthy sources to encourage audiences to trust reliable information over false information. On the other hand, relational approaches give priority to developing relationships or promoting dialogue as a means of addressing misinformation. In this study, we surveyed 416 scientists from U.S. land-grant universities using a self-report questionnaire. We find that scientists' engagement in science communication activities is positively related to their intention to correct misinformation using both strategies. Moreover, the scientists' attitude towards misinformation mediates the relationship between engagement in communication activities and intention to correct misinformation. The study also finds that the deficit model perception-that is, the assumption that scientists only need to transmit scientific knowledge to an ignorant public in order to increase understanding and support for science-moderates the indirect effect of engagement in science communication activities on behavioral intention to correct misinformation using relational strategies through attitude towards misinformation. Thus, the deficit model perception is a barrier to engaging in relational strategies to correct misinformation. We suggest that addressing the deficit model perception and providing science communication training that promotes inclusive worldviews and relational approaches would increase scientists' behavioral intentions to address misinformation. The study concludes that scientists should recognize their dual positionality as scientists and members of their community and engage in respectful conversations with community members about science.
... The perceived expertise and trustworthiness of the source affect how likely someone is to accept the correction (Benegal & Scruggs, 2018;Ecker & Antonio, 2021;Guillory & Geraci, 2013;Vraga & Bode, 2017). Furthermore, Margolin et al. (2017) found that Twitter users are significantly more likely to accept a correction by an account that they follow than a correction by a stranger, indicating that strong connections between fact-checkers and misinformation spreaders are key for the effectiveness of debunking. ...
Full-text available
Developing effective interventions to counter misinformation is an urgent goal, but it also presents conceptual, empirical, and practical difficulties, compounded by the fact that misinformation research is in its infancy. This paper provides researchers and policymakers with an overview of which individual-level interventions are likely to influence the spread of, susceptibility to, or impact of misinformation. We review the evidence for the effectiveness of four categories of interventions: boosting (psychological inoculation, critical thinking, and media and information literacy); nudging (accuracy primes and social norms nudges); debunking (fact-checking); and automated content labeling. In each area, we assess the empirical evidence, key gaps in knowledge, and practical considerations. We conclude with a series of recommendations for policymakers and tech companies to ensure a comprehensive approach to tackling misinformation.
... We note that additional misinformation research is of course conducted on real-world platforms (e.g., Grinberg et al., 2019;Margolin et al., 2018;Mosleh et al., 2021;Shao et al., 2018). However, in addition to potential ethical and legal concerns, this research requires specific data-science skills and comes with issues associated with causal inference and experimental control (e.g., see Aral & Eckles, 2019; E. E. Chen & Wojcik, 2016;K. ...
Full-text available
Given the potential negative impact reliance on misinformation can have, substantial effort has gone into understanding the factors that influence misinformation belief and propagation. However, despite the rise of social media often being cited as a fundamental driver of misinformation exposure and false beliefs, how people process misinformation on social media platforms has been under-investigated. This is partially due to a lack of adaptable and ecologically valid social media testing paradigms, resulting in an over-reliance on survey software and questionnaire-based measures. To provide researchers with a flexible tool to investigate the processing and sharing of misinformation on social media, this paper presents The Misinformation Game—an easily adaptable, open-source online testing platform that simulates key characteristics of social media. Researchers can customize posts (e.g., headlines, images), source information (e.g., handles, avatars, credibility), and engagement information (e.g., a post’s number of likes and dislikes). The platform allows a range of response options for participants (like, share, dislike, flag) and supports comments. The simulator can also present posts on individual pages or in a scrollable feed, and can provide customized dynamic feedback to participants via changes to their follower count and credibility score, based on how they interact with each post. Notably, no specific programming skills are required to create studies using the simulator. Here, we outline the key features of the simulator and provide a non-technical guide for use by researchers. We also present results from two validation studies. All the source code and instructions are freely available online at .
... One plausible explanation for this finding lies in the social dynamics of Facebook. Instant messengers, such as WhatsApp and Facebook Messenger, are predominantly utilized for casual and personal communication (Gil de Zúñiga et al., 2021), whereas Twitter connects individuals not only with their strong ties but also with strangers (Margolin, Hannak, & Weber, 2018). In contrast, Facebook is a semi-public venue that enables users, including healthcare providers, to interact with their intimate social spheres (e.g., family members and close friends) and their known networks with whom they have less frequent communication (e.g., acquaintances and colleagues) (Miller, 2012;Valeriani & Vaccari, 2016). ...
... Moreover, researchers working alone and in groups have attempted various approaches to overcome this problem. To decrease or eliminate exposure to such material, they are improving people's awareness of potential news and modifying the structures [50,51]. Fact-checking is destructive since acquaintance with news articles or rumors fosters approval rather than disapproval [52]. ...
Full-text available
Social media, fake news, and different propaganda strategies have all contributed to an increase in misinformation online during the past ten years. As a result of the scarcity of high-quality data, the present datasets cannot be used to train a deep-learning model, making it impossible to establish an identification. We used a natural language processing approach to the issue in order to create a system that uses deep learning to automatically identify propaganda in news items. To assist the scholarly community in identifying propaganda in text news, this study suggested the propaganda texts (ProText) library. Truthfulness labels are assigned to ProText repositories after being manually and automatically verified with fact-checking methods. Additionally, this study proposed using a fine-tuned Robustly Optimized BERT Pre-training Approach (RoBERTa) and word embedding using multi-label multi-class text classification. Through experimentation and comparative research analysis, we address critical issues and collaborate to discover answers. We achieved an evaluation performance accuracy of 90%, 75%, 68%, and 65% on ProText, PTC, TSHP-17, and Qprop, respectively. The big-data method, particularly with deep-learning models, can assist us in filling out unsatisfactory big data in a novel text classification strategy. We urge collaboration to inspire researchers to acquire, exchange datasets, and develop a standard aimed at organizing, labeling, and fact-checking.
Full-text available
Social networks –and Facebook in particular– have become an important element of the information diet for millions of people around the world. By using them, the traditional media lose control of the distribution channel for their content, whose reach now depends, firstly, on the relevance criteria established by the networks themselves and, secondly, on the interactions generated by the audience with each publication. Very often research on journalism has tackled the issue of reach and how efficient fact-checkers are. To find answer to the sociodemographic features of their audiences or the characteristics of their posts are explored. However, factors such as the influence of the algorithms which choose the content users are shown on the social networks is not often dealt with. This article aims to contribute in both areas. Firstly, it offers a broad perspective on the publications of Ibero-American fact-checkers on Facebook between 2016 and 2021, focuses on the evolution of video production (n=9075) and on the views and engagement achieved by this format with respect to the rest, and relates them to changes in the News Feed algorithm. Secondly, it proposes a content analysis to identify formal and thematic elements in the most popular videos in the same period (n=414) and relates them to previous research. Our results show significant similarities in popular videos, but also changes in video production, a generalized decrease in the ratio of views and a drop in the interaction rate more accentuated than in all the publications of the period. Although the focus of this research does not allow us to make direct causal inferences, the trends identified coincide with the changes in the Facebook News Feed algorithm that were made public in those years.
Full-text available
According to recent studies, most of the Spanish population identifies disinformation as a social problem and believes that it could endanger democracy and the stability of the country. In this context, many institutions point out the need for media literacy campaigns and initiatives that alleviate the possible harmful social effects of the phenomenon, especially among vulnerable audiences. While children and young people are the continuous target of this type of action, few so far have targeted the elderly. This article analyzes the effectiveness of a training action to increase the ability to detect false news in this age group. A 10-day course was designed, and a sample of 1,029 individuals over 50 years of age residing in Spain who are smartphone users was selected. Participants were divided into an experimental group (n=498), who were invited to take the course, and a control group (n=531). An ex ante and ex post study was carried out to determine the effects of the course on their ability to detect false news. The results reveal that those who took the course were more successful in identifying the news as true or false than the members of the control group. The results confirm the opportunity and convenience of designing media literacy actions aimed at those over 50 years of age, a social group particularly exposed to disinformation.
Full-text available
Professional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact, regarding the false and misleading statements of then President Donald J. Trump. Differences in statement selection and deceptiveness scaling were investigated. The Washington Post checked PolitiFact fact-checks 77.4% of the time (22.6% selection disagreement). Moderate agreement was observed for deceptiveness scaling. Nearly complete agreement was observed for bottom-line attributed veracity. Additional cross-checking with other sources (Snopes,, original sources, and with fact-checking for the first 100 days of President Joe Biden's administration were inconsistent with potential ideology effects. Our evidence suggests fact-checking is a difficult enterprise, there is considerable variability between fact-checkers in the raw number of statements that are checked, and finally, selection and scaling account for apparent discrepancies among fact-checkers.
Full-text available
Social media can be a double-edged sword for political misinformation, either a conduit propagating false rumors through a large population or an effective tool to challenge misinformation. To understand this phenomenon, we tracked a comprehensive collection of political rumors on Twitter during the 2012 US presidential election campaign, analyzing a large set of rumor tweets (n = 330,538). We found that Twitter helped rumor spreaders circulate false information within homophilous follower networks, but seldom functioned as a self-correcting marketplace of ideas. Rumor spreaders formed strong partisan structures in which core groups of users selectively transmitted negative rumors about opposing candidates. Yet, rumor rejecters neither formed a sizable community nor exhibited a partisan structure. While in general rumors resisted debunking by professional fact-checking sites (e.g. Snopes), this was less true of rumors originating with satirical sources.
Full-text available
Social interaction is pivotal to the formation of social relationships and groups. Much is known about the importance of interaction content (e.g., the transfer of information). The present review concentrates on the influence of the act of conversing on the emergence of a sense of solidarity, more or less independently of the content. Micro-characteristics of the conversation (e.g., brief silences, smooth turn-taking) can profoundly influence the emergence and the regulation of relationships and of solidarity. We suggest that this might be because the form of a conversation is experienced as an expression of the social structures within the group. Because of its dynamic nature, moreover, the form of conversation provides group members with a continuous gauge of the group?s structural features (e.g., its hierarchy, social norms, and shared reality). Therefore, minor changes in the form and flow of group conversation can have considerable consequences for the regulation of social structure.
This 2003 book assesses the consequences of new information technologies for American democracy in a way that is theoretical and also historically grounded. The author argues that new technologies have produced the fourth in a series of 'information revolutions' in the US, stretching back to the founding. Each of these, he argues, led to important structural changes in politics. After re-interpreting historical American political development from the perspective of evolving characteristics of information and political communications, the author evaluates effects of the Internet and related new media. The analysis shows that the use of new technologies is contributing to 'post-bureaucratic' political organization and fundamental changes in the structure of political interests. The author's conclusions tie together scholarship on parties, interest groups, bureaucracy, collective action, and political behavior with new theory and evidence about politics in the information age.
Conference Paper
The widely unexpected outcome of the 2016 US Presidential election prompted a broad debate on the role played by “fake-news” circulating on social media during political campaigns. Despite a relatively vast amount of existing literature on the topic, a general lack of conceptual coherence and a rapidly changing news eco-system hinder the development of effective strategies to tackle the issue. Leveraging on four strands of research in the existing scholarship, the paper introduces a radically new model aimed at describing the process through which misleading information spreads within the hybrid media system in the post-truth era. The application of the model results in four different typologies of propagations. These typologies are used to describe real cases of misleading information from the 2016 US Presidential election. The paper discusses the contribution and implication of the model in tackling the issue of misleading information on a theoretical, empirical, and practical level.