No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Russian Troll Hunting in a Brexit Twitter Archive
Abstract
Twitter has identified 2,752 accounts that it believes are linked to the Internet Research Agency (IRA), a Russian company that creates online propaganda. These accounts are known to have tweeted about the US 2016 Elections and the list was submitted as evidence by Twitter to the United States Senate Judiciary Subcommittee on Crime and Terrorism. There is no equivalent officially published list of accounts from the IRA known to be active in the UK-EU Referendum debate (Brexit), but we found that the troll accounts active on the 2016 US Election also produced content related to Brexit. We found 3,485 tweets from 419 of the accounts listed as IRA accounts which specifically discussed Brexit and related topics such as the EU and migration. We have been collating an archive of tweets related to Brexit since August 2015 and currently have over 70 million tweets. The Brexit referendum took place on the 23rd June 2016 and the UK voted to leave the European Union. We gathered the data using the Twitter API and a selection of hashtags chosen by a panel of academic experts. Currently we have in excess of fifty different hashtags and we add to the set periodically to accurately represent the evolving conversation. Twitter has closed the accounts that were documented in the Senate list meaning that these tweets are no longer available through the webpage or API. Due to Twitter's terms of service we are unable to share specific tweet text or user profile information but our findings, utilising text and metadata from derived and aggregated data, allows us to provide important insights into the behaviour of these trolls.
... During the US elections in 2016, state-sponsored trolls infused both pro-Trump (Russian) and anti-Trump (Iranian) fabricated and inflammatory content [66]. In a similar way, content with an anti-EU sentiment was disseminated during the Brexit campaign [42]. The troll accounts and social bots were found to interfere with the #BlackLivesMatter movement [56] and the gun control debate on Twitter [35]. ...
... The sample was skewed towards the pro-vaccine: 260 or 82.5% participants indicated they were pro-vaccine, 32 or 10.2% were anti-vaccine, and 23 or 7.3% were ambivalent. Six age brackets in the sample were distributed as: 23.5% [18][19][20][21][22], 22.2% [23][24][25][26][27], 17.8% [28][29][30][31][32], 8.3% [33][34][35][36][37], 4.7% [38][39][40][41][42], and 23.5% [43 or more]. The sample was representative with 175 or 55.6% identified as female, 115 or 36.5% as male, and 25 or 7.9% as non-cis individuals (transgender male, transgender female, gender variant/non-conforming, not listed, or preferred not to answer). ...
... brackets, although with an opposite levels of confidence. The least confident pro-vaccine participants were in the [38][39][40][41][42] (score = 4.8) and [42 and older] (score = 39.15) brackets. ...
Trolling and social bots have been proven as powerful tactics for manipulating the public opinion and sowing discord among Twitter users. This effort requires substantial content fabrication and account coordination to evade Twitter's detection of nefarious platform use. In this paper we explore an alternative tactic for covert social media interference by inducing misperceptions about genuine, non-trolling content from verified users. This tactic uses a malware that covertly manipulates targeted words, hashtags, and Twitter metrics before the genuine content is presented to a targeted user in a covert man-in-the-middle fashion. Early tests of the malware found that it is capable of achieving a similar goal as trolls and social bots, that is, silencing or provoking social media users to express their opinion in polarized debates on social media. Following this, we conducted experimental tests in controlled settings (N=315) where the malware covertly manipulated the perception in a Twitter debate on the risk of vaccines causing autism. The empirical results demonstrate that inducing misperception is an effective tactic to silence users on Twitter when debating polarizing issues like vaccines. We used the findings to propose a solution for countering the effect of the malware-induced misperception that could also be used against trolls and social bots on Twitter.
... During the US elections in 2016, state-sponsored trolls infused both pro-Trump (Russian) and anti-Trump (Iranian) fabricated and inflammatory content [66]. In a similar way, content with an anti-EU sentiment was disseminated during the Brexit campaign [42]. The troll accounts and social bots were found to interfere with the #BlackLivesMatter movement [56] and the gun control debate on Twitter [35]. ...
... brackets, although with an opposite levels of confidence. The least confident pro-vaccine participants were in the [38][39][40][41][42] (score = 4.8) and [42 and older] (score = 39.15) brackets. ...
... brackets. The [42 and older] however were the most authentic ones in their The [38][39][40][41][42] showed the most positive tone (score = 92.40) while the [33][34][35][36][37] showed the most negative tone in their responses (score = 8.95). ...
Trolling and social bots have been proven as powerful tactics for manipulating the public opinion and sowing discord among Twitter users. This effort requires substantial content fabrication and account coordination to evade Twitter's detection of nefarious platform use. In this paper we explore an alternative tactic for covert social media interference by inducing misperceptions about genuine, non-trolling content from verified users. This tactic uses a malware that covertly manipulates targeted words, hashtags, and Twitter metrics before the genuine content is presented to a targeted user in a covert man-in-the-middle fashion. Early tests of the malware found that it is capable of achieving a similar goal as trolls and social bots, that is, silencing or provoking social media users to express their opinion in polarized debates on social media. Following this, we conducted experimental tests in controlled settings (N=315) where the malware covertly manipulated the perception in a Twitter debate on the risk of vaccines causing autism. The empirical results demonstrate that inducing misperception is an effective tactic to silence users on Twitter when debating polarizing issues like vaccines. We used the findings to propose a solution for countering the effect of the malware-induced misperception that could also be used against trolls and social bots on Twitter.
... Another notable aspect of this case is that the Kremlin and its supporters are known to have frequently promoted polarizing topics (Doroshenko and Lukito, 2021;Karlsen, 2019), including those related to refugees and immigrants (Llewellyn, et al., 2018;Suk, et al., 2022), as part of their larger efforts to destabilize Western democracies by exacerbating existing anti-refugee sentiments and isolationist attitudes. For example, in the wake of Russia's full-scale invasion of Ukraine in 2022, pro-Kremlin social media accounts pushed unsubstantiated claims that Ukrainian refugees were taking advantage of welfare services in countries like Germany (Alieva, et al., 2022;Morris and Oremus, 2022). ...
... The Kremlin's IO is also known to rely on anti-immigrant discourse (Llewellyn, et al., 2018), appealing to right-wing users who may amplify messages echoing populist and isolationist views concerning Western support of Ukraine. Twelve percent of posts in our dataset feed into similar narratives characterizing Ukrainian refugees as being parasitic to host countries. ...
Russia’s war of aggression in Ukraine has triggered Europe’s largest refugee crisis since World War II. In this case study, we investigate the prevalence and types of anti-refugee discourse about Ukrainian refugees on Twitter. Previous studies primarily focused on public discourse and attitudes toward racialized refugees and immigrants; the Ukrainian refugee crisis is unique in that it is one of the few instances of a recent refugee crisis involving people who do not come from mostly racialized communities. Using Communalytic, a computational social science tool for studying public discourse on social media, we automatically collected and identified toxic posts mentioning Ukrainian refugees during the first year of Russia’s full-scale invasion of Ukraine. We focused on posts containing toxic language, as this is where we are most likely to find examples of anti-refugee sentiments. Based on a manual analysis of 2,045 toxic posts referencing Ukrainian refugees, the most prevalent ones were politically motivated and included partisan content (33 percent), followed by posts containing expressions countering anti-refugee narratives (20 percent). These findings highlight the escalating politicization and polarization of discussions about Ukrainian refugees both online and offline. Furthermore, 53 percent of the sample aligned with pro-Kremlin narratives against Ukraine. By exploiting anti-refugee sentiments and leveraging existing political and cultural fault lines in the West, pro-Kremlin messages on Twitter contribute to diminishing support for Ukrainian refugees, minimizing the severity of the war, and undermining international support for Ukraine.
... However, the largest and most studied IO was carried out by the Internet Research Agency (IRA), a Russian company operating in the information sector and involved in multiple information manipulation campaigns [21], [22]. In 2016, the IRA exploited thousands of fake humanoperated accounts to influence political events in the US [6] and other countries [23]. Leveraging longitudinal Twitter/X data, [24] qualitatively studied the evolution of the activities and behavior of the IRA accounts, by means of temporal userhashtag graphs [24]. ...
... In turn, better knowledge about the coordinated groups of accounts involved in an IO might inform future strategies for promptly detecting unfolding IOs, which still represents an open problem [20]. Moreover, out of all the IOs detected and shared by Twitter/X, the vast majority of works analyzed the one related to the activities of the IRA [2], [6], [23]- [25], [37]- [39]. Other IOs that received some scholarly attention are those perpetrated by Egypt [20], [39], China [31], [39], and Iran [40]. ...
Online information operations (IOs) refer to organized attempts to tamper with the regular flow of information and to influence public opinion. Coordinated online behavior is a tactic frequently used by IO perpetrators to boost the spread and outreach of their messages. However, the exploitation of coordinated behavior within large-scale IOs is still largely unexplored. Here, we build a novel dataset comprising around 624K users and4M tweets to study howonline coordinationwas used in two recent IOs carried out on Twitter. We investigate the interplay between coordinated behavior and IOs with state-of-the-art network science and coordination detection methods, providing evidence that the perpetrators of both IOs were indeed strongly coordinated. Furthermore, we propose quantitative indicators and analyses to study the different patterns of coordination, uncovering a malicious group of users that managed to hold a central position in the discussion network, and others who remained at the periphery of the network, with limited interactions with genuine users. The nuanced results enabled by our analysis provide insights into the strategies, development, and effectiveness of the IOs. Overall, our results demonstrate that the analysis of coordinated behavior in IOs can contribute to safeguarding the integrity of online platforms.
... truth and facts do not dwindle in supply as more people "consume" them and truth and facts are available to all people in a society) [31]. Misinformation also runs counter the (ii) authority should be mistrusted and decentralization promoted postulate because it is promulgated by a state-sponsored "shadow authority," as evidence confirms in the aftermath of the Brexit and the 2016 US elections [48,73,134]. Surprisingly, the hacktivists never struck back [11], though they clearly poses the capabilities to do so, as witnessed in the Anonymous's #OpI-SIS campaign, for instance, where the collective flagged about 101,000 Twitter accounts attributed to the Islamic-State [49]. ...
... But the "appropriators" -privy of prior campaigns of disinformation and with the support of nation-state governments [113] -need not to look further as "sock puppet" accounts were already utilized for spreading political falsehoods (e.g., Martha Coackey's "twitter bomb" disinformation campaign [85]). Having all the ingredients for exploiting the virality of social media and users' familiarity with emotionally-charged discourse, the "appropriators" established troll farms in the wake of the UK's Brexit campaign and 2016 US elections [73,135]. ...
In this study, we interviewed 22 prominent hacktivists to learn their take on the increased proliferation of misinformation on social media. We found that none of them welcomes the nefarious appropriation of trolling and memes for the purpose of political (counter)argumentation and dissemination of propaganda. True to the original hacker ethos, misinformation is seen as a threat to the democratic vision of the Internet, and as such, it must be confronted on the face with tried hacktivists' methods like deplatforming the "misinformers" and doxing or leaking data about their funding and recruitment. The majority of the hacktivists also recommended interventions for raising misinformation literacy in addition to targeted hacking campaigns. We discuss the implications of these findings relative to the emergent recasting of hacktivism in defense of a constructive and factual social media discourse.
... ns.html). The most famous of these operations is the Internet Research Agency (IRA), also known as a Russian "Troll Farm," The IRA has engaged in online political tactics to sew intergroup conflict and influence US citizens during the 2016 presidential election (34) and British citizens prior to the Brexit vote (35). Similarly, other state-affiliated influence operations have been found in numerous countries, including Iran, Bangladesh, Venezuela, China, Saudi Arabia, Ecuador, the United Arab Emirates, Spain, and Egypt (https://transparency.twitter.com/en/information-operatio ...
... Russian trolls, or anonymous social media accounts that are affiliated with the Russian government, were active around highly contentious political topics around the world, including in the United States and Britain (34,35). With the release of the Twitter Transparency Report, a sample of the Russian and other countries' operations were officially disclosed and used to study the role of trolls in amplifying political polarization (37,38,55). ...
The affective animosity between the political left and right has grown steadily in many countries over the past few years, posing a threat to democratic practices and public health. There is a rising concern over the role that “bad actors” or trolls may play in the polarization of online networks. In this research, we examined the processes by which trolls may sow intergroup conflict through polarized rhetoric. We developed a dictionary to assess online polarization by measuring language associated with communications that display partisan bias in their diffusion. We validated the polarized language dictionary in four different contexts and across multiple time periods. The polarization dictionary made out-of-set predictions, generalized to both new political contexts (#BlackLivesMatter) and a different social media platform (Reddit), and predicted partisan differences in public opinion polls about COVID-19. Then we analyzed tweets from a known Russian troll source (N = 383,510) and found that their use of polarized language has increased over time. We also compared troll tweets from three countries (N = 798,33) and found that they all utilize more polarized language than regular Americans (N = 1,507,300) and trolls have increased their use of polarized rhetoric over time. We also find that polarized language is associated with greater engagement, but this association only holds for politically engaged users (both trolls and regular users). This research clarifies how trolls leverage polarized language and provides an open-source, simple tool for exploration of polarized communications on social media.
... Usually, moderators review suspected trolling activity to ban/mute trolling users and flagging/deleting trolling content, but this kind of manual solution has some major drawbacks, including a delay of actions, subjectivity of judgment, and scalability [12,26]. The need for automated trolling detection on Twitter thus drew the attention of the research community yielding various detection approaches [14,17,24,43]. ...
... Analyzing the state-sponsor trolling linked to the Russian troll farm Internet Research Agency (IRA), a study found that trolls create a small portion of original trolling content (e.g. posts, hashtags, memes, etc.) and heavily engage in retweeting around a certain point in time of interest (e.g. the Brexit referendum) [24]. A detailed investigation into the trolling activity around the 2016 U.S. elections reveals different state-sponsored trolling with varying tactics: IRA trolls were pro-Trump while Iranian trolls were anti-Trump [43]. ...
This paper presents TrollHunter2020, a real-time detection mechanism we used to hunt for trolling narratives on Twitter during and in the aftermath of the 2020 U.S. elections. Trolling narratives form on Twitter as alternative explanations of polarizing events with the goal of conducting information operations or provoking emotional responses. Detecting trolling narratives thus is an imperative step to preserve constructive discourse on Twitter and remove the influx of misinformation. Using existing techniques, the detection of such content takes time and a wealth of data, which, in a rapidly changing election cycle with high stakes, might not be available. To overcome this limitation, we developed TrollHunter2020 to hunt for trolls in real-time with several dozen trending Twitter topics and hashtags corresponding to the candidates' debates, the election night, and the election aftermath. TrollHunter2020 utilizes a correspondence analysis to detect meaningful relationships between the top nouns and verbs used in constructing trolling narratives while they emerge on Twitter. Our results suggest that the TrollHunter2020 indeed captures the emerging trolling narratives in a very early stage of an unfolding polarizing event. We discuss the utility of TrollHunter2020 for early detection of information operations or trolling and the implications of its use in supporting a constrictive discourse on the platform around polarizing topics.
... For some time, trolling was antisocial behaviour characteristic of the gaming communities and fringe discussion forums like "4chan" [42,63]. Trolling quickly spread on social media where the trolls were not simply "amusing themselves by upsetting other users," but intentionally disseminating disinformation as part of a state-sponsored effort to manipulate public opinion about political candidates, public health issues like vaccination and social justice issues [6,46,73,85]. The definition of trolling extends to encompass "users who exhibit a clear intent to deceive or create conflict with the goal to manipulate the public opinion on a polarised topic and cause distrust in the socio-political system" [1]. ...
... Analysing the state-sponsor trolling linked to the Russian troll farm Internet Research Agency (IRA), one study found that trolls create a small portion of an original trolling content (e.g. posts, hashtags, memes, etc) and heavily engage in retweeting around a certain point in time of interest (e.g. the Brexit referendum) [46]. An investigation into the trolling activity around the 2016 US elections reveals different state-sponsored strategies: IRA trolls were pro-Trump while Iranian trolls were anti-Trump. ...
... For some time, trolling was antisocial behaviour characteristic of the gaming communities and fringe discussion forums like "4chan" [42,63]. Trolling quickly spread on social media where the trolls were not simply "amusing themselves by upsetting other users," but intentionally disseminating disinformation as part of a state-sponsored effort to manipulate public opinion about political candidates, public health issues like vaccination and social justice issues [6,46,73,85]. The definition of trolling extends to encompass "users who exhibit a clear intent to deceive or create conflict with the goal to manipulate the public opinion on a polarised topic and cause distrust in the socio-political system" [1]. ...
... Analysing the state-sponsor trolling linked to the Russian troll farm Internet Research Agency (IRA), one study found that trolls create a small portion of an original trolling content (e.g. posts, hashtags, memes, etc) and heavily engage in retweeting around a certain point in time of interest (e.g. the Brexit referendum) [46]. An investigation into the trolling activity around the 2016 US elections reveals different state-sponsored strategies: IRA trolls were pro-Trump while Iranian trolls were anti-Trump. ...
This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% precision and 69.8% recall over a dataset of 1.3 million tweets. Without a final resolution of the pandemic in sight, it is unlikely that the trolls will go away, although they might be forced to evade automated hunting. To explore the plausibility of this strategy, we developed and tested an adversarial machine learning mechanism called TrollHunter-Evader. TrollHunter-Evader employs a Test Time Evasion (TTE) approach in a combination with a Markov chain-based mechanism to recycle originally trolling tweets. The recycled tweets were able to achieve a remarkable 40% decrease in the TrollHunter's ability to correctly identify trolling tweets. Because the COVID-19 infodemic could have a harmful impact on the COVID-19 pandemic, we provide an elaborate discussion about the implications of employing adversarial machine learning to evade Twitter troll hunts.
... The most famous of these operations is the Internet Research Agency (IRA), also known as a Russian 'Troll Farm', The IRA has engaged in online political debates to influence U.S. citizens during the 2016 presidential election (Badawy et al., 2018) and British citizens prior to the Brexit vote (Llewellyn et al., 2018). However, other state-backed influence operations have been found in numerous countries, including Iran, Bangladesh, Venezuela, China, Saudi Arabia, Ecuador, the United Arab Emirates, Spain, and Egypt. ...
... Russian trolls, or anonymous social media accounts that are backed by the Russian government, were active around highly contentious political topics both in the United States and 10 TROLL AND DIVIDE Britain (Badawy et al., 2018;Llewellyn et al., 2018). With the release of the Twitter Transparency Report, a sample of the Russian and other countries' operations were officially disclosed and used to study the role of trolls in amplifying political polarization (Arif et al., 2018;Broniatowski et al., 2018;Walter et al., 2020). ...
Political polarization, or the ideological distance between the political left and right, has grown steadily in recent decades. There is a rising concern over the role that ‘bad actors’ or trolls may play in polarization in online networks. In this research, we examine the processes by which trolls may sow intergroup conflict through polarizing rhetoric. We developed a dictionary to gauge online polarization by measuring language associated with communications that display partisan bias in their diffusion. We validated the polarized language dictionary in three different contexts and across multiple time periods. We found the polarization dictionary made out-of-set predictions, generalized to new political contexts (#BlackLivesMatter), and predicted partisan differences in public polls about COVID-19. Then we analyzed 383,510 tweets from a known Russian troll source (the Internet Research Agency) and found that their use of polarized language has increased over time. We also compared troll tweets from 3 different countries (N = 798,33) and found that they all utilize more polarized language on average than a control dataset of tweets from regular Americans (N = 1,507,300) and trolls have dramatically increased their use of polarized rhetoric over time. These results illustrate how trolls leverage polarized language. We also provide an open-source, simple tool for exploration of polarized communications on social media.
... The against misinformation effort, then, turned to uncover the misinformation producers, be that individual "fake news" mercenaries [85,106] or state-sponsored factories, farms, or outfits [11,18,54]. The operational effect of misinformation was extended to encompass both public opinion manipulation but also abusing and antagonizing ordinary users [14]. ...
... The author of the [31], which analyzes the antecedents of personal trolling behavior, discovered that the use of both the debate and mood context can describe the behavior of trolling better than a person's trolling history. According to a trolling study [15], which links to the Russian troll farm Internet Research Agency (IRA) and is sponsored by the state, a minor part of original trolling content for example hashtags, posts, memes, etc. is generated by trolls and for about a specific time of interest such as the Brexit referendum, they deeply involve in re-tweeting about a specific point in time of interest. Dissimilar statesponsored approaches are found in a study into the activity of trolling during the US election in 2016. ...
In recent years, many troll accounts have emerged to manipulate social media opinion. Detecting and eradicating trolling is a critical issue for social-networking platforms because businesses, abusers, and nation-state-sponsored troll farms use false and automated accounts. NLP techniques are used to extract data from social networking text, such as Twitter tweets. In many text processing applications, word embedding representation methods, such as BERT, have performed better than prior NLP techniques, offering novel breaks to precisely comprehend and categorize social-networking information for various tasks. This paper implements and compares nine deep learning-based troll tweet detection architectures, with three models for each BERT, ELMo, and GloVe word embedding model. Precision, recall, F1 score, AUC, and classification accuracy are used to evaluate each architecture. From the experimental results, most architectures using BERT models improved troll tweet detection. A customized ELMo-based architecture with a GRU classifier has the highest AUC for detecting troll messages. The proposed architectures can be used by various social-based systems to detect troll messages in the future.
... Quantitative analysis of texts can be very useful for understanding the strategies used in online information warfare. In the past few years, a number of studies investigated characteristics of Twitter accounts created to sow discord and polarization before, during, and after the 2016 presidential election in the USA (Ghanem et al., 2019;Llewellyn et al., 2018;Zannettou et al., 2019a;Zannettou et al., 2019b). A common denominator in these studies has been the use of latent Dirichlet allocation (LDA; Blei et al., 2003) to estimate latent topics from posts on social media platforms. ...
The past few years were marked by increased online offensive strategies perpetrated by state and non-state actors to promote their political agenda, sow discord, and question the legitimacy of democratic institutions in the US and Western Europe. In 2016, the US congress identified a list of Russian state-sponsored Twitter accounts that were used to try to divide voters on a wide range of issues. Previous research used latent Dirichlet allocation (LDA) to estimate latent topics in data extracted from these accounts. However, LDA has characteristics that may limit the effectiveness of its use on data from social media: The number of latent topics must be specified by the user, interpretability of the topics can be difficult to achieve, and it does not model short-term temporal dynamics. In the current paper, we propose a new method to estimate latent topics in texts from social media termed Dynamic Exploratory Graph Analysis (DynEGA). In a Monte Carlo simulation, we compared the ability of DynEGA and LDA to estimate the number of simulated latent topics. The results show that DynEGA is substantially more accurate than several different LDA algorithms when estimating the number of simulated topics. In an applied example, we performed DynEGA on a large dataset with Twitter posts from state-sponsored right- and left-wing trolls during the 2016 US presidential election. DynEGA revealed topics that were pertinent to several consequential events in the election cycle, demonstrating the coordinated effort of trolls capitalizing on current events in the USA. This example demonstrates the potential power of our approach for revealing temporally relevant information from qualitative text data.
... Analyzing the state-sponsor trolling linked to the Russian troll farm Internet Research Agency (IRA), a study found that trolls create a small portion of original trolling content (e.g. posts, hashtags, memes, etc.) and heavily engage in retweeting around a certain point in time of interest (e.g. the Brexit referendum) [18]. A detailed investigation into the trolling activity around the 2016 U.S. elections reveals different statesponsored trolling with varying tactics: IRA trolls were pro-Trump while Iranian trolls were anti-Trump [17]. ...
This paper presents TrollHunter2020, a real-time detection mechanism we used to hunt for trolling narratives on Twitter during the 2020 U.S. elections. Trolling narratives form on Twitter as alternative explanations of polarizing events like the 2020 U.S. elections with the goal to conduct information operations or provoke emotional response. Detecting trolling narratives thus is an imperative step to preserve constructive discourse on Twitter and remove an influx of misinformation. Using existing techniques, this takes time and a wealth of data, which, in a rapidly changing election cycle with high stakes, might not be available. To overcome this limitation, we developed TrollHunter2020 to hunt for trolls in real-time with several dozens of trending Twitter topics and hashtags corresponding to the candidates' debates, the election night, and the election aftermath. TrollHunter2020 collects trending data and utilizes a correspondence analysis to detect meaningful relationships between the top nouns and verbs used in constructing trolling narratives while they emerge on Twitter. Our results suggest that the TrollHunter2020 indeed captures the emerging trolling narratives in a very early stage of an unfolding polarizing event. We discuss the utility of TrollHunter2020 for early detection of information operations or trolling and the implications of its use in supporting a constrictive discourse on the platform around polarizing topics.
... In addition to the 2016 US presidential election, researchers discovered other political events that accompanied by computational propaganda managed by social sponsored-trolls, these include the 2016 UK Brexit referendum [18], the 2017 French presidential election [19], and the Gulf crisis [20]. Generally, state-sponsored trolling has become a global phenomenon over the last four years. ...
In recent years, there has been an increased prevalence of adopting state-sponsored trolls by governments and political organizations to influence public opinion through disinformation campaigns on social media platforms. This phenomenon negatively affects the political process, causes distrust in the political systems, sows discord within societies, and hastens political polarization. Thus, there is a need to develop automated approaches to identify sponsored-troll accounts on social media in order to mitigate their impacts on the political process and to protect people against opinion manipulation. In this paper, we argue that behaviors of sponsored-troll accounts on social media are different from ordinary users' because of their extrinsic motivation, and they cannot completely hide their suspicious behaviors, therefore these accounts can be identified using machine learning approaches based solely on their behaviors on the social media platforms. We have proposed a set of behavioral features of users' activities on Twitter. Based on these features, we developed four classification models to identify political troll accounts, these models are based on decision tree, random forest, Adaboost, and gradient boost algorithms. The models were trained and evaluated on a set of Saudi trolls disclosed by Twitter in 2019, the overall classification accuracy reaches up to 94.4%. The models also are capable to identify the Russian trolls with accuracy up to 72.6% without training on this set of trolls. This indicates that although the strategies of coordinated trolls might vary from an organization to another, they are all just employees and have common behaviors that can be identified. INDEX TERMS State-sponsored trolls, disinformation, propaganda, behavioral pattern.
... Quantitative analysis of texts can be very useful for understanding the strategies used in online information warfare. In the past few years, a number of studies investigated characteristics of Twitter accounts created to sow discord and polarization before, during, and after the 2016 presidential election in the U.S. (Ghanem et al., 2019;Llewellyn, Cram, Favero, & Hill, 2018;Zannettou, Caulfield, De Cristofaro, et al., 2019;Zannettou, Caulfield, Setzer, et al., 2019). A common denominator in these studies has been the use of Latent Dirichlet Allocation (LDA; Blei et al., 2003) to estimate latent topics from posts on social media platforms. ...
The past few years were marked by increased online offensive strategies perpetrated by state and non-state actors to promote their political agenda, sow discord and question the legitimacy of democratic institutions in the US and Western Europe. In 2016 the US congress identified a list of Russian state-sponsored Twitter accounts that were used to try to divide voters on a wide range of issues. Previous research used Latent Dirichlet Allocation (LDA) to estimate latent topics in data extracted from these accounts. Howerver, LDA is has characteristics that may pose significant limitations to be used in data from social media: the number of latent topics must be specified by the user, interpretability can be difficult to achieve, and it doesn't model short-term temporal dynamics. In the current paper we propose a new method to estimate latent topics in texts from social media termed Dynamic Exploratory Graph Analysis (DynEGA). We compare DynEGA and LDA in a Monte-Carlo simulation in terms of their capacity to estimate the number of simulated latent topics. Finally, we apply the DynEGA method to a large dataset with Twitter posts from state-sponsored right-and left-wing trolls during the 2016 US presidential election. The results show that DynEGA is substantially more accurate to estimate the number of simulated topics than several different LDA algorithms. Our empirical example shows that DynEGA revealed topics that were pertinent to several consequential events in the election cycle, demonstrating the coordinated effort of trolls capitalizing on current events in the U.S. This demonstrates the potential power of our approach for revealing temporally relevant information from qualitative text data.
... Surprisingly, IRA linked accounts, which have been identified by Twitter as evidence and later on submitted to United States Senate Judiciary Subcommittee on Crime and Terrorism, have also been found to be associated with Brexit (Llewellyn et al., 2018). These accounts attempted to promote discord for various topics regarding the European Union and migration. ...
... Ratkiewicz et al. [60] and Kollanyi et al. [37] examined the role of bots with respect to US politics and found that bots may strategically boost the dissemination of particular messages (e.g., that favor a particular political candidate). Moreover, Llewellyn et al. [49] showed that bots actively used trending hashtags (such as #Brexit) to promote specific political messages. ...
Social bots are software programs that automatically produce messages and interact with human users on social media platforms. In this paper, we provide an analysis of the emotion-exchange patterns that arise from bot- and human-generated Twitter messages. In particular, we analyzed 1.3 million Twitter accounts that generated 4.4 million tweets related to 24 systematically chosen real-world events. To this end, we first identified the intensities of the eight basic emotions (according to Plutchik's wheel of emotions) that are conveyed in bot- and human-generated messages. We then performed a temporal analysis of the emotions that have been sent during positive, negative, and polarizing events. Furthermore, we investigated the effects on user reactions as well as on the message exchange behavior between bots and humans. In addition, we performed an analysis of the emotion-exchange motifs that occur when bots communicate with humans. For this purpose, we performed a systematic structural analysis of the multiplex communication network that we derived from the 4.4 million tweets in our data-set. Among other things, we found that 1) in contrast to humans, bots do not conform to the base mood of an event, 2) bots often emotionally polarize during controversial events and even inject polarizing emotions into the Twitter discourse on harmless events such as Thanksgiving, 3) when bots directly exchange messages with human accounts they are, however, indistinguishable from humans with respect to the emotions they send, 4) direct message exchanges between bots and humans result in characteristic and statistically significant emotion-exchange motifs.
Este libro es la tercera edición de otro que se publicó en 2013 como material de apoyo y contenido en un máster universitario de la Universidad de Alcalá. Fue recomendado por algunos de los más reputados especialistas, con éxito de crítica y uso. En la segunda edición añadimos una parte dedicada a Grupos Estratégicos. Ahora hemos añadido una tercera, con 206 páginas de las 330 que componen el libro, dedicada a Amenazas Híbridas.
Sociedad del Conocimiento, Sociedad de la Información o Sociedad Postindustrial, o como ahora Amenazas Híbridas, son expresiones que se utilizan con frecuencia en los foros y escenarios más diversos. En ellos son sometidos a un proceso de simplificación, cuando no de banalización, relacionando algunos de sus rasgos con los aspectos más llamativos o espectaculares de la actualidad. No espere el lector ver en este trabajo una visión simplificada que, más que aclarar, desvirtúe estos rasgos.
Aquí encontrarán un repertorio sistematizado de referencias y de aportaciones a estos conceptos y a las ideas a ellos vinculadas, que podrán ser útiles en estudios e investigaciones. No es pues un texto de divulgación, pero sí puede ser útil para la profundización en esta área, con aplicaciones en otras o a otros temas.
En todas las épocas se da un hecho en el que coinciden todos: El acceso a la información disponible y su procesamiento para obtener un conocimiento derivado, que es operativo en contextos distintos, constituye una práctica común.
La novedad, el punto de discontinuidad, de la sociedad actual con respecto a la sociedad anterior lo constituye el papel que juega la tecnología y que afecta a como conocemos y a como comprendemos:
La nueva sociedad, que se desarrolla de forma autónoma, encuentra su expresión genuina y se conforma a partir de que aparecen las posibilidades de que la información sea soportada digitalmente, de que se pueda procesar masivamente utilizando algoritmos matemáticos y de que es facilitada de forma personalizada, mediante potentes algoritmos. Esa naturaleza de los procesos, que es común a todas las áreas, es la principal característica de nuestro abordamiento y le confiere un planteamiento pluridisciplinar.
En el trabajo que presentamos queremos ofrecer una perspectiva de cuáles son los rasgos más relevantes y recientes de la nueva sociedad (grupos estratégicos, amenazar híbridas y, en un futuro, la crisis de la edición científica y la Inteligencia Artificial generativa), de forma que la perspectiva que este análisis suministra pueda proyectarse sobre distintas disciplinas y sea útil, una vez incorporada, a sus contenidos como docente o para sus objetivos profesionales.
Más allá de la definición clásica de amenaza híbrida como mezcla de actividad coercitiva y subversiva, métodos convencionales y no convencionales, que pueden ser utilizados de manera coordinada para lograr objetivos específicos mientras se mantiene por debajo del umbral de una guerra formalmente declarada, estas actividades cobran especial relevancia por los factores de la sociedad digital que aumentan su eficacia hasta llegar a que se perciba como uno de los principales riesgos que actualmente enfrentamos.
Uno de esos factores ha sido la realidad líquida y la falta de compromiso que potencian los llamados postpúblicos. Tanto la realidad como la modernidad líquida y customizada —junto con una falsa sensación de personalización y flexibilidad— conllevan, básicamente en la sociedad occidental y liberal, una falta de compromiso y un cambio en la forma de pensar y elaborar el conocimiento o de representar la realidad. Este cambio ha tenido implicaciones sociológicas que abordamos en este libro. En ese contexto veremos que aparecen como resultado los postpúblicos, el punto clave dentro de las estrategias de las amenazas híbridas.
De una forma correlacionada aplicaremos y deduciremos esos patrones conceptuales sobre lo que ha sucedido estos años en Ucrania y su guerra, el Reino Unido y el Brexit, EE. UU. y Trump, Cataluña y el procés, Irán, el asunto Israel/Hamás y el antisemitismo de nuevo cuño. Veremos que, junto a la amenaza híbrida y la teoría estratégica (TE), han emergido conceptos nuevos, como el control reflexivo (CR), los tropos y los tropos recurrentes, estructurados en narrativas adecuadas a fines estratégicos determinados. Hablamos de una realidad nueva, donde la naturaleza y la práctica democrática son otras —cuando no se cuestionan explícitamente—.
Pero sobre todo este libro quiere poner el foco, al elegir este tema, el de la amenaza híbrida, sobre cualquier otro de los muchos en los que se manifiesta la sociedad del conocimiento, es por la importante transcendencia en cómo van a ser los conflictos que las sociedades va a afrontar en los tiempos próximos o los están afrontando ya. Y cuáles van a ser nuestras condiciones de vida y por ende sobre qué ejes oscilará nuestra felicidad, así como la de nuestros hijos. Objetivo último de cualquier trabajo científico que, como tal, y en su componente de Ethos, debe contemplar.
Según la misma Zelenkauskaitė (2022) expone y desarrolla, en una primera fase cuando se descubre el troleo y la posible influencia en el proceso, los perjudicados niegan su existencia, les parece absurdo, o quizá tienen miedo de usar argumentos que, por su debilidad y falta de credibilidad, se vuelvan en su contra. Es lo que Zelenkauskaitė (2022) ha definido, como ya hemos introducido, como pussy State. Así hemos visto que sucedió en el Reino Unido con el troleo del Brexit y en EE. UU. con el troleo ruso en las elecciones de 2016 calificado por la propia candidata Clinton como un absurdo.
El 16 de agosto de 2013 se publicó la edición anterior de este libro. Once años después, en una obra de esta naturaleza y con estos contenidos, cabría suponer que hablaríamos de concepciones nuevas, tanto en el campo del pensamiento y de las ideas como en el de la organización y de las prácticas sociales. Lo que era incipiente, como los grupos estratégicos, se ha desarrollado. Ahora ha adquirido carta de naturaleza y es aceptado, aún sin identificarlo como tal, como algo usual que está en la naturaleza de las cosas. Veremos en esta nueva parte del libro que han aparecido, como consecuencia de ello y merced a radicales cambios en cómo la gente se comunica con las nuevas herramientas tecnológicas, otras formas de relacionarse y de representarse la realidad muy distintas a cómo eran antes.
Unas de ellas ha sido lo que se han dado en llamar los postpúblicos y, aunque en principio parezca poco relacionado, según veremos, es el eje sobre el que van a girar las estrategias que hacen ahora más posibles y fuertes que nunca las amenazas híbridas.
Otro enfoque que también tenemos en cuenta es el de lconsiderar las amenazas híbridas como una plasmación de la acción y de la práctica de los grupos estratégicos, estudiados en ediciones anteriores.
Junto a la amenaza hibrida y la teoría estratégica se han acuñado otros conceptos nuevos como el control reflexivo, tropos y tropos recurrentes estructurados en narrativas adecuadamente a unos determinados fines estratégicos. Hablaremos también de un concepto ya mencionado vinculado a ellos, como es el de postpúblico.
Todo ello lo haremos de forma sistémica. En función de que en conjunto configuran una realidad nueva, donde la naturaleza y la práctica democrática son otras, cuando no es que se cuestionen explícitamente.
The purpose of this article is to explore Arabic-language Tweets based out of Saudi Arabia to investigate the social media landscape. Specifically, this article seeks to address the question, “What thematic issues concerning the U.S. socio-political landscape are present in Arabic-language Twitter postings?” And, “To what extent can these issues be described as propagandic in nature?” To do so, we propose a machine-learning and artificial intelligence span detection approach to identify propaganda Tweets in Middle Eastern Countries, with a focus on Saudi Arabia. As opposed to previous work, this article maps and investigates different propaganda categories using the BEND Social Cyber Security framework. This article then proceeds to a case study analysis of state-sponsored targeted propaganda from Saudi Arabia and briefly describes the categories of propaganda uncovered. We then relate those categories to the BEND Framework and conclude with policy recommendations and discussion.
This project investigates political astroturfing, i.e. hidden propaganda by powerful political actors aimed at mimicking grassroots activity, on social media. We focus on Twitter accounts used by the South Korean secret service to influence the 2012 presidential elections in favor of the eventual winner, Park Geun-hye. Two independent cluster analyses based on activity patterns of the Twitter accounts and textual features of tweets reliably reveal that there are three groups of NIS accounts, including one group that engages mostly in retweeting, and another group focused on posting news articles with a link. We show that these groups reflect different strategic agendas and correspond to several secret service agents identified in the court documents. We argue that these patterns of coordinated tweeting are consistent with predictions derived from principal-agent theory, and should therefore appear in other astroturfing campaigns as well.
From politicians and nation states to terrorist groups, numerous organizations reportedly conduct explicit campaigns to influence opinions on social media, posing a risk to freedom of expression. Thus, there is a need to identify and eliminate "influence bots"--realistic, automated identities that illicitly shape discussions on sites like Twitter and Facebook--before they get too influential.
The emergence of user forums in electronic news media has given rise to the proliferation of opinion manipulation trolls. Finding such trolls automatically is a hard task, as there is no easy way to recognize or even to define what they are; this also makes it hard to get training and testing data. We solve this issue pragmatically: we assume that a user who is called a troll by several people is likely to be one. We experiment with different variations of this definition, and in each case we show that we can train a classifier to distinguish a likely troll from a non-troll with very high accuracy, 82–95%, thanks to our rich feature set.
Twitter is a new web application playing dual roles of online social networking and microblogging. Users communicate with each other by publishing text-based posts. The popularity and open structure of Twitter have attracted a large number of automated programs, known as bots, which appear to be a double-edged sword to Twitter. Legitimate bots generate a large amount of benign tweets delivering news and updating feeds, while malicious bots spread spam or malicious contents. More interestingly, in the middle between human and bot, there has emerged cyborg referred to either bot-assisted human or human-assisted bot. To assist human users in identifying who they are interacting with, this paper focuses on the classification of human, bot, and cyborg accounts on Twitter. We first conduct a set of large-scale measurements with a collection of over 500,000 accounts. We observe the difference among human, bot, and cyborg in terms of tweeting behavior, tweet content, and account properties. Based on the measurement results, we propose a classification system that includes the following four parts: 1) an entropy-based component, 2) a spam detection component, 3) an account properties component, and 4) a decision maker. It uses the combination of features extracted from an unknown user to determine the likelihood of being a human, bot, or cyborg. Our experimental evaluation demonstrates the efficacy of the proposed classification system.
This project investigates political astroturfing, that is, hidden propaganda by powerful political actors aimed at mimicking grassroots activity, on social media. We focus on Twitter accounts used by the South Korean secret service to influence the 2012 presidential elections in favor of the eventual winner, Park Geun-hye. Two independent cluster analyses based on activity patterns of the Twitter accounts and textual features of tweets reliably reveal that there are three groups of NIS accounts, including one group that engages mostly in retweeting, and another group focused on posting news articles with a link. We show that these groups reflect different strategic agendas and correspond to several secret service agents identified in the court documents. We argue that these patterns of coordinated tweeting are consistent with predictions derived from principal-agent theory, and should therefore appear in other astroturfing campaigns as well.
We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.