Content uploaded by Ryan Scrivens
Author content
All content in this area was uploaded by Ryan Scrivens on Feb 02, 2019
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=rirt20
Download by: [Simon Fraser University] Date: 18 January 2017, At: 06:38
Behavioral Sciences of Terrorism and Political Aggression
ISSN: 1943-4472 (Print) 1943-4480 (Online) Journal homepage: http://www.tandfonline.com/loi/rirt20
Searching for signs of extremism on the web: an
introduction to Sentiment-based Identification of
Radical Authors
Ryan Scrivens, Garth Davies & Richard Frank
To cite this article: Ryan Scrivens, Garth Davies & Richard Frank (2017): Searching for
signs of extremism on the web: an introduction to Sentiment-based Identification
of Radical Authors, Behavioral Sciences of Terrorism and Political Aggression, DOI:
10.1080/19434472.2016.1276612
To link to this article: http://dx.doi.org/10.1080/19434472.2016.1276612
Published online: 18 Jan 2017.
Submit your article to this journal
View related articles
View Crossmark data
Searching for signs of extremism on the web: an introduction
to Sentiment-based Identification of Radical Authors
Ryan Scrivens, Garth Davies and Richard Frank
School of Criminology, Simon Fraser University, Burnaby, British Columbia, Canada
ABSTRACT
As violent extremists continue to surface in online discussion
forums, law enforcement agencies search for new ways of
uncovering their digital indicators. Researchers have both
described and hypothesized a number of ways to detect online
traces of potential extremists, yet this area of inquiry remains in
its infancy. This study proposes a new search method that,
through the analysis of sentiment, identifies the most radical users
within online forums. Although this method is applicable to web-
forums of any type, the method was evaluated on four Islamic
forums containing approximately 1 million posts of its 26,000
unique users. Several characteristics of each user’s postings were
examined, including their posting behavior and the content of
their posts. The content was analyzed using Parts-Of-Speech
tagging, sentiment analysis, and a novel algorithm called
‘Sentiment-based Identification of Radical Authors’, which
accounts for a user’s percentile score for average sentiment score,
volume of negative posts, severity of negative posts, and duration
of negative posts. The results suggest that there is no simple
typology that best describes radical users online; however, the
method is flexible enough to evaluate several properties of a
user’s online activity that can identify radical users on the forums.
ARTICLE HISTORY
Received 22 January 2016
Accepted 21 December 2016
KEYWORDS
Sentiment analysis;
discussion forums; extremism
It is widely acknowledged that people around the world are increasingly using computer
technologies and computer-mediated communications to connect with each other. The
Internet’s seamless accessibility and user-friendly platform have revolutionized the
sharing of information and communications, facilitating an international web of virtual
communities. Violent extremists and those who subscribe to radical beliefs have embraced
this changing digital landscape, and their presence in online discussion forums has grown
rapidly (Bowman-Grieve, 2009; Sageman, 2008; Seib & Janbek, 2011; Weimann, 2006).
Online discussion forums, also understood as ‘virtual communities,’are ideal venues in
which supporters and sympathizers of different movements, including radical Right or
Jihadist supporters, can interact with one another, free from the geographic barriers
that once made it difficult to communicate (Bowman-Grieve, 2013). Within the extremist
domain, online forums have also facilitated the ‘leaderless resistance’movement, a decen-
tralized and diffused tactic that has made it increasingly difficult for law enforcement
© 2017 Ryan Scrivens, Garth Davies and Richard Frank
CONTACT Ryan Scrivens rscriven@sfu.ca School of Criminology, Simon Fraser University, Burnaby, British
Columbia, Canada V5A 1S6
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION, 2017
http://dx.doi.org/10.1080/19434472.2016.1276612
officials to detect potentially violent extremists (Brynielsson et al., 2012; Cohen, Johansson,
Kaati, & Mork, 2014; Sageman, 2008).
Radicals have realized the benefits of the Internet, and have increasingly moved to
online discussion forums to discuss their ideologies, current events, and opinions. They
have exploited the Internet’s easy to use, quick, cheap and unregulated, relatively
secure and anonymous platforms, and have in turn created online communities that
encourage the gathering and dissemination of information, inspiration to support and
gain resources, and the demonization of their enemies (Sageman, 2008; Whine, 1999).
Online forums have also enabled radicals to freely –and anonymously –express negative
views that may be perceived as hostile, unpalatable, or even non-credible in other settings.
Such online spaces have offered users encouragement and support, and extremist ideol-
ogies have been reinforced and mirrored, rather than challenged by anti-extremist senti-
ment (Bowman-Grieve, 2013; Kennedy & Weimann, 2011; Seib & Janbek, 2011; Weimann,
2004). In turn, conversations and easy access to the Internet have strengthened the deter-
mination of its forum members by constructing a network of online communities that
embrace a sense of purpose, identity, and a common enemy (Bouchard, Joffres, &
Frank, 2014; Rogan, 2006; Tsfati & Weimann, 2002). These communities are created and
sustained on the basis of common values, goals, norms, commitment, support and associ-
ation, and the sense of belonging. They are held together by the ideological core and
vision of the common identity that it informs (Campana & Ducol, 2014; Ducol, 2012).
While online discussion forums have also enabled those with extremist views to
connect, correspond, plan, target, and control violent activities in what Bouchard and
Nash (2015) describe as ‘a loosely connected, flexible, and decentralized network structure’
(p. 54), little is known about the ways in which such individuals –or even radical users for
that matter –can be identified within these growing online communities.
Uncovering signs of extremism online has been one of the most significant policy issues
faced by law enforcement agencies and security officials worldwide (Cohen et al., 2014;
Weimann, 2008), and the current focus of government-funded research has been on
the development of advanced information technologies to identify and counter the
threat of violent extremism on the Internet (Sageman, 2014). Here scholars have argued
that successfully identifying online signs of extremism, especially on a large scale, is the
first step in reacting to them (Bouchard et al., 2014; Brynielsson et al., 2012; Cohen
et al., 2014; Davies, Bouchard, Wu, Joffres, & Frank, 2015; Frank, Bouchard, Davies, & Mei,
2015; Mei & Frank, 2015). Yet in the last 10 years alone, it is estimated that the number
of individuals with access to the Internet has increased 3-fold (Internet World Stats,
2016), from over 1 billion in 2005 to more than 3.1 billion as of 2015 (Internet Live
Stats, 2016). With these new users, more information has been generated, leading to a
constantly growing deluge of data. As the amount of data has increased, it has become
harder and harder to sift through, and manual methods of research have become increas-
ingly less efficient. These new conditions have necessitated guided data filtering methods,
those that can side step the laborious manual methods that have been classically utilized
to identify the information that is relevant (Brynielsson et al., 2012; Cohen et al., 2014).
It is becoming increasingly difficult –and near impossible –to manually search for
violent extremists, potentially violent extremist, or users who post radical messages on
the Web because it contains an overwhelming amount of information. In response to
this problem, researchers have recommended that machine learning techniques,
2R. SCRIVENS ET AL.
particularly semi-automated techniques that include human research decisions, be used to
aid in the process of analyzing big data (Brynielsson et al., 2012; Cohen et al., 2014). The
study detailed in this paper attempts to address this growing concern. We propose a
method of identifying radical users on large-scale web-forums using semi-automated
computer tools, an undertaking that Cohen et al. (2014) heretofore compared to ‘searching
for a needle in a haystack’(p. 274). Here the purpose was not to identify users who may
engage in acts of violent extremism, but to identify radical users or users of interest
based on their distinct behavior in selected online forums. Identifying the most radical
author may not lead us to those whose activities are the most dangerous or radical
offline, but that is not the only reason to identify radical users online: their messages
are openly available on the Internet, and thus their messages may attract radical
readers, or have an ‘echo chamber’or ‘radicalization’effect on impressionable readers
(Stevens & Neumann, 2009). Engaging these extremist views –and countering their nar-
rative –could lead to a viable intervention or anti-radicalization effort.
Uncovering hidden populations online
There has been a surge in the number of studies on radical uses of the Internet since the
tragic events of 9/11. For example, in effort to better understand how those with extremist
beliefs use the Internet, scholars have provided descriptive accounts of the content fea-
tured on Jihadi-based websites (Amble, 2012; Brown & Korff, 2009; Conway, 2006;
Neumann & Rogers, 2007; Thomas, 2003; Tsfati & Weimann, 2002; Weimann, 2004;
Whine, 1999), as well as measured the hyperlinks posted within and between the sites
(Reid & Chen, 2007; Reid et al., 2005; Sageman, 2008). While a number of studies have
described the content on radical Right websites as well (e.g. Back, 2002; Schafer, 2002;
Thiesmeyer, 1999), a more recent trend in the scholarship has been a shift from identifying
specific content on websites to understanding how virtual communities are developed
and maintained on Web 2.0, a term coined by Tim O’Reilly to understand how technology
has enabled peer-to-peer networks. A number of these interactive corners have received
considerable attention in the past decade, including a social network analysis of how
Jihadists use Twitter (Burnap et al., 2014; Klausen, 2015), YouTube (Bermingham,
Conway, McInerney, O’Hare, & Smeaton, 2009; Conway & McInerney, 2008; Klausen,
Barbieri, Reichlin-Melnick, & Zelin, 2012), and online forums (Campana & Ducol, 2014;
Ducol, 2012; Torres-Soriano, 2013). Weimann (2010) also examined how Jihadists interact
via Twitter,Facebook, chatrooms, and online forums, while Hegghammer (2014) explored
Jihadi issues of trust around online recruitment on web-forums. The study of right-wing
extremists’uses of the Internet, such their use of online discussion forums, has also
received considerable academic attention since the inception of the Internet (e.g.
Bowman-Grieve, 2009; Meddaugh & Kay, 2009; Whine, 1999).
In light of these important contributions to how those with extremist views use the
Internet, an important question has been set aside: how can we uncover the digital indi-
cators of ‘extremist behavior’online, particularly for the ‘most extreme individuals’based
on their online activity? To some extent criminologists have begun to explore this critical
point of departure via a customized web-crawler, extracting large bodies of text from web-
sites featuring extremist material and then using text-based analysis tools to assess the
content (Bouchard et al., 2014;Frank et al., 2015; Mei & Frank, 2015). Similarly, some
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 3
computational-based research has been conducted on extremist content on Islamic-based
discussion forums (Abbasi & Chen, 2005; Chen, 2012; Fu, Abbasi, & Chen, 2010; Zhang et al.,
2010; Zhou, Qin, Lai, Reid, & Chen, 2005). While these studies showcased a number of
textual analysis tools to detect the indicators of extremism online, surprisingly, they did
not focus their analysis on the authors of the online forums specifically. Indeed, systematic
research is needed to identify sentiment that may help to identify radical users based on
their online posting behavior. Both Brynielsson et al. (2012) and Cohen et al. (2014)
hypothesized a number of ways to detect the online traces of lone wolf terrorists, for
example, but they did not apply theory to practice. Arguably, a starting point in identifying
lone wolf terrorists or potentially violent extremists is to identify users who post negative
text on a large scale. Much is still unknown about how to uncover this hidden population
(Cohen et al., 2014; Sageman, 2014).
Methods
The purpose of this research project was to add to the extant literature by using a senti-
ment analysis tool and an algorithm to identify radical authors within selected online dis-
cussion forums. Members of the Intelligence and Security Informatics (ISI) research team
made available, as part of a competition,
1
data that they gathered from four online discus-
sion forums. This dataset provided the test-bed for the evaluation of the algorithm pro-
posed herein. The dataset contained extracted versions of four online discussion
forums: Gawaher,Islamic Awakening,Islamic Network, and Turn to Islam. This research is
not an indictment of these particular sites. Rather, the purpose of this project was to
analyze, as part of a ‘challenge,’the entire dataset for Jihadist forum users. To do this,
we first used Parts-Of-Speech (POS) analysis to develop a list of keywords. A total senti-
ment score for each forum post was derived by summing the sentiment scores for each
of the POS keywords contained in the post. Next, radical authors were identified
through the application of a novel algorithm that calculated a radical score based on
the users’percentile score, volume of negative posts, severity of negative posts, and dur-
ation of negative posts (see Figure 1).
Web-forum data
The current study analyzed 1,000,998 posts from 141,763 threads made by approximately
26,171 authors between 28 April 2004 and 20 May 2013. All author names were assigned
with pseudonyms for the purposes of ensuring user anonymity. The following is a brief
overview of each of the four forums:
Gawaher defines itself as a ‘friendly Islamic international community,’and is designed to
facilitate various discussions pertaining to the Islamic world and Islam.
2
The purpose of the
site is to (1) promote a stronger understanding of Islam as a moderate religion and way of
life; and (2) connect Muslims with other Muslims worldwide (Gawaher, 2015). The current
study analyzed 372,499 posts from 53,235 threads made by approximately 9260 members
between 24 October 2004 and 7 June 2012.
Islamic Awakening has members based in the UK and surrounding countries, and ident-
ifies itself as a site ‘dedicated to the blessed global Islamic awakening.’
3
The current study
4R. SCRIVENS ET AL.
analyzed 201,287 posts from 32,879 threads made by approximately 3964 members
between 28 April 2004 and 22 May 2012.
Islamic Network has the overall goal of bringing dedicated individuals together to prac-
tice their Muslim faith, and the discussion forum includes a number of topics of interest to
Muslims, ranging from theology to current world events (Islamic Network, 2015). The
current study analyzed 91,874 posts from 13,995 threads made by approximately 2082
members between 9 June 2004 and 10 November 2010.
4
Lastly, Turn to Islam is a social networking website for Muslims that is committed to dis-
seminating ‘the true understanding of Islam based upon the Noble Qur’an and the auth-
entic Sunnah’(Turn to Islam, 2015a) and ‘correcting the common misconceptions about
Islam.’It defines itself as ‘the best place to learn about Islam’(ISI-ICDM Workshop on Intel-
ligence and Security Informatics Challenge, 2015), with the aim of strengthening and
uniting the Ummah, promoting Islamic values, and providing support for Reverts and
non-Muslims (Turn to Islam, 2015b).
5
The current study analyzed 335,338 posts from
41,654 threads made by approximately 10,858 authors between 2 June 2006 and 20
May 2013.
POS tagging
The first step in analyzing the forum data was to determine users’topics of discussion. To
do this we began by isolating the particular nouns that had the highest rate of occurrence
within the data, under the assumption that the most frequently discussed topics would
most likely be the ones in which extremist content was likely to be detected. This was
done with POS analysis, a method that collects and labels each word with the part of
speech they belong to. POS tagging has also been used to arrange and classify large
Figure 1. Process of text analysis and the creation of the radical score.
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 5
databases, such as patient data from medical institutions (Ferraro et al., 2013). This method
of data reorganization was used to apply specific tags to medical cases and make the data-
bases easier to query.
We used the POS tagger to analyze the post content and divide the words into mean-
ingful groupings. The tagger first removed the HTML content from the post, then scanned
through the posts of all four web-forums and using OpenNLP (Apache Software Foun-
dation, 2015) produced frequency distributions for each word. Each word was then
added into its appropriate grouping (i.e. noun, verb, etc.) and split into sub-groupings
(i.e. plural, singular, etc.). Once this process was completed, all of the noun groups (i.e.
plural, proper, and standard) were separated from the rest of the results. Nouns were
chosen because they are the words most likely to be surrounded by relevant sentiment
terms (Hu & Bing, 2004); names and places are often described or denoted by the adjec-
tives linked to them. Furthermore, adjectives are often the words that have sentiment
values attached to them (Thet, Na, & Khoo, 2010). By specifying adjectives as keywords,
the sentiment of the word itself would be lost.
From each of the four forums, we selected the top 100 nouns based on frequency.
6
Domain experts may replace or extend the list of keywords with relevant domain-
specific words; however, the exploratory nature of the study suggested that we rely
on the data to identify the relevant keywords. After removing terms that were not in
fact words, including symbols, incorrectly translated characters, and redundant
words, the remaining terms formed the keyword list for the sentiment analysis.
Sentiment analysis
After the keyword list was developed, it was necessary to identify and evaluate the context
surrounding the keywords. To allow a proper analysis of the user’s discussion and to aid in
the automatic discovery of which users could be considered ‘radical,’sentiment analysis
was used to highlight relevant text.
Sentiment analysis is a data collection and analysis method that allows for the appli-
cation of subjective labels and classifications (Feldman, 2013). It can evaluate the opinions
of individuals by organizing data into distinct classes and sections, and assigning an indi-
vidual’sentiment with a negative or positive polarity value (Abbasi & Chen, 2005). It also
allows for a more targeted view of a data set by allowing for the demarcation between
cases that are sought after and those without any notable relevance.
Sentiment analysis has been used in a wide variety of contexts, including customer
review analysis for products (Feldman, 2013), an assessment of attitudes toward events
or products on social media platforms (Ghiassi, Skinner, & Zimbra, 2013), a comparison
of different types of web-forums based on levels of negativity (Chalothorn & Ellman,
2012). As the purpose of the current study was not to push the boundaries of sentiment
analysis algorithm, an established Java-based software, SentiStrength, was used (Thelwall &
Buckley, 2013). SentiStrength allows for a keyword-focused method of determining senti-
ment near a specified keyword, which is a central feature of the software. Values are aug-
mented by characters that can influence the values assigned to the text, such as booster
words, negative words, repeated letters, repeated negative terms, antagonistic words,
punctuation, and other distinctive characters suited for studying an online context (for
more information, see Thelwall & Buckley, 2013).
6R. SCRIVENS ET AL.
Such a tool, however, has yet to be used to assess which online users exhibit the most
extreme (i.e. negative) sentiment values, taking into account the number, severity, and
duration of negative messages that each user posted on discussion forums. This was
done as follows. First, each post was scored for each of the 398 keywords (nouns) ident-
ified in the previous step, resulting in a ∼1 million-by-398 keywords matrix. If a keyword
did not exist in a post, the post was not scored for that keyword. The keyword scores
were then averaged for each post, and the average value was assigned to the post as
the final sentiment score. By analyzing the sentiment surrounding the most frequent
nouns that occur within the data, it was possible to obtain an understanding of the discus-
sions and attitudes that were present on these forums.
Sentiment-based Identification of Radical Authors
There are various definitions that can be used to define someone as an ‘extremist.’For
example, an author could be a long-time member of a discussion forum and post some-
what negative material over that time, which may suggest a long-time dedication to an
extremist movement, for example. However, another author could participate in a discus-
sion forum only for a short amount of time, but during that time espouse very radical views
in their postings, all of which may imply that the user has become ‘radicalized.’Indeed, this
raises a critical question: Are long-term authors with somewhat negative posts considered
more radical than new authors with very negative posts? Determining which aspect of an
author’s online activity should be considered when classifying them as more extreme
depends on the definition used, or the goals of the identification exercise. Rather than
focus on answering this question, it was decided that a measure should be developed
that is capable of being adjusted to measure either type of author.
7
Thus, an overall
‘radical score’was created based on the following components of an author’s online
activity:
Average sentiment score percentile (AS). An author could be defined as extreme if they
posted very negative comments to the forum. To measure how extreme an author was,
with respect to the forum average, the average sentiment score percentile was created. It
was calculated by accounting for the average sentiment score for all posts in a given
forum. The scores for each individual were converted into percentiles scores, and percen-
tile scores were divided by 10 to obtain a score out of 10 points.
Volume of negative posts (VN). An author could also be considered very extreme if they
were prolific posters and a lot of their posts were considered somewhat extreme. Authors
were classified as extreme not due to their very extreme post content, but because of their
long-term dedication to posting extremist material. To measure this, the volume of nega-
tive post was developed in two parts: (1) the number of negative posts for a given member,
and (2) the proportion of posts for a given member that were negative. To calculate the
number of negative posts for a given member, we counted the number of negative
posts for a given member and converted these scores into percentiles scores. Percentile
scores were then divided by 20 to obtain a score out of 5 points. To calculate the pro-
portion of posts for a given member that were negative, the counts from the previous
section were divided by the total number of posts to get the proportion. These scores
were converted into percentiles scores, and the percentile scores were divided by 20 to
obtain a score out of 5 points. Finally, the score of the number of negative posts for a
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 7
given member and the score of the proportion of posts for a given member that were
negative were tallied, thus creating a composite measure (out of 10) of negative volume.
Severity of negative posts (SN). There is a clear distinction between an author who posts
moderately extreme messages and another who posts very extreme messages. One
author may discuss a civil war in the Middle East, while the other celebrates the beheading
of Western militants. Consequently, a measure was needed to discern authors’level of
extremism. To do this, a measure for the severity of negative posts was developed consist-
ing of two parts: (1) the number of very negative posts for a given member and (2) the
proportion of posts for a given member that were very negative. ‘Very negative’was cal-
culated by standardizing the count variable; all posts with a standardized value greater
than three were considered to be ‘very’negative. After the standardization process, the
severity calculation was developed in the same manner as the volume calculation
above. To calculate the number of very negative posts for a given member, we counted
the number of very negative posts for a given member and converted these scores into
percentiles scores. Percentile scores were then divided by 20 to obtain a score out of 5
points. To calculate the proportion of posts for a given member that were very negative,
the counts from the previous section were divided by the total number of posts to obtain
the proportion. These scores were converted into percentile scores, and the percentile
scores were divided by 20 to obtain a score out of 5 points. Finally, the score of the
number of very negative posts for a given member and the score of the proportion of
posts for a given member that were very negative were tallied, thus creating a composite
measure (out of 10) of severity.
Duration of negative posts (DN). An author who posted extreme messages over an
extensive period of time should be classified as more extreme than an author who
posted equally extreme messages over a shorter period of time. Sentiment-based Identi-
fication of Radical Authors (SIRA) accounted for this important component, and the dur-
ation of negative posting was developed by calculating the first and last dates on which
individual members made negative posts. We calculated the difference between these
data, which is the duration of negative posting, and converted these scores into percen-
tiles scores. Percentile scores were divided by 10 to obtain a score out of 10 points.
Radical score. Together, the percentile score for average sentiment score,volume of nega-
tive post,severity of negative posts, and duration of negative posts were tallied to produce an
overall score out of 40 points. This ‘radical score’quantified four unique dimensions of ‘ser-
iousness’to identify radical individuals within the Islamic-based discussion forums. Con-
sider the following example of how the radical score of ICN.8, a hypothetical user,
would be calculated, accounting for all of his online activity within a discussion forum.
To attain ICN.8’s percentile score for average sentiment score percentile (AS), the senti-
ment within each of his posts was calculated and averaged (e.g. ICN.8’s sentiment score for
the word ‘gun’was 0.4, while the score for ‘knife’was 0.16, giving an average sentiment
score of 0.28). His average sentiment score was converted into a percentile score (e.g. per-
centile score = 83.19, meaning that 83.19% of all users in the forum maintained an average
score that was equal to or less than ICN.8’s sentiment value of 0.28).
8
The percentile score
was divided by a value of 10 to obtain a score out of 10 points (e.g. 83.10/10 = 8.319 of 10
points).
Volume of negative posts (VN) was calculated in two parts. First, ICN.8’s total number of
negative posts were counted (e.g. 61 negative posts) and converted into percentile scores
8R. SCRIVENS ET AL.
(e.g. percentile score = 77.56, meaning that 77.56% of all user in the forum maintained an
average score that was equal to or less than a total of 61 negative posts). The percentile
score was divided by a value of 20 to obtain a score out of 5 points (e.g. 77.56/20 = 3.878 of
5 points). Second, the original count of ICN.8’s negative posts (e.g. 61 negative posts) was
divided by the total number of his posts (e.g. 127 total posts) to obtain a proportion of
negative posts (e.g. 61/127 = 0.48) and converted into a percentile score (e.g. percentile
score = 95.82, meaning that 95.82% of all users in the forum had a proportion of negative
posts that was equal to or less than 0.48 of all of their posts). The percentile score was
divided by a value of 20 to obtain a score out of 5 points (e.g. 95.82/20 = 4.791 of 5
points). Lastly, the aggregate scores of the number of negative posts (part 1) and the pro-
portion of posts that were negative (part 2) were tallied to obtain a composite measure of
negative volume for ICN.8 (e.g. 3.878 + 4.791 = 8.669 of a maximum of 10 points).
A measure of the severity of negative posts (SN) was also calculated in two parts. First,
ICN.8’s total number of very negative posts was counted (e.g. 1 very negative post) and
converted into percentiles scores (e.g. percentile score = 99.14%, meaning that 99.14%
of all users in the forum displayed 0 very negative posts). The percentile score was
divided by a value of 20 to obtain a score out of 5 points (e.g. 99.14/20 = 4.957 of 5
points). Second, the original count of negative posts (e.g. 1 very negative post) was
divided by the total number of the user’s posts (e.g. 127 total posts) to obtain a proportion
of negative posts (e.g. 1/127 = 0.008) and converted into a percentile score (e.g. percentile
score = 98.54, meaning that 98.54% of users in the forum had a proportion of very nega-
tive posts that was equal to or less than 0.008 of all of their posts). The percentile score was
divided by a value of 20 to obtain a score out of 5 points (e.g. 98.54/20 = 4.927 of 5 points).
Lastly, the sum scores of the number of very negative posts (part 1) and the proportion of
posts that were very negative (part 2) were tallied to obtain ICN.8’s composite measure of
negative severity (e.g. 4.957 + 4.927 = 9.884 of 10 points).
The duration of negative posts was calculated by identifying the first and last date that
ICN.8 posted negative messages (e.g. the user posted his first negative message on 18 July
2006, and posted negative messages until 11 May 2011). To obtain the duration of nega-
tive posts, the difference between the aforementioned dates were calculated (e.g. 11 May
2011 –19 July 2006 = 4.84 years), and the duration of the user’s negative posts was con-
verted into percentiles scores (e.g. percentile score = 99.02, meaning that 99.02% of all
users in the forum had a duration of negative posts that was equal to or less than 4.84
years). The percentile score was divided by a value of 10 to obtain a score out of 10
points (e.g. 99.02/10 = 9.902 of 10 points).
The final step to obtaining ICN.8’s radical score was to combine his percentile score for
average sentiment score percentile (AS) with the volume of negative post (VN), severity of
negative posts (SN), and duration of negative posts (DN).
Radical score =AS +VN +SN +DN
=8.319 +8.669 +9.884 +9.902
=36.774 of 40 points.
The four percentile components, in combination, were strong indicators of radical users
in an online discussion forum, whereby the higher a user’s radical score, the more likely
they were to be discussing extremely negative content in their posts. Across the full
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 9
sample of respondents, the Cronbach’s alpha for the components comprising the radical
score was .742, indicating an acceptable level of internal consistency.
Results and analysis
The results were interpreted in two ways. First, the 26,171 unique forum users were cate-
gorized by discussion forum. They were then ranked according to their ‘radical score,’and
those with the highest radical scores were assessed. Second, users from the four online
discussion forums were analyzed as a single entity and ranked according to their
‘radical score.’Both the users with the highest radical scores and the forums with the
highest number of radical users were evaluated.
Gawaher
According to the algorithm, the most radical user in the Gawaher discussion forum was
O.M.T. The author posted negative messages over a consistent period of over 6.5 years.
Of the 2874 messages posted, the user’s overall average posting score was −1.02, and
2074 (72.4%) of the total number of messages were negative while 16 (0.6%) of the
messages were ‘very negative.’What separated this user from the other users who
received the highest radical score in the forum was the author’s total number of
radical messages posted, as well as the amount of time the author was active in the
forum. Specifically, O.M.T. was among the most prolific posters (i.e. 2874 posts) over
one of the longest periods of time on the forum (i.e. approximately 6.5 years), and
roughly three-quarters of the user’s messages were classified as negative. For
example, as O.M.T. wrote:
…the us [United States] loves to display its donkey talent in poking its nosey nose into others’
affairs […] merely to satisfy the us’[United States] lust for revenge …[sic].
9
Overall, O.M.T. received a radical score of 38.934 of 40 and was the most radical user on the
Gawaher discussion forum according to SIRA.
The second most radical member in the discussion forum, Ray, posted negative mess-
ages over a consistent period of 3.35 years. Of the 524 messages posted, the user’s overall
average posting score was −1.29, and 392 (i.e. 75%) of the total number of messages were
negative while 6 (i.e. 1.1%) of the messages were ‘very negative.’An example of the
author’s online sentiment included the following posting:
This is about the invasions and bombs that the US military machine has unleashed over
Muslim lands. Most people realise that the entire US military machine has to be destroyed
or stopped and this is what many people pray for.
While Ray was not one of the most prolific users on the forum (i.e. 524 posts), three-quar-
ters of the user’s messages were classified as negative (75%) over a fairly extensive period
of time (i.e. 3.35 years). Furthermore, both the user’s average sentiment score percentile
(i.e. −1.29) and the negative severity of their messages (i.e. 1.1%) were slightly higher
than the majority of users who received the highest radical score in the forum. As a
result, Ray received a radical score of 38.823 of 40 and was the second most radical
user on the Gawaher discussion forum.
10 R. SCRIVENS ET AL.
Another important user was identified as the third most radical member in the discus-
sion forum. 3D posted negative messages over a consistent period of almost four years. Of
the 28 messages posted, the user’s overall average posting score was −2.21, and 24
(85.7%) of the total number of messages were negative while 1 (3.6%) message was
‘very negative.’For example, as the author wrote:
Yes, power does corrupt and the downtrodden, the oppressed and deprived peoples of the
world, Muslim or otherwise, will fight back until they or others overthrow such corrupt
regimes …and put in place more just and equitable governments which then slowly
become corrupt again.
Notably, 3D was not among the most prolific users on the discussion forum (i.e. 28 posts)
nor did the user post radical material over the longest periods of time (i.e. 3.94 years), but
the percentage of the user’s posts that were classified as negative were among the highest
in the forum (i.e. 85.7%). Unlike other forum users who also received the highest ‘radical
scores’on the forum, 3D’s average sentiment score was highest among the most radical
users in the discussion forum (i.e. −2.21), and the author was among the few users who
posted a small number of consistently negative messages over a substantial amount of
time (3.94 years). Accordingly, 3D received a radical score of 38.790 of 40 and was the
third most radical user on the Gawaher discussion forum.
Islamic Awakening
The most radical author in the Islamic Awakening discussion forum was user Prism. This
author posted negative messages over a consistent period of 2.87 years. Of the 139 mess-
ages posted, the user’s overall average posting score was −4.47, and 134 (97.1%) of the
total number of messages were negative while 33 (23.9%) of the messages were ‘very
negative.’What distinguished Prism from the other radical users in the forum was the
remarkably high number of posts that were classified as negative (i.e. 97.1% of all
posts), the high percent of posts that were identified as ‘very negative’(i.e. 23.9% of all
posts were very negative) and the relatively long duration of time that the user posted
negative messages on the forum (i.e. 2.87 years). For example, as Prism explains,
Muslims are the current object of our society’s fear and contempt […] The aggressive Amer-
ican war machine continues to endeavour tirelessly to gag every free, independent and self
respecting media organization in the world that tries to convey a message of truth to
others without distortion or perversion…
The author was not among the most prolific posters in the forum (i.e. 139 messages), but
the average sentiment score percentile value of the author’s content was the most nega-
tive in the forum (i.e. −4.47). Consequently, Prism received a noteworthy radical score of
39.028 of 40 and was the most radical user on the Islamic Awakening discussion forum.
The second most radical member in the discussion forum was Nash. The user posted
negative messages over 5.72 years, and messages were identified as negative during
5.69 of those years. Of the 425 messages posted, the user’s overall average posting
score was −1.05, and 294 (70.3%) of the total number of messages were negative while
5 (1.2%) of the messages were ‘very negative.’Unlike the other users who received the
highest radical scores in the discussion forum, Nash was the second most radical user in
Islamic Awakening for two key reasons. First, although the user was not among the
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 11
most prolific poster on the forum (i.e. 425 messages), nearly three-quarters of the mess-
ages posted by Nash were classified as negative (i.e. 70.3%) and were posted over a sub-
stantial period of time (i.e. 5.69 years). Second, Nash’s overall number of very negative
messages was slightly higher than the severity score of other users who received high
radical scores in the forum (i.e. 1.2% of messages were very negative). An example of
such messages includes:
Amrika [America] […] How can a government that tortures our brothers and sisters, that impri-
sons our brothers and sisters (often without trial), that invades our lands, that has killed tens of
thousands of our brothers and sisters, that struts around like the arrogant cowardly bully it is,
be our friend…
Overall, the author received a radical score of 38.876 out of 40 and was the second most
radical user on the Islamic Awakening discussion forum.
Islamic Network
The most radical member in the Islamic Network discussion forum was prism. The user
posted negative messages over 2.28 years, and messages were identified as negative
during 2.11 of those years. Of the 207 messages posted, the user’s overall average
posting score was −2.36, and 155 (74.9%) of the total number of messages were negative
while 9 (4.3%) of the messages were ‘very negative.’What separated prism from the other
radical users in Islamic Network was a combination of the severity of the user’s negative
posts and the user’s duration of negative activity on the discussion forum. Specifically,
the author was not among the most prolific posters on the forum (i.e. 207 posts), but
the duration of the negative posts (i.e. 2.11 years) was longer than the majority of the
users who scored the highest radical scores in the forum. Moreover, a relatively high pro-
portion of this user’s messages were very negative (i.e. 4.3%). For example, as prism wrote
in one discussion thread:
…inshaa’allaah [God-willing] I hope to keep the brothers and sisters updated about what is
happening to unjustly detained Muslims around the world and what we can do about it
[…] The true believer is the one whom other Muslims are safe from his tongue and hands!
And all success comes from Allah alone…
The percent of very negative messages for this particular user was higher than the majority
of those who received the highest radical scores in the discussion forum. As a result, prism
received a radical score of 38.601 of 40 and was the most radical user on the Islamic
Network discussion forum. Interestingly, this user was also identified as the most radical
user on the Islamic Awakening discussion forum.
The second most radical member in the discussion forum, Fait1, posted negative mess-
ages over 3.16 years, and messages were identified as negative during 2.46 of those years.
Of the 279 messages posted, the user’s overall average posting score was −1.28, and 213
(76.3%) of the total number of messages were negative while 3 (1.1%) of the messages
were ‘very negative.’The factors that distinguished this user from those who received
high radical scores in the same forum were similar to that of prism.Fait1 was not
among the most prolific posters on the forum (i.e. 279 posts), but the number of messages
classified as very negative were among the lowest in the group of the most radical users in
the forum (i.e. a mere 1.1% of all messages were very negative). Furthermore, slightly over
12 R. SCRIVENS ET AL.
three-quarters of the user’s messages were classified as negative (i.e. 76.3%), and the dur-
ation of time that the user posted negative messages was considerably longer than the
amount of time that the majority of the most radical users were active on the forum
(i.e. 2.46 years). For example, in an online discussion about the perpetrator who filmed
himself throwing a bullet-riddled Quran at a mosque in the US, Fait1 noted:
The guy in the video is such an obvious piece of human garbage. He claims that he considers
as a hero anyone that kills Muslims. Of course if this little piece of crap is such a coward he
would never step foot in Iraq or Afghanistan.
Overall, Fait1 received a radical score of 38.555 of 40 and was the second most radical user
on the Islamic Network discussion forum.
Turn to Islam
The most radical member in the Turn to Islam discussion forum was dire who posted nega-
tive messages over a consistent period of approximately 5.5 years. Of the 1524 messages
posted, the user’s overall average posting score was −0.7, and 699 (50.9%) of the total
number of messages were negative while 12 (0.9%) of the messages were ‘very negative.’
What distinguished this user from the other users who received the highest radical scores
in the forum was a combination of the user’s total number of radical messages posted, the
amount of time the author was active on the forum, and the user’s average sentiment
score. While dire’svolume of negative posts and the severity of posts was not overly
high (i.e. 50.9% of the author’s posts were negative, and only 0.9% of the posts were
very negative), the user posted negative material over one of the longest periods of
time on forum (i.e. 5.52 years) and was among the most prolific posters on the forum
(i.e. 1524 posts). For example, in one thread the user wrote:
…US army used chemical weapons agains civilians […] In this world only Allah can judge the
president of the USA [sic].
Consequently, dire received a radical score of 38.288 out of 40 and was the most radical
user on the Turn to Islam discussion forum.
Radical users across web-forums
The five most radical users across all four web-forums were active users on Gawaher and
Islamic Awakening (see Table 1). Prism was the most radical user on Islamic Awakening, and
was also the most radical user across all forums. Again, this author posted very few mess-
ages on the forum (i.e. 139 messages); however, the content found within of these mess-
ages was among the most extreme across all forums (i.e. the average sentiment score was
−4.47, 97.1% of all posts were negative, and 23.9% of all posts were very negative). Fur-
thermore, this user did not post a handful of negative messages over a short period of
time. Instead, Prism posted a small number of extremely negative messages that were
spread out over approximately three years (i.e. 2.87 years). The common theme of these
messages was brutalities of being a prisoner at Guantanamo Bay, with an emphasis on
the Islamic religion under attack by Western Nations. As the user noted,
…everyone knows 9/11 was one of the biggest attacks to be attributed to the Al-Qaida
network against America and it started the […] Global war against Islam […] The beginning
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 13
of that was marked by detaining Muslims, detaining those who practice their religion, have
large amounts of money in their bank accounts, or who have travelled to certain Muslim
countries.
This finding suggests that the extreme nature of the messages was not short lived, nor did
it display unusual sentiment for this particular author; rather, this author showed a con-
sistent pattern of extremely radical online discourse and a high level of dedication to
extremist beliefs.
The second most radical user across all discussion forums, O.M.T., was also the most
radical user on Gawaher. Unlike Prism,O.M.T.’s overall average posting score was not
the most negative among those who also received the highest radical scores across
forums (−1.02), nor was the severity of the user’s negative posts noticeably high (i.e.
0.6% of all posts were very negative). What differentiated O.M.T. from all other authors
across all forums, as well as those who received the highest radical scores across
forums, was the volume of negative messages and the length of time in which these mess-
ages were posted (i.e. 72.4% of 2874 posts were negative over a 6.69-year period). This
user was among the most prolific posters across all discussion forums, and although
these messages were not classified as the most negative across all forums, the volume
of mildly negative messages was constant. For example, O.M.T. typically posted messages
that were classified as moderately negative, such as: ‘it dont matter which non muslim
country gonna respond. the chain reaction gonna lead to doomsday [sic]’or ‘…whats
the “hidden”agenda for superpower usa […] invade irag followed by afghanistan and
now possibly eyeing at iran …[sic].’
Focusing on the top 0.1% and 1% of the most radical users in the sample of 26,171 indi-
viduals across the four discussion forums (i.e. roughly 27 users and 262 users, respectively),
the results indicated that two of the four discussion forums hosted substantially more
radical users. Users who scored the highest ‘radical score’were active users in Gawaher
and Islamic Awakening (See Table 2). While the overall volume of most radical users
varied from 0.1% of the most radical users in the web-forums to 1% of the most radical
users in the web-forums, the results suggested that Gawaher and Islamic Awakening
hosted the highest volume of the most radical users in the sample, and at considerably
higher rates than the Islamic Network and Turn to Islam forums.
Table 1. Most radical authors across four Islamic discussion forums.
Author Web-forum
Negative
posting period
(years)
Total
number of
posts
Average posting
score (sentiment
value)
Negative
posts (%)
Very
negative
posts (%)
Radical
score
Prism Islamic
Awakening
2.87 139 −4.47 97.1 23.9 39.028
O.M.T. Gawaher 6.69 2874 −1.02 72.4 0.6 38.934
Nash Islamic
Awakening
5.69 425 −1.05 70.3 1.2 38.876
Ray Gawaher 3.35 524 −1.29 75 1.1 38.823
3D Gawaher 3.94 28 −2.21 85.7 3.6 38.790
Harley Gawaher 6.28 1525 −0.88 64.0 0.3 38.652
JHJ Islamic
Awakening
2.76 423 −1.27 74.1 0.7 38.625
Lee Islamic
Awakening
2.26 2770 −1.47 76.8 0.7 38.620
prism Islamic
Network
2.11 207 −2.36 74.9 4.3 38.601
14 R. SCRIVENS ET AL.
Discussion
Jihadists are a tech-savvy group who continue to show their presence in online discussion
forums. Uncovering the indicators of these individuals is of utmost importance for law
enforcement and government officials. As the Internet continues to play a major role in
connecting those who subscribe to radical beliefs (Sageman, 2008; Weimann, 2006),
guided data analysis methods –on a large scale that is –are necessary to detect these
users of interest (Brynielsson et al., 2012; Cohen et al., 2014). This study proposed a
method to identify the radical users across approximately 1 million posts found on four
Islamic-based discussion forums. By no means does this study or its results imply that
SIRA can detect radical users who may engage in an act of violent extremism. Rather,
the purpose of this project was to attempt to develop an innovative technique to
measure the online behavior of radical users on selected web-forums.
A few notable results were found. First, the findings do not highlight a particular typol-
ogy that can be used to define the most radical users within a web-forum, nor did we
uncover specific patterns that categorize specific activity as most radical. Although this
research is in its preliminary stages, the results of this study indicate that the process of
identifying the most radical users within a discussion forum must encompass a variety
of key elements. The SIRA algorithm for ‘radical score’is flexible enough to evaluate
several combinations of online sentiment activity that may be used to detect the most
radical users in discussion forums.
Second, the same author was identified across two separate discussion forums as the
most radical. Prism of Islamic Awakening and prism of Islamic Network were deemed the
most radical users within their given discussion forum. Furthermore, this user represented
a very controversial Islamic rights group website that described itself as an advocacy group
campaigning against the ‘War on Terror.’It is headed by former Guantanamo Bay detainee
Moazzam Begg, and it has a history of supporting Islamic extremists (Murray & Simcox,
2014). The site has also been described as fraudulent attempt to promote human rights,
and a front for Taliban enthusiasts and Al-Qaida devotees (Galvin, 2010).
Third, the Islamic Network and Turn to Islam discussion forums contained the fewest
radical authors in the sample. This is not surprising, given that Islamic Network is known
to host only a small number of registered members who sympathize and support Islamist
terrorist organizations, and registered members of Turn to Islam only occasionally show
support for fundamentalist militant groups (ISI-ICDM Workshop on Intelligence and
Security Informatics Challenge, 2015). In contrast, the Gawaher and Islamic Awakening
web-forums contained the most radical users across all four web-forums. Gawaher, for
example, is known to host individuals who sympathize with radical Islamic groups, such
as a 23-year-old man accused of attempting to detonate plastic explosives on an American
airliner in 2009. Prior to the event, Amar Abdulmutallab posted over 300 messages on
Table 2. 0.1% and 1% of most radical authors across four Islamic discussion forums.
Web-forum 0.1% of most radical authors in the sample (%) 1% of most radical authors in the sample (%)
Gawaher 48.15 38.93
Islamic Awakening 37.04 45.8
Islamic Network 11.11 6.11
Turn to Islam 3.7 9.16
Total 100 100
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 15
Gawaher, highlighting his inner struggle with liberalism and extremism (Rucker & Tate,
2009). A number of users on this forum also discuss very controversial issues. A study of
29 international Jihadist forums on the Web found that Gawaher included more discus-
sions around keywords ‘bomb,’‘Iraq,’and ‘kill’than any other web-forum (Zhang et al,
2010).
Islamic Awakening is also known to host a number of extremely radical individuals who
favour of the death penalty, and openly deem the following crimes as grounds for capital
punishment: (1) homosexuality; (2) adultery; and (3) discrediting Muhammad, the final
prophet of Allah (God), to name a few. According to Jihad Watch, the site is known as a
Jihadist discussion forum, offering viewers with ‘Pro-Qaida news’(Spencer, 2004) and
the Azzam news bulletins, a prominent source of information on Jihadi extremist activities
around the world (Nauta, 2013). It comes as little surprise, then, that the Gawaher and
Islamic Awakening web-forums were virtual conduits for the most radical users within
the sample.
Limitations and future research
Combining sentiment analysis tools and the SIRA algorithm appears to be a useful way of
identifying radical users –or, at the very least, users of interest to counter-extremism
agencies –in the selected discussion forums. However, this study is not without limit-
ations. First, work on the SIRA algorithm remains in the early stages; it cannot detect or
even predict the online sentiment that will (or may) result in an act of terror. While this
particular identification exercise was not the intent of this study, future research should
assess this critical point, perhaps by taking into account if a user is active during the
time that the data is captured. Focusing on ‘currently active’authors may highlight how
our strategy can be used to identify specific users who are of interest to counter-extremist
agencies. In other words, future research should incorporate a temporal analysis, examin-
ing how users’radical scores change during their time on a forum. This strategy could
involve splitting the entire dataset into clusters of months and calculating each user’s
radical score for that month. From here, sharp changes in scores, especially ones that
are most current, could be interpreted as possible changes in users’motivation or level
of ‘extremism’or negatively. This could help identify authors of interest and minimize
the possible threat posed by violent Islamists. Further, SIRA could also be used to identify
very negative threads on the web-forums by aggregating the data to thread level, and not
the author level.
Most importantly, an evaluation of the SIRA algorithm and the abovementioned
machine learning tools is needed. This paper describes a technique that may be used
to identify radical users on the forum, but it still begs the question of whether the high-
lighted users in the study are, in fact, the most radical users on the selected forums. In an
effort to evaluate the machine learning tools used in the study, for example, we offer a
number of closing thoughts. First, the sentiment found on the web-forums could be
measured using a wider list of keywords or narrower lists of keywords than our initial
approach. For the former, an analysis of online text via an extensive list of keywords
would cast a wide net, thus providing additional points of analysis. Theoretically, more
online posts could be scored if more keywords are included in the model for an assess-
ment by the sentiment analysis tool. On the other side of the spectrum, the latter
16 R. SCRIVENS ET AL.
approach would be a useful way of providing a more precise measure of ‘radical’behav-
ior based on a users’online sentiment, which could be done by developing lexicon
models build around particular themes (e.g. religion, immigration, family, etc.). Second,
a combination of sentiment analysis tools could be integrated into the analysis, in an
attempt to cross-validate each other. Third, in an effort to understand whether all four
components of SIRA are a suitable means of detecting radical users based on their
online behavior, the four unique dimensions of ‘seriousness’to identify such individuals
could be reweighed, rather than having each component make up an equal proportion
of the overall radical score. Doing so would be a useful way of evaluating whether one
component (or multiple components) is a more appropriate technique of evaluating a
users’online behavior. This could be done by running multiple analyses to compare
and contrast the reweighted models, as well as test for correlation or multi-collinearity
between models.
Notes
1. Data were originally extracted using an automated forum spidering tool by the University of
Arizona Artificial Intelligence Lab (AI Lab), and it was assumed that the four discussion forums
were extracted in their entirety (for more information on the data extraction process, see
Chen, 2012). These data were then provided as part of an ‘IEEE Workshop on Intelligence
and Security Informatics (ISI-ICDM, 2015) Challenge,’which included more than 1 million
messages from a total of 4 Dark Web discussion forums (i.e. Gawaher,Islamic Awakening,
Islamic Network, and Turn to Islam). Here participants were challenged to ‘find the more
radical and infectious threads, members, postings, ideas and ideologies’by developing a
‘novel computational techniques and algorithms, e.g., linguistic analysis, topic extraction,
multilingual text parsing, sentiment analysis, social network analysis, time-series analysis,
etc.’(ISI-ICDM Workshop on Intelligence and Security Informatics Challenge, 2015, p. 1).
Each of the four discussion forums from the ‘Dark Web Forums Collection’was included in
our sample and analysis. All discussion forums were in English. For more information, visit
http://cci.drexel.edu/isi/isi-icdm2015/challenge.html.
2. For more information, visit gawaher.com.
3. For more information, visit forums.islamicawakening.com.
4. This particular forum could not be found during the time of the analysis. It may be the case
that this particular forum is located in the deep web.
5. For more information, visit turntoislam.com.
6. Sentiment scores were assigned to each post, based on the keyword list of 400 nouns.
However, two nouns and their variations were removed from the list because their values
were extreme outliers (i.e. admin, admins, and passport, passports), and scores for the remain-
ing 398 keywords were averaged across each post. Each post was assigned a sentiment score
that was the average across all of the keywords found in that particular post.
7. We did not attempt to match the identification number of each member across each dataset.
8. The average sentiment score was a negative value (e.g. 0.28 = −0.28). We multiplied this
number by negative 1 to make the direction more intuitive, as well as for mathematical
convenience.
9. Users’discussion posts were manually selected. The purpose of this was to illustrate the nega-
tive nature of their sentiment.
Disclosure statement
No potential conflict of interest was reported by the authors.
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 17
Notes on contributors
Ryan Scrivens is a PhD student in the School of Criminology at Simon Fraser University and a
Research Associate with the International CyberCrime Research Centre. He is also the Coordinator
of the Canadian Network of PhD Theses Writers for the Terrorism Research Initiative.
Garth Davies is an Associate Professor in the School of Criminology at Simon Fraser University. He is
also Co-Coordinator of the Terrorism, Risk, and Security Studies Professional Master’s Program.
Richard Frank is an Assistant Professor in the School of Criminology at Simon Fraser University and
Associate Director of the International CyberCrime Research Centre. He holds a PhD in Computing
Science and another in Criminology.
References
Abbasi, A., & Chen, H. (2005). Applying authorship analysis to extremist-group web forum messages.
Intelligent Systems,20(5), 67–75.
Amble, J. C. (2012). Combating terrorism in the new media environment. Studies in Conflict and
Terrorism,35(5), 3–10.
Apache Software Foundation. (2015). Welcome to apache OpenNLP. Retrieved from https://opennlp.
apache.org/
Back, L. (2002). Aryans reading Adorno: Cyber-culture and twenty-first century racism. Ethnic and
Racial Studies,25(4), 628–651.
Bermingham, A., Conway, M., McInerney, L., O’Hare, N., & Smeaton, A. F. (2009). Combining social
network analysis and sentiment analysis to explore the potential for online radicalisation.
Proceedings of the 2009 international conference on advances in Social Network Analysis
Mining, Athens, Greece, pp. 231–236.
Bouchard, M., Joffres, K., & Frank, R. (2014). Preliminary analytical considerations in designing a ter-
rorism and extremism online network extractor. In V. Mago & V. Dabbaghian (Eds.), Computational
models of complex systems (pp. 171–184). New York, NY: Springer.
Bouchard, M., & Nash, R. (2015). Researching terrorism and counter-terrorism through a network lens.
In M. Bouchard (Ed.), Social networks, terrorism and counter-terrorism: Radical and connected
(pp. 48–60). New York, NY: Routledge.
Bowman-Grieve, L. (2009). Exploring “stormfront:”A virtual community of the radical right. Studies in
Conflict and Terrorism,32(11), 989–1007.
Bowman-Grieve, L. (2013). A psychological perspective on virtual communities supporting terrorist &
extremist ideologies as a tool for recruitment. Security Informatics,2(9), 1–5.
Brown, I., & Korff, D. (2009). Terrorism and the proportionality of internet surveillance. European
Journal of Criminology,6(2), 119–134.
Brynielsson, J., Horndahl, A., Johansson, F., Kaati, L., Martenson, C., & Svenson, P. (2012). Analysis of
weak signals for detecting lone wolf terrorist. Proceedings of the European intelligence and security
informatics conference, Odense, Denmark, pp. 197–204.
Burnap, P., Williams, M. L., Sloan, L., Rana, O., Housley, W., Edwards, A., …Voss, A. (2014). Tweeting the
terror: Modelling the social media reaction to the Woolwich terrorist attack. Social Network Analysis
and Mining,4,1–14.
Campana, A., & Ducol, B. (2014). Voices of the “Caucasus emirate”: Mapping and analyzing
north Caucasus insurgency websites. Terrorism and Political Violence.doi:10.1080/09546553.
2013.848797
Chalothorn, T., & Ellman, J. (2012). Using SentiWordNet and sentiment analysis for detecting radical
content on web forums. Proceedings of the 6th conference on software, knowledge, information
management and application, Chengdu, China, pp. 9–11.
Chen, H. (2012). Dark Web: Exploring and data mining the dark side of the web. New York, NY:
Springer.
Cohen, K., Johansson, F., Kaati, L., & Mork, J. (2014). Detecting linguistic markers for radical violence in
social media. Terrorism and Political Violence,26(1), 246–256.
18 R. SCRIVENS ET AL.
Conway, M. (2006). Terrorism and the internet: New media–new threat? Parliamentary Affairs,59(2),
283–298.
Conway, M., & McInerney, L. (2008). Jihad video and auto-radicalisation: Evidence from an exploratory
YouTube study. In D. Ortiz-Arroyo, H. L. Larsen, & D. Zeng (Eds.), Intelligence and security infor-
matics: European conference, EuroISI 2008 (pp. 108–118). New York, NY: Springer.
Davies, G., Bouchard, M., Wu, E., Joffres, K., & Frank, R. (2015). Terrorist and extremist organizations’
use of the internet for recruitment. In M. Bouchard (Ed.), Social networks, terrorism and counter-ter-
rorism: Radical and connected (pp. 105–127). New York, NY: Routledge.
Ducol, B. (2012). Uncovering the French-speaking jihadisphere: An exploratory analysis. Media, War &
Conflict,5(1), 51–70.
Feldman, R. (2013). Techniques and applications for sentiment analysis. Communications of the ACM,
56(4), 82–89.
Ferraro, J., Daumé, H., Duvall, S., Chapman, W., Harkema, H., & Haug, P. (2013). Improving perform-
ance of natural language processing part-of-speech tagging on clinical narratives through
domain adaptation. American Medical Information Association,20(5), 931–939.
Frank, R., Bouchard, M., Davies, G., & Mei, J. (2015). Spreading the message digitally: A look into extre-
mist content on the internet. In R. G. Smith, R. C.-C. Cheung, & L. Y.-C. Lau (Eds.), Cybercrime risks
and responses: Eastern and western perspectives (pp. 130–145). London: Palgrave Macmillian.
Fu, T., Abbasi, A., & Chen, H. (2010). A focused crawler for Dark Web forums. Journal of American
Society for Information Science and Technology,61(6), 1213–1231.
Galvin, T. (2010, February 8). Amnesty international doubles down on appeasement. The National
Post. Retrieved from http://oped.ca/National-Post/terry-glavin-amnesty-international-doubles-
down-on-appeasement/
Gawaher. (2015). Islamic forum. Retrieved from http://www.gawaher.com
Ghiassi, M., Skinner, J., & Zimbra, D. (2013). Twitter brand sentiment analysis: A hybrid system using
n-gram analysis and dynamic artificial neutral network. Expert Systems with Applications,40(16),
6266–6282.
Hegghammer, T. (2014). Interpersonal trust on Jihadi internet forums. Retrieved from http://
hegghammer.com/_files/Interpersonal_trust.pdf
Hu, M., & Bing, L. (2004). Mining and summarizing customer reviews. Proceedings of the ACM SIGKDD
international conference on knowledge discovery and data mining, Seattle, Washington.
Internet Live Stats. (2016). Total number of websites. Retrieved from http://www.internetlivestats.com/
total-number-of-websites/
Internet World Stats. (2016). Internet growth statistics. Retrieved from http://www.internetworldstats.
com/emarketing.htm
ISI-ICDM Workshop on Intelligence and Security Informatics Challenge. (2015). Challenge. Retrieved
from http://cci.drexel.edu/isi/isi-icdm2015/challenge.html
Islamic Network. (2015). Our mission. Retrieved from http://www.islamicnetwork.net/our-mission/
Kennedy, J., & Weimann, G. (2011). The strength of weak terrorist ties. Terrorism and Political Violence,
23(2), 201–212.
Klausen, J. (2015). Tweeting the jihad: Social media networks of western foreign fighters in Syria and
Iraq. Studies in Conflict and Terrorism,38(1), 1–22.
Klausen, J., Barbieri, E. T., Reichlin-Melnick, A., & Zelin, A. Y. (2012). The YouTube jihadists: A
social network analysis of Al-Muhajiroun’s propaganda campaign. Perspectives on Terrorism,6(1),
36–53.
Meddaugh, P. M., & Kay, J. (2009). Hate speech or ‘reasonable racism?’The other in stormfront.
Journal of Mass Media Ethics,24(4), 251–268.
Mei, J., & Frank, R. (2015). Sentiment crawling: Extremist content collection through a sentiment analysis
guided web-crawler. Proceedings of the international symposium on foundations of open source
Intelligence and Security Informatics, Paris, France, pp. 1024–1027.
Murray, D., & Simcox, R. (2014, July 21). The evidence shows that cage is a pro-terrorist group. The
Telegraph.Retrieved from http://www.telegraph.co.uk/news/uknews/law-and-order/10981050/
The-evidence-shows-that-Cage-is-a-pro-terrorist-group.html
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 19
Nauta, A. (2013). Radical Islam, globalisation and social media: Martyrdom videos on the internet. In
M. Gillespie, D. E. J. Herbert, & A. Greenhill (Eds.), Social media and religious change (pp. 121–142).
Hawthorne, NY: Walter de Gruyter.
Neumann, P., & Rogers, B. (2007). Recruitment and mobilisation for the Islamist Militant Movement in
Europe. King’s College London for the European Commission (Directorate General Justice,
Freedom and Security).
Reid, E., & Chen, H. (2007). Internet-savvy U.S. and Middle East extremist groups. Mobilization: An
International Quarterly Review,12(2), 177–192.
Reid, E., Qin, J., Zhou, Y., Lai, G., Sageman, M., Weimann, G., & Chen, H. (2005). Collecting and analyzing
the presence of terrorists on the web: A case study of jihad websites. Proceedings of the IEEE inter-
national conference on intelligence and security informatics, Atlanta, GA.
Rogan, A. (2006). Jihadism online –a study of how Al-Qaida and radical Islamist groups use the internet
for terrorist purposes. FFI Rapport.
Rucker, P., & Tate, J. (2009, December 29). In online posts apparently by Detroit suspect, religious
ideals collide. The Washington Post. Retrieved from http://www.washingtonpost.com/wp-dyn/
content/article/2009/12/28/AR2009122802492.html
Sageman, M. (2008). Leaderless jihad: Terror networks in the twenty-first century. Philadelphia:
University of Pennsylvania Press.
Sageman, M. (2014). The stagnation in terrorism research. Terrorism and Political Violence,26(4),
565–580.
Schafer, J. A. (2002). Spinning the web of hate: Web-based propagation by extremist organization.
Journal of Criminal Justice and Popular Culture,9(2), 69–88.
Seib, P., & Janbek, D. M. (2011). Global terrorism and new media: The post-al Qaeda generation.
London: Routledge.
Spencer, R. (2004, March 20). Jihad at Islamicawakening.com. Jihad Watch. Retrieved from http://
www.jihadwatch.org/2004/03/jihad-at-islamicawakeningcom
Stevens, T., & Neumann, P. R. (2009). Countering online radicalisation: A strategy for action. London:
International Centre for the Study of Radicalisation and Political Violence.
Thelwall, M., & Buckley, K. (2013). Topic-based sentiment analysis for the social web: The role of mood
and issue-related words. Journal of the American Society for Information Science and Technology,64
(8), 1608–1617.
Thet, T., Na, J., & Khoo, C. (2010). Aspect-based sentiment analysis of movie reviews on discussion
board. Journal of Information Science,36(6), 823–848.
Thiesmeyer, L. (1999). Racism on the web: Its rhetoric and marketing. Ethics and Information
Technology,1(2), 117–125.
Thomas, T. L. (2003). Al Qaeda and the internet: The danger of “cyberplanning”.Parameters,33(1),
112–123.
Torres-Soriano, M. R. (2013). The dynamics of the creation, evolution, and disappearance of terrorist
internet forums. International Journal of Conflict and Violence,7(1), 164–178.
Tsfati, Y., & Weimann, G. (2002). www.terrorism.com: Terror on the internet. Studies in Conflict and
Terrorism,25(5), 317–332.
Turn to Islam. (2015a). About us. Retrieved from http://turntoislam.com/about/
Turn to Islam. (2015b). Salamyou Muslim social network. Retrieved from http://turntoislam.com
Weimann, G. (2004). www.terror.net: How modern terrorism uses the internet (United States Institute of
Peace (USIP) Special Report, 116, pp. 1–12).
Weimann, G. (2006). Virtual disputes: The use of the internet for terrorist debates. Studies in Conflict
and Terrorism,29(7), 623–639.
Weimann, G. (2008). The psychology of mass-mediated terrorism. American Behavioral Scientist,52(1),
69–86.
Weimann, G. (2010). Terror on Facebook, Twitter, and YouTube. Brown Journal of World Affairs,16(2),
45–54.
Whine, M. (1999). Cyberspace –a new medium for communication, command, and control by extre-
mists. Studies in Conflict and Terrorism,22(3), 231–245.
20 R. SCRIVENS ET AL.
Zhang, Y., Zeng, S., Huang, C.-N., Fan, L., Yu, X., Dang, Y., …Chen, H. (2010). Developing a Dark Web
collection and infrastructure for computational and social sciences. Proceedings of the 2010 IEEE
international conference on Intelligence and Security Informatics, Atlanta, GA.
Zhou, Y., Qin, J., Lai, G., Reid, E., & Chen, H. (2005). Building knowledge management system for
researching terrorist groups on the web. Proceedings of the 11th Americas conference on infor-
mation systems, Omaha, NE.
BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 21
