Conference PaperPDF Available

The role of sarcasm in hate speech. A multilingual perspective


The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior online. For this reason, hate speech online is a real problem in modern society and the necessity of control of user-generated contents has become one of the priorities for governments, social media platforms and Internet companies. Current methodologies are far from solving this problem. Indeed, several aggressive comments are also disguised as sarcastic. In this perspective, this research proposal wants to investigate the role played by creative linguistic devices, especially sarcasm, in hate speech in multilingual context.
The role of sarcasm in hate speech.
A multilingual perspective
La funci´on del sarcasmo en los discursos de odio.
Una perspectiva multiling¨ue
Simona Frenda1 2
1PRHLT Research Center, Universitat Polit`ecnica de Val`encia, Spain
2Dipartimento di Informatica, Universit`a degli Studi di Torino, Italy
Abstract: The importance of the detection of aggressiveness in social media is due
to real effects of violence provoked by negative behavior online. For this reason, hate
speech online is a real problem in modern society and the necessity of control of user-
generated contents has become one of the priorities for governments, social media
platforms and Internet companies. Current methodologies are far from solving this
problem. Indeed, several aggressive comments are also disguised as sarcastic. In this
perspective, this research proposal wants to investigate the role played by creative
linguistic devices, especially sarcasm, in hate speech in multilingual context.
Keywords: Hate speech, social media, aggressiveness, misogyny, sarcasm
Resumen: La importancia del reconocimiento de la agresividad en las redes so-
ciales es debido al hecho que esas conductas negativas se traducen en violencias en
la vida real tambi´en. Por esa raz´on los discursos de odio online son un problema
real en nuestra sociedad y la necesidad del control de los contenidos generados por
usuarios se ha convertido en una de las prioridades de gobiernos, de las redes so-
ciales y de empresas de Internet. Las metodolog´ıas corrientes est´an lejos de resolver
este problema. De hecho gran parte de los comentarios agresivos son disfrazados
como sarcasticos. En esta perspectiva, esta propuesta de investigaci´on propone de
estudiar la funci´on de las figuras ret´oricas, con particular atenci´on al sarcasmo, en
los discursos de odio en un contexto multiling¨ue.
Palabras clave: Discursos de odio, redes sociales, agressividad, misoginia, sarcasmo
1 Introduction
The web facilitates the large resonance of
hate speech, inciting racism, misogyny or
xenophobia also in the real world. Actually,
it is common that misbehaviours online are
traduced in physical attacks, such as rapes
or bulling. For instance, Fulper et al. (2014)
demonstrated the existence of a correlation
between the number of rapes and the amount
of misogynistic tweets per state in USA, sug-
gesting the fact that social media can be used
as a social sensor of violence.
In addition, the persistence and diffu-
sion of misogynistic or offensive content can
hurt and distress psychologically the victims,
causing sometime their suicide, such as the
case of the teenager Amanda Todd in 20121.
In order to contrast the origin of these hate
events and to monitor the uncontrolled flow
of users texts, several initiatives have been
taken in the last years. An example is the
campaign No Hate Speech Movement2of the
Council of Europe for human rights online.
The growing interest of NLP (Natural
Language Processing) research community is
demonstrated by the proposal of national
and international workshops (such as ALW
20183) or campaigns of evaluation fostering
the research in this issue in various languages,
such as EvalIta 20184, IberEval 20185and Se-
mEval 20196. These initiatives allow to share
Lloret, E.; Saquete, E.; Mart´ınez-Barco, P.; Moreno, I. (eds.) Proceedings of the Doctoral Symposium of the XXXIV
International Conference of the Spanish Society for Natural Language Processing (SEPLN 2018), p. 1317 Sevilla, Spain,
September 19th 2018. Copyright c
2018 by the paper’s authors. Copying permitted for private and academic purposes.
information and results exploring the differ-
ent topics regarding the hate speech online.
As well as, the organizers of these compe-
titions provide resources such as annotated
datasets that are very costly to obtain.
The fact that the majority of data are col-
lected from Twitter or Facebook supports the
analysis of the computer-mediated communi-
cation. As well as, the context of short text
incites the creativity of authors who use fig-
urative devices to express their opinion. One
of the most used figures of speech to manifest
negative opinions is the sarcasm. In fact, it
is used to disguise and, at the same time, to
reinforce the negative thinking, such as:
i) Un pensiero di ringraziamento ogni mat-
tina va sempre ai comunisti che ce li
hanno portati fino a casa musulmani
rom e delinquenti grazie7.
The ironic sharpness of the sarcasm seems
to be appropriated to express contempt and
to offend individuals subtly. In order to
study this correlation between sarcasm and
hate speech, we proposed the shared task
IronITA8at Evalita 2018 that asks partici-
pants to recognize ironic and sarcastic tweets
in a dataset containing also offensive mes-
sages addressed, especially, immigrants (San-
guinetti et al., 2018).
Moreover, we participated in two tasks
proposed at IberEval 2018 about hate speech:
aggressiveness detection in Mexican Spanish
tweets (MEX-A3T)9organized by ´
Carmona et al. (2018) and identification
of misogynistic English and Spanish tweets
(AMI)10 organized by Fersini, Anzovino, and
Rosso (2018). As a confirmation of our in-
tuition, the systems proposed for these tasks
show some difficulties to classify the sarcas-
tic abusive tweets. Indeed, sarcasm, inde-
pendently from the differences between lan-
guages, disguises the real intention of the
message which is with difficulty recognized
by machine. In line with these early experi-
ments, IronITA could be a good step of anal-
7Each morning, I would like to thank communists
who bring home musulmans, roms and delinquents
thanks. Tweet from IronITA corpus.
The rest of the paper is structured as
follows. Section 2 introduces the literature
that inspired our investigation. Section 3
describes our participation in IberEval tasks
with the used approach and obtained results.
In Section 4 we analyze the presence of sar-
casm in analyzed aggressive and offensive
texts. Finally, in Section 5 and 6 we draw
our research proposal and the future work.
2 Related work
The literature about hate speech detection
includes different issues, such as: cyberbul-
lying, misogyny, nastiness and aggressive-
ness. The most commercial methods, cur-
rently, rely on the use of blacklists. However,
filtering the messages in this way does not
provide a sufficient remedy because it falls
short when the meaning is more subtle or al-
tered by sarcasm. Actually, some authors,
such as Justo et al. (2014) and Nobata et
al. (2016), underline the fact that sarcasm
makes the interpretation of the message dif-
ficult, generally requiring world knowledge.
Also Smokey, one of the first systems, im-
plemented by Spertus (1997), uses syntactic
and semantic rules with lexicons to recognize
In this context, the research is ori-
ented at investigating deeply the language
using classical (Samghabadi et al., 2017)
and deep learning methods (Del Vigna et
al., 2017). Differently from Mehdad and
Tetreault (2016) and Gamb¨ack and Sikdar
(2017), for MEX-A3T task in Frenda and
Banerjee (2018) we applied an experimental
technique that combines linguistic features
and Convolutional Neural Network (CNN).
For the first time, Anzovino, Fersini, and
Rosso (2018) propose a classical machine
learning approach to identify misogyny in
English, comparing different classifiers. Tak-
ing into account this previous work and the
psychological studies about sexism (Ford and
Boxer, 2011), in Frenda and Ghanem (2018)
we combined sentiment and stylistic informa-
tion with specific lexicons involving several
aspects of misogyny online.
In the following section we report how we
addressed the identification of aggressiveness
and misogyny in Twitter, the experiments
carried out and the results obtained.
3 Hate speech, aggressiveness
and misogyny
Considering our motivations, our early
experiments focus mainly on hate speech
detection. For this purpose, we participated
at two tasks at IberEval 2018 respectively
about aggressiveness and misogyny detec-
3.1 Aggressiveness detection
The first task aims to classify aggressive and
non-aggressive tweets in Mexican Spanish.
We applied a deep learning approach incor-
porating into CNN architecture a set of lin-
guistic features (DL+FE) concerning: proper
characteristics of a tweet, such as emoticons,
abbreviations and slang words; stylistic infor-
mation, such as the length of tweets, the use
of the punctuation and the uppercase charac-
ters; bags of words weighted with tf-idf; emo-
tive traits of the aggressiveness; and deroga-
tory adjectives and vulgar expressions typical
of Mexican culture.
By means of Information Gain, we no-
ticed that anger and disgust are the princi-
pal emotions that incite the aggressive be-
haviour. We compared this system with a
simple CNN architecture (DL) in order to un-
derstand the contribution of features to deep
learning approach. The measure used for the
competition is F-score for positive class (i.e.
aggressive class). Despite the novel approach,
the results obtained are low and the features
seem not to help deep learning, as showed in
Table 1.
Prec. Rec. F-pos Rank
DL 0.34 0.34 0.34 9
DL+FE 0.27 0.38 0.31 10
Table 1: Results for aggressiveness detection
Therefore, in order to understand what
are the difficulties of DL+FE, we carried
out the error analysis. We mainly noticed
that there are several humorous cases,
especially sarcastic (see Section 4), which
are misclassified.
3.2 Automatic misogyny
The second task proposes to identify misog-
yny in two collection of English and Span-
ish tweets. In the case a tweet is classified
as misogynistic (Task A), we need to distin-
guish (Task B) if the target is an individual
or not (Tar.) and identify the type of misog-
yny, according to the following classes (Cat.):
stereotype and objectification, dominance,
derailing, sexual harassment and threats of
violence, and discredit. This subdivision of
misogyny allows us to explore the different
aspects of misogyny and compare them in
two different languages. Moreover, the data
are not geolocalized. Therefore, in order to
gather the linguistic variations and consider
the various traits of misogyny, we proposed
an approach based on stylistic features cap-
tured by means of the character n-grams, sen-
timent and affective information, and on a set
of lexicons concerning: sexuality, profanity,
femininity, human body and stereotypes. In
addition, we considered slangs, abbreviations
and hashtags.
By means of Information Gain, we discov-
ered some differences between the two lan-
guages: sexual language is more used in En-
glish misogynistic tweets, whereas profanities
or vulgarities are more used in Spanish ones.
For this task, we applied Support Vector Ma-
chine (SVM) and majority voting technique.
To evaluate the Task A the organizers used
Accuracy measure and for Task B the average
Macro-F1 measure. In Table 2 and Table 3
we report the promising results obtained with
better runs for both languages.
Approach Acc Rank
En Ensemble 0.87 2
Sp Ensemble 0.81 3
Table 2: Results for Task A of misogyny de-
Approach F1 Cat. Tar. Rank
En SVM 0.44 0.29 0.59 1
Sp Ensemble 0.44 0.33 0.55 2
Table 3: Results for Task B of misogyny de-
4 Sarcasm
In Trait´e des tropes (1729) Dumarsais has de-
fined the sarcasm as an ironie faite avec ai-
greur et emportement11, that is a kind of ag-
gressive and sharp irony addressed a target to
hurt or criticize him without to exclude the
possibility to amuse. This statement is cor-
roborate by our analyses on English, Spanish,
Mexican and Italian hate speech corpora. As
said above, we carried out the error analysis
for both tasks.
In the first competition we noticed that
our approach fails in the classification of sar-
castic aggressive utterances, such as:
ii) @USUARIO #LOS40MeetAndGreet 9 .
Por q es una mam´a luchona que cuida a
su bendici`on12.
Actually, the sarcasm is a type of figurative
devices that modifies the perception of mes-
sage, hindering the correct detection of hate
speech by automatic systems. We found, in
fact, the same difficulty for the recognition of
misogynistic tweets in both languages, such
iii) ¿Cu´al es la peor desgracia para una mu-
jer? Parir un var´on, porque despu´es de
tener un cerebro dentro durante 9 meses,
van y se lo sacan13;
iv) What’s the difference between a blonde
and a washing machine? A washing ma-
chine won’t follow you around all day af-
ter you drop a load in it.
In virtual as in real life, sexist jokes are
very common. In general, they are considered
innocent by the majority of people. How-
ever, Ford and Boxer (2011) reveal that sex-
ist jokes are experienced by women as sex-
ual harassment as well as offences. Moreover,
Ford, Wentzel, and Lorion (2001) investigate
on the effects of exposure to sexist jokes and
they underline that a continue exposition can
also modify the perception of sexism as norm
and not as misbehavior.
5 Research Proposal
These early observations suggest the neces-
sity to address the use of figures of speech
such as sarcasm, in order to accurate, in
11“type of irony done with sharpness and a fit of
12@User #LOS40MeetAndGreet 9 . Because she is
a fighter mother who takes care of her kid.
13What’s the worst disgrace for a woman? Giving
birth to boy, because after she has got a brain into her
for 9 months, it is taken out
a multilingual perspective, the automated
methods to flag abusive language.
For this purpose, we propose an accurate
analysis of different kinds of hate speech on-
line especially in Italian, English and Span-
ish, taking into account also the geographical
linguistic variations. We focus in particular
on short texts such as tweets, posts or com-
ments, exploring the informal language.
Considering the previous observations, we
propose approaching the hate speech detec-
tion issues taking into account the figurative
dimension of language and especially of abu-
sive language. Moreover, it is necessary to
examine the appropriateness of various com-
putational techniques to solve this problem.
In this line, we want to examine the contribu-
tion of the linguistic features to deep learning
approaches by comparison with the perfor-
mances of classical techniques. Finally, the
multilingual context allows to discover the
typical aspects of hate speech in order to rec-
ognize it independently from the languages.
Indeed, the scope of this investigation is to
propose a methodology for monitoring cor-
rectly the user-generated contents allowing
the system to work as sensor of the violence,
also in real world.
6 Future work
Our research aims to explore the several di-
mensions of hate speech considering, above
all, the use of figurative devices that hinder
the automatic processes of recognition. In
order to investigate the remarks observed in
these first experiments, as future work, we
would like to participate in HaSpeeDe14 and
AMI15 at Evalita 2018 for Italian.
In addition, similar tasks are proposed at
SemEval 2019 concerning: multilingual hate
speech against immigrants and women (Hat-
Eval)16, and the identification and catego-
rization of offensive language in social me-
dia (OffensEval)17. Analyzing different kinds
of abusive language allows to understand the
boundaries between them and their singular
aspects. Finally, multilingual context gives
us the opportunity to delineate the differ-
ences and analogies between the various lan-
guages, inferring general characteristics of
hate speech online.
Alvarez-Carmona, M. ´
A., E. Guzm´an-Falc´on,
M. Montes-y G´omez, H. J. Escalante,
L. Villase˜nor-Pineda, V. Reyes-Meza, and
A. Rico-Sulayes. 2018. Overview of
mex-3at at ibereval: Authorship and ag-
gressiveness analysis in mexican spanish
tweets. In Notebook Papers of 3rd SEPLN
Workshop on Evaluation of Human Lan-
guage Technologies for Iberian Languages
(IBEREVAL), Seville, Spain, September.
Anzovino, M., E. Fersini, and P. Rosso. 2018.
Automatic identification and classification
of misogynistic language on twitter. In In-
ternational Conference on Applications of
Natural Language to Information Systems,
pages 57–64. Springer.
Del Vigna, F., A. Cimino, F. Dell’Orletta,
M. Petrocchi, and M. Tesconi. 2017. Hate
me, hate me not: Hate speech detection on
facebook. In Proceedings of ITASEC17.
Fersini, E., M. Anzovino, and P. Rosso. 2018.
Overview of the task on automatic misog-
yny identification at ibereval. In Note-
book Papers of 3rd SEPLN Workshop on
Evaluation of Human Language Technolo-
gies for Iberian Languages (IBEREVAL),
Seville, Spain, September.
Ford, T. E. and C. F. Boxer. 2011. Sexist
humor in the workplace: A case of subtle
harassment. In Insidious Workplace Be-
havior. Routledge, pages 203–234.
Ford, T. E., E. R. Wentzel, and J. Lorion.
2001. Effects of exposure to sexist humor
on perceptions of normative tolerance of
sexism. European Journal of Social Psy-
chology, 31(6):677–691.
Frenda, S. and S. Banerjee. 2018. Deep
analysis in aggressive mexican tweets. In
Notebook Papers of 3rd SEPLN Work-
shop on Evaluation of Human Lan-
guage Technologies for Iberian Languages
(IBEREVAL), Seville, Spain, September.
Frenda, S. and B. Ghanem. 2018. Explo-
ration of misogyny in spanish and english
tweets. In Notebook Papers of 3rd SEPLN
Workshop on Evaluation of Human Lan-
guage Technologies for Iberian Languages
(IBEREVAL), Seville, Spain, September.
Fulper, R., G. L. Ciampaglia, E. Fer-
rara, Y. Ahn, A. Flammini, F. Menczer,
B. Lewis, and K. Rowe. 2014. Misogy-
nistic language on twitter and sexual vi-
olence. In Proceedings of the ACM Web
Science Workshop on ChASM.
Gamb¨ack, B. and U. K. Sikdar. 2017. Using
convolutional neural networks to classify
hate-speech. In Proceedings of the First
Workshop on Abusive Language Online,
pages 85–90.
Justo, R., T. Corcoran, S. M. Lukin,
M. Walker, and M. I. Torres. 2014. Ex-
tracting relevant knowledge for the detec-
tion of sarcasm and nastiness in the social
web. Knowledge-Based Systems, 69:124–
Mehdad, Y. and J. Tetreault. 2016. Do char-
acters abuse more than words? In Pro-
ceedings of the 17th Annual Meeting of the
Special Interest Group on Discourse and
Dialogue, pages 299–303.
Nobata, C., J. Tetreault, A. Thomas,
Y. Mehdad, and Y. Chang. 2016. Abu-
sive language detection in online user con-
tent. In Proceedings of the 25th interna-
tional conference on world wide web, pages
Samghabadi, N. S., S. Maharjan, A. Sprague,
R. Diaz-Sprague, and T. Solorio. 2017.
Detecting nastiness in social media. In
Proceedings of the First Workshop on
Abusive Language Online, pages 63–72.
Sanguinetti, M., F. Poletto, C. Bosco,
V. Patti, and M. Stranisci. 2018. An
italian twitter corpus of hate speech
against immigrants. In Proceedings of
the Eleventh International Conference
on Language Resources and Evaluation.
Spertus, E. 1997. Smokey: Auto-
matic recognition of hostile messages. In
AAAI/IAAI, pages 1058–1065.
... The proposed system, called AlBERToIS, integrates the knowledge of AlBERTo language model with the weights of linguistic features that aim to introduce stylistic, syntactic, and semantic information. A correct identification of irony and sarcasm is, indeed, crucial for the development of systems aware of irony and sarcasm, especially in hate speech detection (Frenda, 2018;Nobata et al., 2016) and sentiment analysis. In sentiment analysis, for example, Hernández Farías and Rosso (2017) underlined a significant gap between the performance of sentiment analysis systems on non-figurative content and the performance reached on sarcastic content. ...
... The detection of irony and sarcasm is gaining more and more interest in scientific communities and companies. In fact, it proves to be relevant in Sentiment Analysis for recognizing correctly the opinion or orientation of users about a specific subject (product, service, topic, issue, person, organization, or event) as well as on Hate Speech detection (Frenda, 2018;Nobata et al., 2016). Many have been the recent shared tasks on irony/sarcasm detection and figurative language in general: SENTIPOLC 2014 and 2016 subtask Irony detection in Italian tweets (Barbieri et al., 2016;, DEFT2017-Task2 Figurative language detection in French tweets (Benamara et al., 2017), SemEval2018-Task3 Irony detection in English tweets (Van Hee et al., 2018a) that asked participants to distinguish also among four categories of irony (irony by clash, situational irony, other verbal irony and non-irony), IroSvA2019 Irony Detection in Spanish Variants (Ortega-Bueno et al., 2019) where also the context was provided to understand to what ironic comments referred to, ALTA2019 shared task on Sarcastic Target Identification (Molla & Joshi, 2019), and, more recently, FigLang2020-Task2 Sarcasm Detection (Ghosh et al., 2020) focused on sarcastic texts identification in English conversations on Twitter and Reddit. ...
... Emotions and Hatred Another aspect previously investigated in irony and sarcasm detection is the contribution of emotional and sentiment information in various languages (Calvo et al., 2020;Hernández Farías et al., 2016) and in different contexts (Babanejad et al., 2020;Chauhan et al., 2020). With respect to hate information, the intuition about the use of sarcasm to disguise hateful and offensive utterances was preliminary investigated in Frenda (2018), Justo et al. (2014) and Nobata et al. (2016). In Justo et al. (2014) the authors showed differences and analogies in sarcasm and nastiness detection. ...
In the last decade, the need to detect automatically irony to correctly recognize the sentiment and hate speech involved in online texts increased the investigation on humorous figures of speech in NLP. The slight boundaries among various types of irony lead to think of irony as a linguistic phenomenon that covers sarcasm, satire, humor and parody joined by their trend to create a secondary or opposite meaning to the literal one expressed in the message. Although this commonality, in literature sarcasm is defined as a type of irony more aggressive with the intent to mock or scorn a victim without excluding the possibility to amuse. The aggressive tone and the intent of contempt suggest that sarcasm involves some peculiarities that make it a suitable type of irony to disguise negative messages. To investigate these peculiarities of sarcasm, we examined the dataset of the IronITA shared task. It consists of Italian tweets about controversial social issues, such as immigration, politics and other more general topics. Each tweet is annotated as ironic and non-ironic, and, at a deeper level, as sarcastic and non-sarcastic. Qualitative and quantitative analyses of the dataset showed how sarcasm tends to be expressed with hurtful language revealing the aggressive intention with which the author targets the victim. While irony is characterized by being offensive in hateful context and, in general, moved by negative emotions. For a better understanding of the impact of hurtful and affective language on the detection of irony and sarcasm, we proposed a transformer-based system, called AlBERToIS, combining pre-trained AlBERTo model with linguistic features. This approach obtained the best performances on irony and sarcasm detection on the IronITA dataset.
... Tweets that were more difficult to categorize, such as potentially sarcastic messages, were also discussed in coding meetings. Sarcasm can be difficult to detect onlineindeed it is often exploited to deny that particular statements are hate speech (Frenda, 2018) -so, for the purpose of our sample, we defined sarcasm as tweets that seemed to infer one thing but used explicit signifiers to show they meant the opposite. For instance, one of the most widely circulated tweets in our sample appeared to be a positive statement about Islam being a peaceful religion, but this was directly undermined by a meme underneath the tweet that associated Islam with violence. ...
... As detailed within an infamous New Yorker piece on the style guide of neo-Nazi site the Daily Stormer, for those posting on social media, users are instructed to include: 'as much visual stimulation as possible', to 'appeal to the ADHD culture', while passages from mainstream sources must be unaltered, so that 'we can never be accused of "fake news" -or delisted by Facebook as such ' (in Marantz, 2018). Echoing academic research that has foregrounded the tactical role of humour and sarcasm (Frenda, 2018), such approaches are encouraged as the 'unindoctrinated should not be able to tell if we are joking or not ' (in Marantz, 2018) in order to avoid censure while contributing to 'dehumaniz[ing]' rhetoric. These guidelines resonate with our analysis of negative comments that attacked the tweets of would-be allies, in which users drew on a fairly narrow and prescribed repertoire of action (oriented around the circulation of 'humorous' memes, statistics and hyperlinks). ...
Full-text available
This article sets out findings from a project focused on #stopIslam, a hashtag that gained prominence following the Brussels terror attack of 2016. We initially outline a big data analysis which shows how counter-narratives – criticizing #stopIslam – momentarily subverted negative news reporting of Muslims. The rest of the article details qualitative findings that complicate this initial positive picture. We set out key tactics engaged in by right-wing actors, self-identified Muslim users, would-be allies and celebrities and elucidate how these tactics were instrumental in the direction, dynamics and legacies of the hashtag. We argue that the tactical interventions of tightly bound networks of right-wing actors, as well as the structural constraints of the platform, not only undermined the longevity and coherence of the counter-narratives but subtly modulated the affordances of Twitter in ways that enabled these users to extend their voice outwards, reinforcing long-standing representational inequalities in the process.
... 5 A deep learning approach for identifying aggressive and non-aggressive tweets in the context of misogyny detection, misogynistic behaviour and target classification, is applied in [62]. Here, a deep learning approach incorporating a set of linguistic features in to a convoluted neural network architecture is used. ...
... Here, a deep learning approach incorporating a set of linguistic features in to a convoluted neural network architecture is used. Again, reported results have a low accuracy with linguistic features such as sarcasm and humour causing significant classification problems [62]. ...
Conference Paper
Full-text available
Recent moves to consider misogyny as a hate crime have refocused efforts for owners of web properties to detect and remove misogynistic speech. This paper considers the use of deep learning techniques for detection of misogyny in Urban Dictionary, a crowdsourced online dictionary for slang words and phrases. We compare the performance of two deep learning techniques, Bi-LSTM and Bi-GRU, to detect misogynistic speech with the performance of more conventional machine learning techniques, logistic regression, Naive-Bayes classification, and Random Forest classification. We find that both deep learning techniques examined have greater accuracy in detecting misogyny in the Urban Dictionary than the other techniques examined.
... Moreover, the existing surveys on abusive language detection (Schmidt and Wiegand 2017;Fortuna and Nunes 2018) underline the necessity to computationally approach the implicitness of toxic discourses, especially in the cases where these are disguised by sarcasm, euphemism, rhetorical questions, litotes, or where there are no explicit accusations, negative evaluations, or insults. This kind of implicitness eludes the offensiveness of the text, making its recognition hard, especially for machines (Nobata et al. 2016;Frenda 2018;MacAvaney et al. 2019). ...
Full-text available
Abusive language is becoming a problematic issue for our society. The spread of messages that reinforce social and cultural intolerance could have dangerous effects in victims’ life. State-of-the-art technologies are often effective on detecting explicit forms of abuse, leaving unidentified the utterances with very weak offensive language but a strong hurtful effect. Scholars have advanced theoretical and qualitative observations on specific indirect forms of abusive language that make it hard to be recognized automatically. In this work, we propose a battery of statistical and computational analyses able to support these considerations, with a focus on creative and cognitive aspects of the implicitness, in texts coming from different sources such as social media and news. We experiment with transformers, multi-task learning technique, and a set of linguistic features to reveal the elements involved in the implicit and explicit manifestations of abuses, providing a solid basis for computational applications.
... Sarcasm and irony As has been shown in multiple studies, e.g., [8], hateful messages containing sarcasm or irony are problematic for detection. Even though this only occurs in 3% and 4% of the explicit and implicit categories respectively, it is worth pointing out. ...
Full-text available
Over the past years, the amount of online hate speech has been growing steadily. Among multiple approaches to automatically detect hateful content online, ensemble learning is considered one of the best strategies, as shown by several studies on English and other languages. In this paper, we evaluate state-of-the-art approaches for Dutch hate speech detection both under in-domain and cross-domain hate speech detection conditions, and introduce a new ensemble approach with additional features for detecting hateful content in Dutch social media. The ensemble consists of the gradient boosting classifier that incorporates state-of-the-art transformer-based pre-trained language models for Dutch (i.e., BERTje and RobBERT), a robust SVM approach, and additional input information such as the number of emotion-conveying and hateful words, the number of personal pronouns, and the length of the message. The ensemble significantly outperforms all the individual models both in the in-domain and cross-domain hate speech detection settings. We perform an in-depth error analysis focusing on the explicit and implicit hate speech instances, providing various insights into open challenges in Dutch hate speech detection and directions for future research.
... This can affect the results of our model and is therefore, a limitation. Moreover, sarcasm [18] is difficult to detect in text. Since we have not explicitly handled sarcasm, some of the false positives (FP) that we encountered in our model were classified as cyberbullying but were actually normal tweets. ...
Full-text available
Cyberbullying is of extreme prevalence today. Online-hate comments, toxicity, cyberbullying amongst children and other vulnerable groups are only growing over online classes, and increased access to social platforms, especially post COVID-19. It is paramount to detect and ensure minors' safety across social platforms so that any violence or hate-crime is automatically detected and strict action is taken against it. In our work, we explore binary classification by using a combination of datasets from various social media platforms that cover a wide range of cyberbullying such as sexism, racism, abusive, and hate-speech. We experiment through multiple models such as Bi-LSTM, GloVe, state-of-the-art models like BERT, and apply a unique preprocessing technique by introducing a slang-abusive corpus, achieving a higher precision in comparison to models without slang preprocessing.
Cyberbullying is of extreme prevalence today. Online-hate comments, toxicity, and cyberbullying amongst vulnerable groups is only growing over increased access to social platforms, especially post COVID-19. It is paramount to detect and ensure safety across social platforms so that any violence or hate-crime is automatically detected and strict action is taken against it. In our work, we explore binary classification by using a combination of datasets from various social media platforms that cover a wide range of cyberbullying such as sexism, racism, abusive, and hate-speech. We experiment through multiple models such as Bi-LSTM, GloVe, state-of-the-art models like BERT, and apply a unique preprocessing technique by introducing a slang-abusive corpus, achieving a higher precision in comparison to models without slang preprocessing.
Full-text available
This research is the first attempt in Georgia to analyse hate speech emerging in Computer-Meditated Communication. Particular attention is paid to the polylogal, asynchronic remarks made by members of the public reacting to online newspaper articles or press releases concerning the LGPT pride event planned for 18 - 23 June 2019, in Tbilisi, Georgia. The methodology is based on combining methods utilized in CDA and Genre Approach to (im)politeness which is in accord with the general approach to CMDA . At the first stage of the analysis, the examples of hate-speech acts were analysed according to the following criteria: identification of linguistic means and strategies employed while expressing impoliteness and specificity of identity construction (self-asserted versus others -asserted, positive versus negative, roles of participants and strategies of conflict generation or management). Next, linguistic peculiarities of hate speech (for instance, linguistic triggers [threats, insults, sarcasm incitements], wordplay, taboo, swear and derogatory words, metaphors, allusions and similes) were identified and analysed. Quantitative methodology was employed while stating the number of proponents and opponents of the event as well as statistical data referring to the number of linguistic and politeness strategies employed while expressing an opinion. This research shows particular tendencies of how impoliteness can be realised and how social identities can be construed using the example of hate discourse concerning LGBT pride in Georgia. However, to fully explore the genre properties of hate discourse in Georgia further research based on examples of hate-discourse strategies applied when discussing ethnic minorities and gender roles, is needed.
Despite their positive effects in promoting participatory politics, digital publics have also manifested an offensive vernacular culture. This study takes a social network analytic approach to explain the contagion of offensive speech in online discussion contexts. The study examines four social interactional mechanisms underlying a user's adoption of political swearing: generalized reciprocity, direct reciprocity, leader-mimicry, and peer-mimicry. The empirical context of this study is a highly popular online discussion forum in Hong Kong. The study examines the effects of social interactional mechanisms on the occurrences of political swearing by analyzing five years of user comments. Findings show that peer-mimicry contributes to the contagion process the most, followed by generalized reciprocity and direct mimicry. The study demonstrates how individual-level speech behaviors spiral into a collective norm that potentially hinders a healthy discussion culture in mediated social spaces.
Conference Paper
Full-text available
The importance of the detection of aggressiveness in social media is due to real effects of violence provoked by negative behavior on-line. Indeed, this kind of legal cases are increasing in the last years. For this reason, the necessity of controlling user-generated contents has become one of the priorities for many Internet companies, although current methodologies are far from solving this problem. Therefore, in this work we propose an innovative approach that combines deep learning framework with linguistic features specific for this issue. This approach has been evaluated and compared with other ones in the framework of the MEX-A3T shared task at IberEval on aggressiveness analysis in Spanish Mexican tweets. In spite of our novel approach, we obtained low results.
Conference Paper
Full-text available
Nowadays, misogynistic abuse online has become a serious issue due, especially, to anonymity and interactivity of the web that facilitate the increase and the permanence of the offensive comments on the web. In this paper, we present an approach based on stylistic and specific topic information for the detection of misogyny, exploring the several aspects of misogynistic Spanish and English user generated texts on Twitter. Our method has been evaluated in the framework of our participation in the AMI shared task at IberEval 2018 obtaining promising results.
Conference Paper
Full-text available
This paper presents the framework and results from the MEX-A3T track at IberEval 2018. This track considers two tasks, author profiling and aggressiveness detection, both of them using Mexican Spanish tweets. The author profiling task aims to identify the place of residence and occupation of Twitter users. On the other hand, the aggressiveness detection task aims to discriminate between aggressive and non-aggressive tweets. For these two tasks we have built new corpora considering tweets from Mexican Twitter users. This paper compares and discusses the results from the participants.
Conference Paper
Full-text available
While favouring communications and easing information sharing, Social Network Sites are also used to launch harmful campaigns against specific groups and individuals. Cyberbullism, incitement to self-harm practices, sexual predation are just some of the severe effects of massive online offensives. Moreover, attacks can be carried out against groups of victims and can degenerate in physical violence. In this work, we aim at containing and preventing the alarming diffusion of such hate campaigns. Using Facebook as a benchmark, we consider the textual content of comments appeared on a set of public Italian pages. We first propose a variety of hate categories to distinguish the kind of hate. Crawled comments are then annotated by up to five distinct human annotators, according to the defined taxonomy. Leveraging morpho-syntactical features, sentiment polarity and word embedding lexicons, we design and implement two classifiers for the Italian language, based on different learning algorithms: the first based on Support Vector Machines (SVM) and the second on a particular Recurrent Neural Network named Long Short Term Memory (LSTM). We test these two learning algorithms in order to verify their classification performances on the task of hate speech recognition. The results show the effectiveness of the two classification approaches tested over the first manually annotated Italian Hate Speech Corpus of social media text.
Full-text available
The results of an experiment supported the hypotheses that (1) for men high in hostile sexism, exposure to sexist humor creates a perceived social norm of tolerance of sexism relative to exposure to nonhumorous sexist communication or neutral humor, and (2) due to this ‘relaxed’ normative standard in the context of sexist humor, men high in hostile sexism anticipated feeling less self-directed negative affect upon imagining that they had behaved in a sexist manner. Finally, exposure to sexist humor did not affect the evaluative content of men's stereotypes of women relative to exposure to neutral humor or nonhumorous sexist communication for participants high or low in hostile sexism. Copyright © 2001 John Wiley & Sons, Ltd.
Conference Paper
Detection of abusive language in user generated online content has become an issue of increasing importance in recent years. Most current commercial methods make use of blacklists and regular expressions, however these measures fall short when contending with more subtle, less ham-fisted examples of hate speech. In this work, we develop a machine learning based method to detect hate speech on online user comments from two domains which outperforms a state-of-the-art deep learning approach. We also develop a corpus of user comments annotated for abusive language, the first of its kind. Finally, we use our detection tool to analyze abusive language over time and in different settings to further enhance our knowledge of this behavior.
Automatic detection of emotions like sarcasm or nastiness in online written conversation is a difficult task. It requires a system that can manage some kind of knowledge to interpret that emotional language is being used. In this work, we try to provide this knowledge to the system by considering alternative sets of features obtained according to different criteria. We test a range of different feature sets using two different classifiers. Our results show that the sarcasm detection task benefits from the inclusion of linguistic and semantic information sources, while nasty language is more easily detected using only a set of surface patterns or indicators.