Conference PaperPDF Available

Uses of Machine Translation in the Sentiment Analysis of Tweets

  • September 2014
DOI:10.3233/978-1-61499-442-8-126
  • Conference: Baltic HLT 2014
  • At: Kaunas, Lithuania
  • Volume: Frontiers in Artificial Intelligence and Applications: Volume 286. Human Language Technologies – The Baltic Perspective - Proceedings of the Sixth International Conference Baltic HLT 2014
Authors:
Jānis Peisenieks
Jānis Peisenieks
Jānis Peisenieks
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Raivis Skadiņš at Tilde
Raivis Skadiņš
  • Tilde
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

This paper reports on the viability of using machine translation (MT) for determining the original sentiment of tweets, when translating tweets made in internationally less used language into more frequently used ones. The results of the study show that it is possible to use MT and sentiment analysis (SA) systems to produce SA results with significant precision.
Figures - uploaded by Raivis Skadiņš
Author content
All figure content in this area was uploaded by Raivis Skadiņš
Content may be subject to copyright.
Content uploaded by Raivis Skadiņš
Author content
All content in this area was uploaded by Raivis Skadiņš on Sep 30, 2014
Content may be subject to copyright.
Uses of Machine Translation in the
Sentiment Analysis of Tweets
Jānis PEISENIEKSa, 1 and Raivis SKADIŅŠ
b
a
University of Latvia, Latvia
b
Tilde, Latvia
Abstract. This paper reports on the viability of using machine translation (MT) for
determining the original sentiment of tweets, when translating tweets made in
internationally less used language into more frequently used ones. The results of
the study show that it is possible to use MT and sentiment analysis (SA) systems to
produce SA results with significant precision.
Keywords. Machine translation, sentiment analysis, Latvian
Introduction
Sentiment analysis has been a very active field of study lately, and there have been a lot
of high quality tools developed for English. These tools allow for near real-time
analysis of various user generated data sources like online reviews, blogs, news, and
social networks. By using sentiment analysis on these data sources, it is possible to
gain a deeper understanding of social processes and help businesses and governments
make well informed decisions. However the main problem of these tools is that they
are usually created for large, internationally used languages.
We wanted to see whether it was possible to use publicly available MT and SA
systems to discern the sentiment of a tweet that was written in an internationally less
used language, in this case, Latvian.
1. Related Work
One of the main problems in cross-language sentiment translation is the quality of the
translation software and whether translations obtained using MT can be used in
sentiment analysis. To this aim, supervised Machine Translation systems have been
used on the English language [1] to produce training data for other languages. On the
other hand, a recent study [2] has shown, that the quality of such work can be sub-
optimal.
Balahur, Turchi et. al. [3] have also shown that it is possible to use MT systems to
train multilingual sentiment classifiers, where the training of classifiers in one language
also improves the abilities of classifiers in other languages.
1 Corresponding Author: Jānis Peisenieks, 29 Raina Blvd., Riga, Latvia; E-mail: janis@peisenieks.lv
Human Language Technologies – The Baltic Perspective
A. Utka et al. (Eds.)
© 2014 The authors and IOS Press.
This article is published online with Open Access by IOS Press and distributed under the terms
of the Creative Commons Attribution Non-Commercial License.
doi:10.3233/978-1-61499-442-8-126
126
2. Test Corpus
In order to study the viability of the proposed approach, a well annotated test corpus of
tweets written in Latvian is necessary. The authors created such a corpus, because no
such corpus was publicly available.
By using a custom crowd sourcing website, a tweet test corpus with sentiment
polarity annotation was created with 3 sentiment classes: positive, neutral, and negative.
The tweets for this website were gathered from Twitter's real-time API from November
2013 to March 2014, using a rough contour of Latvia as the query for the Twitter API.
Furthermore, the Twitter API provides a language for each tweet, and only tweets with
Latvian or no language were used.
In order to assess the sentiment of a tweet accurately, each tweet was rated 11
times (some tweets have up to 13 ratings due to real time specifics of the website), and
only the ones with strong annotator consensus were included in the test corpus. In total,
~20,000 tweet ratings were processed.
These ~20,000 ratings produced 1,722 adequately rated tweets, and of these, 1,177
had the required annotator consensus to be included in the test corpus.
Table 1. Distribution of tweets in sentiment classes in the test corpus
Sentiment class Tweet count
Positive 383
Neutral 627
Negative 167
The distribution of these tweets (Table 1) reflects the distribution of tweets being
tweeted in Latvian. The whole test corpus has been made publicly available on the code
collaboration website Github.com2.
Additionally, Fleiss’s kappa [4] was used to measure the reliability of agreement
between the annotators and is 0.284, which, according to Landis and Koch [5], can
only be described as fair agreement. This could have been influenced by the lack of the
“skip” functionality in the crowd-sourcing website, which would allow the annotator to
skip the current tweet if it contained no sentiment information.
3. Uses of MT in SA
To study the viability of the proposed approach, 3 publicly available MT and SA
systems were chosen.
The MT systems used in this study were chosen based on their performance of
LV-EN translations [6], and those are Google Translate, Bing Translator, and Tilde
Translator. The whole of the test corpus was translated from Latvian to English using
each of the MT systems.
In order to assess the sentiment of the translated tweets, 3 publicly available SA
systems (AlchemyAPI, Textalytics, and Semantria) were chosen, based on their
performance [7] and ease of use. The publicly available SDKs for each of the SA
2 https://github.com/FnTm/latvian-tweet-sentiment-corpus
J. Peisenieks and R. Skadin
,š / Uses of Machine Translation in the Sentiment Analysis of Tweets 127
systems were used to analyze the sentiment of all of the translations, thus producing
MT+SA system pairs, the performance of which could be evaluated.
4. Evaluation and Results
We used the standard performance metrics, such as precision, recall, and F1-measure
(Table 2), to evaluate the SA results. While processing the SA results, it became
apparent that not all of the SA tools properly process the neutral sentiment class, which
means that the SA tool would either produce results that are indistinguishable from no
sentiment or that are hard to identify. Additionally, the SA systems that could at least
partially recognize the neutral sentiment class had a very low precision and extremely
low recall, which could be caused by the difficulty to distinguish text with a neutral
sentiment from text with no sentiment. Because of this, all of the further
research/interpretation was done only on positive and negative sentiment classes.
Table 2. Precision, recall, and F1-measure of MT+SA system pairs
Sentiment class
MT+SA
system pair
Positive
Negative
Neutral
Prec. Rec. F1 Prec. Rec. F1 Prec. Rec. F1
Google Textalytics 0.64 0.70 0.67 0.62 0.39 0.48 0.20 0.00 0.01
Tilde Textalytics 0.64 0.61 0.63 0.58 0.39 0.47 0.12 0.00 0.01
Bing Textalytics 0.55 0.74 0.63 0.59 0.36 0.45 0.21 0.01 0.02
Google AlchemyAPI 0.46 0.74 0.57 0.38 0.75 0.50 - - -
Tilde AlchemyAPI 0.45 0.74 0.56 0.38 0.69 0.49 - - -
Bing AlchemyAPI 0.46 0.78 0.58 0.39 0.72 0.51 - - -
Google Semantria 0.63 0.59 0.61 0.49 0.40 0.44 0.35 0.02 0.03
Tilde Semantria 0.61 0.47 0.53 0.48 0.35 0.40 0.50 0.02 0.04
Bing Semantria 0.65 0.59 0.62 0.50 0.42 0.45 0.33 0.01 0.02
Additionally, the overall accuracy of the MT+SA pairs was measured (Table 3) as
the total percentage of tweets classified correctly. For easier interpretation, this same
data has been graphed in Figure 1.
Table 3. Overall Accuracy of MT+SA pairs
MT+SA
system pair
Accuracy Confidence interval ±
Tilde Textalytics 54.73% 4.16%
Tilde Semantria 45.64% 4.16%
Tilde AlchemyAPI 72.55% 3.73%
Google Textalytics 61.27% 4.07%
Google Semantria 55.45% 4.15%
Google AlchemyAPI 74.55% 3.64%
Bing Textalytics 63.27% 4.03%
Bing Semantria 54.73% 4.16%
Bing AlchemyAPI 76.00% 3.57%
J. Peisenieks and R. Skadin
,š / Uses of Machine Translation in the Sentiment Analysis of Tweets128
Figure 1. Overall Accuracy of MT+SA pairs
As can be seen from these results and from the standpoint of SA systems,
AlchemyAPI is the clear leader, with the best SA results irrespective of the MT system
used to translate the tweets. From the standpoint of MT systems, Bing Translator is the
MT system that acquired the best results, however the results are not decisive in this
matter.
Interestingly, the MT+SA pair with the highest overall accuracy (Bing Translator +
AlchemyAPI) did not produce the best results in other metrics. One part of the answer
to this question is that the mentioned pair is not the best, however it is consistent.
However, it is possible that the way in which the experiments were conducted,
accompanied with the characteristics of the test set, allowed for an interesting error to
occur.
When looking at the SA systems, only AlchemyAPI does not even try to process
the neutral sentiment class, which allows for a more broad classification in the other
sentiment classes. This means that AlchemyAPI classifies more tweets as having either
a positive or negative sentiment, where the other SA systems would produce no
sentiment for the particular tweet. This can be easily seen in Figure 2 which shows the
percentage of classified tweets that are false-positives. This means that a tweet that has
been classified, for example, as a positive, is in fact from the neutral sentiment class.
Thus, even though AlchemyAPI provides for a higher overall accuracy, it also has
more problems of correctly identifying neutral data. Also interestingly, the percentage
of false positives between the positive and negative sentiment classes is quite small
(~7%), which means that even though a lot of tweets have been incorrectly classified,
the false-positives introduce very little bias to the data. This could mean that these false
positives would have minimal impact in real-world use cases.
J. Peisenieks and R. Skadin
,š / Uses of Machine Translation in the Sentiment Analysis of Tweets 129
Figure 2. Percentage of false positives in classified tweets
5. Conclusions
The evaluation of the proposed approach shows that it is possible to do binary
sentiment analysis on tweets originally written in a less internationally used language,
with a high degree of accuracy. Additionally, it is clear that doing this sort of
classification using 3 sentiment classes would provide results with a low degree of
accuracy.
It should also be noted that during the binary sentiment analysis of tweets, a
significant amount of false positives occur. Depending on the particular use case, this
may or may not present problems and skew the end results.
References
[1] Balahur, A., and Turchi. M. (2012). Multilingual Sentiment Analysis using Machine Translation?. In
Proceedings of the Association for Computational Linguistics: 3rd Workshop on Computational
Approaches to Subjectivity and Sentiment Analysis Workshop (pp. 52-60) Jeju, Republic of Korea:
ACL
[2] Cieliebak, M., Dürr, O., and Uzdilli. F. (2013) Potential and Limitations of Commercial Sentiment
Detection Tools. Proceedings of the First International Workshop on Emotion and Sentiment in Social
and Expressive Media: approaches and perspectives from AI (ESSEM 2013)
[3] Balahur, A., Turchi, M., Steinberger, R., Perea-Ortega, J. M., Jacquet, G., Küçük, D., Zavarella, V., & El
Ghali, A. (2014). Resource Creation and Evaluation for Multilingual Sentiment Analysis in Social
Media Texts. In Proceedings of the 9th edition of the Language Resources and Evaluation Conference
(LREC). Reykjavik, Iceland: European Language Resources Association (ELRA)
[4] Fleiss, J.L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin.
76:378-382.
[5] Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data.
Biometrics. 1977 Mar; 33(1):159-74.
J. Peisenieks and R. Skadin
,š / Uses of Machine Translation in the Sentiment Analysis of Tweets130
[6] Skadiņš, R., Šics, V., Rozis, R. (2014). Building the world’s best general-domain MT for Baltic
languages. In Human Language Technologies – The Baltic Perspective - Proceedings of the Sixth
International Conference Baltic HLT 2014. Kaunas, Lithuania: IOS Press.
[7] Abbasi, A., Ammar, H., and Dhar, M. (2014). Benchmarking Twitter Sentiment Analysis Tools. In
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
(pp. 823-829). Reykjavik, Iceland: European Language Resources Association (ELRA)
J. Peisenieks and R. Skadin
,š / Uses of Machine Translation in the Sentiment Analysis of Tweets 131
... In this setting, it effectively adapts the sentiment information from a resourcerich language (English) to a resource-poor language (Chinese) and helps the improvement of the sentiment classification performance. Moreover, Peisenieks, and Skadins [38] tried to report the viability of using Machine Translation task by translating Tweets text from Latvian to English using Google Translate, Bing Translator, and Tilde Translator, whereas they performed the Sentiment Analysis task based on publicly available SDKs, such as AlchemyAPI, Textalytics, and Semantria. On top of this, they constructed pairs of translation and sentiment analysis tools and evaluated their performance. ...
... To this end, the challenges of performing multilingual neural sentiment analysis have been examined with emphasis on the evaluation of the role and the impact that NMT and pre-trained Word Embeddings tasks can have. In contrary with other studies [38] the implementation of the neural sentiment analysis task was performed by the utilization of different Neural Networks. Furthermore, NMT task and the translation of English reviews in the Greek and German languages was implemented based on the utilization of popular and widely used APIs offered by pioneer providers. ...
An Evaluation of Neural Machine Translation and Pre-trained Word Embeddings in Multilingual Neural Sentiment Analysis
Conference Paper
Full-text available
  • Dec 2020
One of the main elements in several application domains, such as policy making, addresses the scope of public opinion analysis. The latter is recently realized through sentiment analysis and Natural Language Processing, for identifying and extracting subjective information from raw texts. An additional challenge refers to the exploitation and correlation of data from different languages, in order to analyse sentiments by considering all available information in a holistic way. This paper investigates the impact of Neural Machine Translation in sentiment analysis, based on the translation of text for which neural sentiment analysis in English has been already applied and is applied again on the translated German and Greek texts. The latter is performed through the same Neural Network models that were used in the original language, in order to analyse the differences between the performed neural sentiment analysis on the source and the target languages. The outcomes of the proposed approach have a twofold interpretation. While, it is concluded that modern Neural Machine Translation tools are mature enough to provide high accuracy translations and have minor impact on multilingual sentiment analysis, on the other hand it is shown that additional research should be performed on the multilingualism that pre-trained Word Embeddings can provide.
... Milajevs [1] collected and analysed 1.4 million tweets geo-located in Riga, Latvia from April 2017 to July 2018 and 60 thousand tweets [2] from November 2016 to March 2017. Pinnis [3] collected and analysed 3.8 million tweets of Latvian politicians, companies, media, and users who interacted with these entities from August 2016 to July 2018 There are also several data sets of general sentiment-annotated tweets [4,5,3] 3 amounting to 14,781 tweets in total. ...
... We compared a Python implementation of the Naive Bayes classifier from NLTK [14] against Pinnis [3] implementation of the Perceptron classifier. We also experimented with several combinations of training data sets -TE (our Twitter Eater dataset), MP [3], RV [5], PE [4], NI 11 . We found that the highest classification accuracy -61.23% -is achieved by using all but NI data sets for training and only stemming all words. ...
What Can We Learn from Almost a Decade of Food Tweets
Conference Paper
Full-text available
  • Sep 2020
We present the Latvian Twitter Eater Corpus - a set of tweets in the narrow domain related to food, drinks, eating and drinking. The corpus has been collected over time-span of over 8 years and includes over 2 million tweets entailed with additional useful data. We also separate two sub-corpora of question and answer tweets and sentiment annotated tweets. We analyse the contents of the corpus and demonstrate use-cases for the sub-corpora by training domain-specific question-answering and sentiment-analysis models using the data from the corpus.
... Milajevs [1] collected and analysed 1.4 million tweets geo-located in Riga, Latvia from April 2017 to July 2018 and 60 thousand tweets [2] from November 2016 to March 2017. Pinnis [3] collected and analysed 3.8 million tweets of Latvian politicians, companies, media, and users who interacted with these entities from August 2016 to July 2018 There are also several data sets of general sentiment-annotated tweets [4,5,3] 3 amounting to 14,781 tweets in total. ...
... We compared a Python implementation of the Naive Bayes classifier from NLTK [14] against Pinnis [3] implementation of the Perceptron classifier. We also experimented with several combinations of training data sets -TE (our Twitter Eater dataset), MP [3], RV [5], PE [4], NI 11 . We found that the highest classification accuracy -61.23% -is achieved by using all but NI data sets for training and only stemming all words. ...
What Can We Learn From Almost a Decade of Food Tweets
Preprint
Full-text available
  • Jul 2020
We present the Latvian Twitter Eater Corpus - a set of tweets in the narrow domain related to food, drinks, eating and drinking. The corpus has been collected over time-span of over 8 years and includes over 2 million tweets entailed with additional useful data. We also separate two sub-corpora of question and answer tweets and sentiment annotated tweets. We analyse contents of the corpus and demonstrate use-cases for the sub-corpora by training domain-specific question-answering and sentiment-analysis models using data from the corpus.
... Previous work on sentiment analysis for Latvian has been mostly focussed on lexicon-based sentiment analysis methods, i.e. methods where positive and negative word lists (gazetteers) are used to determine the polarity of a text message [1,2]. Cross-lingual sentiment analysis for Latvian has been previously analysed also by Peisenieks and Skadiņš [3]. Because open source sentiment analysis tools for Latvian were not available, however, tools for English had been widely researched (as shown by the numerous shared tasks on sentiment analysis, e.g. ...
... In order to test whether such automatic annotation does not negatively impact machine learning-based methods (by making the models learn to classify based on emoticon presence only), we created two other datasets based on this dataset -the "Auto (no )" dataset has emoticons removed and the "Auto (both)" dataset combines both automatically acquired datasets. 3. Reused the corpus of 1,178 annotated tweets created by Peisenieks and Skadiņš [3]. This dataset is the "Peisenieks" dataset. ...
Latvian Tweet Corpus and Investigation of Sentiment Analysis for Latvian
Conference Paper
  • Sep 2018
We present the Latvian Tweet Corpus and its application in sentiment analysis by comparing four different machine learning algorithms and a lexical classification method. We show that the best results are achieved by an averaged perceptron classifier. In our experiments, the more complex neural network-based classification methods (using recurrent neural networks and word embeddings) did not yield better results.
... This approach is similar to other works, such as Grimminger & Klinger (2021), in which 3,000 tweets were manually annotated for a BERT classification. While there is no universal number for how many tweets should be labeled when using BERT, we observe a range across studies from under 2,000 labeled tweets (Şaşmaz & Tek, 2021;Alomari, ElSherif & Shaalan, 2017;Peisenieks & Skadiņš, 2014) up to several thousands (Golubev & Loukachevitch, 2020;Rustam et al., 2021;Nabil, Aly & Atiya, 2015;Zhang et al., 2020). In line with previous studies on sentiment analysis in tweets Khan et al., 2021), we compared BERT and several other supervised machine learning models from the textblob library (i.e., decision tree, Naive Bayes) as well as scikit-learn (logistic regression, support vector machine). ...
(Re)shaping online narratives: when bots promote the message of President Trump during his first impeachment
Article
Full-text available
  • Apr 2022
Influencing and framing debates on Twitter provides power to shape public opinion. Bots have become essential tools of ‘computational propaganda’ on social media such as Twitter, often contributing to a large fraction of the tweets regarding political events such as elections. Although analyses have been conducted regarding the first impeachment of former president Donald Trump, they have been focused on either a manual examination of relatively few tweets to emphasize rhetoric, or the use of Natural Language Processing (NLP) of a much larger corpus with respect to common metrics such as sentiment. In this paper, we complement existing analyses by examining the role of bots in the first impeachment with respect to three questions as follows. (Q1) Are bots actively involved in the debate? (Q2) Do bots target one political affiliation more than another? (Q3) Which sources are used by bots to support their arguments? Our methods start with collecting over 13M tweets on six key dates, from October 6th 2019 to January 21st 2020. We used machine learning to evaluate the sentiment of the tweets ( via BERT ) and whether it originates from a bot. We then examined these sentiments with respect to a balanced sample of Democrats and Republicans directly relevant to the impeachment, such as House Speaker Nancy Pelosi, senator Mitch McConnell, and (then former Vice President) Joe Biden. The content of posts from bots was further analyzed with respect to the sources used (with bias ratings from AllSides and Ad Fontes) and themes. Our first finding is that bots have played a significant role in contributing to the overall negative tone of the debate (Q1). Bots were targeting Democrats more than Republicans (Q2), as evidenced both by a difference in ratio (bots had more negative-to-positive tweets on Democrats than Republicans) and in composition (use of derogatory nicknames). Finally, the sources provided by bots were almost twice as likely to be from the right than the left, with a noticeable use of hyper-partisan right and most extreme right sources (Q3). Bots were thus purposely used to promote a misleading version of events. Overall, this suggests an intentional use of bots as part of a strategy, thus providing further confirmation that computational propaganda is involved in defining political events in the United States. As any empirical analysis, our work has several limitations. For example, Trump’s rhetoric on Twitter has previously been characterized by an overly negative tone, thus tweets detected as negative may be echoing his message rather than acting against him. Previous works show that this possibility is limited, and its existence would only strengthen our conclusions. As our analysis is based on NLP, we focus on processing a large volume of tweets rather than manually reading all of them, thus future studies may complement our approach by using qualitative methods to assess the specific arguments used by bots.
... This was necessary as the lexicon used to measure tweet-level sentiment only contains English words. While we did not directly assess how tweet-based sentiment scores were impacted by the translation, existing research on machine translation and sentiment analysis has shown that English lexicons applied to translated texts produce comparable results to lexicons applied within a text in the original language as the resulting sentiment scores are word-based, rather than on the semantic structure of text (Peisenieks and Skadins, 2014;Mohammad et al., 2016;Shalunts et al., 2016). The translation was also necessary for our topic modeling approach, in order to make comparisons across countries. ...
Using Twitter to track immigration sentiment during early stages of the COVID-19 pandemic
Article
Full-text available
  • Jan 2021
Large-scale coordinated efforts have been dedicated to understanding the global health and economic implications of the COVID-19 pandemic. Yet, the rapid spread of discrimination and xenophobia against specific populations has largely been neglected. Understanding public attitudes toward migration is essential to counter discrimination against immigrants and promote social cohesion. Traditional data sources to monitor public opinion are often limited, notably due to slow collection and release activities. New forms of data, particularly from social media, can help overcome these limitations. While some bias exists, social media data are produced at an unprecedented temporal frequency, geographical granularity, are collected globally and accessible in real-time. Drawing on a data set of 30.39 million tweets and natural language processing, this article aims to measure shifts in public sentiment opinion about migration during early stages of the COVID-19 pandemic in Germany, Italy, Spain, the United Kingdom, and the United States. Results show an increase of migration-related Tweets along with COVID-19 cases during national lockdowns in all five countries. Yet, we found no evidence of a significant increase in anti-immigration sentiment, as rises in the volume of negative messages are offset by comparable increases in positive messages. Additionally, we presented evidence of growing social polarization concerning migration, showing high concentrations of strongly positive and strongly negative sentiments.
... This was necessary as the lexicon used to measure tweet-level sentiment only contains English words. While we did not directly assess how tweet-based sentiment scores were impacted by the translation, existing research on machine translation and sentiment analysis has shown that English lexicons applied to translated texts produce comparable results to lexicons applied within a text in the original language as the resulting sentiment scores are word-based, rather than on the semantic structure of text (Peisenieks and Skadins, 2014;Mohammad et al., 2016;Shalunts et al., 2016). The translation was also necessary for our topic modelling approach, in order to make comparisons across countries. ...
Using Twitter to Track Immigration Sentiment During Early Stages of the COVID-19 Pandemic
Preprint
Full-text available
  • Jul 2021
In 2020, the world faced an unprecedented challenge to tackle and understand the spread and impacts of COVID-19. Large-scale coordinated efforts have been dedicated to understand the global health and economic implicationsof the pandemic. Yet, the rapid spread of discrimination and xenophobia against specific populations, particularlymigrants and individuals of Asian descent, has largely been neglected. Understanding public attitudes towardsmigration is essential to counter discrimination against immigrants and promote social cohesion. Traditional datasources to monitor public opinion – ethnographies, interviews, and surveys – are often limited due to smallsamples, high cost, low temporal frequency, slow collection, release and coarse spatial resolution. New forms ofdata, particularly from social media, can help overcome these limitations. While some bias exists, social mediadata are produced at an unprecedented temporal frequency, geographical granularity, are collected globally andaccessible in real-time. Drawing on a data set of 30.39 million tweets and natural language processing, this paperaims to measure shifts in public sentiment opinion about migration during early stages of the COVID-19 pandemicin Germany, Italy, Spain, the United Kingdom and the United States. Results show an increase of migration-relatedTweets along with COVID-19 cases during national lockdowns in all five countries. Yet, we found no evidence ofa significant increase in anti-immigration sentiment, as rises in the volume of negative messages are offset bycomparable increases in positive messages. Additionally, we presented evidence of growing social polarisationconcerning migration, showing high concentrations of strongly positive and strongly negative sentiments.
... We adapted the sentiment analysis tool 1 used for general Latvian tweets [11] and used it along with a Latvian-specific language model [13], which we fine-tuned onto the entire LTEC. This allowed us to train a sentiment classifier to distinguish negative, positive and neutral tweets in the sentiment training data subset of LTEC other Latvian tweet data sets 2 [7,8,12]. In total we had 17,255 tweets for training, 1,000 [8] for development and 744 from the LTEC for evaluation. ...
Fragmented and Valuable: Following Sentiment Changes in Food Tweets
Preprint
Full-text available
  • Jun 2021
We analysed sentiment and frequencies related to smell, taste and temperature expressed by food tweets in the Latvian language. To get a better understanding of the role of smell, taste and temperature in the mental map of food associations, we looked at such categories as 'tasty' and 'healthy', which turned out to be mutually exclusive. By analysing the occurrence frequency of words associated with these categories, we discovered that food discourse overall was permeated by `tasty' while the category of 'healthy' was relatively small. Finally, we used the analysis of temporal dynamics to see if we can trace seasonality or other temporal aspects in smell, taste and temperature as reflected in food tweets. Understanding the composition of social media content with relation to smell, taste and temperature in food tweets allows us to develop our work further - on food culture/seasonality and its relation to temperature, on our limited capacity to express smell-related sentiments, and the lack of the paradigm of taste in discussing food healthiness.
... For Latvian, Pinnis [10] performed experiments on Latvian tweets with a wide range of classifiers and features. Peisenieks and Skadiņš [11] analysed machine translation as a viable tool for performing SA for Latvian tweets. A recent study [12] performed pre-training of the BERT model from scratch for classifying sentiment of tweet representations. ...
Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets
Preprint
  • Oct 2020
In this paper, we present various pre-training strategies that aid in im-proving the accuracy of the sentiment classification task. We, at first, pre-trainlanguage representation models using these strategies and then fine-tune them onthe downstream task. Experimental results on a time-balanced tweet evaluation setshow the improvement over the previous technique. We achieve 76% accuracy forsentiment analysis on Latvian tweets, which is a substantial improvement over pre-vious work
Big Data and Its Applications in Smart Real Estate and the Disaster Management Life Cycle: A Systematic Analysis
Article
Full-text available
  • Mar 2020
Big data is the concept of enormous amounts of data being generated daily in different fields due to the increased use of technology and internet sources. Despite the various advancements and the hopes of better understanding, big data management and analysis remain a challenge, calling for more rigorous and detailed research, as well as the identifications of methods and ways in which big data could be tackled and put to good use. The existing research lacks in discussing and evaluating the pertinent tools and technologies to analyze big data in an efficient manner which calls for a comprehensive and holistic analysis of the published articles to summarize the concept of big data and see field-specific applications. To address this gap and keep a recent focus, research articles published in last decade, belonging to top-tier and high-impact journals, were retrieved using the search engines of Google Scholar, Scopus, and Web of Science that were narrowed down to a set of 139 relevant research articles. Different analyses were conducted on the retrieved papers including bibliometric analysis, keywords analysis, big data search trends, and authors' names, countries, and affiliated institutes contributing the most to the field of big data. The comparative analyses show that, conceptually, big data lies at the intersection of the storage, statistics, technology, and research fields and emerged as an amalgam of these four fields with interlinked aspects such as data hosting and computing, data management, data refining, data patterns, and machine learning. The results further show that major characteristics of big data can be summarized using the seven Vs, which include variety, volume, variability, value, visualization, veracity, and velocity. Furthermore, the existing methods for big data analysis, their shortcomings, and the possible directions were also explored that could be taken for harnessing technology to ensure data analysis tools could be upgraded to be fast and efficient. The major challenges in handling big data include efficient storage, retrieval, analysis, and visualization of the large heterogeneous data, which can be tackled through authentication such as Kerberos and encrypted files, logging of attacks, secure communication through Secure Sockets Layer (SSL) and Transport Layer Security (TLS), data imputation, building learning models, dividing computations into sub-tasks, checkpoint applications for recursive tasks, and using Solid State Drives (SDD) and Phase Change Material (PCM) for storage. In terms of frameworks for big data management, two frameworks exist including Hadoop and Apache Spark, which must be used simultaneously to capture the holistic essence of the data and make the analyses meaningful, swift, and speedy. Further field-specific applications of big data in two promising and integrated fields, i.e., smart real estate and disaster management, were investigated, and a framework for field-specific applications, as well as a merger of the two areas through big data, was highlighted. The proposed frameworks show that big data can tackle the ever-present issues of customer regrets related to poor quality of information or lack of information in smart real estate to increase the customer satisfaction using an intermediate organization that can process and keep a check on the data being provided to the customers by the sellers and real estate managers. Similarly, for disaster and its risk management, Big Data Cogn. Comput. 2020, 4, 4 2 of 53 data from social media, drones, multimedia, and search engines can be used to tackle natural disasters such as floods, bushfires, and earthquakes, as well as plan emergency responses. In addition, a merger framework for smart real estate and disaster risk management show that big data generated from the smart real estate in the form of occupant data, facilities management, and building integration and maintenance can be shared with the disaster risk management and emergency response teams to help prevent, prepare, respond to, or recover from the disasters.
Benchmarking Twitter Sentiment Analysis Tools
Conference Paper
Full-text available
  • May 2014
Twitter has become one of the quintessential social media platforms for user-generated content. Researchers and industry practitioners are increasingly interested in Twitter sentiments. Consequently, an array of commercial and freely available Twitter sentiment analysis tools have emerged, though it remains unclear how well these tools really work. This study presents the findings of a detailed benchmark analysis of Twitter sentiment analysis tools, incorporating 20 tools applied to 5 different test beds. In addition to presenting detailed performance evaluation results, a thorough error analysis is used to highlight the most prevalent challenges facing Twitter sentiment analysis tools. The results have important implications for various stakeholder groups, including social media analytics researchers, NLP developers, and industry managers and practitioners using social media sentiments as input for decision-making.
Potential and Limitations of Commercial Sentiment Detection Tools
Conference Paper
Full-text available
  • Dec 2013
In this paper, we analyze the quality of several commercial tools for sentiment detection. All tools are tested on nearly 30,000 short texts from various sources, such as tweets, news, reviews etc. In addition to the quality analysis (measured by various metrics), we also investigate the effect of increasing text length on the performance. Finally, we show that combining all tools using machine learning techniques increases the overall performance significantly.
Building the World's Best General Domain MT for Baltic Languages
Conference Paper
Full-text available
  • Sep 2014
In this paper we present our experience in building machine translation (MT) systems for the languages of the Baltic States: Estonian, Latvian, and Lithuanian. The paper reports on the implementation, research, data, data collection methods, and evaluation of the MT. Results of the evaluation show that it is possible to collect a sufficient amount of data and train MT systems that can compete with Google in quality and even overtake it in general domain MT.
Multilingual sentiment analysis using machine translation?
Conference Paper
Full-text available
  • Jul 2012
The past years have shown a steady growth in interest in the Natural Language Processing task of sentiment analysis. The research community in this field has actively proposed and improved methods to detect and classify the opinions and sentiments expressed in different types of text - from traditional press articles, to blogs, reviews, fora or tweets. A less explored aspect has remained, however, the issue of dealing with sentiment expressed in texts in languages other than English. To this aim, the present article deals with the problem of sentiment detection in three different languages - French, German and Spanish - using three distinct Machine Translation (MT) systems - Bing, Google and Moses. Our extensive evaluation scenarios show that SMT systems are mature enough to be reliably employed to obtain training data for languages other than English and that sentiment analysis systems can obtain comparable performances to the one obtained for English.
Resource creation and evaluation for multilingual sentiment analysis in social media texts
Conference Paper
  • May 2014
Measuring Nominal Scale Agreement Among Many Raters
Article
  • Nov 1971
Introduced the statistic kappa to measure nominal scale agreement between a fixed pair of raters. Kappa was generalized to the case where each of a sample of 30 patients was rated on a nominal scale by the same number of psychiatrist raters (n = 6), but where the raters rating 1 s were not necessarily the same as those rating another. Large sample standard errors were derived.
The Measurement Of Observer Agreement For Categorical Data
Article
  • Apr 1977
This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa-type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.