Conference PaperPDF Available

Studying sentiment in social media conversations using manual vs. automated approaches

Authors:
!
1!
Studying sentiment in social media conversations using manual vs.
automated approaches
Conference theme: Big data, social media analysis, text mining
Author information:
Yuvaraj Padmanabhan, Mindgraph, Founder and Managing Director
Ana Isabel Canhoto, Oxford Brookes University, Senior Lecturer in Marketing
Abstract submitted for oral paper presentation
There is now an abundance of commercial software tools that mine social media data
and produce reports of expressed sentiment. These tools scan textual data and produce
a score that reflects the underlying sentiment – positive or negative – expressed in the
segment of text under analysis. Given the increasing reliance on such automated tools,
it is critical to investigate how accurate they are in studying consumer behaviour. This
paper reports on the findings from a research project exploring the question: To what
extent do automated and manual analysis of sentiment in social media data match;
and why?
200 Twitter posts were analysed manually and using two popular sentiment analysis
tools. The results revealed significant differences between the outcomes of the three
approaches, not only in terms of sentiment polarity but also in terms of the specific
emotion identified. The systems struggled to detect nuance and the target of the
emotion. Differences also arose when the sentiment was implied rather than explicit,
and when abbreviations or slang were used.
The results are very concerning given the popularity of automated sentiment analysis.
They are concerning for academics, particularly the novice user, who may be too
reliant on these tools to analyse large volumes of consumer data; and for practitioners
in search of speedy and inexpensive customer insight and who are unlikely to use two
systems plus manual analysis, as we did, to assess the robustness of the tools.
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.