Adapting an open source social bookmarking system to observe critical information behaviour

Abstract

Constructively dealing with societal problems requires a process of opinion formation that is preceded by a competent and diverse search of information. Alarming are thus, communicative processes that can increasingly be observed in social media. In particular, a tendency of drawing virtual circles around like-minded people seems to characterize users' information behaviour and opinion formation dynamics, leading to separated viewpoints of communicative milieus, often referred to as echo chambers. This development raises concern about the public as a hub of diverse perspectives and also starts entering the agenda setting of educational innovation, which aims to prepare students for a more responsible information behaviour. In order to support such innovation, the goal of the present work is to complement prior research on echo chamber phenomena that has mainly made use of questionnaires to get to know the involved socio-cognitive variables and dynamics. By the example of two application scenarios, we showcase how an adapted social bookmarking system can be applied for a more direct observation technique that derives behavioural indices for variables of interest from log-file recordings. We believe that the observation of students' information seeking behaviour will also inspire the design of teaching strategies, e-learning and learning analytics tools.

Full text

Adapting an open source social bookmarking system to observe
critical information behaviour
Simone Kopeinik
Institute of Interactive Systems and
Data Science, Graz University of
Technology
Graz, Austria
simone.kopeinik@tugraz.at
Almonzer Eskandar
School of Digital Technologies,
Tallinn University
Tallinn, Estonia
eskandar.almonzer@gmail.com
Tobias Ley
School of Educational Science, Tallinn
University
Tallinn, Estonia
tley@tlu.ee
Dietrich Albert
Institute of Interactive Systems and
Data Science, Graz University of
Technology
Graz, Austria
dietrich.albert@tugraz.at
Paul Seitlinger
School of Educational Science, Tallinn
University
Tallinn, Estonia
pseiti@tlu.ee
ABSTRACT
Constructively dealing with societal problems requires a process
of opinion formation that is preceded by a competent and diverse
search of information. Alarming are thus, communicative processes
that can increasingly be observed in social media. In particular,
a tendency of drawing virtual circles around like-minded people
seems to characterize users’ information behaviour and opinion
formation dynamics, leading to separated viewpoints of communica-
tive milieus, often referred to as echo chambers. This development
raises concern about the public as a hub of diverse perspectives and
also starts entering the agenda setting of educational innovation,
which aims to prepare students for a more responsible information
behaviour. In order to support such innovation, the goal of the
present work is to complement prior research on echo chamber
phenomena that has mainly made use of questionnaires to get to
know the involved socio-cognitive variables and dynamics. By the
example of two application scenarios, we showcase how an adapted
social bookmarking system can be applied for a more direct obser-
vation technique that derives behavioural indices for variables of
interest from log-le recordings. We believe that the observation of
students’ information seeking behaviour will also inspire the design
of teaching strategies, e-learning and learning analytics tools.
CCS CONCEPTS
•Human-centered computing →Web-based interaction
;
So-
cial networking sites;User studies;Open source software;
KEYWORDS
critical search, depolarisation, user studies, learning analytics, Se-
manticScuttle
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
Conference’17, July 2017, Washington, DC, USA
©2018 Copyright held by the owner/author(s).
ACM ISBN 123-4567-24-567/08/06. . . $15.00
https://doi.org/10.475/123_4
ACM Reference Format:
Simone Kopeinik, Almonzer Eskandar, Tobias Ley, Dietrich Albert, and Paul
Seitlinger. 2018. Adapting an open source social bookmarking system to
observe critical information behaviour. In Proceedings of ACM Conference
(Conference’17). ACM, New York, NY, USA, Article 4, 4 pages. https://doi.
org/10.475/123_4
1 INTRODUCTION
Recent years have recorded an increase in using Social Networking
Sites (SNSs) [
5
] and other online forums as a space for online discus-
sion, opinion formation and interaction with others. Irrespective
of our geographic location, we can gather online to view, share
and discuss information in a virtual exchange of opinions and par-
ticipate in deliberative democracy [
12
]. During online discussions,
people interact with content shared by others, get inuenced by
this content, and then, inuence others through their own inter-
actions [
3
]. Particular dynamics between user dispositions (e.g.,
open- vs. closed-mindedness) and content of interaction (e.g., con-
troversial vs. consensual topics) can create a public sphere, a notion
coined by Jürgen Habermas. According to Peter Dahlgren, it can
be dened as “a constellation of communicative spaces in society
that permit the circulation of information, ideas, debates, ideally in
an unfettered manner, and also the formation of political will” [
2
].
Though SNSs were not meant in the rst place to support processes
of a public sphere, they are assumed to cause inadvertent exposure
to political dierence [
1
] and thereby, to support more deliberate
decision-making that draws on alternative information sources [
4
].
In contrast to this positive view of SNSs as a public sphere, other
authors contest this scenario of deliberation (e.g., [
11
]). Specically,
they argue that participants in online discussions show selective
attention toward prior viewpoints, mainly engage with like-minded
people and exhibit closed-mindedness about alternatives [
10
]. This
brings about opinion formation dynamics, which move people to-
wards extreme positions or attitudes. One major reason for this
polarising process is conrmatory search, i.e., the selective exposure
to partisan information (e.g., [
9
,
13
]). While, for sure, the tendency
to selectively expose ourselves to the opinion of like-minded people
was present in the pre- digital world [
8
], the communicative means

Conference’17, July 2017, Washington, DC, USA S. Kopeinik et al.
in social media, such as personalized information services, might
amplify our biases. Through such technology-enhanced exposure to
consonant views, initial doubts continuously give way to a growing
condence in one’s own opinion, leading a person to strengthen an
initial position and attitude [
13
]. A prominent cognitive explanation
of such conrmatory information search bias is the psychological
phenomena of cognitive dissonance [
6
], according to which people
feel stressed when faced with divergent opinions. Political scien-
tists who take this pessimistic perspective on SNSs assume that the
functionalities of social media, such as personalized information
lter [
6
], resonate with the human motive of reducing cognitive
dissonance and thus, reinforce people in performing conrmatory
search. As a consequence, users of a SNS run the risk of getting
locked into a perpetual echo chamber [
9
], a metaphor for an in-
terpersonal phenomenon where other people’s opinions become
echoes of one’s own and start reinforcing instead of challenging
prior beliefs (e.g., [
11
]). In many cases, such self-reinforcement fuels
the phenomenon of group polarisation and political extremism [
14
].
Messages in the daily press about hateful Facebook postings (e.g.
[
7
]) make us aware that such excessively cohesive group dynamics
quite often result in emotionalized and derogative stances to alter-
native viewpoints. Due to this development, it became of public
interest to educate people in digital literacy. As a consequence, a
discussion of possibilities and means to teaching technological ap-
plication started that exceeds the traditional level of teaching how
to eectively use tools and programmes.
In this paper we introduce the application of a social bookmark-
ing platform to observe and interpret users’ learning behaviour
on the Web. Our goal is to contribute to a better understanding
of underlying socio-cognitive dynamics that either lead to a delib-
erate, open-minded or a biased, polarised information behaviour
[
15
]. As a demonstration and in a rst step, we make use of the
present online scenario to cross-validate results of a study that
has found a systematic relationship between people’s tendency to
perform conrmatory search on the Web and polarisation [
13
]. As
this nding, however, is based on data gathered through question-
naires, the question remains open whether a similar pattern can
be found if both variables are observed directly by extracting them
from log-le recordings. With this, we also aim to contribute to
the development of teaching strategies of digital competences in
schools. A suitable processing of collected data may lead to design
implications for formative assessment and timely interventions.
2 ENVIRONMENT
The environment that we propose is build upon the open source
social bookmarking tool SemanticScuttle
1
. It constitutes a collabora-
tive platform to collect and share information online. The function-
ality of the platform has been tailored to monitor users’ information
search behaviour. So far, it supports mechanisms to observe users’
collection of resources, their assessment of the trustworthiness
of information, a user’s tendency towards polarisation and their
manifestations of conrmatory search. This has been realized with
adaptations in the platform’s range of functionality, in its user
interfaces and database and the deployment of logging services.
1https://sourceforge.net/projects/semanticscuttle/
First, to avoid unnecessary training periods, the original plat-
form was reduced in functionality. Remaining functions allow for
collecting, annotating and reecting on Web resources as well as
for browsing through bookmarks that are shared among the users.
The Annotation Interface and the Search Interface were extended as
described in the following paragraphs:
2.1 Annotation Interface
To support users’ reection on their Web resources, the Annotation
Interface was adapted as illustrated in Figure 1. It was designed to
enable the observation of students’ ability to assess the credibility
of information, their tendency of polarisation during information
search and information consumption and their ability to embed new
concepts into their knowledge representation. Figure 1 illustrates
the interface that takes basic information about the resource in
input elds labelled with one. It consists of the URL, a name and
freely chosen keywords (tags). Tags assigned by a user can be used
to observe particular semantics of the opinion formation process.
Marked with two is a slider that asks for the user’s perception
of trustworthiness towards the selected resource. The slider ranges
from 0 (“not at all trustworthy”) to 10 (“very trustworthy”). In com-
bination with the resource’s URL, this information can be used to
better understand users’ ability to evaluate the quality of informa-
tion and information sources.
In the last block marked with three, a set of topic aspects is
presented to the user. These aspects vary with the search topic and
therefore, can be congured by the site administrator. A bipolar
rating scale is given by two sliders, ranging from -3 (“very nega-
tive”), over 0 (“neutral”) to 3 (“very positive”). The sliders ask for
the author and user stance towards single aspects and allows for
inferring conrmatory search behaviour and polarisation.
2.2 Search Interface
The Search Interface presented in Figure 2 consists of two parts.
Marked with one is the keyword search that is natively provided by
the system. Beyond that, the environment was extended to enable
the browsing of Web resources according to positive and negative
attitudes towards pre-dened topic aspects (labelled with two). To
this end, each aspect was displayed as a clickable keyword within
the two boxes. That means that if a user clicks on "Cyborgization“
within the "Pro Arguments”, all bookmarks in the system (added
by any group member) with a positively indicated author stance to-
wards "Cyborgization“ are listed to the user. This search aid enables
the observation of collaborative information search behaviour and
further contributes to the assessment of users’ tendencies towards
conrmatory search.
2.3 Technical Facts
SemanticScuttle is implemented in PHP and Java Script. Persistent
data is saved in an SQL database. To implement the adaptation of the
platform, this database was extended to embed information about
topic aspects, user and author opinions. In addition, an apache-
solr
2
based log data server was developed that is exposed through
a Web service interface. It is in place to receive all user interaction
data. This data can further be analysed for research purposes or
2http://lucene.apache.org/solr/

Observing critical information behaviour Conference’17, July 2017, Washington, DC, USA
Figure 1: Adapted Annotation Interface.
Figure 2: Adapted Search Interface.
accessed continuously to feed into dashboards or other learning
analytics tools.
The source code of the adapted SemanticScuttle instance and ad-
ditional background logging services is freely available via github
3
.
3 USE CASES
In this section we describe two application scenarios of the software.
The rst one reports on a pilot approach that was implemented as
an online user study with volunteers collecting resources over a
period of two weeks. The second example provides insight on a
currently running school experiment and its possible application
3https://github.com/meins/ReectiveScuttle
to support formative assessment of students’ information search
processes.
3.1 Study Procedure
Prior to the study, each participant was provided with a brief de-
scription of the study setup and its main research goals. Participants
were informed about tasks they had to complete, the data that was
gathered and potential privacy concerns. Based on this informa-
tion, participants or their legal guardians (if applicable) signed a
consent form. To ensure data protection and anonymity, users were
identied by a pseudonym they created for themselves. For commu-
nication purposes we kept a list of email addresses and associated
pseudonyms for the duration of the experimental phase.
3.2 Online Pilot Experiment
The rst use case presents a pilot study that explores information
search behaviour in a controlled online experiment. To this end, 20
participants were recruited through Facebook, of which 13 com-
pleted the study.
After signing the consent form, they were instructed to research
the topic "transhumanism“ online. This topic was selected as we
expected it to be controversial while interesting for a wide range
of people. At least 20 resources had to be collected continuously
over a period of two weeks. Participants added their bookmarks
to the provided SemanticScuttle instance, using the Annotation In-
terface described in section 2. They had to reect on the selected
resource, assign it to at least one predened topic facet (i.e., "self-
optimization", "cyborgization", "intervene in evolution", "faith in

Conference’17, July 2017, Washington, DC, USA S. Kopeinik et al.
progress") and indicate their personal and the author’s stance to-
wards the selected aspects. The resulting workload for a participant
was about 20 minutes per day and allowed us to look into dierent
aspects of the opinion formation process, such as conrmatory
search or polarisation.
Initial Results. The main research question of this study investi-
gates whether there is a positive correlation between polarisation
and conrmatory search. Surprisingly, results showed a signicant
negative correlation between the two measures indicating that an
increase in conrmatory search on a topic is accompanied by a less
polarised stance towards the aspect.
This counter-intuitive pattern of results can be explained post-
hoc by the fact that most of the participants performing conrma-
tory search had already started with a rather balanced stance to-
wards dierent topic aspects. Thus, by collecting resources authored
by like-minded people, i.e., by performing a conrmatory search,
they had the chance to get to know additional arguments that
supported their already developed (balanced) stance and thereby
helped further decrease their polarization score (i.e., deviation from
a balanced stance).
3.3 Application in the School Context
Currently, the environment is being used in a real-life classroom
study. While monitoring students’ information search behaviour,
the study is part of a participatory design approach to collabora-
tively develop software that supports the teaching of digital literacy.
In this study, a total of 90 high school students between 14 and 18
and three teachers of two schools are taking part. After obtaining
parents’ informed consent, students attended an initial workshop to
become familiar with the problems of echo chambers, lter bubbles
and fake news. Also, they were informed about means to eval-
uate the quality of information. In four school lessons, students
researched the topic "global nutrition" under the topic aspects of
”genetic engineering", “conservation”, “sustainable consumption”
and “development aid”. The topic and its aspects were selected by
the participating teachers in accordance with the curriculum of the
age group.
The environment allows us to monitor Web resources students
collect and interact with, their ability to evaluate the quality of re-
sources and their tendency towards polarisation and biased search
tendencies. In this work, the focus is on understanding students’
struggles in online research to allow formative diagnosis and inter-
vention. For instance, if a student assesses a user comment of an
SNS as trustworthy, this may indicate a lack in information evalu-
ation skill. Collected log data will further be discussed in teacher
workshops to collaboratively design a learning dashboard that will
support the formative evaluation of students’ information search
behaviour.
4 DISCUSSION AND FUTURE WORK
In this paper we have presented an approach to observe information
behaviour and opinion formation dynamics directly by using an
adapted instance of the open source social bookmarking platform
SemanticScuttle. In contrast to prior research that explored the rela-
tionship between conrmatory search and polarisation (e.g. [
15
])
on the basis of survey data, the introduced platform oers an envi-
ronment to investigate this and related phenomena in behavioural
observation studies.
Two initial use cases in dierent contexts (online volunteers,
high school students) give an idea of how to apply the system in
practice. Our current basis of collected data does not allow for
drawing stringent conclusions and the reported results suer from
data loss due to high dropout rates. However, more robust results
can be expected from upcoming laboratory studies, where users
will operate in a more controlled environment. Also, the presented
real-life school study is still in progress, and with 90 participat-
ing students promises to improve our understanding of students’
information behaviour. Furthermore, the goal of the present and
future studies is to derive design implications for further platform
development, depolarising discourse services and learning analytics
visualizations.
ACKNOWLEDGMENTS
This work is supported by the Austrian Science Fund (FWF): P 27709-
G22, TCS-034 and the European Union’s Horizon 2020 research and
innovation programme under grant agreement No. 669074. We are
grateful for the help of Dominik Kowald, Helena Flemming and
Marcel Jud in the realization of the experiments.
REFERENCES
[1]
Jennifer Brundidge. 2010. Encountering "dierence" in the contemporary pub-
lic sphere: The contribution of the Internet to the heterogeneity of political
discussion networks. Journal of Communication 60, 4 (2010), 680–700.
[2]
Peter Dahlgren. 2005. The Internet, public spheres, and political communication:
Dispersion and deliberation. Political communication 22, 2 (2005), 147–162.
[3]
Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec,
and Christopher Potts. 2013. No country for old members: User lifecycle and
linguistic change in online communities. In Proceedings of the 22nd international
conference on World Wide Web. ACM, 307–318.
[4]
Anna De Liddo and Simon Buckingham Shum. 2013. The Evidence Hub: harness-
ing the collective intelligence of communities to build evidence-based knowledge.
(2013).
[5]
Nicole B Ellison et al
.
2007. Social network sites: Denition, history, and scholar-
ship. Journal of computer-mediated Communication 13, 1 (2007), 210–230.
[6]
Leon Festinger. 1954. A theory of social comparison processes. Human relations
7, 2 (1954), 117–140.
[7]
Klenk Florian. 2016. Boris wollte mich verbrennen. Falter 46/16 (11 2016).
https://www.falter.at/archiv/wp/boris-wollte-mich-verbrennen
[8]
William Hart, Dolores Albarracín, Alice H Eagly, Inge Brechan, Matthew J Lind-
berg, and Lisa Merrill. 2009. Feeling validated versus being correct: a meta-
analysis of selective exposure to information. Psychological bulletin 135, 4 (2009),
555.
[9]
Robert Huckfeldt, Jeanette Morehouse Mendez, and Tracy Osborn. 2004. Dis-
agreement, ambivalence, and engagement: The political consequences of hetero-
geneous networks. Political Psychology 25, 1 (2004), 65–95.
[10]
Michael MacKuen, Jennifer Wolak, Luke Keele, and George E Marcus. 2010. Civic
engagements: Resolute partisanship or reective deliberation. American Journal
of Political Science 54, 2 (2010), 440–458.
[11]
Dimitar Nikolov, Diego FM Oliveira, Alessandro Flammini, and Filippo Menczer.
2015. Measuring online social bubbles. PeerJ Computer Science 1 (2015), e38.
[12]
Bryan C Semaan, Scott P Robertson, Sara Douglas, and Misa Maruyama. 2014.
Social media supporting political deliberation across multiple public spheres:
towards depolarization. In Proceedings of the 17th ACM conference on Computer
supported cooperative work & social computing. ACM, 1409–1421.
[13]
Natalie Jomini Stroud. 2010. Polarization and partisan selective exposure. Journal
of communication 60, 3 (2010), 556–576.
[14]
Cass R Sunstein. 2001. Echo chambers: Bush v. Gore, impeachment, and beyond.
Princeton University Press Princeton, NJ.
[15]
Yu Wang, Jiebo Luo, Richard Niemi, and Yuncheng Li. 2016. To Follow or Not to
Follow: Analyzing the Growth Patterns of the Trumpists on Twitter.. In News@
ICWSM.