Conference PaperPDF Available

A Real-Life School Study of Confirmation Bias and Polarisation in Information Behaviour

Authors:

Abstract and Figures

When people engage in Social Networking Sites, they influence one another through their contributions. Prior research suggests that the interplay between individual differences and environmental variables , such as a person's openness to conflicting information, can give rise to either public spheres or echo chambers. In this work, we aim to unravel critical processes of this interplay in the context of learning. In particular, we observe high school students' information behavior (search and evaluation of Web resources) to better understand a potential coupling between confirmatory search and polarization and, in further consequence, improve learning analytics and information services for individual and collective search in learning scenarios. In an empirical study, we had 91 high school students performing an information search in a social book-marking environment. Gathered log data was used to compute indices of confirmatory search and polarisation as well as to analyze the impact of social stimulation. We find confirmatory search and polarization to correlate positively and social stimulation to mitigate, i.e., reduce the two variables' relationship. From these findings, we derive practical implications for future work that aims to refine our formalism to compute confirmatory search and polarisation indices and to apply it for depolar-izing information services.
Content may be subject to copyright.
A Real-Life School Study of Confirmation Bias
and Polarisation in Information Behaviour
Simone Kopeinik1, Elisabeth Lex1,2, Dominik Kowald1, Dietrich Albert2,3, and
Paul Seitlinger3,4
1Know-Center GmbH, Graz, Austria
{skopeinik,dkowald}@know-center.at
2ISDS, Graz University of Technology, Graz, Austria
{elisabeth.lex,dietrich.albert}@tugraz.at
3Institute of Psychology, University of Graz, Graz, Austria
4School of Educational Sciences, Tallinn University, Tallinn, Estonia
pseiti@tlu.ee
Abstract. When people engage in Social Networking Sites, they in-
fluence one another through their contributions. Prior research suggests
that the interplay between individual differences and environmental vari-
ables, such as a person’s openness to conflicting information, can give rise
to either public spheres or echo chambers. In this work, we aim to unravel
critical processes of this interplay in the context of learning. In particular,
we observe high school students’ information behavior (search and eval-
uation of Web resources) to better understand a potential coupling be-
tween confirmatory search and polarization and, in further consequence,
improve learning analytics and information services for individual and
collective search in learning scenarios. In an empirical study, we had 91
high school students performing an information search in a social book-
marking environment. Gathered log data was used to compute indices
of confirmatory search and polarisation as well as to analyze the impact
of social stimulation. We find confirmatory search and polarization to
correlate positively and social stimulation to mitigate, i.e., reduce the
two variables’ relationship. From these findings, we derive practical im-
plications for future work that aims to refine our formalism to compute
confirmatory search and polarisation indices and to apply it for depolar-
izing information services.
Keywords: Learning Analytics ·Real-Life School Study ·Information
Behaviour ·Polarisation ·Confirmatory Search.
1 Introduction
When people engage in online discussions in Social Networking Sites (SNSs)
or different online forums, they interact with content shared by others, get
influenced by this content, and then, influence others through their interac-
tions [6]. Particular dynamics between user dispositions (e.g., open- vs. closed-
mindedness) and content of interaction (e.g., controversial vs. consensual topics)
2 Simone Kopeinik et al.
can create a public sphere [14], i.e., a place where people gather, share informa-
tion and participate in critical debates about public affairs [5]. In principle, SNSs
can support processes of a public sphere as they connect people and expose users
to political differences online [4]. This confrontation with different viewpoints
can encourage decision-making that draws on alternative information sources
[7]. However, as discussed in related work (e.g., [26, 1]), users of SNSs show
a propensity to engage with like-minded others and tend to be closed-minded
about alternative information [23]. One reason for the reinforcement of such pro-
cesses is personalized filtering [27], which helps us find information related to
what we prefer or already know. This caters to people’s tendency to seeking
information that corresponds to their existing beliefs (i.e., confirmatory search).
As a consequence, people move towards extreme positions and attitudes [16, 33]
(i.e., polarization). Messages in the daily press about hateful Facebook post-
ings make us aware that such dynamics quite often result in emotionalized and
derogative stances to alternative viewpoints. Thus, it becomes of public interest
to strengthen peoples’ education in digital literacy. The motivation of this work
is two-fold. On the one hand, our long-term goal is to help increase students’
awareness and competences to consume information online critically, a skill many
students lack to this date [25]. On the other hand, we aim to contribute to the
development of learning analytics services for teaching and improving teaching
strategies of digital competences in schools. We believe the progress towards
these goals should be built upon a thorough understanding of underlying mental
processes.
In this work, we propose means to study confirmatory search (search for
consensual resources) and polarisation (drifting towards extreme positions) dy-
namics in an educational context. Our main aim is to better understand socio-
cognitive dynamics leading to either deliberate, open-minded or biased, polarised
information behavior [35]. To this end, we present a study that observes and in-
terprets students’ information behavior in a semi-controlled online environment.
In particular, we investigate the impact of shared artifacts (i.e., social tags and
bookmarks) on a collective search process and expect two artifact-mediated ben-
efits: (i) the introduction of potentially new ideas (i.e., concepts labeled by freely
chosen tags) will help a student activate new associations to a given topic and
thereby, mitigate a tendency towards monotonous thoughts regarding a given
problem [32], and (ii) the revealing of tags other students have previously cho-
sen to index underlying concepts (e.g., by recommending social tags) will support
the collective of students to mitigate the vocabulary problem, i.e., to agree on a
common terminology of concepts more quickly [34].
We, therefore, raise the following two research questions:
RQ1: What is the impact of shared artifacts (social tags and bookmarks) on
confirmatory search and polarisation in collective search processes?
RQ2: Can shared artifacts (social tags and bookmarks) be applied to reduce
the vocabulary problem in collective search processes?
To examine these questions under natural conditions, we have conducted a
study with 91 high-school students performing an information search task in
Confirmation Bias and Polarisation in Information Behaviour 3
an adapted version of the open-source social bookmarking system SemanticS-
cuttle. This system can be used as a platform to collect and share information
online and, from a research perspective, allows for recording user data related to
information selection and opinion formation processes. Furthermore, to examine
the impact of shared artifacts on these processes, three different conditions have
been varied experimentally: As a baseline for comparisons, we had one group of
students receiving no recommendations at all. In the following, this baseline is
denoted ’None’. By contrast, the other two groups have been supported by tag
recommendations, which we derived either inclusively from the entire group’s
tagging activities (’social’ condition) or exclusively only from the student’s per-
sonal tagging history (’individual’ condition).
The present work contributes to current research on technology-enhanced
learning by demonstrating how students’ search and sharing behavior on the
Web can be observed under natural conditions and how this behavior can be an-
alyzed automatically in cognitive terms. Beyond that, it highlights a depolarizing
impact of shared artifacts and can thus guide future design processes aiming to-
wards more effective recommender systems in computer-supported learning sce-
narios. We, therefore, believe that the study helps to further learning analytics
services for the teaching and training of critical and nuanced search behavior.
2 Related Work
The productive use of online information tools demands teaching strategies that
address relevant competences [22]. To date, students’ competencies and aware-
ness to critically consume information are still widely lacking [25]. There is no
evidence of digital skills that exceed the level of using technologies frequently
[13]. Quite to the contrary, existing research reports on students’ superficial
understanding of new technologies and their lack of information seeking and
analytical skills necessary to assess and learn from online resources (e.g., [3]).
2.1 Supporting Collective Search
A central motive to engage in SNSs is to acquire information, in private, societal-
political, or vocational contexts. Therefore, this engagement can be framed as
participation in a collective search, where the term collective means that differ-
ent individuals act in a common environment and influence each other through
shared artifacts, such as links to external news sites. Prior work has shown that
even simple features, such as shared keywords (i.e., social tags) can become
sources of mutual influences and can alter mental states (e.g., information goals)
through the process of semantic priming (e.g., [11, 31]). The term priming refers
to an increased availability of traces in long-term memory evoked by an environ-
mental stimulus (e.g., the tag ”polarisation”), which is mentally connected to
these traces (e.g., the associations of ”echo chamber” or ”confirmatory search”)
as well as to the subsequent behavioral consequences that follow from such prim-
ing, like performing a keyword-based search or accepting/declining recommended
pieces of information.
4 Simone Kopeinik et al.
When it comes to designing effective learning analytics services, which ob-
serve and support students’ search behavior, the question should be raised, in
which manner shared artifacts need to be (re)presented to facilitate a collective
and open information search. In the context of the present study, we ask for
the extent to which the prominence of other members’ ideas and contributions
should be increased or decreased to reduce the coupling between confirmatory
search and polarisation eventually. Technology-enhanced group creativity pro-
vides some answers to these questions (e.g., [28]), which, e.g., explores the effects
of shared artifacts on individuals’ divergent thinking abilities during a collective
information search (e.g., [32]). Among others, this research demonstrates that
the recommendation of social tags (i.e., tags that are semantically related to a
user’s search but are generated by someone else) are on average more conducive
to each group member’s ideational fluency (i.e., the rate at which new ideas come
to one’s mind) than the recommendation of individual tags (i.e., semantically
related tags drawn from a user’s own tag vocabulary).
From a cognitive-psychological perspective, neurophysiological processes are
stimulated by environmental influences and help trains of thoughts diverge.
These processes should function antagonistically to mental processes that would
otherwise actuate the convergence of contents of consciousness [15], such as the
convergence of a current belief or opinion and an ongoing information goal. Put
differently, cognitive processes during a search that support divergent thinking
should simultaneously counteract confirmatory tendencies (e.g., the conversion
of beliefs into search goals) and in further consequence, mitigate forces driving
polarisation. Therefore, we assume and predict that providing social recommen-
dations in the form of shared artifacts (e.g., social tags and social bookmarks)
will result in a relatively weaker coupling between confirmatory search and po-
larisation than providing individual or even no recommendations.
2.2 Tagging and Semantic Stabilisation
Tagging is a mechanism to annotate resources individually or socially [36]. In
TEL, it has demonstrated its potential to facilitate search, to foster reflection
upon retrieved learning contents [19] and to promote the development of a
metacognitive level of knowledge [2]. Throughout the learning process, structures
of users in a social tagging environment assimilate [12]. Such implicit agreement
on a common vocabulary over time and in meaning is called semantic stabil-
ity [34]. The term semantic stabilization describes the evolution of convergence
in vocabulary choices of different groups [18]. Research has described a mutual
influence between learners’ internal knowledge representation and the tagging
vocabulary that emerges in the social information system, in which they inter-
act [10]. Ley and Seitlinger [20] investigate these dynamics and prove a positive
influence of semantic stabilization on individual learning. Consequently, it can
be argued that a high level of semantic stability provides a structure that sup-
ports individual learning activities and therefore, can be conducive to individual
learning gains [20]. Because students’ typically struggle with the achievement
of a semantically stable vocabulary in their usage and amongst their learning
Confirmation Bias and Polarisation in Information Behaviour 5
peers [20] recommendation mechanisms that introduce shared artifacts (e.g.,
tags) have been proposed [9]. Thus, expending prior research in inquiry-based
learning [18], we explore the impact of shared artifacts (recommended tags) on
semantic stabilization in an information search task.
3 Experimental Setup
For this study, we monitored and explored students’ information search behavior
in a real-life classroom setting. The study took place at Graz University of Tech-
nology, Institute of Interactive Systems and Data Science, as part of a top citizen
science funding program, in which citizens are encouraged to participate in re-
search endeavors actively. Three teachers and four high-school classes from two
schools were recruited to participate in different project stages during the school
terms of 2017 and 2018. In this time, 91 students (60 female and 31 male), aged
between 14 and 18, took part in workshops that included completing worksheets,
questionnaires, interviews, focus groups, and information search tasks. Here, we
report on data insights extracted from the students’ information search task.
3.1 Study Procedure and Design
Before the study, each participating student was provided with a brief descrip-
tion of the study setup and its main research goals. They were informed about
the tasks they had to complete, the data that was gathered and potential privacy
concerns. To ensure data protection and anonymity, students were identified by
a pseudonym they created for themselves. After obtaining guardians’ informed
consent, students attended an introductory workshop to familiarize with the
problems of echo chambers, filter bubbles, and fake news. Also, they were in-
formed about the means to evaluate the quality of information. Before the search
task, teachers selected a topic and associated topic aspects that fit the curriculum
of the age group. This topic was depicted in the environment.
Within the information search task, students were instructed to explore the
topic ”global nutrition” by collecting information to the four defined aspects
”genetic engineering”, ”conservation”, ”sustainable consumption” and “devel-
opment aid”. They had to upload their articles as bookmarks to the study envi-
ronment. Students used the annotation tool shown in Figure 1 to reflect on their
Web resources. They had to select at least one predefined topic aspect, indicate
their attitude and an estimation of the author’s attitude towards the chosen
aspects. The requested set of information provides insights on different facets of
the opinion formation process, such as confirmatory search or polarisation.
To simulate a search environment with social, individual or no stimulation
on appearing information dynamics, students were split into three groups. De-
pending on the group, the environment provided for the social and individual
stimulation tag clouds and tag recommendations based on social or individual
data. Students of the third group were neither presented with a tag cloud nor
tag recommendations. This leads to the independent variable ”search condition”
6 Simone Kopeinik et al.
with the three levels ”Social”, ”Individual” and ”None”. As dependent variables,
we observed semantic stabilization, recommender accuracy, confirmatory search,
and polarisation.
Fig. 1. Study Environment: Annotation Interface.
3.2 Evaluation Measures.
Semantic Stabilisation While there is a multitude of metrics to evaluate se-
mantic stability [34], few methods can deal with narrow folksonomies, where
items are tagged only by the uploading user (as it is in our case). Lin et al. [21]
present the Macro Tag Growth Method (MaTGM) that measures social vocabu-
lary growth at a systemic level, looking at the social tagging system as a whole.
In this study, experimental groups (i.e., ”Social”, ”Individual” and ”None”) are
observed as separate environments. The MaTGM is applied to compare the tag
growth within these systems. For each group, the collected bookmarks (tag as-
signments) are sorted according to their timestamps. The tag growth after each
bookmark, is calculated as a value pair (tgi, f (tgi)), where tgiis the cumulative
number of tags, and f(tgi) is the cumulative number of unique tags occurring in
ibookmarks.
Recommender Accuracy. To evaluate the efficacy of the tag recommendation
algorithms that operate either on social or individual tagging data, the perfor-
mance metrics recall and precision [24] were applied. To calculate recall and
precision, we determined for each bookmark the relation of tags recommended
to a user for a Web resource to the tags that the user assigned to a resource.
Confirmation Bias and Polarisation in Information Behaviour 7
Recall (R) indicates how well the recommendation supported the user, giving
the relation between correctly recommended tags (i.e., the subset of recom-
mended tags that the user assigned to the Web resource) and the set of tags
the user needed to describe the Web resource.
R(Tu,r,ˆ
Tu,r) = |Tu,r ˆ
Tu,r|
|Tu,r|(1)
Precision (P) is the number of tags that have been recommended correctly
divided by the number of recommended tags.
P(Tu,r,ˆ
Tu,r) = |Tu,r ˆ
Tu,r|
|ˆ
Tu,r|(2)
3.3 Behavioral Indicators
Confirmatory Search. Confirmatory search is described as the process of
seeking information that is biased towards existing believes [29]. Prior research
deduces confirmatory search in laboratory studies, by numerical comparisons of
experimental and control groups’ document selections, which confirm current
beliefs or not [30]. With the environments’ Annotation Interface (see Figure 1)
such data is tracked with every resource upload. In Equation 3, we present one
option to calculate confirmatory search (CS) with such data:
CSi,t = (1 |ASti,t U Sti,t |
diffmax
)(1 e−|ASti,t |) (3)
Here, CS with respect to a Web resource iand a topic tis defined as the difference
of a user’s stance USt towards tand the author’s stance ASt towards twith
respect to i. The second term includes an exponential function to increase the
impact of strongly polarised Web resources on the one hand, and to subtract out
resources with a balanced author stance (i.e., ASti, t == 0) on the other hand.
CS of a user uis calculated as the mean value over all observed topic events of
u, as formalized in Equation 4:
CSu,t =
n
X
i=0
CSi, t
n(4)
Polarisation. Equation 5 gives a value for a user’s polarisation. In line with
[8], we understand polarisation as a twofold construct that is characterized by
a state and a process. Polarisation as a state is defined by the distance of an
attitude position to a theoretical maximum of that attitude. The polarisation
process ∆P olu,t describes the development of the attitude position in relation
to this theoretical maximum over time. This is represented by the normalized
difference of the user’s stance towards tcaptured at the first topic event to the
nth one.
∆P olu,t =|UStn,t USt0,t |
diffmax
(5)
8 Simone Kopeinik et al.
Equation 6 calculates a users’ polarisation as a combination of polarisation
change and the extremes of the final user stance UStn.
P olu=w1∆P olu+w2|US tn|
on
2(6)
where onis the number of possible absolute values (except zero) the user or
author stance can capture.
3.4 Study Environment
The study environment is based on the open-source social bookmarking sys-
tem SemanticScuttle5, which is a collaborative platform to collect and share
information online. To fit the requirements of the experimental setting, it was
adapted in its annotation and browsing interfaces and expanded by matching log
data services. This has been realized with adaptations in the platform’s range
of functionality, in its database, user interfaces and the deployment of data log-
ging services. To support users’ reflection on their collected Web resources, the
Annotation Interface was adapted as illustrated in Figure 1. It is designed to
enable the observation of students’ ability in assessing the credibility of informa-
tion, their tendency of polarisation during information search and information
consumption as well as their ability to embed new concepts into their knowledge
representation. Figure 1 illustrates the interface that takes basic information
about the resource in input fields labeled with ”one”. It consists of the URL, a
name and freely chosen keywords (i.e., tags). Tags assigned by a user can be used
to observe particular semantics of the opinion formation process. Marked with
”two” is a slider that asks for the user’s perception of trustworthiness towards
the selected resource. The slider ranges from 0 (”not at all trustworthy”) to 10
(”very trustworthy”). In combination with the resource’s URL, this information
can be used to better understand users’ ability to evaluate the quality of infor-
mation and information sources. In the last block marked with ”three”, a set of
topic aspects is presented to the user. These aspects vary with the search topic
and therefore, can be configured by the site administrator. A bipolar rating scale
is given by two sliders, ranging from -3 (”very negative”), over 0 (”neutral”) to
3 (”very positive”). The sliders ask for the author and user stance towards single
aspects and allow for inferring confirmatory search behavior and polarisation.
Further details on the study environment and its technical adaptations are given
in Kopeinik et al. [17].
3.5 Data Characteristics
Table 3.5 shows the data characteristics separated according to the three exper-
imental conditions: ”Social”, ”Individual” and ”None”.
The final dataset combines collected data from students of four participating
school classes. Students of each class were randomly assigned to one experimental
condition.
5http://semanticscuttle.sourceforge.net/
Confirmation Bias and Polarisation in Information Behaviour 9
#users #bmks #tags Tuser #ET A
Social 35 407 1078 3.86 603
Individual 35 362 753 3.83 527
None 21 276 895 3.76 297
Table 1. Illustration of the data characteristics, given by the number of users (#users),
bookmarks (#bmks) and tags (#tags), the average number of topics covered by a user
(Tuser) and the captured events of topic attitudes (#ET A).
4 Results and Discussion
This section presents the result of our study that examines the impact of shared
artifacts on aspects of information selection and opinion formation processes.
4.1 RQ1: What is the impact of shared artifacts (social tags and
bookmarks) on a coupling between confirmatory search and
polarisation in a collective search?
Based on prior empirical work, we expected a coupling, i.e., systematic relation-
ship between participants’ tendency towards confirmatory search (CS) (Equa-
tions 3 and 4) and polarisation (Equations 5 and 6). According to our theoretical
assumptions (see Section 2.1), we predicted this coupling to be smaller under the
”Social” condition, when users are supported by social tag recommendations and
shared bookmarks, than under the ”Individual” and ”None” search condition. To
test both of these predictions, we performed a linear regression of CS (criterion)
on the continuous predictor ”polarisation” and the categorical predictor ”search
condition”, and included an interaction term to quantify potential differences in
the slope (as an index of the CS-polarisation coupling) across the three search
conditions. 91 data points have entered the regression (NN one = 20, NIndividual
= 35, NSocial = 36 participants) explaining about 50% of variance in polarisation
(adjusted R2= .467, p<.001). This effect is represented well by the scatter plot
of Figure 2, which draws polarisation against CS and whose best fitting regression
lines indicate a positive and moderate slope for each of the three conditions. The
outcome for the “None” condition is represented by the steep red line, for which
we have found a standardized beta coefficient of β= 1.07 (t= 5.86, p < .001).
The other two lines appear to be flatter (βIndividual = 0.65; βSocial = 0.46), sug-
gesting an interaction between the two predictors of CS and search condition.
In line with our expectation, however, this decrease in the CS-polarisation rela-
tionship is significant only under the social condition (t=2.59, p < .05) but
not under the individual (t=1.98, n.s.). We can therefore conclude that (i)
similar to [33], the present study provides evidence of a CS-polarisation coupling
too, which (ii) gets mitigated through the influence of shared artifacts (under
the ”Social” condition).
As we now gained clear evidence that the CS-polarisation coupling is looser
under the ”Social” than the other two conditions, we further examined whether
these group differences are also reflected by differences in the overall range of
10 Simone Kopeinik et al.
Fig. 2. Correlation between confirmatory search and polarisation illustrated in the
three experimental conditions.
(a) Confirmatory Search (CS) (b) Polarisation
Fig. 3. Box plots depicting medians and quartiles of the CS and polarisation scores
separately for the three groups “None”, “Social”, and “Individual”.
values in the two variables. Given that the two variables fuel each other in this
coupling, the main group effect for both polarisation and CS should come about,
with relatively smaller levels under the ”Social” condition.
We find a strong effect in the case of polarisation and a weak effect in the case
of CF. First, the descriptive results, as represented by the plots in Figures 3a and
3b, point towards a pattern that is in line with both expectations, i.e., the median
is relatively lower in the social than in the other two groups. However, the test
of significance, for which we have run a non-parametric, i.e., the Kruskal-Wallis
test, to take into account the apparent violation of the equal variance assumption
(see the box plots’ interquartile ranges), has underlined this pattern only in case
of polarisation (χ2(2) = 7.20, p < .05) but not of CS (χ2(2) = 4.55, n.s.).
We conclude that a relatively stronger CS-polarisation coupling indeed man-
ifests in a higher CS value range and that prospectively, the same can be an-
Confirmation Bias and Polarisation in Information Behaviour 11
(a) Macro Tag Growth Method shows the
semantic stabilization on a system level.
The graphs plot the search conditions:
”None”, ”Social” and ”Individual”.
(b) Recall/Precision plots showing the ac-
curacy of recommendation algorithms in
the ”Social” and the ”Individual” experi-
mental condition.
Fig. 4. The impact of shared artifacts on vocabulary development in the individual
and collective search task.
ticipated for polarisation as well, given a sufficiently long period of observation
and a relatively more extensive sample of participants. Of course, the latter
anticipation needs to be validated in future work.
4.2 RQ2: Can shared artifacts (social tags and bookmarks) be
applied to reduce the vocabulary problem in collective search
processes?
We address this research question considering two angles. First, we look at the
semantic stabilization itself. Second, we investigate which recommendation ap-
proach can best support the process of semantic stabilization in the context of
online information. Figure 4a illustrates the tag growth in the three experimental
conditions represented as Macro Tag Growth Function. Comparing the vocabu-
lary development of the groups, we find that while initially, the graphs overlap
in all three groups, students in the two groups that receive tag recommenda-
tions (i.e., ”Social” and ”Individual”), start to introduce less new vocabulary in
relation to tags than the group with no recommendations. This effect is even
stronger for the group in the ”Social” condition. In other words, we can observe
two phenomena: i) students in the ”Individual” condition reuse their own words
more frequently and thus, apply a more consistent terminology in their personal
resource annotation; ii) students in the ”Social” condition start to reuse and pick
up the vocabulary of their peers faster. This demonstrates the positive effect of
social tag recommendations on semantic stabilization. In summary, results show
the benefit of tag recommendations on semantic stabilization, even when applied
in the context of individual information scenarios, which implies that previous
findings [18] can be generalized to a collective information setting.
Results presented in Figure 4b pay attention to the efficiency of provided tag
recommendations. The recall/precision plot highlights the strong performance of
tag recommendations based on the collaborative vocabulary of a group (”Social”
12 Simone Kopeinik et al.
condition) in comparison to recommendations based on individual tag traces. To
the best of our knowledge, such an effect has not been reported in any other TEL
recommender study. We explain the effect with the open and dynamic nature
of the information search task itself. Students were asked to research a given
topic and related aspects throughout four school lessons. This constitutes an
explorative learning endeavor, where information takes place within a specific
scope, while also developing over time. Consequently, we observe that social tag
recommendations can support the explorative process within the information
task, while tag recommendations that are based on the historic word traces of
an individual are not suited to depict such continuous development.
5 Conclusion
In this paper, we presented an approach to study opinion dynamics in a collabo-
rative search task. In a two-week real-life classroom study, we collected data on
students’ information behavior, their ability to evaluate information, and their
tendencies towards confirmatory search and polarization. Based on the data that
we gathered in the presented semi-controlled study environment, we proposed
a formalism to calculate confirmatory search and polarisation in information
behavior and found a strong correlation between the two constructs. This is in
line with prior research and constitutes a proof of concept of the platform’s field
application. We understand the presented platform with its functionality and
the formalism of behavioral indicators as a starting point for further discussion
and exploration towards understanding and supporting critical information be-
havior in formal and informal learning. Gained insights will contribute to the
prospective design and development of depolarising discourse services, learning
analytics services, and visualizations.
Moreover, we found a positive impact of shared artifacts on polarisation and
semantic stabilization. This highlights the benefit of social influence on the early
ideation process. In the future, we plan to corroborate our findings in long term
studies.
Acknowledgements This work is supported by the Austrian Science Fund
(FWF) TCS-034 Project, the European Union’s Horizon 2020 research and in-
novation programme under grant agreement No.669074 and by the Know-Center.
The Know-Center is funded within the Austrian COMET Program - Competence
Centers for Excellent Technologies. We are grateful for the help of Helena Flem-
ming, Kevin Harkim and Marcel Jud in the realization of the school workshops,
and the Projects Miles and HELI-D funded the Gesundheitsfond Steiermark.
References
1. Bakshy, E., Rosenn, I., Marlow, C., Adamic, L.: The role of social networks in
information diffusion. In: Proc. of WWW’2012. pp. 519–528. ACM (2012)
Confirmation Bias and Polarisation in Information Behaviour 13
2. Bateman, S., Brooks, C., Mccalla, G., Brusilovsky, P.: Applying collaborative tag-
ging to e-learning. In: Proc. of WWW’2007) (2007)
3. Brown, C., Czerniewicz, L.: Debunking the digital native: beyond digital apartheid,
towards digital democracy. Journal of Computer Assisted Learning 26(5), 357–369
(2010)
4. Brundidge, J.: Encountering ”difference” in the contemporary public sphere: The
contribution of the internet to the heterogeneity of political discussion networks.
Journal of Communication 60(4), 680–700 (2010)
5. Dahlgren, P.: The internet, public spheres, and political communication: Dispersion
and deliberation. Political communication 22(2), 147–162 (2005)
6. Danescu-Niculescu-Mizil, C., West, R., Jurafsky, D., Leskovec, J., Potts, C.: No
country for old members: User lifecycle and linguistic change in online communities.
In: Proc. of WWW’2013. pp. 307–318. ACM (2013)
7. De Liddo, A., Buckingham Shum, S.: The evidence hub: harnessing the collective
intelligence of communities to build evidence-based knowledge (2013)
8. DiMaggio, P., Evans, J., Bryson, B.: Have american’s social attitudes become more
polarized? American journal of Sociology 102(3), 690–755 (1996)
9. Font, F., Serr`a, J., Serra, X.: Analysis of the impact of a tag recommendation
system in a real-world folksonomy. ACM Trans. Intell. Syst. Technol. 7(1), 6:1–
6:27 (Sep 2015)
10. Fu, W.T.: The microstructures of social tagging: a rational model. In: Proc. of
CSCW’2008. pp. 229–238. ACM (2008)
11. Fu, W.T., Dong, W.: Collaborative indexing and knowledge exploration: A social
learning model. IEEE Intelligent Systems 27(1), 39–46 (2012)
12. Fu, W.T., Kannampallil, T.G., Kang, R.: A semantic imitation model of social tag
choices. In: Proc. of CSE’09. vol. 4, pp. 66–73. IEEE (2009)
13. Gallardo-Echenique, E.E., Marqu´es-Mol´ıas, L., Bullen, M., Strijbos, J.W.: Lets
talk about digital learners in the digital era. The International Review of research
in open and distributed learning 16(3), 156–187 (2015)
14. Habermas, J., Habermas, J., Mccarthy, T.: The structural transformation of the
public sphere: An inquiry into a category of bourgeois society. MIT press (1991)
15. Hommel, B.: Convergent and divergent operations in cognitive search. Cognitive
search: Evolution, algorithms, and the brain 9, 221–235 (2012)
16. Huckfeldt, R., Mendez, J.M., Osborn, T.: Disagreement, ambivalence, and engage-
ment: The political consequences of heterogeneous networks. Political Psychology
25(1), 65–95 (2004)
17. Kopeinik, S., Eskandar, A., Ley, T., Albert, D., Seitlinger, P.C.: Adapting an open
source social bookmarking system to observe critical information behaviour. In:
Proc. of LILE’2018 (2018)
18. Kopeinik, S., Lex, E., Seitlinger, P., Albert, D., Ley, T.: Supporting collabora-
tive learning with tag recommendations: a real-world study in an inquiry-based
classroom project. In: Proc. of LAK’2017. pp. 409–418. ACM (2017)
19. Kuhn, A., Cahill, C., Quintana, C., Schmoll, S.: Using tags to encourage reflection
and annotation on data during nomadic inquiry. In: Proc. of CHI’2011. pp. 667–
670. ACM (2011)
20. Ley, T., Seitlinger, P.: Dynamics of human categorization in a collaborative tagging
system: How social processes of semantic stabilization shape individual sensemak-
ing. Computers in human behavior 51, 140–151 (2015)
21. Lin, N., Li, D., Ding, Y., He, B., Qin, Z., Tang, J., Li, J., Dong, T.: The dy-
namic features of delicious, flickr, and youtube. Journal of the American Society
for Information Science and Technology 63(1), 139–162 (2012)
14 Simone Kopeinik et al.
22. Livingstone, S.: Critical reflections on the benefits of ict in education. Oxford
review of education 38(1), 9–24 (2012)
23. MacKuen, M., Wolak, J., Keele, L., Marcus, G.E.: Civic engagements: Resolute
partisanship or reflective deliberation. American Journal of Political Science 54(2),
440–458 (2010)
24. Marinho, L.B., Hotho, A., J¨aschke, R., Nanopoulos, A., Rendle, S., Schmidt-
Thieme, L., Stumme, G., Symeonidis, P.: Recommender systems for social tagging
systems. Springer Science & Business Media (2012)
25. McGrew, S., Breakstone, J., Ortega, T., Smith, M., Wineburg, S.: Can students
evaluate online sources? learning from assessments of civic online reasoning. Theory
& Research in Social Education 46(2), 165–193 (2018)
26. Nikolov, D., Oliveira, D.F., Flammini, A., Menczer, F.: Measuring online social
bubbles. PeerJ Computer Science 1, e38 (2015)
27. Pariser, E.: The filter bubble: What the Internet is hiding from you. Penguin UK
(2011)
28. Sarmiento, J.W., Stahl, G.: Group creativity in interaction: Collaborative refer-
encing, remembering, and bridging. Intl. Journal of Human–Computer Interaction
24(5), 492–504 (2008)
29. Schulz-Hardt, S., Frey, D., L¨uthgens, C., Moscovici, S.: Biased information search
in group decision making. Journal of personality and social psychology 78(4), 655
(2000)
30. Schwind, C., Buder, J.: Reducing confirmation bias and evaluation bias: When are
preference-inconsistent recommendations effective–and when not? Computers in
Human Behavior 28(6), 2280–2290 (2012)
31. Seitlinger, P., Ley, T., Albert, D.: Verbatim and semantic imitation in indexing
resources on the web: A fuzzy-trace account of social tagging. Applied Cognitive
Psychology 29(1), 32–48 (2015)
32. Seitlinger, P., Ley, T., Kowald, D., Theiler, D., Hasani-Mavriqi, I., Dennerlein,
S., Lex, E., Albert, D.: Balancing the fluency-consistency tradeoff in collabora-
tive information search with a recommender approach. International Journal of
Human–Computer Interaction 34(6), 557–575 (2018)
33. Stroud, N.J.: Polarization and partisan selective exposure. Journal of communica-
tion 60(3), 556–576 (2010)
34. Wagner, C., Singer, P., Strohmaier, M., Huberman, B.: Semantic stability and
implicit consensus in social tagging streams. IEEE Transactions on Computational
Social Systems 1(1), 108–120 (2014)
35. Wang, Y., Luo, J., Niemi, R., Li, Y.: To follow or not to follow: Analyzing the
growth patterns of the trumpists on twitter. In: Proc. of ICWSM’2016 (2016)
36. Xu, Z., Fu, Y., Mao, J., Su, D.: Towards the semantic web: Collaborative tag
suggestions. In: Proc. of WWW’2006 (2006)
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
Constructively dealing with societal problems requires a process of opinion formation that is preceded by a competent and diverse search of information. Alarming are thus, communicative processes that can increasingly be observed in social media. In particular, a tendency of drawing virtual circles around like-minded people seems to characterize users' information behaviour and opinion formation dynamics, leading to separated viewpoints of communicative milieus, often referred to as echo chambers. This development raises concern about the public as a hub of diverse perspectives and also starts entering the agenda setting of educational innovation, which aims to prepare students for a more responsible information behaviour. In order to support such innovation, the goal of the present work is to complement prior research on echo chamber phenomena that has mainly made use of questionnaires to get to know the involved socio-cognitive variables and dynamics. By the example of two application scenarios, we showcase how an adapted social bookmarking system can be applied for a more direct observation technique that derives behavioural indices for variables of interest from log-file recordings. We believe that the observation of students' information seeking behaviour will also inspire the design of teaching strategies, e-learning and learning analytics tools.
Conference Paper
Full-text available
In online social learning environments, tagging has demonstrated its potential to facilitate search, to improve recommendations and to foster reflection and learning. Studies have shown that as a prerequisite for learning, shared understanding needs to be established in the group. We hy-pothesise that this can be fostered through tag recommendation strategies that contribute to semantic stabilization. In this study, we investigate the application of two tag rec-ommenders that are inspired by models of human memory: (i) the base-level learning equation BLL and (ii) Minerva. BLL models the frequency and recency of tag use while Min-verva is based on frequency of tag use and semantic context. We test the impact of both tag recommenders on semantic stabilization in an online study with 51 students completing a group-based inquiry learning project in school. We find that displaying tags from other group members contributes significantly to semantic stabilization in the group, as compared to a strategy where tags from the students' individual vocabularies are used. Testing for the accuracy of the different recommenders revealed that algorithms using frequency counts such as BLL performed better when individual tags were recommended. When group tags were recommended, the Minerva algorithm performed better. We conclude that tag recommenders, exposing learners to each other's tag choices by simulating search processes on learn-ers' semantic memory structures, show potential to support semantic stabilization and thus, inquiry-based learning in groups.
Article
Full-text available
Creative group work can be supported by collaborative search and annotation of Web resources. In this setting, it is important to help individuals both stay fluent in generating ideas of what to search next (i.e., maintain ideational fluency) and stay consistent in annotating resources (i.e., maintain organization). Based on a model of human memory, we hypothesize that sharing search results with other users, such as through bookmarks and social tags, prompts search processes in memory, which increase ideational fluency, but decrease the consistency of annotations, e.g., the reuse of tags for topically similar resources. To balance this tradeoff, we suggest the tag recommender SoMe, which is designed to simulate search of memory from user-specific tag-topic associations. An experimental field study (N = 18) in a workplace context finds evidence of the expected tradeoff and an advantage of SoMe over a conventional recommender in the collaborative setting. We conclude that sharing search results supports group creativity by increasing the ideational fluency, and that SoMe helps balancing the evidenced fluency-consistency tradeoff. © 2017 Paul Seitlinger, Tobias Ley, Dominik Kowald, Dieter Theiler, Ilire Hasani-Mavriqi, Sebastian Dennerlein, Elisabeth Lex, and Dietrich Albert. Published by Taylor and Francis.
Article
Full-text available
In this paper, we analyze the growth patterns of Donald Trump's followers (Trumpists, henceforth) on Twitter. We first construct a random walk model with a time trend to study the growth trend and the effects of public debates. We then analyze the relationship between Trump's activity on Twitter and the growth of his followers. Thirdly, we analyze the effects of such controversial events as calling for Muslim ban and his 'schlonged' remark.
Article
Full-text available
This paper reports on a literature review of the concept of “Digital Natives” and related terms. More specifically, it reports on the idea of a homogeneous generation of prolific and skilled users of digital technology born between 1980 and 1994. In all, 127 articles published between 1991 and 2014 were reviewed. On the basis of the findings, there appears to be no commonly-accepted definition of a “Digital Native”. The concept varies among individuals, societies, regions and nations, and also over time. Moreover, there are a number of variables other than age that may help us understand the nature of students’ use of digital technologies. The so-called “Digital Native” literature demonstrates that despite students’ high digital confidence and digital skills, their digital competence may be much lower than those of their “digital teachers”. Given the confusion surrounding “Digital Native” and its affiliates, we propose to unify them under the concept “digital learners”.
Article
Full-text available
We study how categories form and develop over time in a sensemaking task by groups of students employing a collaborative tagging system. In line with distributed cognition theories, we look at both the tags students use and their strength of representation in memory. We hypothesize that categories get more differentiated over time as students learn, and that semantic stabilization on the group level (i.e. the convergence in the use of tags) mediates this relationship. Results of a field experiment that tested the impact of topic study duration on the specificity of tags confirms these hypotheses, although it was not study duration that produced this effect, but rather the effectiveness of the collaborative taxonomy the groups built. In the groups with higher levels of semantic stabilization, we found use of more specific tags and better representation in memory. We discuss these findings with regard to the important role of the information value of tags that would drive both the convergence on the group level as well as a shift to more specific levels of categorization. We also discuss the implication for cognitive science research by highlighting the importance of collaboratively built artefacts in the process of how knowledge is acquired, and implications for educational applications of collaborative tagging environments.
Article
To be an informed citizen in today’s information-rich environment, individuals must be able to evaluate information they encounter on the Internet. However, teachers currently have limited options if they want to assess students’ evaluations of digital content. In response, we created a range of short tasks that assess students’ civic online reasoning—the ability to effectively search for, evaluate, and verify social and political information online. Assessments ranged from paper-and-pencil tasks to open Internet search tasks delivered via Google Forms. We outline a process of assessment development in which middle school, high school, and college students in 12 states completed tasks. We present a series of representative tasks and analyses of trends in student performance. Across tasks and grade levels, students struggled to effectively evaluate online claims, sources, and evidence. These results point to a need for curriculum materials that support students’ development of civic online reasoning competencies.
Article
Collaborative tagging systems have emerged as a successful solution for annotating contributed resources to online sharing platforms, facilitating searching, browsing, and organizing their contents. To aid users in the annotation process, several tag recommendation methods have been proposed. It has been repeatedly hypothesized that these methods should contribute to improving annotation quality and reducing the cost of the annotation process. It has been also hypothesized that these methods should contribute to the consolidation of the vocabulary of collaborative tagging systems. However, to date, no empirical and quantitative result supports these hypotheses. In this work, we deeply analyze the impact of a tag recommendation system in the folksonomy of Freesound, a real-world and large-scale online sound sharing platform. Our results suggest that tag recommendation effectively increases vocabulary sharing among users of the platform. In addition, tag recommendation is shown to contribute to the convergence of the vocabulary as well as to a partial increase in the quality of annotations. However, according to our analysis, the cost of the annotation process does not seem to be effectively reduced. Our work is relevant to increase our understanding about the nature of tag recommendation systems and points to future directions for the further development of those systems and their analysis.
Chapter
In this chapter we describe the state-of-the-art in social tagging recommender systems. Many of the algorithms presented here borrow ideas and techniques from other areas such as information retrieval, machine learning, and statistical relational learning. In Section 4.3 we also describe many approaches for exploiting additional sources of information such as the content of resources and the social relations of users.