Conference PaperPDF Available

Recommending Tags with a Model of Human Categorization

Authors:

Abstract and Figures

Social tagging involves complex processes of human catego-rization that have been the topic of much research in the cognitive sciences. In this paper we present a recommender approach for social tags whose principles are derived from some of the more prominent and empirically well-founded models from this research tradition. The basic architecture is a simple three-layers connectionist model. The input layer encodes patterns of semantic features of a user-specific re-source, which are either latent topics elicited through Latent Dirichlet Allocation (LDA) or available external categories. The hidden layer categorizes the resource by matching the encoded pattern against already learned exemplar patterns. The latter are composed of unique feature patterns and as-sociated tag distributions. Finally, the output layer samples tags from the associated tag distributions to verbalize the preceding categorization process. We have evaluated this ap-proach on a real-world folksonomy gathered from Wikipedia bookmarks in Delicious. In the experiment our approach outperformed LDA, a well-established algorithm. We at-tribute this to the fact that our approach processes seman-tic information (either latent topics or external categories) across the three di↵erent layers, and this substantially en-hances the recommendation performance. With this paper, we demonstrate that a theoretically guided design of algo-rithms not only holds potential for improving existing rec-ommendation mechanisms, but it also allows us to derive more generalizable insights about how human information interaction on the Web is determined by both semantic and verbal processes.
Content may be subject to copyright.
Recommending Tags with a Model of Human
Categorization
Paul Seitinger
Knowledge Technologies Institute
Graz University of Technology
Graz, Austria
paul.seitlinger@tugraz.at
Dominik Kowald
Know-Center
Graz University of Technology
Graz, Austria
dkowald@know-center.at
Christoph Trattner
Know-Center
Graz University of Technology
Graz, Austria
ctrattner@know-center.at
Tobias Ley
Institute of Informatics
Tallinn University
Tallin, Estonia
tley@tlu.ee
ABSTRACT
So cial tagging involves complex processes of human catego-
rization that have been the t opic of much research in the
cognitive sciences. In this paper we present a recommender
approach for social tags whose principles are derived from
some of the more prominent and empirically well-founded
mo dels from this research tradition. The basic architecture
is a simple three-layers connectionist model. The input layer
enco des patterns of semantic features of a user-specific re-
source, which are either latent topics elicited through Latent
Dirichlet Allocation (LDA) or available external categories.
The hidden layer categorizes the resource by matching the
enco ded pattern against already learned exemplar patterns.
The latter are composed of unique feature patterns and as-
so ciated tag di stributions. Finally, the output layer samples
tags from the associated tag distributions to verbalize the
preceding categorization process. We have evaluated this ap-
proach on a real-world folksonomy gathered from Wi kipedia
b ookmarks in Delicious. In the experiment our approach
outperform ed LDA, a well-established algorithm. We at-
tribute this to the fact that our approach processes seman-
tic information (either latent topics or external categories)
across the three dierent layers, and this substantially en-
hances the recommendation performance. With this paper,
we demonstrate that a theoretically guided design of algo-
rithms not only holds potential for improving existing rec-
ommendation mechanisms, but it also allows us to derive
more generalizable insights about how human information
interaction on the Web is determined by both semantic and
verbal processes.
Categories and Subject Descriptors
H.2.8 [Database Management]: Database Applications—
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
CIKM’13, Oct 27 - Nov 01, 2013, San Francisco, CA.
Copyright 2013 ACM xxx-x-xxxx-xxxx-x/xx/xx... $15.00
Data mining;H.3.3[Information Storage and Retrieval]:
Information Search a nd Retrieval—Information filtering
General Terms
Measurement, Experimentation
Keywords
tag recommendations, LDA, human categorization model,
Wikipedi a, Delicious
1. INTRODUCTION
There is now broad agreement that in order to support users
in tagging resources on the Web, a good understanding of
the mechanisms that underlie human tagging behavior is ad-
vantageous [6, 9, 35]. Several such mechanisms have been
discovered, including human motivation [18, 17], memory
pro cesses [3], categorization and language production [31].
Several influential papers in this context have looked at the
dynamics underlying the behavioral patterns, such as the
p ower law of a tag distribution (e.g. [9]). Based on mod-
els of information theory [9] and human memory theory [5,
6, 4] generative models of social tagging have b e en devel-
op ed providing much insight into the emergence of the data
that result in social tagging systems. The generative mod-
els implement some aspects of the assumed micro-level in-
formation processing mechanisms (such as the frequency or
recency with which tags have been observed prior to pro-
duction) and provide computational models that produce a
tag distribution. Comparing this distribution to a distri-
bution observed in real-world tagging systems then allows
making claims about the validity of the assumed micro-level
mechanisms. A stricter and theoretically more informing
test about these theoretical claims can be provided by con-
trolled experiments as these allow for testing causal relation-
ships more directly. Such studies have been conducted, for
instance, by Fu et al. [7, 6] to test the semantic imitation
mo del of s ocial tagging, by Cress et al. [4] to test a social
variant of information foraging theory, as well as by our own
group to find evidence for a dual-process memory mechanism
[31]. These studies, on the other hand, have limitations as
they need to necessarily control the setting in which they are
conducted. To generalize findings from these lab settings to
naturally occurring tagging behavior, the models need to be
tested in real-life settings. In this paper, we follow a dier-
ent approach to test our models with real-life data. We have
devised a recommender mechanism which implements some
basic mechanisms of human categorization and sense making
that is assumed to take place in social tagging environments.
Human categorization is one of the fundamental mechanisms
through which persons make sense of their surroundings [28]
, and it has been the subject of study in cognitive science
for several decades. Our recommender, 3Layers, mimics the
user’s categorization and language production of the book-
mark to predict the use r’s tag assignments. The recommen-
dation paradigm allows us to draw on evaluation techniques
that have been applied and refined in recommender systems
research for a number of years. We test the performance of
the algorithm in a dataset drawn from a real-world tagging
system. Rather than comparing outputs of the recommender
to observed overall tag distributions, as research on gener-
ative models has done, the recommender paradigm allows
us to use knowledge acquired in a training dataset and then
to predict individual tag choices in a test dataset. This ap-
proach allows for much stronger claims about the underlying
mechanisms of tag production and it also allows several al-
gorithms to be compared to e ach other. The contributions
of the paper are as follows
1. We present a novel tag recommendation mechanism
that is based on psycholinguistic models of categoriza-
tion and speech production
2. We demonstrate that such a recommendation mecha-
nism performs significantly better than a standard tag
recommendation approach s uch as LDA, and
3. We demonstrate that a theoretically guided design of
a recommender complements purely data-driven ap-
proaches in that it allows for some theoretically in-
forming claims about how humans process information
in sense making tasks on the Web.
The remainder of the paper is structured as follows: In Sec-
tion 2 we discuss related work and in Section 3 we present
our new approach. In Section 4 we shortly introduce our
experi ments and the used dataset. Section 5 presents the
results of our study. Section 6 concludes the paper and dis-
cusses our findings in light of the benefits of connecting data-
driven and theory-driven research for recommender systems
research. Finally, Section 7 outlines future research direc-
tions.
2. RELATED WORK
Compared to the research on generative models of social
tagging, c omputer science research has taken a more prag-
matist stance. The main purpose here is to support persons
in their sense making activities on the Web. Recommender
systems research, in particular, has aimed at helping users
discover useful resources on the Web, or to guide the user in
her categorization process. Employing a diverse number of
paradigms has led to a number of approaches over the past
years that have been very successful in predicting human
b ehavior such as predicting tags for Web resources.
One of the first approaches implemented for the task of rec-
ommending tags to a user for a specific Web resource is
called collaborative filtering [10]. The probably first work
describing such a mechanism for the domain of collabora tive
tagging systems is the work of Xu et al. [36] who introduce
a simple tag co-occurrance approach to recommend tags to
a user.
Sigurbjornsson et al. [33] developed a similar approach and
showed for the photo tagging system Flickr that it is “es-
sential to t ake the co-occurance values of the candidate tags
into account when aggregating the intermediate results in a
ranked list of recommended tags”.
Hotho et al. [14] presented an algorithm called FolkRank
which uses the structure of folksonomies for searching and
ranking. These rankings can also be used to recommend
tags, resources and users or to build communities of inter-
est from the folksonomy. In [15] Jaschke et al. extended
FolkRank to design a graph-based tag recommendation al-
gorithm on t op of it and compared it to collaborative filter-
ing based on users, where they achieved better recall and
precision values.
Another interesting work in the field of tag recommender
systems is the work of Lipczak and Milios [26]. In their work
they introduce a novel scalable and adoptable system that is
able to recommend tags based on the resource’s title, content
and the user’s profile. Furthermore, the system allows to
learn new tags also in an ecient manner.
In [30] Rendle et al. introduce a factorization model called
PITF (Pairwise Interaction Tensor Factorization) which al-
lows the linear runtime for both learning and recommend-
ing tags. Similar to the work of Lipczak and Milios [26]
they address the problem of the cubic runtime of Tensor
Factorization approaches which have been shown to outper-
form for instance other tag recommender algorithms such as
FolkRank, collaborative filtering, etc.
The probably first work extensively studying the tag predic-
tion problem from the perspective of rule mining is the work
of Heyman et al. [11]. In a number of experiments based
on tags from the social tagging system Delicious they show
that their approach achieves high-precision results [11]. In
a follow-up Krestel et al. experimented with LDA for the
use of recommending tags and showed that it reaches signif-
icantly bett er results than association rules [21, 19]. In [20]
they combined LDA with simple language models. These
language models are based on the most frequent tags of the
users and resources in the bookmarks and enhanced the per-
formance of LDA.
These approaches perform more or less well, they are very
economical and motivated by the endeavor of finding algo-
rithms to predict historical user-interaction data as accu-
rately as possible. However, this data-driven approach of-
ten lacks theoretical grounding with respect to the cognitive
pro cesses that result in the data to be predicted. By ap-
plying fo rmal models of human semantic memory, the rec-
ommender presented in the next section should therefore
complement the recommender systems described above and
should thereby integrate current cognitive science results
into recommender s ystems for Social Tagging.
3. APPROACH
We have designed a tag recommender by applying principles
of generative mo del s of Social Tagging, with a focus on hu-
man semantic memory (e.g.[5, 7, 6]). The general idea is to
equip recommendation algorithms with a reasoning mech-
anism emulating human categorization and verbal behav-
ior. That way, tag recommendations should emerge that
resemble a user’s natural indexing behavior. In particular,
our theoretical focus is on formal memory models explain-
ing word (re-)productions and hence, psycholinguistic pro-
cesses that we deem to be in play during tag assignments
and tag imitations. A number of prominent memory mod-
els assume word productions to proceed in dierent steps
on distinct levels of memory: on abstract, categorical levels
encompassing representations of words’ meanings (”Bedeu-
tungsfelder”, i.e., semantic fields or concepts) and on more
language-dep endent levels indexing semantic fields by means
of word forms (e.g. Language and Situated Simulation, [34].
For instance, the Fuzzy Trace Theory (FTT, [2]) postulates
an ac tivation of a gist-trace in response to a stimulus, e.g.
a Web-resource or a set of associated tags, which contains
semantic aspects (concepts, relations, patterns) of the stim-
ulus (e.g. [16]). The gist-trace i n turn reconstructs several,
semantically related word forms verbalizing the activated
gist. By means of a Markov model derived from FTT, [31]
showed that a substantial amount of tag productions can in-
deed be predicted by a two-step memory retrieval involving
b oth categorical and verbal processes. Similar to the FTT,
the Topic Model [8] assumes that if a person is writing a
sequence of words (e.g. tags) or even a whole document,
she firstly samples an underlying latent structure l from a
distribution of latent structures P(l) and - depending on l
- samples a sequence of corresponding words, w ,P(w|l).
Taken together, this internal translation process results in
the joint distribution P(w, l)=P(w|l)P(l).
All these models fit well into the psycholinguistic theory
of Levelt et al. [24] distinguishing between three processes
during the production of words: 1) Categorization (result-
ing in a message or gist to be articulated), 2) Formalization
(accessing the mental lexicon to activate word forms cor-
respondi ng to the categorization) and 3) Articulation (se-
lecting and producing appropriate word forms). The recom-
mender presented here is called 3Layers and is in line with
this proposed translation of latent structures into words. We
assume a set of tagged resources, which are at the same time
assigned to a category. These categories (hereinafter called
semantic features”) are either given a-priori (e.g. because a
page is categorized to a wikipedia category) or are derived
as LDA topics [8] from the tag assignments. The recom-
mender starts with categorizing a user-specific resource by
enco ding and processing semantic features true for the user
and/or resource, then formalizes the categorization by iden-
tifying tag distributions associated with the resource’s se-
mantic features and finally, articulates tags by sampling the
most appropriate tags from the identified tag distributions.
The basic architecture is a feed-forward connectionist net-
work consisting of three layers of nodes realizing a top-
down pattern completion process by means of straightfor-
ward equations derived from formal models of human cat-
egory learning [13, 12, 22, 23]. In response to semantic in-
formation on the input layer (two patterns of LDA-topics or
external categories, one characterizing a user and one a spe-
cific resource), the hidden layer categorizes and formalizes
the resource by calculating the input’s similarity to already
stored exemplars that are unique topic (or category) pat-
terns and associated tag distributions. In other words, an
exemplar is a previously categorized and tagged resources of
the user. Finally, the output layer articulates the preceding
categorization and formalization processes by sampling tags
from the tag distributions of the identified, similar exem-
plars.
On the input layer, there are two input vectors representing
semantic features that are true for the user u, a
in
u
, and the
resource r, a
in
r
. Within each vector, each of the N nodes
represents a single semantic feature f
i
(in our case a topic
identified by LDA or a category). Its activation (denoted
a
in
i
) indicates the extent to which that feature applies to
the user, a
in
iu
, and resource in question, a
in
ir
. In other words,
the parameters a
in
iu
(and a
in
ir
) represent the association of the
semantic feature to the user (and resource). a
in
iu
is given by
a
in
iu
=
c(f
i
,u)
P
N
j=i
c(f
j
,u)
(1)
where c(f
i
,u) represents the counted frequency of the se-
mantic feature in t he user’s personomy (i.e., her or his book-
mark collection). Correspondingly, a
in
ir
represents the associ-
ation of the semantic feature to the resource and is estimated
in a similar way from the counted frequency of the feature
f
i
in all bookmarks of the resource r, c(f
i
,r). The activa-
tions across the N input nodes constitute the vectors a
in
u
=(a
in
1u
,a
in
2u
,...,a
in
Nu
) and a
in
r
=(a
in
1r
,a
in
2r
,...,a
in
Nr
) that rep-
resent the extent to which each of the N topics/categories
is true for the given user and resource, respectively. In Fig-
ure 1, the left semantic feature pat tern at t he input layer
corresp onds to the input vector a
in
u
=(.04,.24,...,.01,.00)
indicating that, for instance, the topics/categories 1 and 2
have the relative frequencies .04 and .24, respectively, across
the user u’s personomy.
The nodes on the hidden layer store information ab out ex-
amples e
j
extracted from the training set. Figure 1 illus-
trates two such examples (e
6
and e
8
) that are composed of
unique, semantic feature patterns, h
j
=(h
j1
,h
j2
,...,h
jN
),
and associative weights w
tj
. The latter are maintained be-
tween each of all m tags t and the unique feature pattern h
j
and are illustrated in form of diagrams plotting the weights
against the tags. The estimates of each h
ji
in h
j
are calcu-
lated in a similar way as the activation of each input feature
(either a
in
iu
or a
in
ir
), and the associative weight w
tj
enco des
the relative frequency of each tag t in e
i
and is estimated as
w
in
tj
=
c(t, e
j
)
P
k2e
j
c(t
k
,e
j
)
(2)
where c(t, e
i
) is the counted frequency of tag t in exemplar
e
j
.
Step 1 in Figure 1 is based on a simple pattern matching pro-
cess and results in probability estimates of each exemplar.
To conduct this step, we firstly calculate the distance of a
given exemplar e
j
to the input vectors a
in
u
and a
in
r
, denoted
!
"
#$!%
&'()*+%!',-%!
"
#$!%
Input Layer
Encoding user
and resource
information
Hidden Layer
Categorization and
Formalization
a
u
in
= (0.04,0.24,...,0.01,0.00)
Step 1: Categorization - Matching
input
against stored examples
Step 2: Formalization - Multiplying
a
j
hid
by w
tj
Step 3: Summing products (w
tj
*a
j
hid
)
over all exemplars
Step 4: Articulation - Simulating Tag-Choices
by drawing tags from rank-t
x
out
distribution
a
8
hid
topic pattern h
6
of example e
6
activation of e
6
tags' associative
weights with
respect to e
6
Output Layer
Articulation
h
8
of example e
8
activation of e
8
associative
weights with
respect to e
4
19
2
16
8
11
17
1
7
4
13
5
15
10
3
18
14
12
6
9
19
2
16
8
11
17
1
7
4
13
5
15
10
3
18
14
12
6
9
a
r
in
= (0.04,0.24,...,0.01,0.00)
a
6
hid
a
3
hid
a
5
hid
a
4
hid
a
2
hid
a
1
hid
a
7
hid
!"
!"
"#$%&"
'&
!"
!"
"#$%&"
'&
Figure 1: Basic architecture of 3Layers (Note that
only two of the six examples at the hidden layer are
illustrated completely).
d
u
ji
and d
r
ji
, respectively, by applying the cosine similarity
measure and subtracting the result from 1, i.e.,
d
u
ij
=1
ha
in
u
,h
j
i
ka
in
u
kkh
j
k
(3)
Correspondi ngly, d
r
ji
is calculated by subtracting the simi-
larity between a
in
r
and h
j
from 1. The distances are linearly
combined to a single distance, w hich is then transformed to
an activation (or similarity) estimate a
hid
j
falling exponen-
tially with the distance between the hidden node and the
input [32], and yielding a probability estimate for e
j
:
a
hid
j
=
exp[(d
u
ji
+ d
r
ji
)]
P
k
exp[(d
u
ki
+ d
r
kj
)]
(4)
For example, the Figure 1 schematically illustrates that e
6
receives higher activation than e
8
(illustrated by the black-
and grey-filled rhombic form, respectively) since e
6
’s topic
pattern h
6
is more similar to both input vectors a
in
u
and a
in
r
than e
8
’s topic pattern h
8
.
We then form response strengths for each of the tags, t
out
x
.
In step 2 (see Figure 1), each hidden node’s activation a
hid
j
is multiplied by the corresponding tags’ associative weights,
i.e., a
hid
j
·w
tj
, and in step 3, these products are summed over
!
"
#$!%
Input Layer
Encoding user and
resource information
Hidden Layer
Categorization and
Formalization
Step 1
Equations (3) and (4)
Steps 2
and 3
Equation
(5)
Step 4
Equation (6)
t
1
t
4
Output Layer
Articulation
a
u
in
a
i
in
h
ji
!
!
t
4j
t
mj
X
Y
d
ji
u
a
j
hid
w
1j
t
1j
...
...
!j 0.12
!j 0.05
a
r
in
d
ji
r
e
1
e
6
e
8
e
9
...
...
...
w
1j * a
j
hid
w
4j
w
4j * a
j
hid
0.14
...
...
0.00
...
0.00
...
0.00
0.71
...
...
0.19
...
0.00
...
0.14
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
0.00
...
...
0.00
...
0.00
...
0.00
0.00
...
...
0.00
...
0.00
...
0.00
0.27
...
...
0.22
...
0.80
...
0.63
0.74
...
...
0.59
...
1.00
...
0.00
0.12
...
...
0.14
...
0.05
...
0.17
0.09
0.00
0.11
0.11
0.00
0.25
0.11
0.11
0.25
0.01
0.00
0.01
0.01
0.00
0.03
0.01
0.01
0.04
...
...
...
...
...
...
...
...
...
0.09
0.00
0.11
0.11
0.00
0.00
0.11
0.11
0.00
0.01
0.00
0.01
0.01
0.00
0.00
0.01
0.01
0.00
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
0.04
0.24 ... ... 0.01 0.00
0.00
0.14 ... ... 0.00 0.00
Figure 2: Example for the dierent 3Layers calcula-
tion steps.
all hidden nodes, given by
t
out
x
=
X
j
wt
j
· a
hid
j
(5)
where each t
out
x
is a realization of a discrete random variable
X since
P
m
x
Pr(X = t
out
x
)=1.
In a last step 4, we make use of this probability distribution
to simulate the user’s tag assignments by drawing y random
numbers and mapping them into events, i.e. t
out
x
. Finally,
the observed count of tag t
x
in the simulation, c(t
x
), deter-
mines its ranking for being recommended. If the parameter
l specifies the number of tags to be selected, the subset of
tags to be recommended RecT ags is given by
RecT ags := {t
x
|rank[c(t
x
)] l} (6)
For the sake of clarification, in Figure 2 we also provide a
concrete example of the calculation steps. Note that in this
example, all exemplars’ semantic feature patterns are repre-
sented in the e
j
by h
ji
matrix X 2 [0, 1]. Similar to Figure 1,
the vectors a
in
u
and a
in
r
at the input layer encode the extent,
to which the topics or categories are true for the user and
the resource, respectively. Step 1, the categorization, starts
with the equation (3), which calculates the distance of a
in
u
and a
in
r
to each of the row vectors e
j
in X. The third and
second columns from right (d
u
ji
and d
r
ji
, respectively) include
Prop erties Wikipedi a-full Wikipedia-15-core
Book marks 1,700,929 11,540
Resources 386,337 559
Tags 361,603 648
Users 304,063 548
Tag assignments 4,942,100 47,725
Table 1: Properties of the full and the p-core pruned
Wikipedia dataset.
the resulting distance measures. Equation (4) finalizes the
categorization by transforming these distances into proba-
bility estimates a
hid
j
(see first column from right). Steps 2
and 3 formalize the preceding categorization by means of
equation (5): The activation a
hid
j
of each row vector in X
is multiplied by the tags’ associative weights w
tj
that cor-
respond to the given exemplar e
j
. For instance, in Figure
2 , the activation of e
1
amounts to a
hid
1
= .12 and the cor-
respondi ng associative weight of tag t
1
, w
11
,is.09 (see first
cell entry in the e
j
by t
xj
matrix Y 2 [0, 1]). Hence, step 2
(w
tj
· a
hid
j
) results in .01. After conducting step 2 for the re-
maining exemplars, step 3 (summing over all products and
yielding t
out
1
) amounts to .12. Since the tag t
4
is not as-
so ciated with the highly activated exemplars e
6
and e
9
,its
respons e strength only amounts to t
out
4
= .05. Finally, step
4 articulates the preceding categorization and formalization
pro cesses by drawing l tags from the rank-t
out
x
distribution
that is the result of equation (5). Referring to equation (6),
lower-ranked tags (e.g. t
1
) will be chosen for output, i.e.,
for recommendation, with a higher probability than higher-
ranked tags (e.g. t
4
) since they are more likely to belong to
the subset RectT ag s.
4. EXPERIMENTAL SETUP
In order to evaluate our approach, we compared it w ith
a popular tag recommendation approach based on Latent
Dirichlet Allocation [20, 21].
Latent Dirichlet Allocation (LDA) is a probability model
that helps to find latent topics for documents where each
topic is described by words in these documents. This can be
formalized as [21]:
P (t
i
|d)=
Z
X
j=1
P (t
i
|z
i
= j)P (z
i
= j|d)(7)
Here P (t
i
|d) is the probability of the ith word for a document
d and P (t
i
|z
i
= j) is the probability of t
i
within the topic
z
i
. P (z
i
= j|d) is the probability of using a word from topic
z
i
in the document. In LDA the number of latent topics
Z has to be chosen in advance, which defines the level of
sp ecialization of the topics.
When using LDA for tag recommendation, documents are
resources which are described by tags. This means that each
resource, or more specified each bookmark of a resource, can
also be represented with the top tags of topics identified by
LDA.
We implemented the LDA tag recommendation algorithm
with Gibbs sampling using the Java framework LingPipe
1
.
1
LingPipe: http://alias-i.com/lingpipe/
Number of latent topics MRR±STD MAP±STD
24 .767±.025 .289±.013
100 .899±022 .363±.011
250 .980±.029 .409±.016
500 .991±020 .415±.012
750 .967±0.021 .401±.011
1000 .931±.020 .381±.010
Table 2: MRR and MAP values wi th s tandard d e-
viations for LDA with dierent numbers of latent
topics.
Therefore we calculated the probability of a tag tP(t|r, u)
based on a given user uP(t|u) and based on a given resource
rP(t|r) and combined these two values based on Krestel and
Frankhauser [20], where P (t) is the prior-probability of this
tag:
P (t|r, u) /
P (t|r)P (t|u)
P (t)
(8)
The prior probability P (t) can be estimated as the relative
frequency of this tag in all bookmarks and P (t|u) and P (t|r)
are smoothed based on this value using the following formu-
las, where |u| are the number of tags for a user u and |r| are
the number of tags for resource r [20]:
´
P (t|r) / log
2
(|r| +1)P (t|r)+log
2
(|u| +1)P (t)(9)
´
P (t|u) / log
2
(|u| +1)P (t|u)+log
2
(|r| +1)P (t)(10)
The smoothing ensures that the two probabilities are weighted
according to their imp ortance and that no tag gets a prob-
ability value of 0.
4.1 Dataset
For our experimentation we used a large-scale social t agging
dataset crawled from Delicious
2
and provided by Arkaitz et
al. [37]. This dataset was crawled between 2003 and March
2011 and contains nearly 340 million bookmarks, 119 mil-
lion unique resources, 15 million unique tags and 2 million
unique users. In order to get a dataset where all resources
are categorized and freely available, we parsed out all book-
marks of Wikipedia
3
articles, which resulted in 1,7 million
b ookmarks, 386 thousand unique resources, 361 thousand
tags, 304 thousand unique users and 4,9 million tag assign-
ments. This focus on the Wikipedia domain gives us not only
the possibility to test our approach with external knowledge
such as category information, but also increases the repro-
ducibility of our experiments.
In order to get a dense fraction of this dataset we used p-core
pruning as proposed by Batagelj and Zaversnik [1]. This p-
core pruning is an iterative process where in each iteration
all resources, tags and users are deleted that occur less t han
p times in the dataset. The algorithm terminates when no
more tag assignments can be deleted w hich ensures that all
resources, tags and users can be found at least p times in
the remaining core [15, 25]. We tried this on dierent lev-
els and for p = 15 we received enough bookmarks a nd tag
2
Delicous: https://delicious.com/
3
Wikipedi a: http://en.wikipedia.org/
Figure 3: Recall/precision plots for LDA with
24 topics, LDA with 500 topics, 3Layers with
Wikipedia categories and 3 Layers with LDA tags
on 1 - 10 recommended tags.
assignments for our evaluations. Table 1 summarizes the
prop erties of the dataset.
In order to extend the resources in our dataset with semantic
features that can be used as external data for the input layer
of our approach, we fetched the category information of the
Wikipedi a articles from that time latest Wikipedia dump
4
.
Since these articles categories are very specific, we only fo-
cused on the 24 Wikipedia top-level categories of each arti-
cle which we obtained from a Wikipedia category-taxonomy
which we created following the approach of [27].
4.2 Evaluation Methode and Metrics
To evalute the performance of our tag recommender ap-
proach we used a leave-one-out hold-out method. For each
user we randomly eliminated one b ookmark and added it to
test set, the remaining bookmarks were used as the train-
ing set [15, 29]. Using this method we generated 20 dif-
ferent training and test sets that were used for our evalua-
tions. With the training set we examined then whether a
tag recommender algorithm could recommend the tags for a
user associated with a particular resource within the top-N
ranked list in the test set [29].
As evaluation metrics we used dierent well-established met-
rics for tag recommendations in order to obtain the perfor-
mance of our approach compared to LDA [15, 25]. All these
metrics are reported as an average over our 20 training and
test sets.
Recall is calculated as the number of correctly recommended
tags divided by the number of relevant tags, where tags
u
denotes the list of recommended tags and T
u
the list of rele-
vant tags of a bookmark of user u. This can be calculated for
dierent numbers of recommended tags n, where recall@n
should be growing with n because the number of relevant
tags is constant. This is averaged on all known bookmarks
in a test set, which corresponds to the numb er of users |U|.
Recall =
1
|U|
X
u2U
|tags
u
\ T
u
|
|T
u
|
(11)
4
Dump: http://dumps.wikimedia.org/enwiki/20121101/
Figure 4: F1-score values for LDA with 24 topics,
LDA with 500 topics, 3Layers with Wikipedia cate-
gories and 3Layers with LDA tags on 1 - 10 recom-
mended tags.
Precision is calculated as the number of correctly recom-
mended tags divided by the number of recommended tags.
This can also be done based for dierent n, which means that
the precision@n should decrease with n as it is the divisor.
Precision =
1
|U|
X
u2U
|tags
u
\ T
u
|
|tags
u
|
(12)
Recall/precision plot is a popular visualization method
to show the relation between recall and precision. It shows
the recall on the x-axis and the precision on the y-axis for a
dierent number of recommended tags [25].
F1-score combines precision and recall into one score and
unlike these two metrics it is not monotonic with the number
of recommended tags n [25].
F 1 score =2
pr ecision recall
pr ecision + recall
(13)
Precision, Recall and F1-score are calculated for n =1-10.
Mean reciprocal rank (MRR) is the sum of the recipro-
cal ranks of all relevant tags in the list of the recommended
tags. This means that a higher MRR is achieved if the rel-
evant tags occur at the beginning of the recommended tag
list [29].
MRR =
1
|U|
|U|
X
u=1
(
X
t2T
u
1
rank(t)
)(14)
Mean average precision (MAP) is an extension of the
precision metric that also looks on the ranking of the recom-
mended tags. It is described in the formula below where B
n
is 1 if the recommended tag at position n is relevant [29].
MAP =
1
|U|
|U|
X
u=1
(
1
|T
u
|
|tags
u
|
X
n=1
B
n
precision@n)(15)
MRR and MAP increase with n and are reported for n =
10.
Algorithm MRR±STD MAP±STD
LDA 24 .767±.026 .289±.014
LDA 500 .991±.020 .415±.012
3Layers-Categories 1.152±.027 .513±.015
3Layers-LDA 500 1.285±.027 .596±.013
Table 3: MRR and MAP values with standard devi-
ations for LDA with 24 topics, LDA with 500 topics,
3Layers with Wikipedia categories and 3Layers with
LDA tags on 10 recommended tags.
5. RESULTS
In this section we present the results of our approach com-
pared to LDA based on the previously mentioned evaluation
metrics and the Wikipedia dataset.
As reported in Se ction 4, the number of latent topics for
LDA has to be set in advance. Therefore we tried dierent
numbers of topics to get the best values. Table 2 shows
the MRR and MAP values with standard deviations for 24,
which corresp onds to the number of top-level categories in
Wikipedi a, 100, 250, 500, 750 and 1000 topics based on 10
recommended tags.
It can be seen that we received the best results for 500
topics with MRR = .991 and MAP = .415 (visualized in
b old). Based on these results, we configured our 3Layers
approach with two dierent data sources for its input layer,
(i) Wikipedia categories as described in Section 4.1 and (ii)
tags based on LDA with 500 topics. For the second config-
uration we used the top 10 tags identified by LDA for each
b ookmark in the training set.
Figure 3 shows the recall/precision plot for LDA with 24
topics, LDA with 500 topics, 3Layers with Wikipedia cate-
gories and 3Layers with LDA tags calculated for 500 topics
on 1 - 10 recommended tags. LDA with 24 topics is used
here as a simple baseline based on the number of top-level
categories in Wikipedia. Furthermore, Figure 4 also shows
the F1-score values for these algorithms on 1 - 10 recom-
mended tags. It can bee seen that both 3Layers approaches
outperform LDA on all values where the maximum values
are reached for recall@10 = .808, precision@1 = .686 and
F1-score@4 = .456 for 3Layers with LDA tags identified for
500 topics.
The MRR and MAP values with standard deviations are
shown in Table 3 for all the algorithms on 10 recommended
tags. Also on these metrics the two 3Layers approaches
outperform s LDA on all values. The maximum vales are
reached by 3Layers with LDA tags based on 500 topics for
MRR = 1.285 and MAP = .596 (visualized in bold). These
estimates clearly imply that independent of the measure the
probability estimates vary with the conditions, i.e. the tag
recommenders, in a constant ordering.
To check for statistical significance we performed two one-
way ANOVAs on MRR and MAP for 10 recommended tags
with Algorithm as a between-subjects factor. The statisti-
cal prerequisites of normal distribution and equal variances
were met. The results of both ANOVAs are shown in Table
4 and are well in line with the descriptive pattern of Table 3.
Metric Source SS DF MS F p-value
MRR Algorithm 2.981 3 .994 1586.651 .000
Error .048 76
MAP Algorithm 1.048 3 .349 1972.186 .000
Error .013 76
Table 4: Summary of one-way ANOVA for MRR
and MAP on 10 recommended tags with Algorithm
as between-subjects factor.
In particular, the overall dierence between the four recom-
menders proved highly significant and yielded the large eect
sizes of
2
MRR
= .984 and
2
MAP
= .987. Additionally, pair-
wise comparisons conducted by means of the Tukey’s HSD
test corresponded to the ordering described above. First, the
dierence between the two best performing recommenders,
i.e. 3Layers-LDA 500 and 3Layers-Categories (MRR: q =
16.63, p<.001; MAP: q =20.75,p<.001), second, the
dierence between 3Layers-Categories and LDA 500 (MRR:
q =20.00,p<.001; MAP: q =24.50,p<.001) and third,
the dierence between LDA 500 and LDA 24 (MRR: q =
64.75, p<.001; MAP: q =31.50,p<.001) all proved large
and highly significant.
Table 5 shows an example of the top 4 relevant and recom-
mended tags for 4 users given on 6 dierent resources. This
is done for LDA with 500 topics, 3Layers with Wikipedia
categories and 3Layers with LDA tags based on 500 topics.
All correctly recommended tags are visualized in bold.
6. DISCUSSION AND CONCLUSION
In this paper we have presented and evaluated a model of
human categorization implemented in form of a tag recom-
mender. The model takes into account semantic informa-
tion about a user-specific bookmark, which is either a set
of available Wikipedia categories or a set of t opics derived
by LDA. The semantic information is further processed in
a connectionist network of dierent layers that mimics the
user’s categorization and formalization of the bookmark to
predict the user’s tag assignments. We have presented an in-
depth discussion of the theoretical principles on which this
approach is based and we think this has introduced some
new perspectives into recommender systems research for so-
cial tagging environments. Particularly, the dierentiation
b etween semantic processing and language production has
a much wider applicability as we will discuss in more detail
below.
We were able to show that the 3Layers-model holds potential
of realizing a strongly performing recommender system. In
particular, 3Layers-LDA that utilizes LDA-topics as input
significantly outperforms the LDA-recommender introduced
by [21]. The same applies to 3Layers-Categories, which
makes use of Wikipe dia categories and therefore, operates
indepe ndently of the LDA-approach. From that we conclude
that mechanisms mimicking human categorization and ver-
bal behavior through a serial processing of semantic informa-
tion (either in form of LDA-topics or Wikipedia categories)
and verbal information (in form of tags) can substantially
contribute to the development of eective recommendations.
Of course, several limitations of these results need to be ad-
dressed. As we have only tested the performance in one data
User Resource Relevant tags LDA 24 LDA 500 3Layers-Categories 3Layers-LDA 500
1001901 Resource
Description
Fram ework
xml, rdf, se-
manticweb,
metadata
semanticweb,
rdf,ontology,
semantic
rdf, metadata,
programming,
reference
rdf, xml, seman-
ticweb,web
rdf, seman-
ticweb, xml,
metadata
1001901 Web-Service article, web-
services, web,
soap
webservices,rest,
http, soa
programming,
soap, xml,soft-
ware
web,webservice,
xml, webservices
development,
webservices,
xml,software
1921331 Lua-
Programming
Language
games, gui,
programming,
reference
glossary, func-
tional, program-
ming,philosophy
lua, language, gui,
scripting
reference, pro-
gramming,lua,
language
reference, pro-
gramming,lua,
scripting
1921331 Functional-
Programming
programming,
reference,
scala, software
language, concur-
rency, maths, pro-
gramming
functional, refer-
ence, program-
ming,invention
programming,
reference, soft-
ware,functional
programming,
reference,func-
tional, software
212165 M essage-
Passing-
Interface
api, parallel,
programming,
api
programming,
google, philosophy,
parallel
api, parallel, pro-
gramming,wiki
programming,
api, parallel, api
programming,
parallel,psychol-
ogy, api
1106599 Clojure lisp, clojure,
java, program-
ming
programming,
concurrency, lan-
guage, lisp
jvm, java,
program-
ming,program-
ming language
work, program-
ming, clojure,
lisp
lisp, clojure, pro-
gramming, java
Table 5: Top 4 relevant and recommended tags for 4 users on 6 dierent technical resources in the English
Wikipedia for LDA with 24 topics, LDA with 500 topics, 3Layers with Wikipedia categories and 3Layers with
LDA tags.
set, generalizability to other cases ne eds to be demonstrated.
Also without a doubt, there is nowadays a much larger set of
recommender algorithms available than we could take into
account in our study. With this paper, however, our main
intention was to integrate some of the current research from
cognitive science into current approaches of recommender
systems for social tagging.
We take the results as a promising outcome. First of all, the
pro cessing of semantic categories (either explicitly given, or
latent) can alleviate the cold start problem that other ap-
proaches are suering from (such as Collaborative Filtering
or those based on popularity, for instance). Reliance on
these categories should also improve the robustness as the
algorithm does not only depend on word-level imitation but
takes into account shared semantic interpretations (e.g. [7,
6]).
Additionally, our approach significantly enhances the LDA-
recommender [21, 20] by further operating on the identified
latent topic patterns. We attribute the latter result to the
calculation steps of formalization where prior tag distribu-
tions are weighted according to the preceding categorization
steps. The result is a distribution at the output layer ex-
hibiting fewer ties and allowing fo r a more accurate selection
of relevant tags. Therefore, our approach provides an appro-
priate theoretical framework and an eective recommender
that integrates top-down and bottom-up generated data.
Our approach therefore should transfer well to other related
Web interaction paradigms where both top down classifi-
cation systems and bottom-up categorization co-exist. For
example, Web curation is a recent trend in which Web users
can create collections of resources and share these collec-
tions with others. These usually employ mechanisms of so-
cial bookmarking and tagging, but also employ classification
systems to which collections are assigned.
With Web interaction paradigms changing quickly and data
sets pro duced w ithin them diering considerable, a purely
data-driven strategy has its limitations. It is then more dif-
ficult to understand, why certain approaches perform very
well in certain datasets, but not very well in others. The
reason is that datasets are products of very complex pro-
cesses [9] and they depend on a number of factors that the
mo dels would need to take into account.
While the datasets will look dierent, many of the funda-
mental processes that underlie the interaction in these new
environments (such as human categorization or language
pro duction) will be very similar. Hence, the danger of a
predominantly data-driven res earch strategy is that with ev-
ery new paradigm, we have to start from zero as the earlier
algorithms are not directly transferable.
With the current work, we demonstrate how a connection
b etween a data-driven and theory-driven approach can be
realized. The model underlying 3Layers corresponds to the
dierentiation between pro cesses on a categorical and word
level as suggested by [7, 6] and is validated in form of a well-
p erforming tag-recommender. The model also formalizes the
so cial influence of the tag assignments by other users (by
means of the second input vector r, a
in
r
) as suggested by the
so cial foraging model by Cress and colleagues (e.g. [4]). Be-
cause 3Layers implements fundamental cognitive principles
that are independent of interaction paradigms, the approach
holds potential of being applicable in dierent contexts. For
instance, regardless of the semantic input that can either
b e extracted from the folksonomy (through LDA) or from
available, taxonomic categories, the recommender’s output
resembles the user’s indexing behavior.
7. FUTURE WORK
In future work we will address the previously mentioned is-
sues by testing the recommender mechanism in other tagging
datasets as well as with other Web interaction paradigms,
such as Web curation. Additionally, we will compare 3Lay-
ers’ performance to other well-established approaches, such
as FolkRank [14] or Collaborative Filtering [36]. A distinc-
tive benefit of our theory-driven approach in designing tag
recommendation mechanisms is that it opens up fruitful di-
rections for conceptual, future research. For instance, we
hypothesize that our approach relates to the distinction of
categorizers and describers that was introduced in [18, 17]
to explain dierent tagging motivations. We suspect that
the 3Layers will especially work well for the categorizers
who draw on a more refined system on personal categories
when assigning tags. Additionally, we will refine the current
3Layers version by further exploiting aspects of the under-
lying human memory theories (e.g. [22, 23]). For instance,
in future user-studies we will equip 3Layers with a back-
propagation learning mechanism that will take into account
whether a user has adopted the recommended tags or not.
Dep ending on the user’s responses, the feed-forward network
of 3Layers will change the strength of associations between
the nodes across the dierent layers and thereby, adopt to
the user’s categorization and verbal behavior.
8. ACKNOWLEDGMENTS
The authors would like to thank Andreas Hotho for his fruit-
ful discussions on this work. This work is supported by the
Know-Center and the EU funded project Learning Layers.
The Learning Layers project is supported by the European
Commission within the 7th Framework Programme under
Grant Agreement 318209, under the DG Information society
and Media (E3), unit of Cultural heritage and technology-
enhanced learning.The Know-Center is funded within the
Austrian COMET Program - Competence Centers for Ex-
cellent Technologies - under the auspices of the Austrian
Ministry of Transport, Innovation and Technology, the Aus-
trian Minis try of Economics and Labor and by the State
of Styria. COMET is managed by the Austrian Research
Promotion Agency (FFG).
9. REFERENCES
[1] V. Batagelj and M. Zaverˇsnik. Generalized cores.
arXiv preprint cs/0202039,2002.
[2] C. Brainerd and V. Reyna. Recollective and
nonrecollective recall. Journal of memory and
language,63(3):425445,2010.
[3] C. Cattuto, V. Loreto, and L. Pietronero. Semiotic
dynamics and collaborative tagging. Proceedings of the
National Academy of Sciences,104(5):14611464,
2007.
[4] U. Cress, C. Held, and J. Kimmerle. The collective
knowledge of social tags: Direct and indirect
influences on navigation, learning, and information
pro cessing. Comput. Educ., 60(1):59–73, Jan. 2013.
[5] W.-T. Fu. The microstructures of social tagging: a
rational model. In Proceedings of the 2008 ACM
conference on Computer supported cooperative work,
pages 229–238. ACM, 2008.
[6] W.-T. Fu, T. Kannampallil, R. Kang, and J. He.
Semantic imitation in social tagging. ACM
Transactions on Computer-Human Interaction
(TOCHI),17(3):12,2010.
[7] W.-T. Fu, T. G. Kannampallil, and R. Kang. A
semantic imitation model of social tag choices. In
Computational Science and Engineering, 2009.
CSE’09. International Conference on, volume 4, pages
66–73. IEEE, 2009.
[8] T. L. Griths, M. Steyvers, J. B. Tenenbaum, et al.
Topics in semantic representation. Psychological
review,114(2):211,2007.
[9] H. Halpin, V. Robu, and H. Shepherd. The complex
dynamics of collaborative tagging. In Proceedings of
the 16th international conference on World Wide Web,
WWW ’07, pages 211–220, New York, NY, USA,
2007. ACM.
[10] J. L. Herlocker, J. A. Konstan, L. G. Terveen, and
J. T. Riedl. Evaluating collaborative filtering
recommender systems. ACM T r ans. Inf. Syst.,
22(1):5–53, Jan. 2004.
[11] P. Heymann, D. Ramage, and H. Garcia-Molina.
So cial tag prediction. In Proceedings of the 31st
annual international ACM SIGIR conference on
Research and development in information retrieval,
pages 531–538. ACM, 2008.
[12] D. Hintzman. Judgments of frequency and recognition
memory in a multiple-trace model. Psychological
Review,95:528551,1988.
[13] D. L. Hintzman. Schema abstraction in a
multiple-trace memory model. Psychological review,
93(4):411–428, 1986.
[14] A. Hotho, R. J
¨
aschke, C. Schmitz, and G. Stumme.
Information retrieval in folksonomies: Search and
ranking. In The semantic web: research and
applications, pages 411–426. Springer, 2006.
[15] R. J
¨
aschke, L. Marinho, A. Hotho,
L. Schmidt-Thieme, and G. Stumme. Tag
recommendations in folksonomies. In Knowledge
Discovery in Databases: PKDD 2007, pages 506–514.
Springer, 2007.
[16] B. T. Johns, M. N. Jones, and D. J. Mewhort. A
synchronization account of false recognition. Cognitive
psychology,2012.
[17] C. K
¨
orner, D. Benz, A. Hotho, M. Strohmaier, and
G. Stumme. Stop thinking, start tagging: tag
semantics emerge from collaborative verbosity. In
Proceedings of the 19th international conference on
World wide web, WWW ’10, pages 521–530, New
York, NY, USA, 2010. ACM.
[18] C. K
¨
orner, R. Kern, H.-P. Grahsl, and M. Strohmaier.
Of categorizers and describers: an evaluation of
quantitative measures for tagging motivation. In
Proceedings of the 21st ACM conference on Hypertext
and hypermedia, HT ’10, pages 157–166, New York,
NY, USA, 2010. ACM.
[19] R. Krestel and P. Fankhauser. Tag recommendation
using probabilistic topic models. ECML PKDD
Discovery Challenge 2009 (DC09), page 131, 2009.
[20] R. Krestel and P. Fankhauser. Language models and
topic models for personalizing tag recommendation. In
Web Intelligence and Intelligent Agent Technology
(WI-IAT), 2010 IEEE/WIC/ACM International
Conference on, volume 1, pages 82–89. IEEE, 2010.
[21] R. Krestel, P. Fankhauser, and W. Nejdl. Latent
dirichlet allocation for tag recommendation. In
Proceedings of the third ACM conference on
Recommender systems, pages 61–68. ACM, 2009.
[22] J. K. Kruschke et al. Alcove: An exemplar-based
connectionist model of category learning. Psychological
review,99(1):2244,1992.
[23] M. D. Lee and D. J. Navarro. Extending the alcove
mo del of category learning to featural stimulus
domains. Psychonomic Bulletin & Review,9(1):4358,
2002.
[24] W. J. Levelt. A theory of lexical access in speech
pro duction. In Proceedings of the 16th conference on
Computational linguistics-Volume 1, pages 3–3.
Asso ciation for Computational Linguistics, 1996.
[25] M. Lipczak. Hybrid Tag Recommendation in
Collaborative Tagging Systems. PhD thesis, Dalhousie
University, 2012.
[26] M. Lipczak and E. Milios. Learning in ecient tag
recommendation. In Proceedings of the fourth ACM
conference on Recommender systems, RecSys ’10,
pages 167–174, New York, NY, USA, 2010. ACM.
[27] D. Milne and I. H. Witten. Learning to link with
wikipe dia. In Proceedings of the 17th ACM conference
on Information and knowledge management, pages
509–518. ACM, 2008.
[28] G. L. Murphy. The Big Book of Concepts. MIT Press,
Cambridge, 2002.
[29] M. Rawashdeh, H.-N. Kim, J. M. Alja’am, and
A. El Saddik. Folksonomy link prediction based on a
tripartite graph for tag recommendation. Journal of
Intelligent Information Systems, pages 1–19, 2012.
[30] S. Rendle and L. Schmidt-Thieme. Pairwise
interaction tensor factorization for personalized tag
recommendation. In Proceedings of the third ACM
international conference on Web search and data
mining, WSDM ’10, pages 81–90, New York, NY,
USA, 2010. ACM.
[31] P. Seitlinger and T. Ley. Implicit imitation in so cial
tagging: familiarity and semantic reconstruction. In
Proceedings of the 2012 ACM annual conference on
Human Factors in Computing Systems, pages
1631–1640. ACM, 2012.
[32] R. N. Shepard et al. Toward a universal law of
generalization for psychological science. Science,
237(4820):1317–1323, 1987.
[33] B. Sigurbj
¨
ornsson and R. van Zwol. Flickr tag
recommendation based on collective knowledge. In
Proceedings of the 17th international conference on
World Wide Web, WWW ’08, pages 327–336, New
York, NY, USA, 2008. ACM.
[34] W. K. Simmons, S. B. Hamann, C. L. Harenski, X. P.
Hu, and L. W. Barsalou. fmri evidence for word
assoc iation and situated simulation in conceptual
pro cessing. Journal of Physi ology-Paris,
102(1):106–119, 2008.
[35] L. Steels. Collaborative tagging as distributed
cognition. Pragmatics & Cognition,14(2):287292,
2006.
[36] Z. Xu, Y. Fu, J. Mao, and D. Su. Towards the
semantic web: Collaborative tag suggestions. In
Proceedings of Collaborative Web Tagging Workshop at
15th International World Wide Web Conference,2006.
[37] A. Zubiaga, V. Fresno, R. Mart´ınez, and A. P.
Garc´ıa-Plaza. Harnessing folksonomies to produce a
so cial classification of resources. IEEE Transactions on
Knowledge and Data Engineering, 99(PrePrints), 2012.
... is a research strand in cognitive science, which uses artificial neural networks to study cognition and to model cognitive processes (Buckner and Garson, 2019). In this vein, Seitlinger et al. (2013) use the connectionist human memory simulation model ALCOVE (Kruschke, 1992) to implement a novel tag recommendation algorithm termed 3Layers. Kowald et al. (2013) enhance the 3Layers algorithm with recency effects by combining it with the BLL equation mentioned before. ...
Book
Full-text available
Personalized recommender systems have become indispensable in today’s online world. Most of today’s recommendation algorithms are data-driven and based on behavioral data. While such systems can produce useful recommendations, they are often uninterpretable, black-box models that do not incorporate the underlying cognitive reasons for user behavior in the algorithms’ design. This survey presents a thorough review of the state of the art of recommender systems that leverage psychological constructs and theories to model and predict user behavior and improve the recommendation process – so-called psychology-informed recommender systems. The survey identifies three categories of psychology-informed recommender systems: cognition-inspired, personality-aware, and affectaware recommender systems. For each category, the authors highlight domains in which psychological theory plays a key role. Further, they discuss selected decision-psychological phenomena that impact the interaction between a user and a recommender. They also focus on related work that investigates the evaluation of recommender systems from the user perspective and highlight user-centric evaluation frameworks, and potential research tasks for future work at the end of this survey.
... is a research strand in cognitive science, which uses artificial neural networks to study cognition and to model cognitive processes (Buckner and Garson, 2019). In this vein, Seitlinger et al. (2013) use the connectionist human memory simulation model ALCOVE (Kruschke, 1992) to implement a novel tag recommendation algorithm termed 3Layers. Kowald et al. (2013) enhance the 3Layers algorithm with recency effects by combining it with the BLL equation mentioned before. ...
... The focus of this paper is on the textual content, and we divide existing methods into two steps: text representation and candidate tag generation. Text representation methods include TF-IDF, topic models (Ramage et al. 2009;Krestel, Fankhauser, and Nejdl 2009;Seitlinger et al. 2013;Wu et al. 2016;, and neural networks (Weston, Chopra, and Adams 2014;Li et al. 2016a;Gong and Zhang 2016;Li et al. 2016b;Sun et al. 2018). A recent trend is to apply neural networks such as RNNs to learn the text representations. ...
Article
Recommending suitable tags for online textual content is a key building block for better content organization and consumption. In this paper, we identify three pillars that impact the accuracy of tag recommendation: (1) sequential text modeling meaning that the intrinsic sequential ordering as well as different areas of text might have an important implication on the corresponding tag(s) , (2) tag correlation meaning that the tags for a certain piece of textual content are often semantically correlated with each other, and (3) content-tag overlapping meaning that the vocabularies of content and tags are overlapped. However, none of the existing methods consider all these three aspects, leading to a suboptimal tag recommendation. In this paper, we propose an integral model to encode all the three aspects in a coherent encoder-decoder framework. In particular, (1) the encoder models the semantics of the textual content via Recurrent Neural Networks with the attention mechanism, (2) the decoder tackles the tag correlation with a prediction path, and (3) a shared embedding layer and an indicator function across encoder-decoder address the content-tag overlapping. Experimental results on three realworld datasets demonstrate that the proposed method significantly outperforms the existing methods in terms of recommendation accuracy.
... Prior efforts on tag recommendation for textual content are mainly based on topic models [19,24,27,39]. Some recent studies try to adopt deep neural networks since they have shown promising results in many language processing tasks [16,29]. ...
Conference Paper
Full-text available
In modern cities, complaining has become an important way for citizens to report emerging urban issues to governments for quick response. For ease of retrieval and handling, government officials usually organize citizen complaints by manually assigning tags to them, which is inefficient and cannot always guarantee the quality of assigned tags. This work attempts to solve this problem by recommending tags for citizen complaints. Although there exist many studies on tag recommendation for textual content, few of them consider two characteristics of citizen complaints, i.e., the spatio-temporal correlations and the taxonomy of candidate tags. In this paper, we propose a novel Spatio-Temporal Taxonomy-Aware Recommendation model (STAR), to recommend tags for citizen complaints by jointly incorporating spatio-temporal information of complaints and the taxonomy of candidate tags. Specifically, STAR first exploits two parallel channels to learn representations for textual and spatio-temporal information. To effectively leverage the taxonomy of tags, we design chained neural networks that gradually refine the representations and perform hierarchical recommendation under a novel taxonomy constraint. A fusion module is further proposed to adaptively integrate contributions of textual and spatio-temporal information in a tag-specific manner. We conduct extensive experiments on a real-world dataset and demonstrate that STAR significantly performs better than state-of-the-art methods. The effectiveness of key components in our model is also verified through ablation studies.
... They generated joint item-tag recommendations for users, where the tags represent topics from an item (i.e., a web resource) in which the user may be interested. Seitlinger et al. (2013) used a model of human category learning (i.e., ALCOVE) for social tags recommendation. The model uses semantic information regarding a user-specific bookmark (e.g., Wikipedia categories or LDA topics). ...
Article
Online images’ tags are very important for indexing, sharing, and searching of images, as well as surfacing images with private or sensitive content, which needs to be protected. Social media sites such as Flickr generate these metadata from user-contributed tags. However, as the tags are at the sole discretion of users, these tags tend to be noisy and incomplete. In this article, we present a privacy-aware approach to automatic image tagging, which aims at improving the quality of user annotations, while also preserving the images’ original privacy sharing patterns. Precisely, we recommend potential tags for each target image by mining privacy-aware tags from the most similar images of the target image, which are obtained from a large collection. Experimental results show that, although the user-input tags compose noise, our privacy-aware approach is able to predict accurate tags that can improve the performance of a downstream application on image privacy prediction and outperforms an existing privacy-oblivious approach to image tagging. The results also show that, even for images that do not have any user tags, our proposed approach can recommend accurate tags. Crowd-sourcing the predicted tags exhibits the quality of our privacy-aware recommended tags. Our code, features, and the dataset used in experiments are available at: https://github.com/ashwinitonge/privacy-aware-tag-rec.git.
... We can consider our method to be better than labelled LDA when the number of tags or topics being handled is more than 10,000. Consequently, the results indicate that methods based on LDA, such as Tag2Word (Wu et al., 2016) and a method of Seitlinger et al. (2013), do not gain high performance in this task. ...
... We can consider our method to be better than labelled LDA when the number of tags or topics being handled is more than 10,000. Consequently, the results indicate that methods based on LDA, such as Tag2Word (Wu et al., 2016) and a method of Seitlinger et al. (2013), do not gain high performance in this task. ...
... However, these methods only recommend tags that the user has used and cannot capture complex features of users, items, and tags. In [22], the topic model LDA is applied to process the semantic information, which is to mimic the human category learning for better representations of users and items. ...
Chapter
Personalized tag recommender systems suggest tags to users when annotating specific items. Usually, recommender systems need to take both users’ preference and items’ features into account. Existing methods like latent factor models based on tensor factorization use low-dimensional dense vectors to represent latent features of users, items and tags. The problem with these models is using the static representation for the user, which neglects that users’ preference keeps evolving over time. Other methods based on base-level learning (BLL) only use a simple time-decay function to weight users’ preference. In this paper, we propose a personalized tag recommender system based on neural networks and attention mechanism. This approach utilizes the multi-layer perceptron to model the non-linearities of interactions among users, items and tags. Also, an attention network is introduced to capture the complex pattern of the user’s tagging sequence. Extensive experiments on two real-world datasets show that the proposed model outperforms the state-of-the-art tag recommendation method.
Article
It has been the historic responsibility of the social sciences to investigate human societies. Fulfilling this responsibility requires social theories, measurement models and social data. Most existing theories and measurement models in the social sciences were not developed with the deep societal reach of algorithms in mind. The emergence of ‘algorithmically infused societies’—societies whose very fabric is co-shaped by algorithmic and human behaviour—raises three key challenges: the insufficient quality of measurements, the complex consequences of (mis)measurements, and the limits of existing social theories. Here we argue that tackling these challenges requires new social theories that account for the impact of algorithmic systems on social realities. To develop such theories, we need new methodologies for integrating data and measurements into theory construction. Given the scale at which measurements can be applied, we believe measurement models should be trustworthy, auditable and just. To achieve this, the development of measurements should be transparent and participatory, and include mechanisms to ensure measurement quality and identify possible harms. We argue that computational social scientists should rethink what aspects of algorithmically infused societies should be measured, how they should be measured, and the consequences of doing so. This Perspective discusses the challenges for social science practices imposed by the ubiquity of algorithms and large-scale measurement and what should—and should not—be measured in societies pervaded by algorithms.
Article
Full-text available
In this paper, we propose the Social Semantic Server (SSS) as a service-based infrastructure for workplace and professional learning analytics (LA). The design and development of the SSS have evolved over eight years, starting with an analysis of workplace learning inspired by knowledge creation theories and their application in different contexts. The SSS collects data from workplace learning tools, integrates it into a common data model based on a semantically enriched artifact-actor network, and offers it back for LA applications to exploit the data. Further, the SSS design’s flexibility enables it to be adapted to different workplace learning situations. This paper contributes by systematically deriving requirements for the SSS according to knowledge creation theories, and by offering support across a number of different learning tools and LA applications integrated into the SSS. We also show evidence for the usefulness of the SSS extracted from 4 authentic workplace learning situations involving 57 participants. The evaluation results indicate that the SSS satisfactorily supports decision making in diverse workplace learning situations and allow us to reflect on the importance of knowledge creation theories for this analysis.
Article
Full-text available
In our daily lives, organizing resources like books or webpages into a set of categories to ease future access is a common task. The usual largeness of these collections requires a vast endeavor and an outrageous expense to organize manually. As an approach to effectively produce an automated classification of resources, we consider the immense amounts of annotations provided by users on social tagging systems in the form of bookmarks. In this paper, we deal with the utilization of these user-provided tags to perform a social classification of resources. For this purpose, we have created three large-scale social tagging data sets including tagging data for different types of resources, webpages and books. Those resources are accompanied by categorization data from sound expert-driven taxonomies. We analyze the characteristics of the three social tagging systems and perform an analysis on the usefulness of social tags to perform a social classification of resources that resembles the classification by experts as much as possible. We analyze six different representations using tags and compare to other data sources by using three different settings of SVM classifiers. Finally, we explore combinations of different data sources with tags using classifier committees to best classify the resources.
Article
Full-text available
The multiple-trace simulation model, MINERVA 2, was applied to a number of phenomena found in experiments on relative and absolute judgments of frequency, and forced-choice and yes-no recognition memory. How the basic model deals with effects of repetition, forgetting, list length, orientation task, selective retrieval, and similarity and how a slightly modified version accounts for effects of contextual variability on frequency judgments were shown. Two new experiments on similarity and recognition memory were presented, together with appropriate simulations; attempts to modify the model to deal with additional phenomena were also described. Questions related to the representation of frequency are addressed, and the model is evaluated and compared with related models of frequency judgments and recognition memory.
Article
Full-text available
The multiple-trace simulation model, {minerva} 2, was applied to a number of phenomena found in experiments on relative and absolute judgments of frequency, and forced-choice and yes–no recognition memory. How the basic model deals with effects of repetition, forgetting, list length, orientation task, selective retrieval, and similarity and how a slightly modified version accounts for effects of contextual variability on frequency judgments were shown. Two new experiments on similarity and recognition memory were presented, together with appropriate simulations; attempts to modify the model to deal with additional phenomena were also described. Questions related to the representation of frequency are addressed, and the model is evaluated and compared with related models of frequency judgments and recognition memory. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Nowadays social tagging has become a popular way to annotate, search, navigate and discover online resources, in turn leading to the sheer amount of user-generated metadata. This paper addresses the problem of recommending suitable tags during folksonomy development from a graph-based perspective. The proposed approach adapts the Katz measure, a path-ensemble based proximity measure, for the use in social tagging systems. We model a folksonomy as a weighted, undirected tripartite graph. We then apply the Katz measure to this graph, and exploit it to provide tag recommendations for individual users. We evaluate our method on two real-world folksonomies collected from CiteULike and Last.fm. The experimental results demonstrate that the proposed method improves the recommendation performance and is effective for both active taggers and cold-start taggers compared to existing algorithms.
Article
Tag clouds generated in social tagging systems can capture the collective knowledge of communities. Using as a basis spreading activation theories, information foraging theory, and the co-evolution model of cognitive and social systems, we present here a model for an extended information scent, which proposes that both collective and individual knowledge have a significant influence on link selection, incidental learning, and information processing. Two experimental studies tested the applicability of the model to a situation in which individual knowledge and collective knowledge were contradictory to each other. The results of the first experiment showed that a higher individual strength of association between a target in demand and a tag led to a higher probability of selecting corresponding links, combined with less thorough information processing for non-corresponding links. But users also adapted their navigation behavior to the collective knowledge (strength of associations of tags) of the community and showed incidental learning during navigation, which resulted in a change of their individual strength of associations. The second experiment confirmed these results and showed, in addition, that the effects also occurred for indirect associations. Altogether, the results show that the extended information scent is an appropriate and fertile model for describing the interplay of individual knowledge and the collective knowledge of social tags.
Article
The paper discusses recent developments in web technologies based on collaborative tagging. This approach is seen as a tremendously powerful way to coordinate the ontologies and views of a large number of individuals, thus constituting the most successful tool for distributed cognition so far.