Content uploaded by Stylianos Sergis
Author content
All content in this area was uploaded by Stylianos Sergis on Oct 09, 2015
Content may be subject to copyright.
Enhancing Learning Object Recommendations for Teachers
Using Adaptive Neighbor Selection
Stylianos Sergis1,2 and Demetrios G. Sampson1,2
1Department of Digital Systems
University of Piraeus
Piraeus, Greece
2Information Technologies Institute
Centre for Research and Technology Hellas
Thessaloniki, Greece
e-mail: steliossergis@iti.gr, sampson@iti.gr
Abstract—Recommender Systems (RS) have been implemented
in the Technology enhanced Learning (TeL) field for
facilitating, among others, Learning Object (LO) selection by
teachers to support their daily teaching practice. In particular,
memory-based collaborative filtering (CF) approaches have
demonstrated promising results for real-life implementations
of web-based Learning Object Repositories (LOR). Building
on this, the contribution of this paper is an enhancement to
existing memory-based CF RS methods, by adaptively selecting
the teacher neighbors based on their co-rated LOs and the
attribute similarity of the latter to the LO to be recommended.
The evaluation results show a significant increase in the
predictive accuracy of the adaptive RS approaches compared
to their "traditional" benchmarks, signifying the proposed
approach's capacity to enhance the accuracy of existing
memory-based CF approaches.
Keywords- recommender systems; technology enhanced
learning; teachers; learning objects; adaptive neighbor selection;
I. INTRODUCTION
Recommender Systems (RS) are adaptive software
applications aimed at providing suggestions to their users for
potentially useful items [1]. They have been utilized in a
wide range of applications, including Technology enhanced
Learning (TeL) for the provision (among others) of Learning
Object (LO) recommendations to teachers for supporting
their learning designs and daily teaching practice [2], [3]. To
generate these recommendations, many techniques have been
exploited by RS, with the collaborative filtering (CF)
technique [1] being one the most prominent.
In the domain of CF technique, existing approaches are
mainly divided in two strands, namely the model-based
approaches and the memory-based approaches [4]. Model-
based approaches aim to process all available user-generated
interaction data (e.g., ratings) in order to create predictive
user models. Despite their high level of accuracy, these
approaches usually come with a costly need for creating and
maintaining such models and the resulting scalability issues
[4], [5]. On the other hand, memory-based approaches work
directly on the provided user-item interaction data in order to
evaluate potential correlations and predict user preferences.
This approach has been attributed with simplicity of
implementation and immunity in terms of scalability [4].
In the context of TeL, the provision of accurate LO
recommendations to teachers for supporting their teaching
practice is an important issue considering the ever-increasing
availability of such resources on the web and the resulting
difficulty in locating them [3]. Under this light, the
aforementioned benefits of memory-based approaches could
allow them to be more easily deployed in real-life LORs
towards facilitating teachers in the above process. Thus,
efforts should be placed on further enhancing the level of
accuracy provided by these approaches, in order to reap their
reported simplicity and scalability benefits.
Towards this goal, a potential drawback of existing
memory-based CF RS is the manner in which the teachers'
"like-minded" neighbors are selected. More specifically, the
existing neighbor selection processes consider all candidate
teacher neighbors to be equally important as a preference
proxy for the active teacher (i.e., the teacher using the system
and receiving the recommendations). Therefore, the active
teacher's final neighbors are those that they share the greater
rating-based similarity with. This process, however, does not
take into account the (metadata) attributes of the LOs that
have been co-rated by the teachers. This shortcoming could
affect the actual similarity between the active teacher and the
selected neighbors in cases where the co-rated LOs are
significantly dissimilar (in terms of attributes) to the one that
is currently being recommended.
Therefore, in this paper, we propose an enhancement to
existing memory-based CF RS methods. It involves selecting
the teacher neighbors in an adaptive manner by assigning
more weight to those candidate neighbors who have co-rated
LOs that are more similar to the LO to be recommended.
The remainder of the paper is structured as follows.
Section II presents the background of our work. Section III
presents the proposed adaptive neighbor selection
mechanism for teacher-oriented memory-based CF RS.
Section IV describes the evaluation methodology adopted
and reports on the predictive accuracy evaluation results.
Finally, Section V discusses our conclusions and presents
proposed future work in this agenda.
II. BACKGROUND
In previous work, a literature review of the landscape of
TeL RS was performed towards identifying approaches that
were focused on teachers, directly or indirectly [2]. This
review highlighted 22 such approaches, each utilizing a
specific recommendation technique. As this review process
revealed, the CF technique, either on its own or in
combination with other techniques (i.e., hybrid), is very
widely used in the existing TeL teacher-oriented RS.
Moreover, a further analysis of the identified TeL teacher-
oriented RS systems highlighted the fact that none have
utilized mechanisms for selecting neighbors adaptively
depending on the (metadata) attributes of the LO to be
recommended. As aforementioned, this can be a drawback
when recommending LOs to teachers using "traditional" CF
approaches, considering that the LO attributes (and
indirectly, affordances) could be used to refine the similarity
calculation process between teachers.
Outside the TeL context, a similar approach has been
introduced by [6]. More specifically, Choi and Suh in [6]
developed and positively evaluated a method for selecting
the active user's neighbors based on the item to be
recommended, by considering the rating-based similarity of
their co-rated items. When applying this approach within the
TeL domain (to reap its reported accuracy-related benefits),
however, the rating-based LO similarity should be replaced
with an attribute-based LO similarity. The rationale for this
is that rating-based LO similarity might not be very
meaningful in the context of TeL, where teachers usually
select LOs in terms of their attribute appropriateness to
support a specific learning design. Therefore, these attributes
should be used as a similarity basis between teachers, instead
of their overall "crude" rating-based similarity.
Under the light of the above, a research problem is
identified, i.e. how can adaptive neighbor selection be
modeled and utilized by considering LOs attribute similarity,
towards facilitating teachers in selecting LOs for their
teaching practice. The contribution of this paper is (a) the
presentation of a method for extending memory-based CF
RS with adaptive neighbor selection, and (b) the evaluation
of the proposed approach in terms of predictive accuracy.
III. ADAPTIVE NEIGHBOR SELECTION IN TEACHER-
ORIENTED RECOMMENDER SYSTEMS
In our approach, we alter the methodology of [6] by
calculating LOs similarity in terms of their (metadata)
attribute profiles, using the commonly used Jaccard
similarity coefficient [7]. The proposed approach of
attribute-based similarity exploits the LO metadata records,
usually depicted as a vector space model. Moreover, it is not
limited to a specific metadata schema, but can be adapted to
accommodate any LO Metadata Application Profile utilized
in a LOR, e.g., the Open Discovery Space Repository LO
Metadata Application Profile presented in [8].
As aforementioned, the adaptive neighbor selection
approach is implemented as an enhancement to memory-
based CF methods. For the context of this paper, the
extensively used Pearson correlation coefficient (PCC) and
Cosine coefficient of similarity (CC) [7] were used (a) as a
basis for implementing the adaptive approach and (b) as a
performance benchmark. More specifically, based on the
original PCC, we define the Adaptive Pearson correlation
coefficient (APCC) by introducing the "adaptive neighbor"
factor simij which is the Jaccard –based similarity index
between the LO j to be recommended and the each LO i that
has been co-rated by the two teachers (1).
The "adaptive neighbor" factor (which is calculated off-
line to reduce response time) aims to consider the attribute
similarity of the LO j (to be recommended) to the LO i (that
has been co-rated by the two teachers u and w). This Jaccard-
based similarity between any LO pair is calculated for each
element of the LO metadata vector space model of this LO
pair (e.g., educational objectives and technical format),
assigning equal weights to each LO metadata element. In this
way, when measuring the Pearson-based similarity between
teachers towards formulating the active teacher's
neighborhood, more weight will be assigned to candidate
neighbors who have provided common ratings to LOs more
similar to the one being considered for recommendation.
In a similar vein, based on the original CC we define the
Adaptive Cosine coefficient of similarity (ACC) by
introducing again the "adaptive neighbor" factor simij in (2).
Similarly, the introduced factor aims to capture the
attribute Jaccard-based similarity between all LOs co-rated
by the teachers u and w and assign the corresponding weight
to the co-rating candidate neighbor, towards formulating a
more informed teacher neighborhood.
IV. EVALUATION
The evaluation process utilized two educational datasets
retrieved from the Discover the Cosmos (DtC) Repository
1
and the Open Science Resources (OSR) Repository
2
. An
overview of the two datasets is provided in Table 1.
Aggregation Level (AL) 1 LOs refer to standalone
educational resources (e.g., flash simulations), while
Aggregation Level 2 LOs refer to Lesson Plans and/or
Educational Scenarios [9]. Overall Sample Size (N) refers to
the number of LOs in each AL category, while Rating Data
refers to the number of ratings provided by the Contributing
Users for each AL category. Finally, Rating Density is
defined as the logarithm of the ratio of known ratings against
the number of all possible ratings that can be provided [10].
This metric has been attributed with a high positive
correlation to CF RS accuracy [10].
TABLE I. EDUCATIONAL DATASETS OVERVIEW
DtC Dataset
OSR Dataset
AL1
AL2
AL1
Overall Sample Size (N)
92709
629
1545
Rating Data
835
7469
1148
Contributing Users
84
127
175
Rating Density
-3.97
-1.03
-2.37
The focus of the evaluation of the proposed approach was
its predictive accuracy measured using the common Root
Mean Squared Error (RMSE) metric [11]. More specifically,
the two adaptive versions (APCC and ACC) were
benchmarked against their "original" versions (PCC and CC
respectively) for each dataset using an increasing
neighborhood size (1 to 20), towards evaluating their added
value in each occasion. Therefore, for each dataset, a set of
four RS methods were run, i.e., PCC, APCC, CC and ACC.
1
http://portal.discoverthecosmos.eu (Accessed 21Apr 2015)
2
http://www.osrportal.eu/en/repository (Accessed 21 Apr 2015)
=
,
,
,
=
,
,
, ()
Figure 1: Evaluation Results for the DtC AL1 (a), DtC AL2 (b) and OSR (c) datasets
Finally, the standard "Leave-N-out" technique [11] was
adopted, using a 70%-30% data split between the "training"
set and the "test" set, respectively.
Fig. 1 presents the evaluation results for each dataset. As
the Fig. 1 depicts, both adaptive RS approaches outperform
their respective benchmarks for all datasets, i.e., DtC AL1
(Fig.1a), DtC AL2 (Fig. 1b) and OSR (Fig. 1c). Moreover,
the increase in predictive accuracy is evident for all
neighborhood sizes in all datasets, especially for small sizes
(N<6). These findings, therefore, support the hypothesis of
the paper regarding the proposed adaptive approach's
capacity to enhance the predictive accuracy of existing CF
approaches. Moreover, an important finding deriving from
the results of Fig. 1 is that the adaptive RS provide a
significant accuracy increase in datasets showing a lower
level of rating density (i.e., DtC AL1 and OSR). This could
prove to be very useful in real-life deployments of the
proposed approach, considering that educational datasets
usually have low rating density levels, a fact that can hinder
the performance of existing CF RS approaches [7].
V. CONLUSIONS AND FUTURE WORK
This paper presented a method of enhancing the
predictive accuracy of memory-based CF teacher-oriented
RS. This method includes the adaptive selection of the most
appropriate active teachers' neighbors not only based on their
overall rating-based similarity, but also considering the
similarity of the co-rated LOs to the one that is being
considered for recommendation. Moreover, the paper
presented evaluation results based on two real-life
educational datasets, highlighting the proposed approach's
added value in terms of predictive accuracy.
Future work can examine the potential of adaptive
neighbor selection beyond memory-based approaches, i.e., in
RS systems utilizing more elaborate teacher personal
characteristics for generating neighborhoods, such as the
teachers’ ICT competence profiles which have been
attributed with very promising potential to enhance the level
of accuracy in the LO recommendation process [3].
ACKNOWLEDGMENTS
The work presented in this paper has been partially supported by (a) the
“Open Discovery Space: A socially-powered and multilingual open learning
infrastructure to boost the adoption of eLearning Resources” Project that is
funded by the European Commission's CIP-ICT Policy Support Programme
(Project Number: 297229) and (b) “Inspiring Science: Large Scale
Experimentation Scenarios to Mainstream eLearning in Science,
Mathematics and Technology in Primary and Secondary Schools” Project
that is funded by the European Commission's CIP-ICT Policy Support
Programme (Project Number: 325123). Moreover, the authors would like to
acknowledge the Discover the Cosmos and the Open Science Resources
Projects for providing the corresponding educational datasets. Finally, the
publication of this paper has been partly supported by the University of
Piraeus Research Center.
REFERENCES
[1] J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez,
"Recommender Systems Survey," Knowledge-Based Systems, vol. 46,
pp. 109-132, 2013.
[2] S. Sergis, P. Zervas, and D. G. Sampson, "Towards Learning Object
Recommendations based on Teachers' ICT Competence Profiles,"
Proc. of the 14th International Conference on Advanced Learing
Technologies (ICALT2014), pp. 534 - 538, 2014.
[3] S. Sergis, P. Zervas, and D.G. Sampson, "ICT Competence-based Learning
Object Recommendations for Teachers," Proc. of the 11th International
Conference on Cognition and Exploratory Learning in Digital Age
(CELDA2014), pp. 150-157, 2014.
[4] C. Desrosiers, and G. Kapyris, "A Comprehensive Survey of Neighborhood-
based Recommendation Methods," Recommender Systems Handbook, F.
Ricci, L. Rokach, B. Shapira and P.B Kantor, Eds., US: Springer, pp.
107-144, 2011.
[5] Y. Koren, and R. Bell, "Advances in Collaborative Filtering,"
Recommender Systems Handbook, F. Ricci, L. Rokach, B. Shapira and
P.B Kantor, Eds., US: Springer, pp. 145-186, 2011.
[6] K. Choi, and Y. Suh, "A New Similarity Function For Selecting
Neighbors For Each Target Item In Collaborative Filtering,"
Knowledge-Based Systems, vol. 37, pp. 146–153, 2013.
[7] K. Verbert, H. Drachsler, N. Manouselis, M. Wolpers, R. Vuorikari,
and E. Duval, "Dataset-Driven Research For Improving
Recommender Systems For Learning," Proc. of the 1st International
Conference on Learning Analytics and Knowledge, pp. 44-53, 2011.
[8] N. Athanasiadis, S. Sotiriou, P. Zervas, and D.G. Sampson, "The
Open Discovery Space Portal: A Socially-Powered and Open
Federated Infrastructure," Digital Systems for Open Access to Formal
and Informal Learning, D.G. Sampson, D. Ifenthaler, J.M. Spector, P.
Isaias, Eds., International Publishing: Springer, pp. 11-23, 2014.
[9] D.G. Sampson, P. Zervas, and S. Sotiriou, "Science Education
Resources Supported with Educational Metadata: The Case of the
OpenScienceResources Web Repository," Advanced Science Letters,
vol. 4, no. 11-12, pp. 3353-3361, 2011.
[10] G. Adomavicius, and J. Zhang, "Impact Of Data Characteristics On
Recommender Systems Performance," ACM Transactions on
Management Information Systems, vol 3, no. 1, pp. 3:1-3:17, 2012.
[11] G. Shani, and A. Gunawardana, "Evaluating Recommendation
Systems," Recommender systems handbook, F. Ricci, L. Rokach, B.
Shapira, and P.B Kantor, Eds., US: Springer, pp. 257-297, 2011.
