May I suggest? Comparing Three PLE Recommender Strategies

Abstract

Personal learning environment (PLE) solutions aim at empowering learners to
design (ICT and web-based) environments for their learning activities, mashingup
content and people and apps for different learning contexts. Widely used in
other application areas, recommender systems can be very useful for supporting
learners in their PLE-based activities, to help discover relevant content, peers
sharing similar learning interests or experts on a specific topic. In this paper we
examine the utilization of recommender technology for PLEs. However, being
confronted by a variety of educational contexts we present three strategies for
providing PLE recommendations to learners. Consequently, we compare these
recommender strategies by discussing their strengths and weaknesses in general.

Full text

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
1
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
May I Suggest? Comparing Three PLE Recommender Strategies
!
Felix Mödritscher
felix.modritscher@wu.ac.at
Barbara Krumay
barbara.krumay@wu.ac.at
Vienna University of Economics and Business, Austria
Sandy El Helou
sandy.elhelou@epfl.ch
Denis Gillet
denis.gillet@epfl.ch
Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland
Alexander Nussbaumer
alexander.nussbaumer@tugraz.at
Dietrich Albert
dietrich.albert@tugraz.at
Graz University of Technology, Asutria
Ingo Dahn
dahn@uni-koblenz.de
University of Koblenz-Landau, Germany
Carsten Ullrich
ullrich_c@sjtu.edu.cn
Shanghai Jiao Tong University, China
!

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
2
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Abstract
Personal learning environment (PLE) solutions aim at empowering learners to
design (ICT and web-based) environments for their learning activities, mashing-
up content and people and apps for different learning contexts. Widely used in
other application areas, recommender systems can be very useful for supporting
learners in their PLE-based activities, to help discover relevant content, peers
sharing similar learning interests or experts on a specific topic. In this paper we
examine the utilization of recommender technology for PLEs. However, being
confronted by a variety of educational contexts we present three strategies for
providing PLE recommendations to learners. Consequently, we compare these
recommender strategies by discussing their strengths and weaknesses in general.
.
Keywords
Personal learning environments; Recommender technology; Federated search;
collaborative recommendations; Community-based recommendations; Psycho-
pedagogical recommender; Technology review.
I. Introduction
Over the last decades, recommender systems have been successfully applied in various areas, like
online retailing (cf. Amazon) or social networking (cf. Facebook). Due to the success of this kind of
technology, research on technology-enhanced learning (TEL) has started to deal with recommender
strategies for learning, as documented by workshop proceedings (Manouselis et al., 2010) and
special issues in journals (Santos and Boticario, 2011). Addressing more learner-centric TEL
developments, recommendations seem to be a powerful tool for personal learning environment
(PLE) solutions (Mödritscher, 2010).
Unlike traditional LMS (Learning Management Systems) where content is predefined, PLEs are
based on “soft” context boundaries (Wilson et al., 2007), with resources and apps being added at
run time. In such “open corpus” environments (Brusilovsky and Henze, 2007), personalized
recommendations give learners the opportunity to take the best of an environment where shared
content differs in quality, target audience, subject matter, and is constantly expanded, annotated,
and repurposed (Downes, 2010).
As being addressed within the EU project ‘ROLE’ (abbreviation for ‘Responsive Open Learning
Environments’, cf. http://www.role‐project.eu), this paper deals with the generation and provision of
recommendations which should support learners in using PLE technology. Such recommendations
could comprise artifacts, peers or experts, pre-defined PLE designs as well as shared experiences
which are helpful in designing or making use of PLEs for learning (Mödritscher, 2010). As the ROLE
project deals with a wide range of educational scenarios, we present three different strategies,
each one aiming at supporting certain needs of learners.
The paper is structured as follows. The next section summarizes our understanding of personal
learning environments and gives a brief overview of recommenders for TEL and PLEs. Then, we

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
3
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
describe the three recommender approaches developed in the ROLE project. A discussion of the
benefits and limitations of applying each approach in PLEs follows, before the paper is concluded
and future work is summarized.
II. PLEs, PLE recommendations, and related work
According to Henri et al. (2008), personal learning environments (PLEs) refer to a set of learning
tools, services, and artifacts gathered from various contexts to be used by the learners. Figure 1
depicts a scenario where a PLE is used for student collaboration. A learner is involved in two
activities, an individual tutoring session in which she consults the facilitator via Facebook and a
task in which she collaboratively works on an outcome together with a peer actor using four
different tools (RSS Feed, Google, YouTube, and Twitter). This example illustrates how learners
interact with their PLEs consisting of different entities, i.e. tools, content artifacts (like emails or
Tweets), peer actors, etc.
Figure 1. Example scenario for PLE-based collaboration (see also Wild et al. (2008))
According to Van Harmelen (2008), web-based PLEs aim at empowering learners to design (ICT-
based) environments for their activities by enabling them to build their own learning environments,
where they can discover, create, reuse, and share content and applications as well as easily
expand their learner networks in order to collaborate on shared outcomes and acquire necessary
(professional and rich professional) competences based on their self-defined learning goals.
However user studies in the fields of higher education (Ullrich et al., 2010) and workplace learning
(Kooken et al., 2007) evidence that learners – and even teachers! (Windschitl and Sahl, 2002) –
have varying attitudes towards hand-on skills in using ICT, like PLE technology, for learning.
Against this background, PLE solutions should provide facilities for empowering learners in using
this kind of technology. One possible solution is the application of recommender technology, as
Resnick and Varian (1997) state that recommendations are useful if users have to make choices
without sufficient personal experiences of alternatives. In that, recommendation services could be
valuable for various aspects of PLE-based learning activities especially informal ones, where they
help formulate concrete learning goals or needs, discover relevant artifacts and tools, and find new
interactions opportunities with peers and experts sharing similar interests.

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
4
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Coming to fame particularly by their application in eCommerce (like Amazon.com) or social
networking platforms (like Facebook.com), recommender systems are “systems that produce
individualized recommendations as output or have the effect of guiding the user in a personalized
way to interesting or useful objects in a large space of possible options” (Burke, 2002).
Recommenders can adopt different strategies, such as item-based ones (e.g. content artifacts or
links to users), model-based ones (e.g. by applying probabilistic models or networked structures),
user-based collaborative filtering (based on user-related data-sets), or hybrid strategies
(Mödritscher, 2010). Verbert et al. (2011) emphasize the importance of building upon real-world
data-sets, e.g. in the form of user interaction data or (implicit and explicit) user feedback, to
develop and improve TEL recommender systems.
A lot of research on recommender systems for TEL has been done in the last few years. Amongst
others, theoretical work on this issue proposes models and ontologies for recommendations in the
educational domain (Santos and Boticario, 2010) or recommendation frameworks based on content
and context (Broisin et al., 2010). On a more practical level, other approaches deal with concrete
facilities like social navigation elements for educational libraries (Brusilovsky et al., 2010), ranking
algorithms for lecture slides (Wang and Sumiya, 2010), people finder for workplace learning
(Beham et al., 2010) or even algorithms for predicting student performance (Thai-Nghe et al.,
2010).
However, in the ROLE project we are facing new challenges that have led to the development of
different recommender strategies for PLE settings.
III. Three different PLE recommender approaches
A grand challenge of the EU project ROLE concerns the wide range of learning contexts to be
supported through responsive open learning environments. As being targeted by the vision of the
project (cf. http://www.role-project.eu/?page_id=406), ROLE claims to support learners in
different educational contexts, starting with formal and informal learning scenarios at universities
and at workplaces and reaching to the many contexts of lifelong learning. Moreover, it is even a
goal to support transitions between these contexts, as indicated by the five test-beds (‘university
to company’ transition, ‘individual to shared competences’ transition, ‘formal to informal learning’
transition etc). Consequently, the project focuses on integrating flexible infrastructures, i.e. widget
technology, into existing learning platforms and on different approaches to personalize learning,
amongst others by providing context-sensitive PLE recommendations to the learners.
In the upcoming subsections we briefly describe three of these recommender strategies being
developed in the project and following different paradigms.
a. Federated Search and Collaborative Recommendation Widget
The first approach developed within the ROLE project is implemented as a federated search and
recommendation widget exploiting the usage of resources by people sharing the same learning
and/or social context. The ‘Binocs’ widget (see Figure 2) employs a federated search engine that
aggregates heterogeneous resources and forwards them to a recommender system. Recommended
resources ranging from wiki pages, videos, to presentations can be saved, shared, assessed, and
re-purposed according to each user’s interest.

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
5
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Figure 2. Federated search and collaborative recommendation widget ‘Binocs’ displaying the results for
the query ‘learn french introduction’ and the opened settings menu
To rank resources, the recommender system takes the following user actions into account: (1)
selecting a resource from a search result, (2) liking or disliking a search result (using a thumbs up
and down feature) and (3) previewing a search result. The learning and social context can be
derived from the course (e.g. all students from a course share similar interests), the business
setting (e.g. all employees of the sales department) or from the user’s friends and contacts in the
widget container (via the OpenSocial API (Mitchell-Wong et al., 2007)). The recommender system
relies on an algorithm influenced by Google's original PageRank algorithm (Page et al., 1999) and
based on the 3A interaction model (El Helou et al., 2010). In the absence of previous user
interaction with a resource, ranking is still possible based on the resource relevance to the search
query.
A preliminary evaluation of the widget’s usability and recommendation usefulness is summarized in
Govaerts et al. (2011a). The evaluation helped to improve the user interface, and revealed that
users prefer Google results due to their diversity. The widget’s results were biased to media, while
Google provides a wider range of Web pages. This can be remedied by adding more repositories to
the federated search engine to drive the recommendations. On the other hand, pilot users agreed
on the usefulness of the collaborative recommendations on top of the search results. We plan to
evaluate the use of the recommender system further through the analysis of user online feedback
(by clicking on top N recommended items) and through user surveys in real-life scenarios.
Two more usability and usefulness evaluation studies of the Binocs widget being used in a PLE
were conducted (Govaerts et al., 2011b). One was done in the context of Business English courses
at the Shanghai Jiao-Tong University (SJTU, http://www.sjtu.edu.cn) where the widget is used to
provide access to social media resources (e.g. YouTube and SlideShare). The second evaluation
was conducted in a business setting, more specifically within an international corporation, FESTO

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
6
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
(http://www.festo.de) where the widget is used to assist sales people by offering more efficient
search over multiple product databases. The results for the widget in the business setting are more
positive than in the university. Potential explanations are the higher stability of the learning
environment at FESTO and the slow internet connection perceived at the SJTU, which could have
biased the evaluation of our federated search and recommendation services. Moreover it was noted
that extending the available repositories would be helpful to get richer search results.
b. Community-based PLE recommender
A second recommender going beyond collaborative recommendations within a single widget is
implemented as part of a practice sharing approach for learning communities (Mödritscher et al.,
2011). Basically, the idea is to integrate a pattern repository into existing PLE solutions so that
users can voluntarily share their PLE usage experiences as ‘good practices’ with peers. Thereby, a
pattern repository is a web-based service (with a RESTful API) which allows storing and retrieving
patterns of PLE-based activities, i.e. recordings of learner interactions with a tool mash-up used for
a specific situation (see also right-hand side of Figure 3). Overall, this practice sharing approach is
intended to be for informal learning settings, thus supporting life-long learners in achieving their
personal needs but also in succeeding at the workplace or in further education.
Figure 3. Client-sided PLE solution PAcMan (left) and proposed architecture of a PLE practice sharing
infrastructure (right, taken from (Mödritscher et al., 2011))
The data for this recommender approach is captured through facilities of the PLE which enables
users to share such an activity pattern in a simply way. A prototypic version of the pattern
repository has been integrated in two different PLE like solutions, a client-sided one (PAcMan add-
on, cf. https://addons.mozilla.org/en-US/firefox/addon/176479) and in OpenSocial-based widget
containers (like iGoogle or Liferay). The format of the activity patterns to be shared has to be
specified by the PLE developers who aim at integrating the pattern repository. For the PAcMan add-
on, the shared data is given as JSON which consists of web resources being structured according to
a simple activity model (an activity is a list of user-tagged URLs; see also left-hand side of Figure

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
7
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
3). Data capturing in OpenSocial containers is realized through a widget which records all events
triggered by the widget on a mash-up page if it has been added to this page. After pressing the
‘Share’ button, the recording of learner-triggered events (user interactions) is stored to the
repository on the basis of the Contextualized Attention Metadata (CAM) schema.
As the format of the shared activity patterns depends on the PLE solution submitting the data, a
recommender strategy has to be implemented for each data format. Currently, the standard
algorithm available can be characterized as a collaborative filtering (CF) technique, as it measures
the occurrences of each item (pattern titles, users having shared patterns, user-generated tags,
and URLs). The recommendations can be retrieved by the PLE solutions through the RESTful API
and according to different entities (patterns, peers, user tags, tools, and artifacts) and different
strategies. Next to the default strategy (‘global top-n’) it is planned to provide local top-n
recommendations. Hereby, locality could refer to the patterns used for generating the
recommendations, e.g. by using the patterns of a clique or for a specific search term only. For the
first case, Mödritscher (2010) describes a study in which a few patterns of a research group were
captured for a (work-related) scenario. Results showed that the distribution of item occurrences
follows a power law, and the network of activities, resources (URLs) and user-generated tags tend
to have characteristics of a scale-free network, which is an indicator that this collaboratively
created data-set is suitable for generating useful recommendations for users (cf. experiences on
music recommendations by Cano et al. (2006)).
Overall, this strategy for generating and providing PLE recommendations seems to be reasonable,
as it already works with smaller sets of data and allows personalizing recommendations e.g.
according to learner’s clique, a search term, or other contextual information. So far,
recommendations are only provided on the level of activity patterns – if a user opens the ‘Pattern
Store’ of the PAcMan add-on (see Figure 3) she can either query the patterns or receives
recommendations in terms of the most frequent downloaded patterns. A more sophisticated
strategy would be to suggest items (peers, artifacts, tools, or resource tags) according to specific
situations, e.g. for a certain clique or a given goal of a learner. As retrieved sub-sets of activity
patterns lead to scale-free networks, it is planned to provide two kinds of recommendations: (a)
the must-sees which comprise the hubs in the PLE network structure and are always displayed to
the user; (b) the might-be-of-interest suggestions, i.e. items of the long tail which are
recommended from time to time or also triggered by a certain context or user interaction.
c. Psycho-pedagogical recommender
In contrast to collaborative filtering strategies, the psycho-pedagogical recommender is not based
on large, community-generated data-sets. However, it is developed according to a theoretical
model and relevant taxonomies (Fruhmann et al., 2010) on the one hand and user data on the
other hand. In order to empower learners to build their learning environments and to use those for
learning, this recommender strategy deals with providing guidance in self-regulated learning
situations. While experienced learners are capable to use PLE technology without getting external
support, many learners need some kind of guidance and support to go through the learning
process (Dabbagh and Kitsantas, 2004; Efklides, 2009). The main aim of the psycho-pedagogical
recommender is to provide guidance especially with respect to self-regulated learning and to find
appropriate resources (artifacts, tools, peers) fitting to the competences of the learner.
Psycho-pedagogical recommendations are generated by taking into account two different
information sources. First user model data is used, comprising learning goals and competences

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
8
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
required at the moment. Also preferences, such as the degree of guidance needed, are considered.
A second information source is given in the form of learning models which serve as basis for the
recommender algorithm. The SRL process model describes how learning should ideally happen in a
self-regulated way in the context of ROLE. This model includes taxonomies of general and concrete
learning activities on the cognitive and meta-cognitive level. Learning tools are related to these
learning activities, which describes the way of learning possible with certain tools. These relations
are specified in advance and form an important basis of the recommendation strategy.
The recommendation strategy is closely related to these learning models and to each of their
elements. The recommender tries to guide the learner though the learning process according to the
SRL process model. Therefore (cognitive and meta-cognitive) learning activities are recommended
depending on what the learner has already done. The learner has to give feedback on what has
been done (which recommended learning activity has been performed). In order to recommend
learning resources (at the moment only tools), the learning goals and competences are taken into
account. Tools are recommended if they fit to the goals of the learner and if learners can actually
use them for successful learning.
This is realized by recommending at first activities to achieve the envisaged goals. Learning
techniques and recommended activity pattern can provide a basis for this step. Each of these
activities requires certain tool functionalities. Then tool descriptions, listing the functionalities of the
respective tools, allow recommendation of specific tools. In this way learning spaces can be
dynamically adapted to the current activities, thus avoiding a cognitive overload of the learner by
tools that are actually not needed. Preferences such as the degree of guidance are also taken into
account, which has an effect on the level of detail of the recommendations.
Figure 4. Psycho-pedagogical recommender realized and provided in the form of widgets (left:
guidance widget, right: learning planning widget)
According to the recommendation strategy the learner is provided with two kinds of
recommendations, learning activities and learning resources. Both are presented as a list of
possible choices. The choices are recorded and used for further recommendations, because this

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
9
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
information is needed to guide the learner through the learning process. In addition to these
recommendations the learner also gets explanations, which should help the learner to adopt the
concept of self-regulated learning. Furthermore, the learner gets an automatically generated
learning plan which is updated each time an interaction takes place. So the learner gets visual
feedback and orientation on what has been planned or completed and a general overview on this
state in the SRL process. The user interface has been implemented as a widget (see Figure 4). It
uses a service in the background where the models and user data are stored and where the
recommendation strategy is implemented.
Further work will concentrate on the integration of artifacts and peers to be recommended, usage
of log data as input data, and on an improved user interface.
IV. Comparison and discussion of the PLE recommender strategies
Considering the different goals and techniques of the three PLE recommenders adopted in ROLE, it
becomes clear that each one has its own benefits and shortcomings. Basically, user scenario for
our recommenders could look like this. In the beginning a learner has a specific need and decides
to start a new activity to address this need and achieve some goal, e.g. creating an outcome like a
document together with some colleagues. In a first step, a PLE recommender has to support the
learner by formulating her learning need and suggesting PLE designs so that she gets an idea of
what an environment for fulfilling the need could look like. Then, after reusing and adjusting such a
PLE design or creating a new one from scratch, a PLE recommender should provide links to
artifacts, peer users, or tools deemed relevant to her current activity.
Binocs widget
PLEShare
Psycho-pedag.
recommender
Recommender
strategy
Collaborative Filtering
(CF), PageRank-like,
content-based
CF and information
retrieval/clustering
(cliques, topics)
Rule and profile-based
(competences)
Data & data
gathering
On entering search
terms, automated
Tagged bookmarks,
voluntarily shared
Questionnaires,
automated (profile)
Estimated
accuracy
High (works well in
specialized scopes;
fallback through IR)
Average (requires
‘initialization’, cf. cold
start & sparsity)
Average (rules and
profile must be given)
PLE scenario
support &
usability
Average (not
considering PLE design
phase); good usability
Good (currently only
focusing on PLE
designs); usable
prototypes
Good (no cold-start
problem but restricted
to pre-def. domains);
average usability
Privacy
concerns
Sufficient
anonymization
Privacy statement,
anonymized activity
recordings (=patterns)
Raw usage data not
used; user profiles not
addressed yet
Preliminary
experiences
Preferences for Google
results; uptake in
business setting better
3 studies; works but
requires pilot users
sharing patterns (e.g.
teachers)
Internal evaluations;
efforts to integrate new
data; requires
modeling expertise
Table 1. Comparison of the three PLE recommender strategies developed in ROLE

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
10
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Table 1 gives an overview of the strengths and weaknesses of the three recommender solutions
which are being developed within the ROLE project. The comparison is conducted along six
dimensions, namely (a) the recommender strategy, (b) data and data gathering, (c) the estimated
accuracy, (d) the usefulness for the PLE scenario and the usability, (e) privacy issues, and (f)
experiences. The comparison evidences that the three recommender approaches in ROLE are based
on very different techniques and data-sets, which consequently leads to certain advantages and
disadvantages of each PLE recommender. In the following paragraphs we briefly summarize and
discuss the characteristics of the three developments.
Collaborative recommendations are realizable with a certain degree of accuracy without
threatening the users’ privacy (Machanavajjhala et al., 2011). However, this recommender is
highly tailored to a specific context, namely information retrieval, as the Binocs widget enables
federated search in different media and content repositories. In the scope of PLEs, this
recommender supports learners in finding appropriate artifacts for their different activities.
Additionally it is possible that the widget points to peers that are relevant to query terms, if privacy
policy allows it. However, the widget does not recommend learning activities and does not
take learner network structures into account. So, the usefulness of the federated search and
collaborative recommendation widget supports learners in the second phase of PLE-based
collaboration rather than in designing their environment.
The community-based PLE recommender, on the other hand, has been developed on top of a
simple semantic model, namely the notion of activities, which are used to structure one’s learning
context and to capture information on user interactions and the context. Following a collaborative
filtering (CF) approach, the pattern repository provides both recommendations of pre-given
(shared) PLE designs in the form of tagged bookmarking collections as well as recommendations on
artifacts, tools, and peers generated according to contextual information. Both kinds of
recommendations can be requested by a PLE solution through the Web-API, whereby items can be
differentiated between ‘must-haves’ (most frequent items) and ‘might-be-of-interest’ (items from
the long tail; see also (Mödritscher, 2010)). Although perfectly supporting the two phases of the
before-mentioned PLE scenario, this recommender suffers from typical weaknesses of CF
techniques, namely the cold-start problem (no data on new user and items) and sparsity (no or
less user ratings (Adomavicius and Tuzhilin, 2005)). The application of clustering techniques and
usage data is currently evaluated in order to refine the recommender algorithm.
Finally, the psycho-pedagogical recommender also supports the two phases of PLE-based learning.
On the one hand, a learner can use the planning widget to start an activity and determine her goal.
On the other hand, she can use the guidance widget to design and adjust the environment for her
current activity. As this recommender is based on a complex and structured semantic model and
pre-processed usage data, it has clear advantages if less or no data is given. In this case, the
psycho-pedagogical recommender claims to use expert-given rules to suggest goals and/or
widgets. On the negative side, it can identify and recommend new items much slower, as the
generation of recommendations is at least a semi-controlled process involving pedagogical experts.
With these recommender approaches we believe that we cover the most critical issues for
supporting learners in designing and using their PLEs. The most positive aspect of developing these
three strategies next to each other concerns the weaknesses of single recommenders we have
highlighted before. In case of lacking good recommendations for a specific case - e.g. if the
community-based recommender does not have enough data on items or users – the learner can try
to make use of suggestions of another recommender. This multi-approach also gives us flexibility
to support different scenarios in the very heterogeneous test-beds of the ROLE project. While some

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
11
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
test-beds are based on instructions and organizational driven learning (SJTU, FESTO) others have a
strong focus on informal settings and collaboration. Here we can vary the strategies for learner
support.
V. Conclusions and future work
To conclude, at this point the three recommenders are at rather different maturity levels. While
Binocs is being used by end-users, the pattern repository approach relies on the integration within
existing PLE systems to give recommendations to end users, and the psycho-pedagogical
recommender lacks the full implementation of all its features. So, next to finishing development
work on the latter two recommenders future work also comprises a user study for evaluating the
recommenders ‘in action’.
Acknowledgements
The authors would like to thank Erik Duval and Sten Govaerts for their valuable comments and
suggestions to improve the paper. The research leading to these results has received funding from
the European Community's Seventh Framework Programme (FP7/2007-2013) under grant
agreement no 231396 (ROLE project).
References
Adomavicius, G., and A. Tuzhilin. 2005. Toward the Next Generation of Recommender Systems: A
Survey of the State-of-the-Art and Possible Extensions. IEEE Transactions on Knowledge
and Data Engineering 17(6): 734-749.
Beham, G., B. Kump, T. Ley, and S. Lindstaedt. 2010. Recommending knowledgeable people in a
work-integrated learning system. Procedia Computer Science 1(2): 2783-2792.
Broisin, J., M. Brut, V. Butoianu, F. Sedes, and P. Vidal. 2010. A personalized recommendation
framework based on cam and document annotations. Procedia Computer Science 1(2):
2839-2848.
Brusilovsky, P., L.N. Cassel, L.M.L. Delcambre, E.A. Fox, R. Furuta, D.D. Garcia, F.M. Shipman III,
and M. Yudelson. 2010. Social navigation for educational digital libraries. Procedia
Computer Science 1(2): 2889-2897.
Brusilovsky, P., and Henze, N. 2007. Open Corpus Adaptive Educational Hypermedia. In P.
Brusilovsky, A. Kobsa, and W. Nejdl, eds.: The Adaptive Web. LNCS Vol. 4321, Springer,
Berlin, pages 671-696.
Burke, R. 2002. Hybrid recommender systems: Survey and experiments. User Modeling and User-
Adapted Interaction 12: 331-370.
Cano, P., O. Celma, M. Koppenberger, and J.M. Buldú. 2006. Topology of music recommendation
networks. An Interdisciplinary Journal of Nonlinear Science (CHAOS) 16(1): 013107/1-6.
Dabbagh, N., and A. Kitsantas. 2004. Supporting Self-Regulation in Student-Centered Web-Based
Learning Environments. International Journal on e-Learning 3(1): 40-47
Downes, S. 2010. New Technology Supporting Informal Learning. Journal of Emerging
Technologies in Web Intelligence 2(1): 27-33.

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
12
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Efklides, A. 2009. The role of metacognitive experiences in the learning process. Psicothema 21(1):
76-82.
El Helou, S., D. Gillet, and C. Salzmann. 2010. The 3A Personalized, Contextual and Relation-based
Recommender System. Journal of Universal Computer Science 16(16): 2179-2195.
Fruhmann, K., A. Nussbaumer, and D. Albert. 2010. A Psycho-Pedagogical Framework for Self-
Regulated Learning in a Responsive Open Learning Environment. Proceedings of the
International Conference eLearning Baltics Science (eLBa Science 2010), Rostock,
Germany.
Govaerts, S., S. El Helou, E. Duval, and D. Gillet. 2011a. A Federated Search and Social
Recommendation Widget. Proceedings of the 2nd International Workshop on Social
Recommender Systems (SRS 2011), Hangzhou, China.
Govaerts, S., K. Verbert, D. Dahrendorf, C. Ullrich, M. Schmidt, M. Werkle, A. Chatterjee, A.
Nussbaumer, D. Renzel, M. Scheffel, M. Friedrich, J.-L. Santos, E. Law, and E. Duval.
2011b. Towards Responsive Open Learning Environments: the ROLE Interoperability
Framework. Proceedings of the EC-TEL 2011 Conference, Palermo, Italy, pp. 125-138.
Henri, F., B. Charlier, and F. Limpens. 2008. Understanding PLE as an Essential Component of the
Learning Process. Proceedings of ED-Media 2008 Conference, Vienna, Austria, pages 3766-
3770.
Kooken, J., T. Ley, and R. De Hoog. 2007. How Do People Learn at the Workplace? Investigating
Four Workplace Learning Assumptions. In E. Duval, R. Klamma, and M. Wolpers, eds.:
Creating New Learning Experiences on a Global Scale. LNCS Vol. 4753, Springer,
Heidelberg, pages 158-171.
Machanavajjhala, A., A. Korolova, and A. Das Sarma. 2011. Personalized Social Recommendations
- Accurate or Private? Proceedings of the VLDB Endowment, Seattle, Washington, pages
440-450.
Manouselis, N., H. Drachsler, K. Verbert, and O.C. Santos, eds. 2010. Proceedings of the 1st
Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL).
Procedia Computer Science 1(2): 2773-2998.
Mitchell-Wong, J., R. Kowalczyk, A. Roshelova, B. Joy, and H. Tsai. 2007. Opensocial: From social
networks to social ecosystem. Proceedings of the Digital EcoSystems and Technologies
Conference (DEST 2007), Cairns, Australia, pages 361-366.
Mödritscher, F. 2010. Towards a recommender strategy for personal learning environments.
Procedia Computer Science 1(2): 2775-2782.
Mödritscher, F., Z. Petrushyna, and E.L.-C. Law. 2011. The Application of Pattern Repositories for
Sharing PLE Practices in Networked Communities.! Journal of Universal Computer Science
17(10): 1492-1510.
Page, L., S. Brin, R. Motwani, and T. Winograd. 1999. The pagerank citation ranking: Bringing
order to the web. Technical Report 1999-66, Stanford InfoLab.
Resnick, P., and H.R. Varian. 1997. Recommender systems. Communications of the ACM 40: 56-
58.
Santos, O.C., and J.G. Boticario, eds. 2011. Educational Recommender Systems and Technologies:
Practices and Challenges. IGI Global, Hershey.

!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! !
13
May I Suggest? Comparing Three PLE Recommender Strategies
!
F.Mödritscher, B.Krumay, S. El Helou, D. Gillet, A. Nussbaumer, D. Albert, I. Dahn and C. Ulrich
Digital Education Review - http://greav.ub.edu/der/
!
Santos, O.C., and J.G. Boticario. 2010. Modeling recommendations for the educational domain.
Procedia Computer Science 1(2): 2793-2800.
Thai-Nghe, N., L. Drumond, A. Krohn-Grimberghe, and L. Schmidt-Thieme. 2010. Recommender
system for predicting student performance. Procedia Computer Science 1(2): 2811-2819.
Ullrich, C., R. Shen, and D. Gillet. 2010. Not Yet Ready for Everyone: An Experience Report about a
Personal Learning Environment for Language Learning. In X. Luo, M. Spaniol, L. Wang, Q.
Li, W. Nejdl, and W. Zhang, eds.: Advances in Web-Based Learning - ICWL 2010. LNCS
Vol. 6483, Springer, Berlin, pages 269-278.
Van Harmelen, M. 2008. Design trajectories: Four experiments in PLE implementation. Interactive
Learning Environments 16(1): 35-46.
Verbert, K., H. Drachsler, N. Manouselis, M. Wolpers, R. Vuorikari, and E. Duval. 2011. Dataset-
driven Research for Improving Recommender Systems for Learning. Proceedings of the 1st
International Conference on Learning Analytics and Knowledge (LAK 2011), Banff, Canada.
Wang, Y., and K. Sumiya. 2010. Semantic ranking of lecture slides based on conceptual
relationship and presentational structure. Procedia Computer Science 1(2): 2801-2810.
Wild, F., F. Mödritscher, and S. Sigurdarson. 2008. Designing for Change: Mash-Up Personal
Learning Environments. eLearning Papers 9(2008): 1-15,
http://www.elearningeuropa.info/out/?doc_id=15055&rsr_id=15972 (accessed March 24,
2011).
Wilson, S., P.O. Liber, M. Johnson, P. Beauvoir, and P. Sharples. 2007. Personal Learning
Environments: Challenging the dominant design of educational systems. Journal of e-
Learning and Knowledge Society 3(2): 27-28.
Windschitl, M., and K. Sahl. 2002. Tracing teachers’ use of technology in a laptop computer school:
The interplay of teacher beliefs, social dynamics, and institutional culture. American
Educational Research Journal 39: 165-205.
Recommended citation
Mödritscher,F., Krumay,B., El Helou,S., Gillet,D., Nussbaumer,A., Albert,D., Dahn,I. and Ullrich,D.
(2011). May I Suggest? Comparing Three PLE Recommender Strategies. Digital Education Review,
20, 1-13. [Accessed: dd/mm/yyyy] http://greav.ub.edu/der
Copyright
The texts published in Digital Education Review are under a license Attribution-Noncommercial-No
Derivative Works 2,5 Spain, of Creative Commons. All the conditions of use in:
http://creativecommons.org/licenses/by-nc-nd/2.5/es/deed.en_US
In order to mention the works, you must give credit to the authors and to this Journal. Also,
Digital Education Review does not accept any responsibility for the points of view and statements
made by the authors in their work.
Subscribe & Contact DER
In order to subscribe to DER, please fill the form at http://greav.ub.edu/der