ArticlePDF Available

Abstract and Figures

Recommender Systems (RS) offer personalized services for facilitating the process of appropriate item selection. To perform this task, user profiling mechanisms should be implemented to automatically construct and update meaningful user profiles. These profiles can drive the RS in providing informed recommendations suited to the unique characteristics of each user. In the context of Technology enhanced Learning (TeL) Recommender Systems, the majority of research focus directly on learners’ profiling and ignore the potential benefits of profiling teachers’ professional capacities too. As a result, limited previous works exist on effectively capturing and utilizing individual teachers’ particular professional characteristics, such as their Digital Competences (commonly referred to as ICT Competences) and exploiting these in systems that support their teaching preparation and practice, for example in the selection of appropriate educational resources. This paper proposes a RS which targets to support teachers in selecting Learning Objects (LO) from existing LO Repositories (LORs) in a unified manner, namely by (a) automatically constructing their ICT Competence Profiles based on their actions within these LORs and (b) exploiting these profiles for more efficient LO selection. Experiments with data from 3 real-life LORs are presented and evaluation results are discussed to demonstrate the benefits of the proposed system
Content may be subject to copyright.
Learning Object Recommendations For
Teachers Based On Elicited ICT Competence
Stylianos Sergis, Student Member, IEEE and Demetrios G. Sampson, Golden Core and Senior
Member, IEEE
Abstract Recommender Systems (RS) offer personalized services for facilitating the process of appropriate item selection.
To perform this task, user profiling mechanisms should be implemented to automatically construct and update meaningful user
profiles. These profiles can drive the RS in providing informed recommendations suited to the unique characteristics of each
user. In the context of Technology enhanced Learning (TeL) Recommender Systems, the majority of research focus directly on
learners’ profiling and ignore the potential benefits of profiling teachers’ professional capacities too. As a result, limited previous
works exist on effectively capturing and utilizing individual teachers’ particular professional characteristics, such as their Digital
Competences (commonly referred to as ICT Competences) and exploiting these in systems that support their teaching
preparation and practice, for example in the selection of appropriate educational resources. This paper proposes a RS which
targets to support teachers in selecting Learning Objects (LO) from existing LO Repositories (LORs) in a unified manner,
namely by (a) automatically constructing their ICT Competence Profiles based on their actions within these LORs and (b)
exploiting these profiles for more efficient LO selection. Experiments with data from 3 real-life LORs are presented and
evaluation results are discussed to demonstrate the benefits of the proposed system.
Index Terms Learning Objects, Personalized E-Learning, Web Services, Recommender Systems
—————————— ——————————
ECOMMENDER Systems (RS) are software tools de-
signed to assist users in tackling the information over-
load problem by highlighting suitable items in a personal-
ized manner [1]. They have been used in a wide range of
application contexts, spanning from the movie industry
[2] to e-commerce [3] and Technology enhanced Learning
(TeL) [4].
RS have been implemented based on an increasing
number of techniques, the most prominent and common-
ly used being the content-based filtering, the collaborative
filtering, the demographic filtering and the hybrid ap-
proaches [1]. The content-based filtering systems generate
recommendations based on users' past choices and the
content similarity between the items that were favored in
the past to novel ones that have not yet been discovered
[5]. The collaborative filtering systems utilize the users'
ratings over the available items for providing recommen-
dations based on information provided by the "like-
minded" neighbors of the active user (i.e., the user cur-
rently using the system and receiving the generated rec-
ommendations) [6]. The demographic filtering relies on
the assummed commonalities that users with similar de-
mographic backgrounds will have [7]. Finally, hybrid
approaches combine techniques from the other approach-
es in order to reap the benefits of all, while tackling their
individual drawbacks [8].
In order to provide their personalized services, RS are
usually implemented to automatically gather data from
the users towards capturing their unique attributes, creat-
ing and updating individual profiles [9]. The reason for
this is that users tend to either not manually provide per-
sonal data, or they do not provide them in an accurate
manner, at a great cost to the RS prediction capacity in
both occassions [9]. Therefore, and in-line with the gen-
eral need for constructing and exploiting highly granulat-
ed user profiles in RS applications [10], intelligent mecha-
nisms should be employed in order to collect and process
meaningful user data for infering essential user character-
istics that could provide added value in the quality of
In the context of TeL, RS have been primarily utilized
for the recommendation of different types of Learning
Objects (LO) based on individual teachers and learners
profiles. Nevertheless, although learners’ profiling has
been extensively considered, teachers’ profiling is almost
neglected [11]. Taking into consideration, the important
role of teachers in adopting technology-supported school-
based educational innovations [12], the complexity of
technology-supported teaching practice [13] and the cur-
rent diversity of individual teachers’ digital competences
[14], it is safe to assume that individual teachers’ profes-
xxxx-xxxx/0x/$xx.00 © 200x IEEE Published by the IEEE Computer Society
S.Sergis is with the Department of Digital Systems, University of Pi-
raeus, Piraeus, GR-18532, Greece, and the Information Technologies In-
stitute, Centre for Research and Technology Hellas, Greece. E-mail: ste-
D. G. Sampson is with the School of Education, Curtin University
(since October 2015) and the Information Technologies Institute, Centre
for Research and Technology Hellas, Greece. Until September 2015 he
was with the Department of Digital Systems, University of Piraeus,
Greece, E-mail:
Manuscript received 5 January 2015.
sional ICT competences profiling is an important element
to be considered in educational RSs and, thus, it is worthy
to investigate this topic.
In our previous work, we have proposed and evaluat-
ed a LO recommendation approach that aimed to support
teachers in identifying and selecting educational re-
sources for their course design and delivery, taking into
consideration the individual teacher’s current ICT compe-
tence status [15]. Furthermore, we have proposed and
evaluated a mechanism for eliciting Fuzzy teachers’ ICT
Competence Profile (ICT-CP) based on their behavior
patterns within Learning Object Repositories (LOR) [16].
Following these initial works, in this paper we pro-
pose and study a Recommender System that supports
teachers, in a holistic manner, to select appropriate Learn-
ing Objects from existing LO Repositories (LORs) based
on their dynamically updated ICT competence profile.
The contribution of the proposed system is twofold. First,
the RS utilizes the teachers' ICT Competence profiles,
which are automatically elicited and dynamically updat-
ed based on the teachers' continuous usage behavior pat-
terns within LORs. Second, the recommendation process
itself adopts a new method for generating the LO recom-
mendations by exploiting the previously mentioned elic-
ited ICT Competence profiles, i.e. (a) by creating an ICT
Competence-based neighborhood and (b) by promoting
the LOs that the active teacher is currently competent in
The proposed system is presented and evaluated fol-
lowing a two-layer presentation in line with the common-
ly accepted layered evaluation methodology [17] of adap-
tive learning systems, which has been extended to RS in
[18]. More specifically, a separate presentation and evalu-
ation of the "Teacher ICT Competence Profile Elicitation"
Layer and the "Learning Object Recommendation Generation"
Layer is performed.
The remainder of this paper is structured as follows.
Section 2 presents the background of the present study, as
well as the identified research problem. Section 3 presents
the proposed approach to teacher ICT Competence-based
LO recommendations. This approach includes both a
method for eliciting the teachers' ICT Competence Profile
(ICT-CP) based on their usage behavior within Learning
Object Repositories as well as the manner in which these
profiles are exploited towards providing more informed
ICT Competence-based LO recommendations, compared
to existing, commonly used RS methods (described in
Section 4). Section 4 presents a detailed analysis of the
evaluation methodology employed. Section 5 presents the
evaluation results for both Layers of the proposed RS.
Finally, Section 6 contains the conclusions of the present
work and the future work in this research agenda.
2.1 User Profiling
User profiling is a technique that has been widely applied
in a range of software applications, including Adaptive
Web Systems [19] and Recommender Systems [20]. It in-
volves gathering data from the users in order to create a
profile for each of them, depicting their unique attributes
in the context of the system's application [21]. The latter
could involve, for example, movie genre preferences in a
movie recommender system [2], or learner preferences in
an educational RS [22].
The process of creating individual user profiles in
software applications is essential because each user has
his/her own characteristics and needs. Therefore, captur-
ing these user attributes is necessary for enabling the pro-
vision of personalized services [23]. A user profile, there-
fore, presents the system’s full interpretation of the users'
preferences and personal characteristics.
In many cases, user profiles are not provided by the
users themselves but they are being elicited automatically
through the identification, collection and processing of
relevant user actions’ data [24]. The reason for this is that
users are usually either unwilling to provide such infor-
mation or when they do, the validity and completeness of
the provided data cannot be ensured [25]. Therefore, the
elicitation process is usually based on the users' relevance
feedback data [26].
Relevance feedback data can be attained in two ways,
namely explicit feedback and implicit feedback [27]. The
former requires users to perform specific actions that will
inform and update their profile attributes, e.g., assign
ratings to items or download items. This approach pro-
vides a set of benefits for the hosting RS system, such as
increased development simplicity and enhanced accuracy
in the profile update process [28]. Users' implicit feedback
refers to mechanisms that monitor the users' interaction
with the system in an unobtrusive manner. Such ap-
proaches have been developed in order to completely
detach the user from the explicit feedback providing pro-
cess and maximize the amount of data that are being har-
vested by the system [29]. Examples of user actions that
are being monitored for profiling purposes include
browsing time in each item and type of items accessed,
uploaded or ignored [21].
2.2 User Profiling in Teacher-Oriented
Recommender Systems
In the context of Technology-enhanced Learning (TeL),
the majority of the implemented RS targets the learners
and aims to provide them with personalized learning
material and sequences of learning activities towards spe-
cific educational goal attainment [30]. The main learner
attributes that are being used in such processes include
their prior knowledge, learning preferences/styles, indi-
vidual goals and other cognitive characteristics [31]. Sub-
sequently, user profiling approaches have been primarily
focused on accommodating the attributes of the learners
However, despite the fact that learners are indeed the
main focus of learning processes, other actors are also
important for the successful implementation of effective
learning procedures. More specifically, teachers also play
a vital part in the educational processes and, amongst
other reasons, their ICT competences [12], [14] and per-
sonal attitudes towards ICT use [33] can greatly affect the
level and the quality of their technology-supported teach-
ing practice. Therefore, systematic accommodation of
teachers’ professional characteristics should be an im-
portant design consideration for educational RS. Within
this context, our previous work [11] included a literature
review of existing teacher-oriented educational RS. The 22
identified approaches are summarized in Table 1. Table 1
also contains information on the types of relevance feed-
back data that are being harvested. These relevance feed-
back data include Explicit and Implicit data. The former
include Social data (e.g., ratings and bookmarks) and user
Demographic data, while the latter include, for example
number of views of LO. Moreover, Table 1 presents in-
formation on whether these data are being utilized only
for ad-hoc similarity calculations (i.e., they do not incor-
porate profile creation) or if they are being automatically
processed and exploited to create dynamic and adaptive
user profiles for providing personalized recommenda-
tions (i.e., they do incorporate profile creation).
Relevance Feedback Data
Profile Creation
Explicit: Demographic
Explicit: Social
Explicit: Social
Explicit: Demographic
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Social
Explicit: Demographic
Explicit: Social
Explicit: Social
Explicit: Demographic
Explicit: Social
Explicit: Social
Only (completed) systems utilizing the above techniques
were considered. Table values designated as "no" either a
lack of user profiling mechanism in the corresponding
system or lack of specific presentation of relevant data in
the presenting paper.As Table 1 depicts, existing teacher-
oriented RS utilize a wide range of "raw" relevance feed-
back data types for creating user profiles. More specifical-
ly, most RS utilize "Explicit: Social" relevance feedback
data (N=17, x=77%). "Implicit" (N=5, x=23%) and "Explic-
it: Demographic" (N=4, x=18%) relevance feedback data
are used less often. However, regardeless of the type of
the relevance feedback utilized, the Table 1 data signify
that only a small portion of existing teacher-oriented RS
(N=6, x=27%) create, maintain and exploit user profiles
for providing enhanced recommendations to teachers,
based on these relevance feedback data. This can be con-
sidered as potential drawback, since systematic and effi-
cient user modeling is an essential element of successful
RSs [10].
More specifically, Shelton et al. [40] describe a system
that utilizes the teachers' clicks and time on each webpage
to detect their preferences and alter the recommendations
accordingly. Brusilovsky et al. [41] propose a social navi-
gation approach for assisting teachers to identify useful
educational resources within web-based repositories,
through manipulation of their usage history and explicit
feedback. Schoeffeger et al. [42] (and [50]) proposed a
method for identifying emergent topics that teachers are
dealing with and are interested in. Fazeli et al. [52] pro-
posed a method for collecting both demographic and so-
cial data from teachers towards utilizing them for build-
ing trust networks. Finally, Ferreira-Satler et al. [22] pro-
pose an ontology-based fuzzy teacher profile inference
system. The main idea is to elicit the preferences of the
teachers based on the semantic importance of each LO
that they create or browse. This importance is derived
from lexical analysis of the LOs and is calculated against
the existing teacher profile.
A careful analysis of the existing approaches highlights
the fact that key teachers’ individual professional capacity
characteristics, such as their aforementioned ICT Compe-
tences, are not being currently exploited for providing
personalized recommendations to support their daily
teaching practice of course design and delivery. Incorpo-
rating such information can lead to more focused recom-
mendations by forming better teacher neighborhoods and
by filtering candidate LOs to identify the most appropri-
ate for each individual teacher's ICT competences. Our
previous work [15] has presented initial evidence that
incorporating this type of teacher characteristics in the LO
recommendation process depicted in a formal manner
(i.e., the UNESCO ICT Competency Framework for
teachers [55]) and appropriately mapping them (e.g., [11])
to the specific LO metadata schema that each dataset em-
ploys (e.g., the Open Discovery Space Repository LO
Metadata Application Profile [56]) can have a significant
positive impact on the predictive accuracy of the RS.
Moreover, due to the fact that (a) competences are not
reified characteristics, but evolve over time, and (b) that
such data are rarely (much less correctly) provided by the
teachers themselves, profiling mechanisms should strive
to construct, and maintain up-to-date, highly granulated
ICT Competence profiles for teachers based on their rele-
vance feedback data [26].
In the light of the above, a research problem can be
identified, namely how can a teacher ICT Competence
profile be (a) elicited based on the implicit and explicit
relevance feedback data of the teachers and (b) utilized
for delivering more informed LO recommendations. To
the best of our knowledge, such an approach has not been
previously considered in the field of RS. Such a system
could provide a unified solution for providing personal-
ized LO recommendations to individual teachers, since it
would both automatically create and update teacher ICT
competence profiles and exploit them towards informed
LO suggestions.
The contribution of this paper, therefore, is the presen-
tation of a proposed solution towards tackling this identi-
fied research problem. The first step, namely, teachers'
ICT-CP elicitation, has already been analysed in our pre-
vious work [16]. In the context of this paper, however an
additional evaluation of this step was performed in order
to further enhance validattion of its robustness. The se-
cond step of the research problem, namely the exploita-
tion of the generated ICT-CP for enhanced LO recom-
mendations, is the original contribution of this paper.
This section presents the proposed solution to the
identified research problem, i.e. a new RS for providing
enhanced LO recommendations to teachers based on their
elicited ICT-CP. The proposed system is schematically
depicted in Fig. 1.
Fig. 1. Proposed Recommender System overview
It essentially comprises two main layers, as follows:
1. Teacher ICT Competence Profile Elicitation Lay-
er. The first Layer, which is described in detail in
Section 3.1, relates to the elicitation of the active
teacher's ICT-CP based on their relevance feed-
back data. More specifically, the system creates a
unified representation of the active teacher by
harvesting his/her "Usage Data" from the LOR.
Following that, this unified representation is fed to
the ICT-CP Elicitation mechanism and the fuzzy
ICT-CP of the teachers is generated.
2. ICT Competence-based Learning Object Rec-
ommendation Generation Layer. The second Lay-
er of the proposed RS is the LO recommendation
generation. This is performed by utilizing the out-
put data generated from the first Layer. More spe-
cifically, as it will be further described in Section
3.2, it utilizes these output data in a dual manner,
i.e., (a) for the selection of the active teacher's
neighbors, in terms of their ICT-CP similarity to
the active teacher and (b) for weighting each can-
didate LO in terms of its appropriateness for the
active teacher's ICT-CP. This is a promising ap-
proach, which was introduced in our previous
work [15] and was shown to result to more accu-
rate LO recommendations.
The presentation of the RS will be performed in two steps
in accordance to the two Layers of the RS, i.e. the teacher
ICT Competence Profile Elicitation Layer and the ICT
Competence-based LO Recommendation Generation
3.1 Teachers' ICT Competence Profile Elicitation
3.1.1 Teacher Relevance Feedback Data
The first Layer of the proposed RS relates to the elicitation
of the teachers' ICT Competence profiles based on their
relevance feedback data from their interaction within any
Towards tackling this issue, a specific and appropriate
set of relevance feedback data had to be selected. More
specifically, this set should provide a valid proxy of
teachers' ICT competence over the the available LO type
in the LOR. More specifically, the LO types referred to the
distinct metadata attributes which are used to character-
ize LO in LORs. It should be noted that these attributes
are flexible and can be adapted to meet the IEEE LOM
Application Profile that each LOR adopts, in line with the
official IANA MIME type extensions
A set of four types of relevance feedback data (from
now on referred to as "Usage Data"), commonly used in
the literature, was selected, as follows:
1. Rating History. This category included the ratings
teachers had provided over the available LO. To
accommodate the unique rating patterns of each
teacher (e.g., some tend to rate high by default), all
ratings were first normalized by subtracting the
rating mean of the active teacher. Rating data
could provide a solid proxy of ICT competence
due to the fact that a high rating could imply that
the user has either actually used the LO or, at least,
is comfortable with it [5]. A low rating on the other
hand, was not be considered as an indicator of low
competence, because a person would probably not
rate an item low just because they would not be
able to use it.
2. Bookmarking History. In the same vein as before,
bookmarking a specific LO type repeatedly could
provide insight on the fact that the active teacher
utilizes this type of resources frequently in their
daily teaching practice. Therefore, it could provide
a solid proxy for inferring the level of relevant ICT
competence of teachers.
3. Learning Object Access History. The access pat-
terns of teachers could be exploited for capturing
their ICT competences in the accessed LO types.
More specifically, if specific LO types had been re-
peatedly accessed from a teacher, they could be
regarded as commonly used by him/her. This
could infer useful information about the teacher's
ICT competence profile.
4. Learning Object Creation History. The last "Us-
age Data" category referred to active teachers'
sharing history, i.e. the number of LOs that each
teacher had uploaded to the LOR. These data
could provide insight on the teachers’ ICT Compe-
tence based on the assumption that a teacher, who
had created (and shared) a specific LO type, would
be competent in actually using them.
These "Usage Data" types were utilized by the system
for eliciting the teachers' ICT Competence profiles. More
specifically, for each teacher, this process included an
initial harvesting of all LOs that any of the "Usage Data"
were available for. This information was then fed to the
ICT Competence Profile Elicitation mechanism, towards
inferring the teachers' ICT-CP.
3.1.2. The Teacher ICT Competence Profile Elicitation
The ICT-CP Elicitation mechanism comprised two
phases, namely the "Aggregation" Phase and the "Fuzzifi-
cation" Phase (Fig. 2).
Fig. 2. Overview of the proposed RS Layer 1
1. The Aggregation Phase. The aim of this phase is
to create a unified representation of the teachers'
interactions within the Learning Object Reposito-
ries (LOR). More specifically, for each teacher, all
four "Usage Data" information per LO metadata
attribute are merged in one "Interactions" metric.
This aggregated metric provides an overview of
the level of interaction that each teacher had with
the diverse LO metadata types, by considering all
four "Usage Data".
In order to calculate this metric, each LO metadata
attribute type received a weight representing the
level of significance that it had for each teacher,
i.e., the level of preference that the active teacher
demonstrated to this particular LO attribute type
compared to the rest within the same "Usage Data"
The weights are calculated using the following
formula for each teacher :
 
where refers to each metadata attribute type
within the same "Usage Data" category and re-
fers to the overall cardinality of the metadata at-
tribute set (which can vary between different LOR)
within the same "Usage Data" category. The possi-
ble values for each weight range from [0 ... 1]. A
weight value of "zero" for a specific LO metadata
type signifies that the teacher has interacted with
no LO of this type regarding the specific "Usage
Data" category (e.g, no ratings provided for any
LO of type "video"). On the other hand, a weight
value of "one" for a specific LO metadata type sig-
nifies that the teacher has interacted solely with
LO of this particular type regarding a specific "Us-
age Data" category (e.g., the teacher has provided
bookmarks only for LO of type "video").
Moreover, (1) also normalizes the available "Usage
Data", since the different "Usage Data" categories
would probably not have the same cardinality of
LO (i.e., there would probably be more "Access"
data than "Bookmark" data).
Therefore, building on the weights calculated in
(1) for each LO metadata type, the combined "In-
teractions" metric is computed using the following
formula, for each teacher :
 
where,  is the resulting "Interactions"
metric for each LO metadata attribute type , is
the cardinality of the “Usage Data” set and is
the weight of the LO metadata attribute, as calcu-
lated by (1).
Therefore, for each teacher , the "Interactions"
metric presents a unified and combinative depic-
tion of their usage pattern in a LOR, for each LO
metadata attribute type by considering all four
"Usage Data". This is done by considering together
(a) the significance that each LO metadata attrib-
ute type had to each teacher (i.e., the weights) and
(b) the number of LO per metadata attribute type
that the teacher had interacted with.
2. The Fuzzification Phase. After the completion of
the Aggregation Phase process, and the construc-
tion of the active teacher's unified "Interactions"
metric, the Fuzzification Phase is activated. This
step is responsible for inferring the level of the
teacher's ICT competence (their ICT-CP) by trans-
lating the aggregated "Interactions" metrics for
each teacher to their fuzzy equivalents. The latter
would be the final proxy of the teachers' ICT
Competence. The reason for employing fuzzy logic
is due to the fact that it is considered as an appro-
priate means of depicting data that are "vague"
and difficult to assign to crisp categories [57].
Therefore, apart from the extensive use of this ap-
proach for creating user profiles in different appli-
cation contexts [22], [58], fuzzy logic is selected
due to the "fluid" nature of competences. More
specifically, competences are not crisp characteris-
tics, but can be attained in a continuum of profi-
ciency levels and can be constantly updated [59].
The adopted fuzzification process uses a set of five
linguistic variables depicting levels of competence,
i.e., Very Low, Low, Medium, High and Very
High. The use of five linguistic variables is a typi-
cal method in the literature (e.g., [29]). Addition-
aly, a triangular membership function was uti-
lized, which is presented in Fig. 3. The latter was
used due to the fact that it is a commonly used
membership function type for depicting character-
istics similar to ours [24].
Fig. 3. Membership functions of the proposed RS
where μ(α) represents the membership function
value and it is calculated for all five linguistic var-
iables each time,
represents the "Interactions"
metric value and the
depict the mini-
mum and maximum marginal values for each
Fuzzy linguistic variable.
The membership func-
tion is schematically depicted in Fig. 4.
Fig. 4. Schematic Representation of the Membership func-
tion of the proposed RS
To give a brief example of the above, supposing
that a teacher had a resulting aggregated "Interac-
tions" value of "0.9" (from the Aggregation Phase
of the ICT Competence Elicitation mechanism) re-
garding a specific type of LO (e.g., "inquiry-based"
Lesson Plans), their generated fuzzy ICT-CP for
this LO type would be:
More specifically, this means that the particular
teacher is 40% Highly Competent in utilizing in-
quiry-based lesson plans and 60% Very Highly
competent. Therefore, upon completion of the pro-
cess of the Fuzzification Phase, each teacher's ICT-
CP has been modeled in a fuzzy manner for all LO
metadata types, based on his/her "Usage Data"
within the LOR. The utilization of fuzzy logic for
capturing and representing teachers' ICT Compe-
tences allows for more granulated construction of
profiles. In turn, this enhanced level of teacher
profile granularity can assist in providing more in-
formed recommendations [22].
The exact manner in which this is realized in the
proposed RS is described in the next section.
3.2 ICT Competence-based Recommendations
After the completion of the processes of the first Layer,
the active teacher has been assigned with a unique fuzzy
ICT Competence profile depicting their personal compe-
tences in each of the distinct LO metadata attribute types
stored in the LOR. At this stage, the second Layer of the
RS is activated, i.e. the LO recommendation generator
(Fig. 5).
Fig. 5. Overview of the proposed RS Layer 2
As aforementioned, the contribution in the proposed
approach in this Layer related to the novel manner in
which these recommendations were generated. More spe-
cifically, the elicited fuzzy ICT-CPs were utilized (a) as a
means for alternative neighbor selection and (b) as a
weighting factor for determining the appropriateness of
each candidate LO to the active teacher's level of ICT
The second Layer is divided in three phases, namely:
(a) the Neighbor Selection Phase, (b) the ICT Competence
Defuzzification Phase and (c) the Recommendation Gen-
eration Phase.
1. The Neighbor Selection Phase: this phase aims at
selecting the most suitable set of neighbors for the
active teacher based on the similarity of their
fuzzy ICT competence profiles instead of the
commonly used rating similarity. In this way, the
active teacher is provided with recommendations
based on the opinions of colleagues that have the
same ICT competence background with him/her,
and thus (s)he is recommended with LO which
will potentially be both useful and appropriate for
The method employed for calculating this similari-
ty is the Euclidean Distance, which is commonly
used for similar processes [60].
2. The Defuzzification Phase: this phase aims to
translate the unified fuzzy teacher ICT-CP in a
single factor depicting the combined level of ICT
competence of the teacher and incorporating all
fuzzy levels. This factor will be utilized for
weighting each candidate LO in terms of its suita-
bility for the active teacher. The method employed
for this Phase is the commonly used Center of
Gravity method [61], which computes the center of
gravity of the area under the membership func-
tion. The resulting factor of the Defuzzification
Process is utilized in the second Phase of the ICT
Competence-based Recommendations Layer of the
3. The Recommendation Generation Phase: this
phase aims to generate and deliver the ICT Com-
petence-based LO recommendations. As afore-
mentioned, the proposed recommendation meth-
od utilizes the teachers' ICT-CP profiles in a dual
First, the active teacher's neighbors are selected in
Phase 1 ("Neighbor Selection Phase"), based on the
similarity between their fuzzy ICT-CP. Second, the
defuzzified ICT-CP factor is utilized as a filtering
method of candidate LOs in terms of their appro-
priateness for the active teacher's level of compe-
Based on the above, for each LO that is being
considered for recommendation, the system as-
sesses its appropriateness score (AS) for the active
teacher t using the following formula:
 
where N is the amount of neighbors that had rat-
ed the specific LO, rj i is the rating provided for
LO by neighbor teacher j and wj is the Euclidean
Distance between the active teacher and neighbor
j. The Euclidean Distance is, therefore, utilized as
a weighting factor in order to assign more gravi-
ty to the opinions of "closer" neighbors. Finally,
the COGtype of (i) is the output of the defuzzification
process of the active teacher t. The resulting AS
value represents the predicted rating for the ac-
tive LO for the active teacher.
This section presented the proposed solution to the
identified research problem, i.e. a unified approach for
eliciting and exploiting teacher ICT-CP towards more
informed LO recommendations. The next section presents
the methodology employed and results generated for
evaluating the performance of the proposed system, in
terms of (a) the accuracy of the ICT-CP elicitation mecha-
nism and (b) the predictive accuracy of the RS for gener-
ating LO recommendations.
4.1 Methodology
The evaluation methodology adopted in this paper fol-
lows the layered evaluation approach [17]. More specifi-
cally, monolithic evaluation of RS has been identified to
lead to limited remedying potential in case of low RS ac-
curacy. Essentially, this means that a potentially low ac-
curacy of the RS cannot be linked to the specific element
of the RS that is causing this level of performance. To al-
leviate this issue, the layered evaluation process proposes
separate evaluation Phases for each of the RS Layers,
namely Phase 1 refers to the evaluation of the Teacher
Profiling Layer and Phase 2 refers to the evaluation of the
ICT Competence-based Recommendation Layer. In this
way, any underperforming Layer can be highlighted and
focused remedying actions can be performed, towards
increasing the accuracy of the specific Layer, and by ex-
tension, of the overall RS.
In the context of the present study, the Phase 1 evalua-
tion focuses on the proposed system's accuracy in re-
creating existing teacher ICT-CP. More specifically, the
system generates a set of teacher ICT-CP based on their
relevance feedback data. The generated ICT-CPs will then
be contrasted against the existing ICT-CPs, which have
already been provided voluntarily by the teachers them-
selves in the context of the ODS Project. Moreover, it
should be mentioned that the existing ICT-CPs were pro-
vided by the teachers in a manual manner, using a web-
form and following the well-known UNESCO ICT Com-
petency Profile for teachers
. The metric used for imple-
menting this benchmark was the Jaccard co-efficient [62].
The Phase 1 evaluation has already been performed
and reported in our previous work [16]. In the context of
this paper, however, we retrieved an updated version of
the same dataset and further validated the results, to-
wards enhancing the robustness of the proposed method.
The updated evaluation results of Phase 1, as well as a
comparison to the initial evaluation results described in
[16], are discussed in Section 5.1 of this paper.
Regarding the Phase 2 evaluation, the main focus was
assessing the accuracy of the Layer 2 of the proposed ap-
proach to predict the ratings of the active teacher on LO
they had not interacted with, using the process described
in (3). This evaluation phase aims at providing evidence
on the positive added value and the increased accuracy of
the proposed RS (depicted as Fuzzy Hybrid, abbreviated
as FH) compared to existing approaches. The commonly
used Root Mean Squared Error (RMSE) metric was select-
ed for measuring the predictive accuracy [63]. This metric
is calculated based on the formula:
 
 
where  is the set of generated predicted ratings for
users on items ,  is the set of known ratings and is
the set of users and items for which the ratings are
known. It should be noted that since this metric aims to
capture errors in the predictions of the RS, lesser values of
RMSE designate a better predictive accuracy of the RS.
As aforementioned, our proposed FH approach was
benchmarked against a set of "control" recommendation
methods, which are (a) commonly used in the literature
and (b) have been reported to provide high levels of accu-
racy. More specifically, the control set included two types
of recommendation approaches, namely user-based col-
1 [Ac-
cessed 10 May 2015]
laborative filtering (U) and item-based collaborative filter-
ing (I) [62]. The former computes similarities between
users to find the most similar users and predicts ratings
based on how the item was rated by the most like-minded
users. The latter approach shares the same idea but it is
based on similarity between items rather than between
users. For each of these two recommendation approaches
one similarity measure was implemented, namely the
Pearson correlation coefficient (PCC) for the user-based
collaborative filtering [64] and the Adjusted Cosine corre-
lation coefficient (CCC) for the item-based collaborative
filtering [65]. Both similarity measures have been selected
due to their reported high performance in terms of pre-
dictive accuracy in the context of their specific recom-
mendation approach [64], [65].
The Pearson correlation coefficient is calculated based
on the following formula:
 
 
where is the set of items that both users and have
rated,  and  denote the ratings of the users and
on the item i¸ while
denote the average ratings
of the two users respectively.
The Adjusted Cosine correlation coefficient is calculat-
ed based on the following formula:
 
 
where is the set of users that have rated both items
and j,  and  denote the ratings of the user on items
and j respectively while
denotes the average ratings
of the user.
Therefore, the overall control recommendation method
set consisted of two alternatives, i.e. User-based Pearson
Correlation Coefficient (UPCC) and the Item-based Ad-
justed Cosine Correlation Coefficient (ICCC).
For all three cases (two control approaches plus the
proposed approach), the predictive accuracy evaluation
process was based on the standard "Leave-N-out" tech-
nique [66]. More specifically, this method includes split-
ting the available dataset in two sub-sets, namely the
"training" set and the "test" set. The former contained 70%
of the overall data and was used for training the RS and
generating the recommendations. The latter contained the
remaining 30% of the overall data and was used for eval-
uating the system accuracy, i.e., the recommendations
generated from the training set [62]. Finally, the evalua-
tion experiment was run for an increasing number of
neighborhood size (one through twenty) in order to mon-
itor the behavior of each approach in each occasion.
4.2 Datasets
The present study utilized three educational datasets for
supporting the two evaluation Phases, all of which were
retrieved from existing Learning Object Repositories
which are currently in use. More specifically, the LORs
used in our experiments were the Open Discovery Space
(ODS) Repository [],
the Discover the Cosmos (DtC) Repository
[], and the Open Sci-
ence Resources (OSR) Repository
The ODS dataset, an earlier version of which was used
in previous work [11], [15], [16], contained (a) existing,
manually provided teachers' ICT Competence profiles,
depicted using the UNESCO ICT Competency Frame-
work for teachers (ICT-CFT) [55], (b) LO metadata records
characterized using a specific IEEE LO Metadata Applica-
tion Profile [56] and (c) "Usage Data" of the teachers as
they were defined in Section 3.1.1. Due to the unique
availability of existing teachers' ICT-CP, the ODS dataset
was utilized for the Phase 1 evaluation process, for
benchmarking the proposed teacher ICT Competence
elicitation method. As aforementioned, this evaluation
process was initially described in [16], where it was per-
formed on an earlier instance of the ODS dataset. In the
context of the work presented in this paper, an up-to-date
version of the ODS dataset was retrieved towards further
validating these initial promising evaluation results. In
both experiments, towards the evaluation of the re-
creation accuracy of the proposed method, mapping rules
were utilized for connecting ICT Competences as they
were described in ICT-CFT and LO metadata attributes as
they were described in the IEEE LOM Application Profile
of the ODS dataset [56]. These mapping rules were pre-
sented and evaluated in previous work [11].
The ODS dataset was not utilized in the Phase 2 evalu-
ation process, in order to fully adhere to the layered eval-
uation framework adopted, namely to not only evaluate
the different layers of the proposed RS individually, but
also to utilize unique datasets for evaluating each layer.
The DtC dataset []
and the OSR dataset [67] contained (a) LO metadata rec-
ords and (b) "Usage Data" of teachers that had been regis-
tered to the corresponding portals. The DtC and OSR da-
tasets did not contain existing teacher ICT Competence
profiles. Therefore, they were used only in the Phase 2
evaluation process, i.e. for evaluating the predictive accu-
racy of the proposed ICT Competence-based RS for
teachers, by exploiting teachers’ ICT-CP elicited by the
first Layer of the proposed RS.
An overview of all three datasets is provided in the fol-
lowing Table 2. Aggregation Level (AL) 1 LOs refer to
standalone Educational Resources (e.g., flash simulations,
educational games, text documents), while Aggregation
Level 2 LOs refer to Lesson Plans and/or Educational
Scenarios (i.e., flows of learning activities supported by
Educational Resources).
Overall Sample Size (N) refers to the total number of
LOs in each AL category. The number of unique users
refers to the cardinality of the set of teachers that had con-
tributed at least in one "Usage Data" category. The re-
maing four data categories, namely Rating Data, Access
Data, Creation Data and Bookmark Data, refer to the
number of LOs that the corresponding "Usage Data" were
provided for. As aforementioned, these four "Usage Data"
categories were exploited by the proposed RS for eliciting
the teachers' ICT-CP.
ODS Dataset
DtC Dataset
OSR Dataset
Overall Sample
Size (N)
Rating Data
Access Data
Creation Data
Bookmark Data
Unique Users
The OSR dataset only contained usable data for Ag-
gregation Level 1 Learning Objects. Finally, as "unique
users" for the ODS dataset were only considered those
who had contributed their ICT-CP, since the purpose of
this dataset was to evaluate the first Layer of the pro-
posed RS.
Furthermore, regarding the Phase 2 evaluation process, in
order to allow for a more informed overview of the eval-
uation results (presented in Section 5), an analysis of spe-
cific characteristics of the datasets utilized (i.e. DtC AL1,
DtC AL2 and OSR) was performed. More specifically, this
analysis was performed due to the reported high depend-
ence of all RS's performance to the dataset to which they
are fed [62],[68] and the resulting need to robustly under-
stand the shift in performance of the selected RS bench-
mark approaches between the different datasets. Based on
the works of [68] and [69], we selected a set of two dataset
characteristics that have been reported to greatly influ-
ence the level of performance of RS. These are:
1. Rating Density. It is defined as the ratio of known
ratings against the number of all possible ratings
that can be provided (i.e., ,
where U and I are the number of users and items
respectively). A logarithmic transformation was
performed in order to normalize its values. Rating
Density has been attributed with a high positive
correlation to collaborative filtering RS perfor-
mance, i.e., denser datasets can allow for more ac-
curate recommendations [68].
2. Rating Standard Deviation. It is defined as the
standard deviation of the ratings provided by the
users in the dataset. Rating Standard Deviation
has been attributed with a high negative correlation
to collaborative filtering RS performance, i.e., low-
er levels of Rating Standard Deviation can lead to
more accurate recommendations [68].
The instantiations of the abovementioned dataset char-
acteristics for the selected datasets for the Phase 2 Evalua-
tion are depicted in Table 3.
DtC Dataset
OSR Dataset
Rating Density
Rating Standard
As Table 3 depicts, the densest dataset is the DtC AL2,
followed by the OSR and the DtC AL1. Therefore, based
on the conclusions of [68], we should expect that evalua-
tion result accuracy in each of the three datasets should
follow the same correspondingly descreasing order. This
intuition is supported by the Rating Standard Deviation
data, which (given its negative correlation to RS accuracy)
also signify the same correspondingly descreasing order
in terms of expected evaluation accuracy.
Table 4 presents an overview of the evaluation meth-
odology and datasets described in Sections 4.1 and 4.2.
RS Layer
Focal Point
Layer 1
teacher ICT-
Layer 2
The following section presents the results for Phase 1 and
Phase 2 of the evaluation methodology described in this
Following the layered evaluation approach, the results for
each of the two Layers of the proposed RS are presented
separately in the following sections.
5.1 Teacher ICT Competence Profile Elicitation
Method Evaluation
As aforementioned, an initial evaluation of the Phase 1
was conducted in our previous work [16]. These prelimi-
nary evaluation results (Experiment #1) are presented in
Fig. 6 along with the new Phase 1 evaluation results gen-
erated from the updated version of the ODS dataset (Ex-
periment #2). Both sets of results provide evidence of the
proposed method's high efficiency to elicit the ICT-CP of
teachers based on their "Usage Data".
More specifically, for Educational Resources (AL1
LOs) the proposed system re-created the existing ICT-CP,
which, as aforementioned, where provided by the teach-
ers themselves in a manual manner through a web-form,
with 70% accuracy in the initial Experiment #1 and with
73% accuracy in the updated Experiment #2. For Lesson
Plans (AL2 LOs) the ICT-CP re-creation accuracy level
was 81% in the initial evaluation Experiment #1 and 79%
accuracy in the new evaluation Experiment #2.
Regarding the evaluation results' accuracy levels in
their own regard, they are very promising for both evalu-
ation experiments. The reported deviations from the orig-
inal, user-provided ICT-CPs can be mainly attributed to
the prominent reason that teachers could have provided
incorrect descriptions of their ICT competences to begin
Fig. 6. Accuracy results for the two evaluation experiments of the
Phase 1 of the teacher ICT Competence Elicitation Mechanism
This is a very common issue in the user profiling litera-
ture, where users provide incomplete or false profiling
data [21], [25]. Therefore, the manually-provided data
would not always correctly represent the current level of
ICT competence of the teacher that provided them, as
opposed to the automatic elicitation method, which cap-
tures the teachers' actual usage patterns in using LOs and
can therefore provide a more solid indicator of preference
and competence [29]. Nonetheless, the re-creation accura-
cy results are significantly high in all experiments and for
both Aggregation Levels despite this known shortcoming.
Moreover, regarding the result differences between
Experiment #1 and Experiment #2, despite their small
size, they could be attributed to the updated sets of “Us-
age Data" that the ODS dataset contained. These potential
data alterations could have triggered these minor accura-
cy changes of the system, since the actual teachers’ usage
patterns could have been slightly altered since the initial
version, without a corresponding manual change in the
ICT-CP by the teacher. In any case, however, as Fig. 6
depicts, the evaluation result deviations between the two
Experiments are very small and the absolute evaluation
results are consistent between Experiments, indicating the
proposed elicitation method's robustness.
Overall, the evaluation results for the first Layer of the
proposed system were promising and provided with evi-
dence that the teacher ICT-CP Elicitation Method could
be utilized within Learning Object Repositories, in order
to elicit the teachers ICT-CP. As aforementioned, this in-
formation could allow for enhanced LO recommenda-
tions. The evaluation results for the latter, which repre-
sents the second Layer of the proposed system, are pre-
sented in the following section.
5.2 ICT Competence-based Learning Object
Recommendations Evaluation
The second Phase in the layered evaluation of the pro-
posed approach to LO recommendations focused on the
added value it could provide in terms of predictive accu-
racy, by utilizing the generated teacher ICT-CP from the
first Layer. As aforementioned, the DtC (for AL1 and AL2
LO) and OSR (for AL1 LO) datasets were utilized in this
process. The evaluation results for each dataset will be
presented in separate.
1. Discover the Cosmos Dataset. Fig. 7 and 8 depict
the evaluation results of the DtC dataset for the
two Aggregation Levels of Learning Objects that
the dataset contained. As the Fig. 7 depicts, the
proposed approach outperfoms both control rec-
ommendation methods for the AL1 LOs.
Fig. 7. Preditive accuracy evaluation results for AL1 Learn-
ing Objects in the DtC dataset.
Even though the accuracy of the two benchmark
methods increases considerably as the neighbor-
hood size (NS) grows, they are outperformed by
the proposed FH approach for all NS values.
Moreover, in the user-based control method, the
threshold neighborhood size is around 10, mean-
ing that increasing the neighbors above that
threshold does not offer significant increase in ac-
This is due to the fact that relatively few ratings
and users were available in the DtC dataset for the
AL1 LOs. Therefore, the user neighborhoods had
limited data to be built on. This is closely related
to the influence of dataset characteristics described
in Section 4.2, and more specifically, the Rating
Density [68]. For the ICCC method this occurs at a
smaller degree, since, regarding candidate neigh-
bors at least, there is a larger pool to choose from
(i.e., the AL1 LOs of the dataset). The ICCC accu-
racy (as well as the proposed FH's accuracy),
therefore, continues to improve for a larger NS
span, and begins to plateu at around 15 neighbors.
Therefore, the findings from this dataset conclude
that the proposed FH approach provides more ac-
curate LO recommendations, throughout the NS
Regarding AL2 LOs (Fig. 8), the proposed ap-
proach similarly outperfoms both control recom-
mendation methods for all NS values. Moreover, it
is important to notice that all three methods' pre-
dictive accuracy is considerably increased, com-
pared to their AL1 equivalents.
Fig. 8. Preditive accuracy evaluation results for AL2 Learn-
ing Objects in the DtC dataset
The reason for this universal performance im-
provement can be attributed to the dataset charac-
teristics of the DtC AL2 dataset and its comparison
to its AL1 equivalent. More specifically, as Table 3
depicted, the DtC AL2 dataset was much denser
(Rating Density) and had a lesser value of Rating
Standard Deviation. Therefore, building on (and
confirming) the findings of [68], we mainly attrib-
ute the difference of the results of Fig. 7 and Fig. 8
(and the enhanced predictive accuracy of the RS in
the DtC AL2 dataset), to these two dataset charac-
teristics' values.
Moreover, the significant improvement of Rating
Density (compared to the DtC AL1 dataset) in-
creased the NS threshold that provided better pre-
dictive accuracy for all three recommendation
methods. More specifically, the NS threshold for
the UPCC control method was increased to
around 17 and for the proposed FH, as well as for
the ICCC control method, the NS threshold is
around 18.
Overall, despite the significant improvement in all
three RS methods in the DtC AL2 datasets (com-
pared to the DtC AL1), the proposed FH still out-
performed the two benchmark methods and, es-
sentially, delivered more accurate LO recommen-
2. Oper Science Resources Dataset. As aforemen-
tioned, the OSR dataset only contained usable data
for AL1 LOs. The accuracy evaluation results are
presented in Fig. 9.
Fig. 9. Preditive accuracy evaluation results for AL1 Learn-
ing Objects in the OSR dataset
As the Fig. 9 depicts, the proposed approach again
ourperforms both benchmark methods for all
neighborhood sizes.
Moreover, the predictive accuracy of all approach-
es is decreased compared to the DtC AL2 dataset,
but are better that the DtC AL1 dataset. This find-
ing can be explained by considering the dataset
characteristics presented in Table 3. More specifi-
cally, as the Table 3 data depict, the values for both
OSR dataset charasteristics (i.e., Rating Density
and Rating Standard Deviation) are better that the
DtC AL1 and worse than the DtC AL2. Therefore,
the predictive accuracy results depicted in Fig. 9
(and their connection to the results of Fig. 7 and
Fig. 8) can be attributed to these facts. Regarding
the NS thresholds, the results follow a similar pat-
tern, i.e., they are increased compared to the DtC
AL1 dataset and decreased compared to the DtC
AL2 dataset. More specifically, the UPCC control
method plateus at around 16 neighbors, and the
ICCC control method, as well as the proposed FH
approach plateu at around 17 neighbors.
Therefore, the OSR dataset re-validated the prom-
ising results of the two DtC datasets, i.e. that the
proposed FH approach can deliver more accurate
LO recommendations to teachers, for all NS.
Overall, a consolidated overview of the predictive ac-
curacy of the proposed FH approach provides with evi-
dence of the added value that it can offer. More specifical-
ly, the reported promising results from Phase 2 evalua-
tion experiments in the DtC (for both AL1 and AL2 LOs)
and OSR datasets show that the proposed approach can
deliver more accurate LO recommendations to teachers
based on their elicited ICT Competence profiles. This is in
line with the findings of our previous work [15] which
presented initial evidence that incorporating (existing)
teachers' ICT-CP in the LO recommendation process can
provide with increased accuracy and, essentially, better
This paper introduced and evaluated a new RS for
providing LO recommendations to teachers based on
their ICT-CP elicited from their relevance feedback data.
More specifically, the proposed system was divided in
two Layers, namely (a) the teacher ICT Competence Pro-
file Elicitation Layer, which was targeted at eliciting and
constructing the teachers' ICT-CP based on their rele-
vance feedback data, and (b) the ICT Competence-based
LO Recommendation Layer, which utilized the elicited
teacher ICT-CP from Layer 1 for providing more in-
formed LO recommendations.
The RS evaluation followed the same layered approach
and provided with evaluation results for each system
Layer. Regarding the Layer 1 evaluation, the expreriment
presented in this paper was the second one being con-
ducted and the consistency between the two Experiments
provides with additional evidence of the proposed elicita-
tion's method robustness. Overall, both individual Layer
evaluations, as well as in combination (as a holistic sys-
tem), provided with evidence that the proposed approach
can generate LO recommendations to teachers, which are
personalized to their level of ICT competence. Moreover,
the increased granularity offered by the fuzzy depiction
of ICT competences was shown to have provided an add-
ed value in the overall RS performance.
Therefore, the evaluation results show that the pro-
posed RS has the potential to be used for assisting teach-
ers in their everyday tasks of course design, lesson plan-
ning as well as their implementation and delivery, by
facilitating the process of selecting and retrieving appro-
priate LOs that match the teachers' ICT competence lev-
Future work in this research agenda could include the
investigation in a granulated manner of the level of influ-
ence that the different levels of teachers’ ICT competence
can have on the predictive accuracy of the proposed sys-
tem. Moreover, it would be interesting to investigate the
potential of adaptive neighbor selection. More specifical-
ly, this strand will aim to extend the work presented in
this paper which included selecting the active teacher
neighbors based on their ICT-CP similarity. The next step
will be to select neighbors in an adaptive manner depend-
ing on (a) the specific metadata attribute type of candi-
date LO to be recommended and (b) the specific ICT
competences of the candidate neighbors in this particular
LO type. By performing this dynamic neighborhood crea-
tion, the "gravity" of each neighbor in the recommenda-
tion process will be properly adjusted to reflect not only
their similarity to the active teachers' ICT-CP, but also
their "credibility" to recommend this particular LO
metadata attribute type. Put simply, from the initial pool
of ICT competence-based neighbors (proposed in the pre-
sent paper), those that are more highly competent in the
particular LO type (that is to be recommended) will have
an increased "gravity" of opinion.
Finally, further research should be focused on incorpo-
rating additional contextual factors in the LO recommen-
dation process for teachers, such as the ICT Competences
of their school [70], [71]. The reason for this is that such
contextual factors (e.g., school infrastructure and school
leader ICT competences) have been repeatedly reported
as significant in the level of uptake of ICT in the formal
learning processes [72]. Therefore, accommodating them
within the LO selection process would potentially extend
the scope of the proposed RS and improve its capacity to
facilitate teachers in selecting appropriate LOs for their
unique context of work and, therefore, to optimize their
instructional design and delivery efforts.
The work presented in this paper has been partially supported by (a)
the “Open Discovery Space: A socially-powered and multilingual
open learning infrastructure to boost the adoption of eLearning Re-
sources” Project that is funded by the European Commission's CIP-
ICT Policy Support Programme (Project Number: 297229) and (b)
“Inspiring Science: Large Scale Experimentation Scenarios to Main-
stream eLearning in Science, Mathematics and Technology in Prima-
ry and Secondary Schools” Project that is funded by the European
Commission's CIP-ICT Policy Support Programme (Project Number:
325123). Moreover, the authors would like to acknowledge the Dis-
cover the Cosmos and the Open Science Resources Projects for
providing the corresponding educational datasets.
[1] J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez,
"Recommender Systems Survey," Knowledge-Based Systems, vol. 46,
pp. 109-132, 2013.
[2] J. Bobadilla, F. Ortega, A. Hernando, and J. Alcala, "Improving
Collaborative Filtering Recommender System Results And
Performance Using Genetic Algorithms," Knowledge-Based Systems,
vol. 46, no. 8, pp. 1310-1316, 2011.
[3] Z. Huang, D. Zeng, and H. Chen, "A Comparison Of Collaborative-
Filtering Recommendation Algorithms For e-Commerce," IEEE
Intelligent Systems, vol. 22, no. 5, pp. 68-78, 2007.
[4] N. Manouselis, H. Drachsler, K. Verbert, and E. Duval, "Introduction
and Background," Recommender Systems for Learning, N.
Manouselis, H. Draschler, K. Verbert, and E. Duval, eds., New York:
Springer, pp. 1-20, 2013.
[5] P. Lops, M. de Gemmis, and G. Semeraro, "Content-Based
Recommender Systems: State Of The Art And Trends," Recommender
systems handbook, F. Ricci, L. Rokach, B. Shapira and P.B Kantor,
eds., US: Springer, pp. 73-105, 2011.
[6] J.B. Schafer, D. Frankowski, J. Herlocker, and S. Sen, "Collaborative
Filtering Recommender Systems," The adaptive web, P. Brusilovsky,
A. Kobsa, W. Nejdl, eds., Berlin Heidelberg: Springer, pp. 291-324,
[7] R. Burke, "Hybrid Recommender Systems: Survey And
Experiments," User modeling and user-adapted interaction, vol. 12,
no. 4, pp. 331-370, 2002.
[8] R. Burke, "Hybrid Web Recommender Systems, " The adaptive web,
P. Brusilovsky, A. Kobsa, W. Nejdl, Eds., Berlin Heidelberg:
Springer, pp. 377-408, 2007.
[9] L. Marin, D. Isern, and A. Moreno, "Dynamic Adaptation of
Numerival Attributes in a User Profile," Applied intelligence, vol. 39,
no. 2, pp. 421-437, 2013.
[10] G. Adomavicius, and A. Tuzhilin, "Toward The Next Generation Of
Recommender Systems: A Survey Of The State-Of-The-Art And
Possible Extensions," Knowledge and Data Engineering, IEEE
Transactions on, vol. 17, no. 6, pp. 734-749, 2005.
[11] S. Sergis, P. Zervas, and D. G. Sampson, "Towards Learning Object
Recommendations based on Teachers' ICT Competence Profiles,"
Proc. of the 14th IEEE International Conference on Advanced
Learing Technologies (ICALT2014), pp. 534-538, 2014.
[12] Y. Goktas, N. Gedik, and O. Baydas, "Enablers and barriers to the use
of ICT in primary schools in Turkey: A comparative study of 2005
2011," Computers & Education, vol. 68, pp. 211-222, 2013.
[13] M. Solar, J. Sabattin and V. Parada, "A Maturity Model for Assessing
the Use of ICT in School Education," Educational Technology &
Society, vol. 16, no.1, pp. 206218, 2013.
[14] G. Sang, M. Valcke, J. van Braak, and J. Tondeur, "Student teachers’
thinking processes and ICT integration: Predictors of prospective
teaching behaviors with educational technology," Computers &
Education, vol. 54, no. 1, pp. 103-112, 2010.
[15] S. Sergis, P. Zervas, and D.G. Sampson, "ICT Competence-Based
Learning Object Recommendations For Teachers," Proc. of the IADIS
11th International Conference on Cognition and Exloratory Learning
in Digital Age (CELDA2014), pp. 150-157, 2014.
[16] S. Sergis, and D.G. Sampson, "Eliciting Teachers' ICT Competence
Profiles Based On Usage Patterns Within Learning Object
Repositories," Proc. of the 6th IEEE International Conference on
Technology in Education 2014 (T4E2014), pp. 99-105, 2014.
[17] P. Brusilovsky, C. Karagiannidis, and D. Sampson, "Layered
Evaluation of Adaptive Learning Systems," International Journal of
Continuing Engineering Education and Lifelong Learning, Special
issue on Adaptivity in Web and Mobile Learning Services, vol. 14, no.
4/5, pp. 402-421, 2004.
[18] N. Manouselis, C. Karagiannidis, and D. Sampson, "Layered
Evaluation in Recommender Systems: A Retrospective Assessment,"
Computer Science and Information Systems, vol. 10, no.1, pp 109-
127, 2014.
[19] P. Brusilovsky, and E. Millan, "User Models for Adaptive
Hypermedia and Adaptive Educational Systems," The Adaptive Web
Lecture Notes in Computer Science Volume 4321, P. Brusilovsky, A.
Kobsa, and W. Nejdl, eds., Berlin Heidelberg: Springer, pp. 3-53,
[20] F. Ricci, L. Rokach, and B. Shapira, "Introduction To Recommender
Systems Handbook," Recommender Systems Handbook, F. Ricci, L.
Rokach, B. Shapira, and P.B. Kantor, eds., US: Springer, pp. 1-35,
[21] S. Schiaffino, and A. Amandi, "Intelligent User Profiling," Artificial
Intelligence An International Perspective, M. Bramer, ed., Berlin
Heidelberg : Springer, pp. 193-216, 2009.
[22] M. Ferreira-Satler, F.P. Romero, V.H. Menendez-Dominguez, A.
Zapata, and M.E. Prieto, "Fuzzy Ontologies-Based User Profiles
Applied To Enhance E-Learning Activities," Soft Computing, vol. 16,
no. 7, pp. 1129-1141, 2013.
[23] X. Qi, "Research On User Profiling Technology For Personalized
Demands," Proc. of the 2010 International Conference on Intelligent
Computation Technology and Automation, vol. 3, pp. 198-201, 2010.
[24] M.Y.H. Al-Shamri, and K.K. Bharadwaj, "Fuzzy-Genetic Approach
To Recommender Systems Based On A Novel Hybrid User Model,"
Expert systems with applications, vol. 35, no. 3, pp. 1386-1399, 2008.
[25] M. Belk, E. Papatheocharous, P. Germanakos, and G. Samaras,
"Modeling Users On The World Wide Web Based On Cognitive
Factors, Navigation Behavior And Clustering Techniques," Journal of
Systems and Software, vol. 86, no. 12, pp. 2995-3012, 2013.
[26] L. Marin, A. Moreno, and D. Isern, "Automatic Preference Learning
On Numeric And Multi-Valued Categorical Attributes," Knowledge-
Based Systems, vol. 56, pp. 201-215, 2014.
[27] M. de Gemmis, L. Iaquinta, P. Lops, C. Musto, F. Narducci, and G,
Semeraro, "Learning Preference Models In Recommender Systems,"
Preference Learning, F. Fürnkranz, and E. Hüllermeier, eds.,
Springer Berlin Heidelberg, pp. 387-407, 2011.
[28] M. Zanker, and M. Jessenitschnig, "Case-Studies On Exploiting
Explicit Customer Requirements In Recommender Systems," User
Modeling and User-Adapted Interaction, vol. 19, no. 1-2, pp. 133-
166, 2009.
[29] L. Marin, D. Isern, A. Moreno, and A. Valls, "On-Line Dynamic
Adaptation Of Fuzzy Preferences," Information Sciences, vol. 220, pp.
5-21, 2013.
[30] N. Manouselis, H. Drachsler, K. Verbert, and E. Duval, "TEL As A
Recommendation Context," Recommender Systems for Learning, N.
Manouselis, H. Drachsler, K. Verbert, and E. Duval, eds. New York:
Springer, pp. 21-36, 2013.
[31] K. Chrysafiadi, and M. Virvou, "Student Modeling Approaches: A
Literature Review For The Last Decade," Expert Systems with
Applications, vol. 40, no. 11, pp. 4715-4729, 2013.
[32] S. Garcia-Martinez, and A. Hamou-Lhadj, "Educational
Recommender Systems: A Pedagogical-Focused Perspective,"
Multimedia Services in Intelligent Environments, G.A. Tsihrintzis, M.
Virvou and L.C. Jain, eds., International Publishing: Springer, pp.
113-124, 2013.
[33] J. Tondeur, M. Cooper, and C.P. Newhouse, "From ICT Coordination
To ICT Integration: A Longitudinal Case Study," Journal of
Computer Assisted Learning, vol. 26, no. 4, pp. 296306, 2010.
[34] A. Zapata, V.H., Menéndez, M.E. Prieto, and C. Romero, "A
Framework For Recommendation In Learning Object Repositories:
An Example Of Application In Civil Engineering," Advances in
Engineering Software, vol. 56, pp. 1-14, 2013.
[35] N. Manouselis, R. Vuorikari, and F. Van Assche, "Collaborative
Recommendation of ELearning Resources: An Experimental
Investigation," Journal of Computer Assisted Learning, vol. 26, no. 4,
pp. 227-242, 2010.
[36] C. Limongelli, A. Miola, F. Sciarrone, and M. Temperini, "Supporting
Teachers to Retrieve and Select Learning Objects for Personalized
Courses in the Moodle_LS Environment," Proc. of the 12th
International Conference Advanced Learning Technologies
(ICALT2012), pp. 518-520, 2012.
[37] J. Bozo, R. Alarcón, and S. Iribarra, "Recommending Learning
Objects According to a Teachers’ Contex Model". Sustaining TEL:
From Innovation to Learning and Practice, M. Wolpers, P.A.
Kirschner, M. Scheffel, S. Lindstaedt, and V. Dimitrova, eds., Berlin
Heidelberg: Springer, pp. 470-475, 2010.
[38] A. Walker, M.M. Recker, K. Lawless, and D. Wiley, "Collaborative
Information Filtering: A Review And An Educational Application,"
International Journal of Artificial Intelligence in Education, vol. 14,
no. 1, pp. 3-28, 2004.
[39] S. Rafaeli, M. Barak, Y. Dan-Gur, and E. Toch, "QSIAA Web-Based
Environment For Learning, Assessing And Knowledge Sharing In
Communities," Computers & Education, vol. 43, no. 3, pp. 273-289,
[40] B.E. Shelton, J. Duffin, Y. Wang, and J. Ball, "Linking Open Course
Wares And Open Education Resources: Creating An Effective Search
And Recommendation System," Procedia Computer Science, vol. 1,
no. 2, pp. 2865-2870, 2010.
[41] P. Brusilovsky, L.N. Cassel, L.M. Delcambre, E.A. Fox, R. Furuta, D.
D. Garcia, ... and M. Yudelson, "Social Navigation For Educational
Digital Libraries," Procedia Computer Science, vol. 1, no.2, pp. 2889-
2897, 2010.
[42] K. Schoefegger, P. Seitlinger, and T. Ley, "Towards A User Model
For Personalized Recommendations In Work-Integrated Learning: A
Report On An Experimental Study With A Collaborative Tagging
System," Procedia Computer Science, vol. 1, no. 2, pp. 2829-2838,
[43] Y. Wang, and K. Sumiya, "Semantic Ranking Of Lecture Slides
Based On Conceptual Relationship And Presentational Structure,"
Procedia Computer Science, vol. 1, no. 2, pp. 2801-2810, 2010.
[44] V.A. Zaldivar, and D. Burgos, "Meta-Mender: A Meta-Rule Based
Recommendation System For Educational Applications," Procedia
Computer Science, vol. 1, no. 2, pp. 2877-2882, 2010.
[45] H. Avancini, and U. Straccia, "User Recommendation For
Collaborative And Personalised Digital Archives," International
Journal of Web Based Communities, vol. 1, no.2, pp. 163-175, 2005.
[46] S. El Helou, C. Salzmann, and D. Gillet, "The 3A Personalized,
Contextual and Relation-based Recommender System," Journal of
Universal Computer Science, vol. 16, no.16, pp. 2179-219, 2010.
[47] K.H. Tsai, T.K. Chiu, M.C. Lee, and T.I. Wang, "A Learning Objects
Recommendation Model Based On The Preference And Ontological
Approaches," Proc. of the Sixth International Conference on
Advanced Learning Technologies, pp. 36-40, 2006.
[48] G. Sielis, C. Mettouris, A. Tzanavari, and G. Papadopoulos, "Context-
Aware Recommendations Using Topic Maps Technology For The
Enhancement Of The Creativity Process," Educational Recommender
Systems and Technologies: Practices and Challenges, O. Santos and
J. Boticatio, eds. Hershey, PA: IGI-Global, pp. 43-66, 2012.
[49] M. García-Valdez, and B. Parra, "A Hybrid Recommender System
Architecture for Learning Objects," Evolutionary Design of Intelligent
Systems in Modeling, Simulation and Control, O. Castillo and W.
Pedrycz, eds., Berlin Heidelberg: Springer, pp. 205-211, 2009.
[50] R. Schirru, S. Baumann, M. Memmel, and A. Dengel, "Extraction of
Contextualized User Interest Profiles in Social Sharing Platforms,"
Journal of Universal Computer Science, vol. 16, no.16, pp. 2196-
2213, 2010.
[51] H. Drachsler, D. Pecceu, T. Arts, E. Hutten, L. Rutledge, P. Van
Rosmalen, ... and R. Koper, "RemashedRecommendations For
Mash-Up Personal Learning Environments," Learning in the synergy
of multiple disciplines, U. Cress, V. Dimitrova and M. Specht, eds.,
Berlin Heidelberg: Springer, pp. 788-793, 2009.
[52] S. Fazeli, H., Drachsler, F., Brouns, and P. Sloep, "Towards a Social
Trust-aware Recommender for Teachers," Recommender Systems for
Technology Enhanced Learning, N. Manouselis, H. Drachsler, K.
Verbert and O.C. Santos, eds., New York: Springer, pp. 177-194,
[53] J. Dron, R. Mitchell, P. Siviter, and C. Boyne, "CofindAn
Experiment In N-Dimensional Collaborative Filtering," Journal of
Network and Computer Applications, vol. 23, no. 2, pp. 131-142,
[54] C. Cobos, O. Rodriguez, J. Rivera, J. Betancourt, M. Mendoza, E.
Leon, and E. Herrera-Viedma, "A Hybrid System Of Pedagogical
Pattern Recommendations Based On Singular Value Decomposition
And Variable Data Attributes," Information Processing &
Management, vol. 49, no. 3, pp. 607-625, 2013.
[55] P. Zervas, K. Chatzistavrianos, and D. Sampson, "Towards Modeling
Teachers’ ICT Competence Profile in Europe," in R. Huang, Kinshuk
and J.K. Price eds., ICT in Education in Global Context, International
Publishing: Springer, pp. 163-184, 2014.
[56] N. Athanasiadis, S. Sotiriou, P. Zervas, and D.G. Sampson, "The
Open Discovery Space Portal: A Socially-Powered and Open
Federated Infrastructure," Digital Systems for Open Access to Formal
and Informal Learning, D.G. Sampson, D. Ifenthaler, J.M. Spector, P.
Isaias eds., International Publishing: Springer, pp. 11-23, 2014.
[57] T.-C. Hsieh, T.-I. Wang, C.-Y. Su, and M.-C. Lee, "A Fuzzy Logic-
Based Personalized Learning System For Supporting Adaptive
English Learning," Educational Technology & Society, vol. 15, no. 1,
pp. 273288, 2012.
[58] D. Anand, and B.S. Mampilli, "Folksonomy-Based Fuzzy User
Profiling For Improved Recommendations," Expert Systems with
Applications, vol. 41, no. 5, pp. 2424-2436, 2014.
[59] D. Sampson, and D. Fytros, "Competence Models In Technology-
Enhanced Competence-Based Learning," Handbook on Information
Technologies for Education and Training, H.H. Adelsberger, Kinshuk
and J.M. Pawlowski, eds. Berlin Heidelberg: Springer, pp. 155-177,
[60] L. Candillier, F. Meyer, and M. Boullé, "Comparing State-Of-The-
Art Collaborative Filtering Systems," Machine Learning and Data
Mining in Pattern Recognition, P. Perner ed., Berlin Heidelberg:
Springer, pp. 548-562, 2007.
[61] W.V. Leekwijck and E.E. Kerre, "Defuzzification: Criteria And
Classification," Fuzzy sets and systems, vol. 108, no. 2, pp. 159-178,
[62] K. Verbert, H. Drachsler, N. Manouselis, M. Wolpers, R. Vuorikari,
and E. Duval, "Dataset-Driven Research For Improving
Recommender Systems For Learning," Proc. of the 1st International
Conference on Learning Analytics and Knowledge, pp. 44-53, 2011.
[63] G. Shani, and A. Gunawardana, "Evaluating Recommendation
Systems," Recommender systems handbook, F. Ricci, L. Rokach, B.
Shapira and P.B Kantor, eds., US: Springer, pp. 257-297, 2011.
[64] J. Herlocker, J.A. Konstan and J. Riedl, An Empirical Analysis Of
Design Choices In Neighborhood-Based Collaborative Filtering
Algorithms,‖ Information Retrieval, vol. 5, no. 4, pp. 287310, 2002.
[65] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, "Item-Based
Collaborative Filtering Recommendation Algorithms," ACM Proc. of
the 10th international conference on World Wide Web, pp. 285-295,
[66] J.L. Herlocker, J.A. Konstan, L.G. Terveen, and J.T. Riedl,
"Evaluating Collaborative Filtering Recommender Systems," ACM
Transactions on Information Systems, vol. 22, no. 1, pp. 5-53, 2004.
[67] D.G. Sampson, P. Zervas, and S. Sotiriou, "Learning Object
Repositories for Science Education: The Case of the Open Science
Resources Repository," 11th IEEE International Conference
Advanced Learning Technologies (ICALT2011), pp. 358-359, 2011.
[68] G. Adomavicius, and J. Zhang, "Impact Of Data Characteristics On
Recommender Systems Performance," ACM Transactions on
Management Information Systems, vol 3, no. 1, pp. 3:1-3:17, 2012.
[69] J, Bobadilla, and F. Serradilla, "The Effect of Sparsity on
Collaborative Filtering Metrics," Proc. of the Twentieth Australasian
Conference on Australasian Database-Volume 92, pp. 9-18, 2009.
[70] S. Sergis, I. Sholla, P. Zervas, and D.Sampson, "Supporting School
ICT Uptake: The ASK School ICT Competence Management
System," Proc. of the International Conference on Interactive Mobile
Communication Technologies and Learning, pp. 359 - 363, 2014.
[71] S. Sergis, P. Zervas, and D. Sampson, ―A Holistic Approach For
Managing School ICT Competence Profiles Towards Supporting
School ICT Uptake, International Journal of Digital Literacy and
Digital Competence, vol 5, no 4, pp. 33-46, 2014.
[72] S. Sergis, and D.G. Sampson, "From Teachers’ to Schools’ ICT
Competence Profiles," Digital Systems for Open Access to Formal
and Informal Learning, D. Sampson, D. Ifenthaler, J. M. Spector and
P. Isaias, Eds., International Publishing: Springer, pp. 307-327, 2014.
Stylianos Sergis received a B.Sc. in "Informatics
and Telecommunications" (June 2010) from the
Department of Informatics and
Telecommunications of the National and
Kapodistrian University of Athens, Greece and a
M.Sc. in "Informatics in Education" (June 2012)
from the Faculty of Primary Education of the
National and Kapodistrian University of Athens,
Greece. Since July 2013 he is a Ph.D. student at the Department of
Digital Systems, University of Piraeus. His PhD research focuses on
Recommender Systems for Technology enhanced Education. He is
an IEEE Student Member.
Demetrios G Sampson (SM’04), has received a Diploma in Electrical
Engineering from the Democritus University of
Thrace, Greece in 1989 and a Ph.D. in Electronic
Systems Engineering from the University of Essex,
UK in 1995. From October 2015 he is a Professor of
Learning Technologies at the School of Education,
Curtin University, Western Australia. He was a
Professor of Digital Systems for Learning and
Education at the Department of Digital Systems, University of Pirae-
us, Greece (2003-2015) and a Research Fellow at the Information
Technologies Institute (ITI), Centre of Research and Technology
Hellas (CERTH) (since 2000). He is the Founder and Director of the
Advanced Digital Systems and Services for Education and Learning
(ASK) since 1999. His main scientific interests are in the area of
Learning Technologies. He is the co-author of 345 publications in
scientific books, journals and conferences with at least 1620 known
citations (h-index: 21). He has received 8 times Best Paper Award in
International Conferences on Advanced Learning Technologies. He
is Co-Editor-in-Chief of the Educational Technology and Society
Journal (5-year impact factor 1.34). He has served or serves as Mem-
ber of the Steering Committee of the IEEE Transactions on Learning
Technologies, Member of the Advisory Board of the Journal of King
Saud University - Computer and Information Sciences and the Inter-
national Journal of Digital Literacy and Digital Competence, Mem-
ber of the Editorial Board of 23 International/National Journals and
a Guest Co-Editor in 28 Special Issues of International Journals. His
participation in the organization of scientific conferences involves:
General and/or Program Committee Chair in 40 International Con-
ferences, Program Committees Member in 400 Internation-
al/National Scientific Conferences. He has been a Keynote/Invited
Speaker in 65 International/National Conferences. He has been pro-
ject director, principle investigator and/or consultant in 65 R&D
projects with external funding at the range of 14 Million € (1991-
2016). He is a Senior and Golden Core Member of IEEE and was the
elected Chair of the IEEE Computer Society Technical Committee on
Learning Technologies (2008-2011). He is the recipient of the IEEE
Computer Society Distinguished Service Award (July 2012).
... The majority of existing TEL recommendation systems are focused on learners as there is a continuous need to scaffold them with personalized learning experiences [2] [3] [4]. On the other hand, a review of the literature has shown that teachers are not having the support they require to develop their capabilities and their teaching practices, especially in the realm of ICT Information and Communication Technology (ICT) in education [5], preventing them from engaging in more ongoing professional development harnessing the power of ICT. In the same way, existing teachers' recommendation systems that can play a pivotal role to let teachers getting efficiently suitable training resources aligned to their needs and training purposes, are also missing and still mostly confined to research labs [3] [6]. ...
... Sergis el al. [5] tackle the lack of consideration of teachers' ICT competencies when recommending Learning Objects (LOs). In order to address this issue, a mapping of teachers' ICT competencies and LO metadata schema that each dataset employs was first ensured. ...
... Results showed the high level of ranking accuracy of the proposed ICT competence-based RS in delivering RVL recommendations compared to the initial RS without incorporating the ICT supported teaching competency profiles. In another work, Sergis el al. [5] propose a RS that uses teachers' ICT profiles which are automatically generated and dynamically updated based on the teachers ongoing behaviour patterns within LOs repositories. The authors reported in their evaluation experiments that the proposed approach can deliver more accurate LO recommendations to teachers based on their ICT competency profiles. ...
Conference Paper
Full-text available
In recent years, there has been an increasing interest in Educational Recommender Systems aiming at addressing the lack of personalized learning for online learners, providing them with adequate learning experiences meeting their needs and preferences. Similarly, teachers having also different needs, competencies and preferences, can avail recommendation systems in order to develop their capacities and to improve their teaching practices. In this paper, we propose a recommender system that provides suitable materials for teachers, retrieved from collections of open educational resources OER aligned to the UNESCO Information and Communication Technologies Competency Framework for Teachers ICT-CFT, based on teachers' competencies and preferences.
... However, these works do not consider important elements of teachers' profiles such as their ICT competences. In [10], the authors highlighted the fact that teachers ICT competencies are not being enough exploited for providing personalized recommendations to support their daily teaching practices. Moreover, the authors asserted that the elicitation of the individual teachers' ICT competences profile is an essential component that educational recommender systems have to take into consideration in order to provide personalized learning or training for teachers. ...
... The teacher's competency profile is assessed using an online form where each teacher provides his/her profile by selecting whether he/she possesses each of the ICT-supported teaching related to RVLs. Another assessment tool integrated into a recommender system based on elicited ICT competence profile is presented in [10]. The teacher ICT competence profiles are assessed by the teachers themselves using a web form and following the UNESCO ICT-CFT framework. ...
... Results related to the first research question show that the objective of around 46% of the reviewed papers is to identify teachers' ICT competencies. Some studies like [14], [10] and [15] address competencebased recommendation to teacher objective. The recommender systems build teachers' profiles by assessing their ICT competencies before ensuring the recommendation phase. ...
Conference Paper
Full-text available
This paper presents a systematic review intending to summarize the state of the art in the field of teachers’ information and communication technology ICT competency assessment by identifying and analyzing existing tools and approaches for assessing teachers’ ICT competency level. By conducting a systematic mapping study, a total of 11 papers has been selected and reviewed. Our results deliver some important findings focusing on the following key features: the objectives addressed by the assessment tools, the approaches used to assess teachers’ ICT competencies, the ICT competency framework that the tool relies on, and the related technical environment. This review demonstrates that there is a lack in terms of assessing teachers’ competencies required to mainstream effectively ICT in education. Designing and developing an assessment tool intended for identifying and measuring Teachers’ ICT Competencies pursuant an ICT competency framework would be the cornerstone towards recommending the most appropriate materials according to the teacher training needs and requirements.
... A recomendação baseada em conteúdo se vale de outros itens que um determinado usuário já acessou e avaliou, para então recomendar novos itens, a partir da análise de semelhança dos atributos que houver entre eles (Venkatesh & Sathyalakshmi, 2020). Por sua vez, a filtragem colaborativa consiste em prover a recomendação de um item a um usuário a partir de avaliações que usuários com perfil semelhante fizeram acerca daquele item (Sergis & Sampson, 2016). ...
... Também há a recomendação por contexto, considerando, por exemplo, o horário e o dia em que se está utilizando o sistema. Existem outras abordagens que aplicam, dentre outras estratégias, Lógica Fuzzy (Sergis & Sampson, 2016), Redes Neurais (Yao, 1999), Algoritmo Genético (Katarya & Verma, 2018) e Computação Bioinspirada (Hassan & Hamada, 2016). ...
Full-text available
Existem grandes repositórios de conteúdos na Internet que podem ser utilizados como fonte de recursos para o e-learning. Para facilitar a utilização desses conteúdos, eles são organizados como Objetos de Aprendizagem (OAs), que além do conteúdo propriamente dito, possuem vários metadados que os descrevem. Devido ao grande volume de materiais disponíveis, torna-se difícil escolher aqueles OAs que melhor atendam às necessidades de cada indivíduo. Para se sugerir os OAs mais adequados, utilizam-se os Sistemas de Recomendação de OAs, responsáveis por encontrar e recomendar os itens mais úteis aos estudantes. Neste contexto, o presente trabalho propõe uma abordagem de otimização multiobjetivo do problema de recomendação de OAs. Nesse problema, tem-se um conjunto de conceitos a serem ensinados, de acordo com um Design Instrucional, um repositório de OAs, no qual cada OA aborda um ou mais conceitos, além de dependências pedagógicas que obrigam determinados conceitos a serem apresentados previamente para compreensão de outros. No problema proposto, não se buscam soluções que necessariamente cubram todos os conceitos estabelecidos. Na verdade, buscam-se por soluções que cumpram dois objetivos distintos, a saber: i) menor custo, dado pela soma do custo de cada OA presente na solução; ii) maior número de conceitos cobertos, respeitadas as dependências pedagógicas estabelecidas. Para resolução do problema, utilizou-se o NSGA-II, um Algoritmo Genético multiobjetivo implementado no framework MOEA. As instâncias do problema foram geradas artificialmente, simulando repositórios de 1000 OAs, cursos com 200 conceitos, com até 20 dependências. Os resultados obtidos abrangeram conjuntos de diferentes configurações de soluções, com números distintos de conceitos cobertos e custos das soluções, o que traz maiores possibilidades para que os conteúdos entregues estejam de acordo com os objetivos, necessidades e realidade dos alunos.
... In regard to the inputs consumed in the recommendation process, collected data shows that the main parameters are attributes related to users' educational profile. Examples are ICT competences (Sergis & Sampson, 2016); learning objectives (Wan & Niu, 2018;Wu et al., 2015), learning styles (Nafea et al., 2019), learning levels (Tarus et al., 2017) and different academic data (Yanes et al., 2020;Fernández-García et al., 2020). Only 25% (n = 4) of the systems apply item-related information in the recommendation process. ...
... = 5) of the studies. Implicit data collection methods identified are user's data usage tracking, as access, browsing and rating history (Rahman & Abdullah, 2018;Sergis & Sampson, 2016;Wan & Niu, 2018), data extraction from another system (Ismail et al., 2019), users data session monitoring (Rahman & Abdullah, 2018) and data estimation (Nabizadeh et al., 2020). ...
Full-text available
Recommender systems have become one of the main tools for personalized content filtering in the educational domain. Those who support teaching and learning activities, particularly, have gained increasing attention in the past years. This growing interest has motivated the emergence of new approaches and models in the field, in spite of it, there is a gap in literature about the current trends on how recommendations have been produced, how recommenders have been evaluated as well as what are the research limitations and opportunities for advancement in the field. In this regard, this paper reports the main findings of a systematic literature review covering these four dimensions. The study is based on the analysis of a set of primary studies (N = 16 out of 756, published from 2015 to 2020) included according to defined criteria. Results indicate that the hybrid approach has been the leading strategy for recommendation production. Concerning the purpose of the evaluation, the recommenders were evaluated mainly regarding the quality of accuracy and a reduced number of studies were found that investigated their pedagogical effectiveness. This evidence points to a potential research opportunity for the development of multidimensional evaluation frameworks that effectively support the verification of the impact of recommendations on the teaching and learning process. Also, we identify and discuss main limitations to clarify current difficulties that demand attention for future research. Supplementary information: The online version contains supplementary material available at 10.1007/s10639-022-11341-9.
... In order to conclude the essential criteria for constructing an efficient RS, four critical points should be discussed; user personalized experience, RS model, data sources, and items measurability as shown in Fig. 3 [12]. Every teacher gets resources recommendations according to the teacher profile and activities creating a realistic experience for the teacher [13]. From the surveyed papers, 4 ERSs only provide a personalized experience for teachers [7,[14][15][16]. ...
... As our problem has multiple aspects, therefore it is not sufficient to use one RS strategy to address all these aspects. A hybridization of two strategies is a necessity to achieve acceptable results [17] which is followed by 3 ERSs from our list [13,18,19] but none of them considered the geographical dimension of the problem. All 11 ERSs use only one source of educational resources datasets which limits the availability and variety of the resources. ...
... For example, Cechinel et al. [18] used two learner learning behavior data sets (with implicit and explicit ratings) obtained from the MERLOT database to evaluate the effectiveness of learning object recommendations produced by different memory-based collaborative filtering algorithms. Unlike providing common recommendation services for learners, Sergis et al. [19] proposed a collaborative filtering recommendation model for teachers to choose teaching materials. Using tags marked by learning resources in the user learning process, Huang et al. [20] designed a novel cross-user domain collaborative filtering algorithm to realize a learning object recommendation system based on domain migration. ...
Massive Open Online Courses (MOOCs) have received unprecedented attention, in which learners can obtain a large number of learning objects anytime. However, the increasing information overload on MOOCs inhibits the appropriate choice of learning objects by learners, leading to a low efficiency and high dropout rates in the learning process. E-learning recommendation systems have been studied to present learning objects directly to learners, thereby relieving such problem. However, in MOOC platforms, recommendation network structures which can selectively extract implicit feature such as heterogeneous learning preference and knowledge organization of learning objects are still not comprehensively studied. To this end, we propose a learning object recommendation model based on heterogeneous learning behavior and knowledge graph. To generate a unified representation of each entity and relation, we first propose an Attentive Composition based Graph Convolutional Network (ACGCN). By introducing an attention mechanism, information is amplified when updating the representation of the heterogeneous graph, which eliminates the impact of noise and improves the robustness of the model. Then, a Dense Feature based Operation-Aware Network (DFOAN) is utilized to capture implicit and complex learners interactive behaviors, and generate recommendation. Experimental results using two real-world datasets revealed our proposed model has the optimal performance.
... Educational recommender systems aim basically (1) to help learners/teachers to find different types of learning materials through the learning/teaching process based on their individual interests; (2) to recommend learning pathways or educational scenarios; (3) to suggest users with same profiles (i.e., peers); (4) to predict appropriate rates and ranks, etc. (Khribi et al., 2015). The majority of existing TEL RS targets learners and attempts to integrate their personal attributes in the recommendation process (Sergis et al., 2014a). On the other hand, as there is an increased interest by teachers to improve their information and communication technology (ICT) in education competencies and their learning and teaching practices availing learning technologies, some existing researches have been conducted attempting to recommend adequate training content and educational resources for teachers based on their profiles and needs. ...
Conference Paper
Full-text available
Recommender Systems (RS) in e-learning has attracted several researchers aiming basically at scaffolding learners in locating relevant learning resources that meet their learning needs and profiles. Whereas a number of review studies have been carried out in the area of educational recommender systems, there is a limited information in the literature review in the specific domain of recommendation systems for teachers. The objective of this work is to summarize the current research efforts in the field of teacher-oriented recommender systems. By performing this systematic review, 32 papers were selected for further analysis. The obtained results show that educational portals and repositories are widely used as learning environments where recommendations occur. The finding of this review further show that personalized recommendations for teachers and teaching practices improvements are the main issues addressed by RS for teachers. Furthermore, the hybrid approach for recommendation and the evaluation by experiment are the most used, occurring in 43,75% of the selected reviewed papers. We are availing the key findings of this work to propose a teacher recommender system that provides teachers with the most relevant open educational resources (OER) retrieved from collections of resources aligned to the UNESCO ICT Competency Framework for Teachers (CFT).
Full-text available
To scientifically and accurately recommend suitable teachers for university courses and improve teaching quality, designing an effective recommendation algorithm is necessary. Therefore, we construct quantitative models of teacher characteristics, course characteristics, and teaching evaluations under the theories and methods of education and build a sparse experimental data matrix based on the quantified data. On this basis, we propose a teacher recommendation algorithm (PRLFM) based on the improved latent factor model (LFM) and the improved PersonalRank algorithm. Firstly, the improved LFM is used to predict the evaluation scores of those courses that teachers have not taught. The scores which are higher than the specified threshold are used to fill the corresponding missing items in the sparse matrix to reduce the matrix’s sparsity. Then, the bipartite graph model based on the teacher set and course set is constructed according to the filled experimental data matrix. The weight of edges in the bipartite graph is replaced by the teacher and course’s evaluation score multiplied by the course difficulty, which reflects the correlation between course and evaluation score. Next, an improved probability transition matrix based on the bipartite graph is constructed. The access probability in the matrix is replaced by the node’s out degree’s reciprocal multiplied by the edge’s weight. The correlation degree between the course and all teachers is quickly calculated using the matrix algorithm of PersonalRank. Finally, a teacher recommendation model is constructed to realize teachers’ top-N recommendation by combining the correlation degree with teachers’ characteristics. Experiments show that the PRLFM algorithm can effectively improve the accuracy of prediction and top-N recommendation. It solves the problem of lack of scientific basis in recommending suitable teachers for university courses and improving the teaching quality.
Full-text available
RESUMO: A definição de competência docente traz consigo uma carga social, política e cultural que muitas vezes vai de encontro a alguns conceitos educacionais, e por este motivo o termo “competência docente” é ainda muito discutido. A relevância atual dessa discussão pode ser explicitada quando observamos que as políticas públicas brasileiras utilizam o termo como embasamento para as diretrizes de formação de professores do ensino básico, sem uma consulta pública mais abrangente. Este artigo tem, portanto, como objetivo, apresentar o resultado de uma Revisão Sistemática de Literatura, buscando visões, caracterizações e construções deste conceito, aprofundando sua discussão de forma democrática e oferecendo uma reconstrução desse mesmo conceito direcionada e específica à profissão docente. Foram realizadas, em bases de dados nacionais e internacionais, pesquisas de artigos, dissertações e teses, que foram analisados sob dois aspectos; em primeiro lugar, uma análise crítica, buscando os diversos vieses, concordâncias e discordâncias, bem como a apresentação de classificações de competências que organizam e trazem um enfoque prático e direcionado à profissão docente. Em segundo lugar, uma análise de Corpus Textual com o uso do software Iramuteq identificando que palavras tais como Conhecimento, Habilidades, Atitudes, Capacidade, Recurso, Experiência e Ação, dentre outras, no campo das construções analisadas, têm relação com o conceito investigado. Finalmente, é apresentada uma proposta de consolidação e reconstrução do conceito de competência docente.
Due to the large amount of data that is available on the Web, it has become increasingly difficult to locate educational resources that satisfy specific learning needs. Furthermore, the searching process can become increasingly frustrating, time-consuming and little accurate when users do not know how to perform a search. Recommender systems aim at reducing this burden by predicting and recommending users relevant elements of interest. In the educational domain, recommender systems can take advantage of user cognitive states and emotions to generate more personalized recommendations. This work proposes EduRecomSys, an educational recommender system that combines collaborative filtering with emotion detection techniques to suggest users educational resources based on the preferences/interests of other users and the user’s emotion previously detected through face recognition technologies. Likewise, EduRecomSys allows users to retrieve educational resources from multiple sources, including social networks, linked data and learning object repositories. EduRecomSys was evaluated in qualitative and quantitative terms. The qualitative evaluation relied on the participation of three domain experts: a teacher, a pedagogue and a software engineer. The quantitative evaluation was conducted with the help of 20 graduate students. The evaluation results seem encouraging and suggest that EduRecomSys has the potential to provide effective support to the teaching-learning process.
Full-text available
With the growing number of students in the classroom and the switch to online environments, instructors are beginning to integrate collaborative learning approaches in the classroom. However, many times in large collaborative environments and large social networks, students are overwhelmed by the amount of available information; it is often challenging to select the most appropriate sources of information. A promising way to deal with this challenge and enhance social interaction in collaborative learning environments is by introducing recommender systems. The main goal of this article is, through a literature review, explore the differences between general recommender systems and educational recommender systems, and to provide a general overview about the benefits, challenges and limitations of recommender systems in educational settings.
Conference Paper
Full-text available
User profiling is a technique aimed at capturing and exploiting significant characteristics of the users towards the provision of personalized services within adaptive systems such as Recommender Systems (RS). In the context of Technology enhanced Learning (TeL), from a teachers' perspective, the unique ICT competence characteristics of individuals have not been considered when providing Learning Object (LO) recommendations, despite their vital contribution to the level of ICT uptake of teachers. Moreover, there is a lack of mechanisms for automatically eliciting and updating such personal characteristics within Learning Object Repositories (LOR) in order to exploit them for enhanced LO recommendations. Towards tackling this issue, this paper proposes a teacher ICT Competence elicitation mechanism utilizing fuzzy logic for inferring teachers' ICT Competences based on their usage patterns within LOR and presents the results of its preliminary accuracy evaluation. The results indicate that the proposed approach provides high accuracy and can, therefore, construct reliable depictions of the teachers' ICT Competence Profiles
Full-text available
Globally, large-scale national initiatives are being implemented towards promoting the level and quality of Information and Communication Technologies (ICT) use in school education. However, despite these efforts, the current level of ICT uptake from schools remains low. A wide range of factors have been identified as barriers, including lack of teachers' ICT Competences and lack of ICT infrastructure. Typically, these barriers are tackled separately without taking into consideration the ecosystemic nature of schools as organizations. This paper introduces a holistic approach on School ICT Competence based on combining both individual teachers' ICT Competences and schools' eMaturity. Furthermore, it presents the design of a web-based School ICT Competence Management System which aims to capture and monitor schools' overall level of ICT uptake and facilitate effective management of resources and strategic planning towards improvement, by jointly processing these data
Conference Paper
Full-text available
Recommender Systems (RS) have been applied in the Technology enhanced Learning (TeL) field for facilitating, among others, Learning Object (LO) selection and retrieval. Most of the existing approaches, however, aim at accommodating the needs of learners and teacher-oriented RS are still an under-investigated field. Moreover, the systems that focus on teachers, do not explicitly exploit their ICT competence profiles when providing LO recommendations. This is a significant drawback, since it can result in LO recommendations that are beyond the teachers' competence to use. Towards tackling this issue, this paper extends previous work and proposes a teacher ICT Competence-based RS that considers teachers' ICT Competence Profiles when recommending Learning Objects. Moreover, the results of its accuracy evaluation are presented. The results indicate that the proposed approach provides high predictive accuracy and outperforms commonly used, existing RS approaches
In this chapter, we start with a short introduction to the increase that has been witnessed in the past few years in applications of recommender systems at the TEL domain. Then we provide some background on the area of recommender systems, by defining recommender systems and outlining their basic types. A comparison with relevant work in TEL is tried, particularly focusing on adaptive educational hypermedia, learning networks, educational data mining, and learning analytics. A discussion on their similarities and differences is also made, so that relevant work can be better positioned in the TEL research landscape.
This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches. This paper also describes various limitations of current recommendation methods and discusses possible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. These extensions include, among others, an improvement of understanding of users and items, incorporation of the contextual information into the recommendation process, support for multicriteria ratings, and a provision of more flexible and less intrusive types of recommendations.
Recommender systemsare now popular both commercially and in the research community, where many approaches have been suggested for providing recommendations. In many cases a system designer that wishes to employ a recommendater system must choose between a set of candidate approaches. A first step towards selecting an appropriate algorithm is to decide which properties of the application to focus upon when making this choice. Indeed, recommender systems have a variety of properties that may affect user experience, such as accuracy, robustness, scalability, and so forth. In this paper we discuss how to compare recommenders based on a set of properties that are relevant for the application. We focus on comparative studies, where a few algorithms are compared using some evaluation metric, rather than absolute benchmarking of algorithms. We describe experimental settings appropriate for making choices between algorithms. We review three types of experiments, starting with an offline setting, where recommendation approaches are compared without user interaction, then reviewing user studies, where a small group of subjects experiment with the system and report on the experience, and finally describe large scale online experiments, where real user populations interact with the system. In each of these cases we describe types of questions that can be answered, and suggest protocols for experimentation. We also discuss how to draw trustworthy conclusions from the conducted experiments. We then review a large set of properties, and explain how to evaluate systems given relevant properties. We also survey a large set of evaluation metrics in the context of the property that they evaluate.
In all European countries many activities are undertaken to promote the use of ICT in school education. Nevertheless, the full potential of ICT is not being exploited. One of the main reasons for this is the lack of teachers’ familiarization with the use of ICT. Therefore, there are increased efforts worldwide for the design of teachers’ professional development programmes that aim to enhance teachers’ competences for the use of ICT in school education. However, these programmes are mainly designed to address ICT competences that are based on different teachers’ ICT competence frameworks that have been recently developed. Although these competence frameworks aim to tackle the same problem, the diversity in their approaches leads to heterogeneous competence descriptions that cannot be formally described and represented in a unified manner among different European Union Member States. To this end, it is important to model teachers’ ICT competence profile at a European Level in a common and systematic way. This can facilitate the design of teachers’ professional development programmes, which will address universal competences descriptions. Within this context, in this chapter, we focus on (a) modelling teachers’ ICT competence profile in Europe by considering existing teachers’ ICT competence frameworks and selecting the most appropriate and (b) describing, in a machine readable way, the proposed teachers’ ICT competence profile by exploiting existing specifications for competences description.
Adaptive web sites may offer automated recommendations generated through any number of well-studied techniques including collaborative, content-based and knowledge-based recommendation. Each of these techniques has its own strengths and weaknesses. In search of better performance, researchers have combined recommendation techniques to build hybrid recommender systems. This chapter surveys the space of two-part hybrid recommender systems, comparing four different recommendation techniques and seven different hybridization strategies. Implementations of 41 hybrids including some novel combinations are examined and compared. The study finds that cascade and augmented hybrids work well, especially when combining two components of differing strengths.