Conference PaperPDF Available

Tracing Self-Regulated Learning in Responsive Open Learning Environments


Abstract and Figures

Self-Regulated Learning (SRL) and related meta-cognitive learning competences help to increase learning progress. However, facilitating the acquisition of such competences with learning technologies is challenging. Training requires an individualized approach and the right balance between the learner’s freedom and guidance. To support SRL, we applied personalisation and adaptive technologies in the development of an Open Source toolkit for Responsive Open Learning Environments (ROLE). In this paper we present a conceptual foundation for the operationalization of self-regulated learning in personal learning environments as a cyclic process model. Furthermore, we present results of a long-term usage data analysis of the ROLE Sandbox, an open and free Web-based hosting environment for personal learning environments. In particular, we trace self-regulated learning activities in three years of productive operation. We conclude our findings with guidelines for self-regulated learning in personal learning environments.
Content may be subject to copyright.
Tracing Self-Regulated Learning in
Responsive Open Learning Environments
Dominik Renzel1, Ralf Klamma1, Milos Kravcik1, and Alexander Nussbaumer2
1Advanced Community Information Systems (ACIS) Group
Chair of Computer Science 5, RWTH Aachen University
Ahornstr.55, 52056 Aachen, Germany
2Knowledge Technologies Institute, Graz University of Technology
Inffeldgasse 13, 8010 Graz, Austria
Abstract. Self-Regulated Learning (SRL) and related meta-cognitive
learning competences help to increase learning progress. However, facili-
tating the acquisition of such competences with learning technologies is
challenging. Training requires an individualized approach and the right
balance between the learner’s freedom and guidance. To support SRL,
we applied personalisation and adaptive technologies in the development
of an Open Source toolkit for Responsive Open Learning Environments
(ROLE). In this paper we present a conceptual foundation for the opera-
tionalization of self-regulated learning in personal learning environments
as a cyclic process model. Furthermore, we present results of a long-term
usage data analysis of the ROLE Sandbox, an open and free Web-based
hosting environment for personal learning environments. In particular,
we trace self-regulated learning activities in three years of productive
operation. We conclude our findings with guidelines for self-regulated
learning in personal learning environments.
Keywords: self-regulated learning, personal learning environments, long-
term evaluation, guidelines
1 Introduction
Self-Regulated Learning (SRL) has become a very active research field [17, 2, 15]
with contributions from various disciplines, including pedagogy, psychology and
neuroscience. The cyclic phase structure of the SRL process as well as meta-
cognitive, motivational and behavioural aspects have been in the focus of educa-
tional research. Our main interest is the use of Personal Learning Environments
(PLE) [13, 16] to support SRL processes. PLE support learners in defining their
own learning goals and managing their learning contents, tools, and peers with
a high degree of autonomy. Pedagogical scaffolding can be provided by suitable
recommenders in such community-based adaptive systems [8].
We argue that SRL processes in reality are embedded in many learning set-
tings ranging from formal to informal settings, even within formal educational
2 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
institutions. We also argue that in many learning situations the learner is not
ready for autonomously performed learning processes. Current research under-
estimates the complexity stemming from differences in technology and learning
culture as well from the real learner needs.
We have performed extensive research in realistic learning settings ranging
from learning in higher education organizations in different countries, in work-
place learning and in self-organized social software-supported learning commu-
nities. We found that besides the collection of Web 2.0 tools, learning tools must
be integrated into existing learning platforms like organizational or institutional
Learning Management Systems (LMS). We also found that learner support must
range from total freedom to detailed control of the learning process, contents,
tools, and peers. These dimensions have been underestimated in research so far
and led to coining the term Responsive Open Learning Environments (ROLE)
for a new class of SRL-enabled personal learning environments.
This paper makes the following contributions. First, we present our con-
ceptual self-regulated learning process model with an emphasis on its opera-
tionalization in PLE. Second, based on usage data sampled from over three
years of operation, we analyzed activity in the ROLE Sandbox, a PLE man-
agement platform including SRL support. The goal was to trace self-regulated
learning over a long time in naturalistic informal learning settings as supple-
ment to rather short-termed controlled laboratory studies. Our results suggest
that only a small fraction of ROLE Sandbox users were or have become self-
regulated learners. However, these few self-regulated learners are PLE power
users in terms of serendipity and repeated frequent use, thus suggesting that
the included SRL tools provide appropriate, however yet improvable support for
different SRL phases. Third, resulting from our observations, we derive a set of
design guidelines for better support of self-regulated learning in PLE.
In Section 2 we discuss related work on self-regulated learning and personal
learning environments. In Section 3 we introduce our SRL Process Model and
its operationalization in widget-based responsive open learning environments. In
Section 4 we present the results of our analysis on SRL activity in the ROLE
Sandbox, based on a long-term collection of usage data. In Section 5 we conclude
five key requirements for PLE management platforms to achieve support for self-
regulated learning .
2 Related Work
In the last decades, much research has been conducted in the field of SRL, which
overlaps with different disciplines, including pedagogy, psychology, neuroscience,
and technology enhanced learning. In particular from a psychological point of
view, self-regulated learning is a complex field of research that combines moti-
vational, cognitive and personality theories. Components of SRL are cognition,
meta-cognition,motivation,affects, and volition [10], accompanied by six key
processes essential for self-regulated learning [3]. These are goal setting,self-
monitoring,self-evaluation,task strategies,help seeking, and time management.
Tracing SRL in Responsive Open Learning Environments 3
A cyclic approach to model SRL has been given by Zimmerman [17]. SRL is a
process of meta-cognitive activities consisting of three phases, namely the fore-
thought phase, the performance phase, and the self-reflection phase. According to
this model, learning performance and behavior consist of both cognitive activities
(e.g. attaining new knowledge) and meta-cognitive activities for controlling the
learning process. Aviram et al. [1] extended this model towards a self-regulated
personalised learning (SRPL) approach by adding a self-profile enabling learners
to indicate own preferences.
PLE [13] empower learners to control and arrange their own learning tool
sets, contents, and processes [6]. They support individuals to combine own suit-
able learning environments from a global ecosystem of generic Web 2.0 services,
e.g. blogs, links, wikis, social software and RSS feeds. The goal of widget-based
PLE platforms is to technically empower learners without strong technical back-
grounds to realize such combinations on their own with steep learning curves.
In such platforms, learners select multiple widgets from an existing ecosystem
(e.g. widget store) and combine them on a desktop surface metaphor. Thereby,
widgets are conceived as small-scale Web applications clearly focused on specific
simple tasks and activities. However, most existing widget platforms and Web
2.0 tools do not meet the complex needs of self-regulated learners in PLE with
respect to planning, learning, and reflection activities. We addressed these short-
comings from conceptual, methodological and technical perspectives with the
help of the ROLE SDK, a development environment and management platform
for PLE with built-in conceptual and technical support for self-regulated learn-
ing. Due to the comparably short existence of widget-based PLE, little research
has been conducted to observe and analyze self-regulated learning processes in
naturalistic PLE settings over longer periods.
3 Self-Regulated Learning in Widget-Based PLE
From a user (i.e. learner or teacher) perspective, the central concept of the
ROLE PLE management platform is the space, which also provides the user
interface for the learner. A space serves as container context for widgets, i.e.
small learning tools usually consisting of a Web front-end and a Web-service in
the backend. Examples of such widgets are search tools for learning material or
tools for planning learning activities. The user can select widgets from a ROLE
widget store [4] and add them to a space. The ROLE implementation provides
several core features of the space concept on top of plain PLE management.
First, it provides a dynamic storage of space and user resources where context
and learner-specific information can be stored. Second, it enables both widgets
and users in the same space to interact with each other, thus enabling a seamless
orchestration of learning tools and real-time communication and collaboration
among learners. Third, the platform is augmented with rich logging facilities,
thus enabling learning analytics [14]. Altogether, a ROLE space can be defined
as a bundle of widgets that includes storage for user and context information
4 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
and widget data. Spaces can also be pre-configured and shared with others, thus
allowing rich re-use of learning environments across learning contexts.
From a conceptual point of view, ROLE spaces are the contexts in which we
model self-regulated learning as a cyclic process. Our SRL process model [9] is
based on Zimmerman’s original SRL conception [17] and the SRPL approach [1].
Our contribution is its adaptation for operationalization in PLE in two ways.
First, we introduced an additional phase primarily related to the creation of the
own learning environment (preparation phase). Second, the phases are consti-
tuted not only on the meta-cognitive level, but also on the cognitive level. These
adaptations are necessary to consider active PLE creation and management as
part of the SRL process. Our learner-centric SRL process model thus consists of
four cyclic phases (planning,preparing,learning, and reflecting). Each of these
phases is associated with concrete learning strategies, in turn established by
learning activities on a finer granularity level. Figure 3 depicts our four-phase
process model including the related learning strategies. In order to operationalize
our model, we defined a taxonomy of learning strategies and techniques assigned
to the learning phases. In this way learning phases are described with clearly de-
fined learning activities, thus establishing a connection between the pedagogical
constructs and concrete learning tools. Instead of directly assigning pedagogi-
cal constructs to widgets, widget functionalities are used as mediator construct.
This mediator approach has the advantage that widget creators simply describe
widget functionality without the need to describe pedagogical purpose. By con-
trast, pedagogical experts can make the assignment of learning techniques with
functionalities without knowing which widgets are available.
Fig. 1. The ROLE Self-Regulated Learning Process Model
Tracing SRL in Responsive Open Learning Environments 5
4 Tracing SRL in the ROLE Sandbox
In 2012, we launched the ROLE Sandbox as a Web deployment of the ROLE
SDK with widget and widget-bundle developers being the intended target group.
Equipped with usage data collection, cleaning, and enrichment [14], ROLE Sand-
box supports the observation of arbitrary human interaction with the system’s
APIs. Entries in our data set include origin IP address (who), timestamp (when),
request URL and operation (what), along with context enrichments. In partic-
ular, the data set provides logs of PLE management operations such as create
space,join/leave space,add/remove widget to/from space, including context in-
formation on widget categories from the ROLE Widget Store [12, 4]. Apart from
typical repeating patterns of widget development and testing, we found emergent
patterns of system interaction indicating learning activity wrt. our operational-
ization of SRL in PLE (cf. Section 3), thus motivating deeper analysis.
We collected our data set over more than three years of operation (2012 -
2015). For this work, we selected the first 1.5 year sample (03/2012 - 09/2013)
including 1.72 million API requests from >3900 IP addresses in >600 cities and
>80 countries. Figure 2 provides an overview of spatio-temporal platform usage
distribution. The map on top indicates geospatial usage distribution limited to
IPs having accessed the system >10 times. The bottom timeline chart shows
temporal usage distribution in terms of request frequency (blue/red graphs) and
data transfer (yellow/green graphs).
Dots on the map aggregate amounts of distinct IP addresses resolved to the
same geo-coordinates, encoded with different dot symbols and colors. High num-
bers of IP addresses with same geo-location hint to usage in larger institutions,
Fig. 2. Spatio-temporal distribution of ROLE Sandbox use from 2012/03 - 2013/09
6 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
while low numbers indicate private use by individuals. Thus, we find that the sys-
tem was used by both individuals and institutions with varying size and varying
intensity. Usage mainly concentrates on European countries, but also includes
large institutions in both China and the US.
From our data sample we find, that users created 974 spaces in total, where
682 (70%) can be considered active spaces, i.e. spaces loaded frequently. About
1324 (33.5%) different users interacted with the ROLE Sandbox in different ways.
178 (4.5%) users created new spaces, 251 (6.4%) joined spaces, 320 (8.1%) added
widgets to spaces. 1231 (31.2%) users loaded spaces designed by others. These
statistics confirm the usual observation in social software systems, that only small
fractions of users become active in terms of designing learning environments
on their own, while most other users only benefit from learning environments
previously created by others. According to our SRL process model extension,
self-regulated learning includes the design of personal learning environments as
well as refinements after repeated self-reflection and evaluation. Thus, we can
assume that only a small amount of ROLE Sandbox users are self-regulated
learners in that sense.
Furthermore, we analyzed the widgets used in the ROLE Sandbox with re-
gard to SRL. In total, users employed 634 distinct widgets in personal or collab-
orative learning environments. In order to support self-regulated learning, the
ROLE consortium designed and implemented a set of 15 SRL widgets for dif-
ferent purposes such as recommendation, self-assessment, self-monitoring, self-
evaluation, and SRL-style work. In our data we found the use of 40 (6.3%)
SRL-related widgets, most of them being further customized deployments of the
Fig. 3. SRL-related and non-SRL widget categories
Tracing SRL in Responsive Open Learning Environments 7
15 original SRL widgets. Furthermore, another 40 (6.3%) widgets were assigned
to one or more of the categories Search & Get Recommendation,Plan & Or-
ganize,Communicate & Collaborate,Create & Modify,Train & Test,Explore
& View Content and Reflect & Evaluate, which relate to typical functionality
groups assigned to the four phases of our SRL process model. Figure 3 provides
an overview of SRL-related (red dots) vs. non-SRL-related widgets (grey dots)
assigned to the categories (blue dots). Widget node size encodes how often the
particular widget was added to a space. Category node size encodes the number
of widgets assigned to the particular category. Most widgets are not assigned to
any category, including the most influential SRL widgets. Most of the assigned
widgets are only associated with exactly one category, thus indicating their clear
purpose for one of the SRL learning phases. We also find that most of the widgets
assigned to categories were used more frequently than those not assigned to any
category. This difference is explained by category-based widget recommendations
issued to space designers. We conclude that category metadata is helpful for a
better overview of the available widget ecosystem and ultimately for improved
guidance and tool selection. The majority of unassigned widgets should thus be
assigned post-hoc to make them accessible for targeted recommendations.
In order to examine the relation widget added by user we constructed a bi-
partite graph of users and widgets. We thereby distinguished SRL widgets vs.
non-SRL widgets and SRL widget adders (i.e. users who added at least one
SRL widget to any space) vs. non-SRL widget adders. The resulting graph (not
shown due to space restrictions) exhibits 22 connected components with one
giant component. Most of the widgets are used by multiple users, and most users
added more than one widget. Users in the smaller components mostly represent
widget developers working on widget prototypes never used by others. Notably,
all SRL widgets and SRL widget adders were part of the giant component. 63
(1.5%) users added SRL widgets to spaces, 29 (0.7%) more than one. 123 (3.1%)
users actually used SRL widgets in space contexts, 34 (0.8%) more than one. In
general, we find a correlation between the number of widget add operations and
the number of distinct widgets added by a user. This correlation is significantly
stronger for SRL-widget adders. At the same time, we find that SRL widget
adders are significantly more active in designing own PLE.
We furthermore analyzed our data from the space perspective with the goal
to investigate significant differences between SRL spaces, i.e. spaces in which at
least one SRL widget was added or loaded, and non-SRL spaces. In total, we
found 138 (20.1%) SRL spaces. In a first step we aggregated widget add frequen-
cies, distinct widget categories, IP addresses and widgets per space. Regarding
widget add frequency per space, we clearly observed distribution differences for
SRL and non-SRL spaces. Non-SRL spaces span a wider range of widget add
frequencies. However, a majority of spaces exhibits very few widget add opera-
tions. In contrast, SRL spaces span a closer range, but tend to better distribute
widget add frequencies in this range. Regarding the number of distinct categories
per space, we again found significant differences. SRL spaces tend to cover more
distinct categories to higher extents and less often exhibit widget constellations
8 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
not assigned to any category. Since categories more or less directly map to the
different phases of our SRL model, we can conclude, that SRL spaces tend to
cover more learning phases than non-SRL spaces. Again, categories seem to fulfill
their role as guidance support.
Regarding the number of distinct IP addresses per space, we find that SRL
spaces involve less learner collaboration than non-SRL spaces. The number of
SRL spaces with only one learner involved is significantly higher than for non-
SRL spaces. The maximum numbers of distinct IP addresses per space also
significantly differ with 25 for non-SRL spaces vs. 13 for SRL spaces. Regarding
the number of distinct widgets, we find that non-SRL spaces concentrate on lower
numbers, while SRL spaces exhibit a larger range with a better distribution.
Thus, we can conclude that more exploration of potentially valuable widgets for
learning happens in SRL spaces than in non-SRL spaces, again attributable to
the SRL recommendation tools we developed.
Finally, we examined widget add operations from the category perspective
to receive an overview which categories showed to be most influential in SRL
spaces and in spaces in general. Again, we found significant differences, as de-
picted in Figure 4. First, a significantly lower percentage of widgets added to
spaces belonged to no specific category for SRL spaces (58.8%) in comparison
to non-SRL spaces (64.8%). Furthermore, we find differences regarding the im-
portance of categories for SRL and non-SRL spaces. While non-SRL spaces put
the strongest focus on the category Collaborate & Communicate, SRL spaces are
stronger in support for SRL-relevant categories such as Plan & Organize (13.0%
vs. 8.7%) and Reflect & Evaluate (4.7% vs. 2.6%). In other categories, differences
between SRL and non-SRL spaces were insignificant.
In total, we found significant differences in terms of PLE management be-
haviour between SRL-related entities (users and spaces). Category metadata as
part of our conceptual SRL process model was helpful for guidance and recom-
mendation. It should be noted that our analysis cannot prove that SRL took
place after all. The data merely captures PLE management operations, and not
concrete learning activities. However, we consider PLE management operations
as integral part of SRL, in particular in planning and exploration phases. We
thus see our analysis as relevant evidence that SRL happened in the ROLE
Fig. 4. Categories of widgets added to SRL spaces vs. all spaces
Tracing SRL in Responsive Open Learning Environments 9
5 Conclusions
From our observations, we conclude five key aspects of self-regulated learning in
PLE (cf. [7]): personalisation,guidance & freedom,meta-cognition & awareness,
motivation and collaboration & good practice sharing.
Learners should be empowered to personalize and adapt their PLE to own
needs and preferred learning techniques, content, tools, services, peers, and
communities. The widget space concept provides a context for the creation
of own learning environments by adding widgets relevant to respective learn-
ing goals. However, personalisation requires an understanding of how to create
pedagogically-enabled widget spaces.
The degree of guidance and freedom is important regarding optimal learner
support. While proficient learners prefer freedom and unobtrusiveness, novice
learners need guidance. Therefore, different levels and types of guidance are
needed for learners with different SRL skills. Guidance should never be prescrip-
tive, but leave the decision to the learners. Initial guidance helps in assembling
or finding widget bundles to build initial PLE. In-learning-process guidance in-
cludes support for performing learning activities, using specific widgets, finding
content, and developing awareness of their own learning process. In ROLE, such
guidance is realized with widgets for personal (i.e. teachers, peers) or algorithmic
recommendations, based on usage data and category metadata.
Meta-cognition is a kind of self-monitoring, self-observation and self-regulation
related to cognitive and information processing. It is stimulated by engaging
learners with the key processes of the self-regulated learning process model.
To support meta-cognitive processes, we recommend widgets or widget bundles
designed to perform meta-cognitive activities (e.g. reflection or progress visu-
alization) and to give learners feedback about their own learning process, thus
stimulating awareness. Such feedback requires support for learning analytics.
For stimulation of intrinsic motivation, PLE must enable learners to develop
autonomy, competence, and relatedness. Motivation is a consequence of the right
balance between guidance and freedom, thus being an implicit feature of our
framework. However, initial extrinsic motivation facilitated by teachers or peers
is advisable in any case.
Collaborative learning comprises extra activities generated by interaction
among peers [5]. These collaborative activities trigger additional cognitive mech-
anisms and appear more frequently in collaborative learning situations than in
individual learning [11]. ROLE spaces include a range of ready-to-use collabora-
tion and communication features, including facilities to share and re-use personal
learning environments for arbitrary learner constellations and learning contexts.
ROLE Sandbox will continue to operate and receive improvements in an
Open Source manner to even better enable self-regulated learning and analytics
thereof in naturalistic informal settings over even longer evaluation periods.
Acknowledgements. This research was supported by the European Com-
mission in the 7th Framework Programme project Learning Layers, grant no.
10 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
1. Aviram, A., Ronen, Y., Somekh, S., Winer, A., Sarid, A.: Self-Regulated Person-
alized Learning (SRPL): Developing iClass’s Pedagogical Model. elearning Papers
9(9), 1–17 (2008)
2. Cassidy, S.: Self-regulated learning in higher education: identifying key component
processes. Studies in Higher Education 36(8), 989–1000 (2011)
3. Dabbagh, N., Kitsantas, A.: Supporting Self-Regulation in Student-Centered Web-
Based Learning Environments. International Journal on E-Learning 3(1), 40–47
4. Dahrendorf, D., Dikke, D., Faltin, N.: Sharing Personal Learning Environments
for Widget Based Systems using a Widget Marketplace. In: Pedro, L., Santos, C.,
Almeida, S. (eds.) The PLE Conference 2012, Proceedings. pp. 17:1–17:7 (2012)
5. Dillenbourg, P.: Introduction: What do you mean by ’collaborative learning’? In:
Dillenbourg, P. (ed.) Collaborative-learning: Cognitive and Computational Ap-
proaches, pp. 1–15. Elsevier, Oxford (1999)
6. Downes, S.: Learning networks in practice. Emerging Technologies for Learning 2,
19–27 (2007)
7. Efklides, A.: The role of meta-cognitive experiences in the learning process. Psi-
cothema 21(1), 76–82 (2009)
8. Farzan, R., Brusilovsky, P.: Social Navigation Support in a Course Recommenda-
tion System. In: Wade, V.P., Ashman, H., Smyth, B. (eds.) Adaptive Hypermedia
and Adaptive Web-Based Systems, LNCS, vol. 4018, pp. 91–100. Springer, Berlin,
Heidelberg (2006)
9. Fruhmann, K., Nussbaumer, A., Albert, D.: A Psycho-Pedagogical Framework for
Self-Regulated Learning in a Responsive Open Learning Environment. In: Ham-
bach, S., Martens, A., Tavangarian, D., Urban, B. (eds.) 3rd International Confer-
ence eLearning Baltics Science. Fraunhofer Verlag, Rostock, Germany (2010)
10. Kitsantas, A.: Test Preparation and Performance: A Self-Regulatory Analysis. The
Journal of Experimental Education 70(2), 101–113 (2002)
11. McConnell, D.: Implementing Computer Supported Cooperative Learning. Kogan
Page Limited, London, UK, 2nd edn. (2000)
12. Nussbaumer, A., Berthold, M., Dahrendorf, D., Schmitz, H.C., Kravcik, M., Albert,
D.: A Mashup Recommender for Creating Personal Learning Environments. In:
Popescu, E., Li, Q., Klamma, R., Leung, H., Specht, M. (eds.) Advances in Web-
based Learning. LNCS, vol. 7558, pp. 79–88. Springer, Berlin Heidelberg (2012)
13. Olivier, B., Liber, O.: Lifelong Learning: The Need for Portable Personal Learning
Environments and Supporting Interoperability Standards. Tech. rep., The JISC
Centre for Educational Technology Interoperability Standards, Bolton (2001)
14. Renzel, D., Klamma, R.: From Micro to Macro: Analyzing Activity in the ROLE
Sandbox. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) The Third
ACM International Conference on Learning Analytics. pp. 250–254. ACM (2013)
15. Sabourin, J., Mott, B., Lester, J.: Utilizing Dynamic Bayes Nets to Improve Early
Prediction Models of Self-Regulated Learning. In: Carberry, S., Weibelzahl, S.,
Micarelli, A., Semeraro, G. (eds.) User Modeling, Adaptation, and Personalization,
LNCS, vol. 7899, pp. 228–241. Springer, Berlin, Heidelberg (2013)
16. van Harmelen, M.: Personal Learning Environments. In: Kinshuk, Koper, R., Kom-
mers, P., Kirschner, P.A., Sampson, D.G., Didderen, W. (eds.) 6th IEEE Interna-
tional Conference on Advanced Learning Technologies. pp. 815–816. IEEE (2006)
17. Zimmerman, B.J.: Becoming a Self-Regulated Learner: An Overview. Theory Into
Practice 41(2), 64–70 (2002)
... SRL can be understood as a learner's ability to self-monitor and adjust their learning strategies or behaviors in response to the current learning context (Littlejohn et al., 2016). Numerous studies have investigated the models, frameworks and fundamental phases of SRL (Kirschenmann et al., 2010;Kopeinik et al., 2014;Renzel et al., 2015;Shih et al., 2010). On this basis, educators have made various attempts to help learners develop SRL skills and apply these strategies to their learning. ...
... Some educators and online learning platform developers assume that learners are able to self-regulate. However, studies have shown that in many online learning contexts, learners are not well prepared for self-regulation (Renzel et al., 2015). According to previous research, many learners report struggling with self-regulation when learning online (Kizilcec et al., 2017). ...
... On the other hand, if no guidance is provided at all, students will experience a lack of support and feel discouraged, as mentioned previously. Thus, it is vital to strike a balance between learner autonomy and guidance in supporting learners' SRL development (Renzel et al., 2015). A recommender system has the potential to achieve this balance by means of developing adaptive recommendations and allowing learners to choose freely. ...
Self-regulated learning (SRL) plays a significant role in promoting academic success in online education. In recent years, attention has focused on using new techniques to promote SRL—one of which is the recommender system. However, there has been little discussion of the actual effects of using recommender systems to facilitate SRL skills among online learners. This paper aims to elucidate the role that recommender systems play in assisting learners to gain self-regulation skills. The main topics addressed in this paper are as follows: (1) SRL strategies that are supported by recommender systems, as well as the techniques used by these recommenders to promote SRL strategies; and (2) evaluations conducted on the use of recommender systems and results. Our analysis of 20 empirical articles indicates that various features of recommender systems were designed to promote SRL strategies in different phases, and students were generally positive about using such systems to help them self-regulate. Five key knowledge gaps related to existing research on SRL recommender systems were identified. The conclusions suggest that future studies could be improved by demonstrating a more comprehensive understanding of the design of recommender systems, as well as by placing more emphasis on the evaluation process.
... In this context, driven by the near real-time Web, research efforts are needed to investigate which Web technologies enable seamless integration of collaborative technologies on 3D Objects for learning scenarios and how rendering of 3D Objects on the Web and their collaborative consumption can foster learning processes for specialized domains such as medicine. In this paper, we present a Personal Learning Environment(PLE) [1] that allows the NRT collaborative visualization of 3D objects on the Web and the instant generation of annotations on these objects. Annotations are created collaboratively and correspond to specific needs of the tutors and learners in terms of what information needs to be captured and retrieved. ...
... We propose a lightweight approach based on standards such as HTML5, x3dom for 3D Objects visualization and a microservice oriented architecture for storing the annotations. Conceptually, our approach is based on the learner-centric self-regulated learning process model, consisting of four cyclic phases (planning, preparing, learning, and reflecting) [1]. ...
... The annotations are performed at the meso level (i.e. at the 3D Object level) in the PLE [5]. The community uses the annotations in order to exploit knowledge (annotations are used as a method of underlining relevant or interesting aspects of an object) and as reflection possibility for the case where a learner explores others' annotations [1]. As can be observed in the figure, we use simple text annotations containing a title, author and a description. ...
Conference Paper
Web-based collaborative learning environments enable groups of learners to negotiate meaning around shared digital artefacts, e.g. by annotating them collaboratively. This particularly applies for complex digital artefacts such as multimedia or 3D objects and is mostly achieved by using metadata description standards, understandable to both user and machines for queries, context detection and retrieving relevant details. However, current approaches lack the ability to rapidly prototype courses by using lightweight Web technologies on the server and the browser side. In this paper, we present a customizable and lightweight approach for designing and performing Web-based collaborative courses using 3D Objects in the medical domain. These artefacts and the annotations are shared using near real-time updates between learners and tutors. In principle, we solve the problem of different annotation standards that can be used in the same environment by providing an API for using simple contextualized annotations. The evaluations and collected user feedback show that our collaborative browser-based approach simplifies access to digital artefacts and enables more collaboration.
... Previous research has developed a range of tools to support online learners in applying SRL skills, such as regulation of behaviour (for a review see PérezÁlvarez, MaldonadoMahauad, & PérezSanagustín, 2018). Tools incorporating time management and effort regulation include ROLE (Nussbaumer et al., 2014), mCALS (Yau & Joy, 2008) and LearnTracker (Tabuenca, Kalz, Drachsler, & Specht, 2015). In ROLE (Nussbaumer et al., 2014), which is a framework for a personal learning environment, learners are supported to plan their learning activities through the selection of learning resources to be used. ...
... Tools incorporating time management and effort regulation include ROLE (Nussbaumer et al., 2014), mCALS (Yau & Joy, 2008) and LearnTracker (Tabuenca, Kalz, Drachsler, & Specht, 2015). In ROLE (Nussbaumer et al., 2014), which is a framework for a personal learning environment, learners are supported to plan their learning activities through the selection of learning resources to be used. mCALS is a mobile context-aware adaptive learning schedule that support learners to organise their time to study and make recommendations of learning activities based on their learning history. ...
One of the main challenges for online learners is knowing how to effectively manage their time. Highly autonomous settings, such as Massive Open Online Courses (MOOCs), put additional pressure on learners in this regard. However, little is known about how learners organize their time in terms of sessions or blocks of time across a MOOC. This study examined session behavioural data of 9272 learners in a MOOC and its relation to their engagement, grade and self-report data measuring aspects of self-regulated learning (SRL). From an exploratory temporal approach using clustering and group comparison tests, we examined how learners distributed sessions in relation to their length and frequency across the course (macro aspect), and which types of activities they prioritised within these sessions (micro aspect). We then investigated if these patterns of sessions were related to learners' level of engagement, achievement and use of self-regulated learning (SRL) skills. We found that successful learners had more frequent and longer sessions across the course, mixed up activities within sessions, and changed the focus of activities within sessions across the course. In addition, session distribution was found to be a meaningful proxy for learners’ use of SRL skills related to time management and effort regulation. That is, learners with higher levels of time management and effort regulation had longer and more sessions across the course. Based on the results, implications for supporting specific session behaviours to promote effective learning in MOOCs are discussed.
... It built on the outcomes of the former ROLE FP7 EU project (, especially the technological platform that facilitates design and development of Personal Learning Environments (PLEs) [36]. The overall aim was to support employees in training activities and to facilitate their personal development. ...
... We thus preliminarily focused our attention to two external MobSOS case studies. In the ROLE Sandbox (, a result from the EU FP7 IP Responsive Open Learning Environments, we demonstrated the application of MobSOS technology for informal community learning analytics purposes, tracing self-regulated learning practice in Personal Learning Environments in over three years of real-life operation [36]. To clarify the role of CIS success awareness for community-oriented design research, we published an earlier case study in which we applied MobSOS in an aphasia community using online chat environments as training tool supplementary to their clinical therapy [12]. ...
Technical Report
Full-text available
The overall objective of WP6 is to co-ordinate and synchronize all technology development tasks across the project, as well as providing a base architecture to support this. In Y3, we have addressed this overall goal in the following ways: In T6.2 (Distributed process and platform for Layers software engineering and integration) we have started to incorporate the lessons learnt in Y1 and Y2 in the emerging industrial DevOps approach by creating and supporting our DevOpsUse approach. The DevOpsUse approach has a strong commitment to participatory design and evaluation activities carried out in communities of practice. We have re-launched several supporting platforms like the Requirements Bazaar which have been re-designed and re-developed according to the methodology. In cooperation with WP4 (SeViAnno) and WP5 (WebOCD) we re-launched or started several services based also on the massive scaling architecture from T6.3. We observed an ongoing uptake of our main instruments, the Layers Developer Task Force (LTDF), the Open Developer Library and the Architecture board. We have supported also the availability of software engineering knowledge beyond the project consortium by starting a series of Webinars on T6.2 related tools and processes. In T6.3 (Development and deployment of Layers networking infrastructure) we have started to roll-out Layers Boxes to our application partners within the project. We re-configured the Layers Adapter for better scalability and worked intensively on the automation of the deployment of Layers Boxes while increasing the flexibility and the configurability using the sky-rocketing deployment tools Docker and Docker Compose. By analyzing different delivery models we draw conclusions for the future delivery of Layers Boxes on a larger scale. We have started to build up the massive scaling architecture consisting of federated Layers Boxes combining the best of two worlds, peer-to-peer and cloud computing. For server-side peer-to-peer computing we developed the las2peer core and services while for client-side peer-to-peer computing we developed the award-winning Yjs library. Both have already found interest outside the project consortium in companies and open source development communities. In T6.4 (Content store and services) we developed a first working prototype of the Layers App and Content Store (LAPPS) currently serving mobile apps. Based on the Layers Box and the LAPPS first business models were developed in cooperation with WP7. In T6.5 (Integration with end-user tools and testbeds) we provided pre-configured services for authentication and authorization of services for the Web and for mobiles based on OpenID Connect (OIDC) for all end-user tools in the consortium. These have been adopted by almost all partners in Y3. Altogether the project has an implemented community learning analytics (WP1, WP5, WP6), design (WP2-WP4) & engineering approach (WP6). For the first time, we have a conceptual integration of learning analytics, engineering of learning environments, requirements engineering and the necessary software development processes to create, deploy and utilize large-scale informal learning environments. We are on the way to finalizing an integrated demo for the two Layers application domains Healthcare and Construction.
... Description A1 [36] A system that aims to improve learner performance through several theory-based functionalities, such as real-time screen-sharing, synchronous demonstration and learner portfolio monitoring. A2 [37], [67]- [69] This framework provides 15 SRL widgets to support learners to search for information, activity planning, goal setting, etc. A3 [38], [70] Learning environment designed to detect, model, trace and foster learner SRL with regard to the human body system. Learners can generate several sub-goals for the session, self-evaluate their knowledge and monitor their learning process. ...
Self-regulated learning (SRL) is a crucial higher-order skill required by learners of the 21st century, who will need to become lifelong learners to adapt to the continually changing environments. Literature provides examples of tools for scaffolding SRL in online environments. In this article, we provide the state-of-the-art concerning tools that support SRL in terms of theoretical models underpinning development, supported SRL processes, tool functionalities, used data and visualizations. We reviewed 42 articles published between 2008 and 2020, including information from 25 tools designed to support SRL. Our findings indicate that: 1) many of the studies do not explicitly specify the SRL theoretical model used to guide the design process of the tool; 2) goal setting, monitoring, and self-evaluation are the most prevalent SRL processes supported through functionalities, such as content navigation, user input forms, collaboration features, and recommendations; 3) the relationship between tool functionalities and SRL processes are rarely described; and 4) few tools assess the impact on learners’ SRL process and learning performance. Finally, we highlight some lessons learned that might contribute to implementing future tools that support learners’ SRL processes.
... Each course is divided first into several modules, which again are divided into multiple learning units. The courses generated by the LMS of our platform are represented as "learning spaces", realized using the "Responsive Open Learning Environment" (ROLE) platform [5]. Fig. 2 shows such a learning space of a module of the course "Social Entrepreneurship". ...
Full-text available
Virtual training centers are hosted solutions for the implementation of training courses in the form of e.g. Webinars. Many existing centers neglect the informal and social dimension of vocational training as well as the legitimate business interests of training providers and companies sending their employees. In this paper, we present the virtual training center platform V3C that blends formal, certified virtual training courses with self-regulated and social learning in synchronous and asynchronous learning phases. We have developed an integrated learning analytics approach to collect, store, analyze and visualize data for different purposes like certification, interventions and gradual improvement of the platform. The results given here demonstrate the ability of the platform to deliver data for key performance indicators like learning outcomes and drop-out rates as well as the interplay between synchronous and asynchronous learning phases on a very large scale. Since the platform implementation is open source, results can be easily transferred and exploited in many contexts.
... At the same time, they should be responsive, providing the right mix of adaptivity and recommendations of available options, in order to facilitate various degrees of guidance and freedom (Nussbaumer et al. 2014). Effective support for SRL must integrate advice in the form of personalized nudges (alerts that can be easily avoided) and reflection facilities in a suitable way . ...
Full-text available
The rapid progress in the development of information and communication technologies opens new opportunities in education, which go hand in hand with new risks that may be difficult to foresee. Our aim here is to focus mainly on the Internet of Things and related technologies, in order to investigate how they can improve this field. We claim a proper analysis and interpretation of the big educational data can enable more precise personalization and adaptation of learning and training experiences, in order to make them more effective, efficient and attractive. Nevertheless, it will require new approaches to implement novel tools and services for more effective knowledge acquisition, deeper learning and skill training, which can take place in authentic settings and stimulate motivation of learners.
Full-text available
The massive and open nature of MOOCs contribute to attracting a great diversity of learners. However, the learners who enroll in these types of courses have trouble achieving their course objectives. One reason for this is that they do not adequately self-regulate their learning. In this context, there are few tools to support these strategies in online learning environment. Also, the lack of metrics to evaluate the impact of the proposed tools makes it difficult to identify the key features of this type of tools. In this paper, we present the process for designing NoteMyProgress, a web application that complements a MOOC platform and supports self-regulated learning strategies. For designing NoteMyProgress we followed the Design Based Research methodology. For the evaluation of the tool, we conducted two case studies using a beta version of NoteMyProgress over three MOOCs offered in Coursera. The findings of these two case studies are presented as a set of lessons learned that inform about: (1) a list of requirements to inform the design of a second version of the tool; (2) a list of requirements that could serve as a reference for other developers to design new tools that support self-regulated learning in MOOCs.
Conference Paper
Recognizing the need for addressing the rather fragmented character of research in this field, we have held a workshop on learning analytics for workplace and professional learning at the Learning Analytics and Knowledge (LAK) Conference. The workshop has taken a broad perspective, encompassing approaches from a number of previous traditions, such as adaptive learning, professional online communities, workplace learning and performance analytics. Being co-located with the LAK conference has provided an ideal venue for addressing common challenges and for benefiting from the strong research on learning analytics in other sectors that LAK has established. Learning Analytics for Workplace and Professional Learning is now on the research agenda of several ongoing EU projects, and therefore a number of follow-up activities are planned for strengthening integration in this emerging field.
Conference Paper
Full-text available
This paper presents an approach and an integrated tool that supports the creation of personal learning environments suitable for self-regulated learning. The rationale behind this approach is an ontology of cognitive and meta-cognitive learning activities that are related to widgets from a Widget Store. Patterns of such learning activities allow for providing the user with appropriate recommendations of widgets for each learning activity. The system architecture follows a Web-based approach and includes the Mashup Recommender widget and its backend service, the ontology available through a Web service, the Widget Store with its interface to retrieve widgets, and the integration into the learning environment framework. The pedagogical approach regarding the usage of this technology is based on self-regulated learning taking into account different levels between guidance and freedom. A quantitative and qualitative evaluation with teachers describes advantages and ideas for improvement.
Conference Paper
Full-text available
Student engagement and motivation during learning activities is tied to better learning behaviors and outcomes and has prompted the development of learner-guided environments. These systems attempt to personalize learning by allowing students to select their own tasks and activities. However, recent evi-dence suggests that not all students are equally capable of guiding their own learning. Some students are highly self-regulated learners and are able to select learning goals, identify appropriate tasks and activities to achieve these goals and monitor their progress resulting in improved learning and motivational ben-efits over traditional learning tasks. Students who lack these skills are markedly less successful in self-guided learning environments and require additional scaf-folding to be able to navigate them successfully. Prior work has examined these phenomena within the learner-guided environment, CRYSTAL ISLAND, and iden-tified the need for early prediction of students' self-regulated learning abilities. This work builds upon these findings and presents a dynamic Bayesian ap-proach that significantly improves the classification accuracy of student self-regulated learning skills.
Conference Paper
Full-text available
To empower the learner for true lifelong and personalised learning with a responsive open learning environment (ROLE) is one aim of the ROLE project. A psycho-pedagogical integration model (PPIM) towards supporting learning has been developed by facilitating the concept of personalised self-regulated learning. The first version of the ROLE PPIM is presented in this paper and gives a general view of the components of this model. The central part of the PPIM is the description of the self-regulated learning process and how it can be personalised by learners using adaptive guidance of ROLE.
Full-text available
The concept of self-regulated learning is becoming increasingly relevant in the study of learning and academic achievement, especially in higher education, where quite distinctive demands are placed on students. Though several key theoretical perspectives have been advanced for self-regulated learning, there is consensus regarding the central role played by student perceptions of themselves as learners. There are two general aims of this positional article. The first is to emphasise self-regulated learning as a relevant and valuable concept in higher education. The second is to promote the study of those constituent elements considered most likely to develop our understanding beyond a mere description of those processes thought to be involved in self-regulated learning. A case is presented for learning style, academic control beliefs and student self-evaluation as key constructs which contribute to an increased understanding of student self-regulated learning, and which facilitate the application of self-regulated learning in pedagogy by enhancing its tangibility and utility.
Conference Paper
Current learning services are increasingly based on standard Web technologies and concepts. As by-product of service operation, Web logs capture and contextualize user interactions in a generic manner, in high detail, and on a massive scale. At the same time, we face inventions of data standards for capturing and encoding learner interactions tailored to learning analytics purposes. However, such standards are often focused on institutional and management perspectives or biased by their intended use. In this paper, we argue for Web logs as valuable data sources for learning analytics on all levels of Bronfenbrenner's Ecological System Theory and introduce a simple framework for Web log data enrichment, processing and further analysis. Based on an example data set from a management service for widget-based Personal Learning Environments, we illustrate our approach and discuss the applicability of different analysis techniques along with their particular benefits for learners.
Numerous benefits of student-centered web-based learning environments have been documented in the literature; however the effects on student learning are questionable, particularly for low self-regulated learners primarily because these environments require students to exercise a high degree of self-regulation to succeed. Currently few guidelines exist on how college instructors should incorporate self-regulated strategies using web-based pedagogical tools. The scope of this paper is to (a) discuss the significance of self-regulation in student-centered web-based learning environments; (b) demonstrate how instructional designers and educators can provide opportunities for student self-regulation using web-based pedagogical tools; and (c) redefine the role of the instructor to support the development of independent, self-regulated learners through the use of web-based pedagogical tools.
The effect of self-regulatory processes on test preparation and performance was examined. The author used a structured 1-to-1 interview to query 62 college students enrolled in the same psychology class about their self-regulatory processes. The author expected that (a) high test scorers would use more self-regulatory processes to enhance their test preparation and performance than would low test scorers; (b) self-regulation would positively affect test performance; and (c) self-regulatory skill, self-efficacy beliefs, and perceived instrumentality would predict subsequent test performance. All hypotheses were supported by the data. The results of this study are discussed from a sociocognitive perspective.
First published in 1994. Excerpts available on Google Books (see link below). For integral book, go to publisher's website :