Content uploaded by Dominik Renzel
Author content
All content in this area was uploaded by Dominik Renzel on Jun 13, 2016
Content may be subject to copyright.
Tracing Self-Regulated Learning in
Responsive Open Learning Environments
Dominik Renzel1, Ralf Klamma1, Milos Kravcik1, and Alexander Nussbaumer2
1Advanced Community Information Systems (ACIS) Group
Chair of Computer Science 5, RWTH Aachen University
Ahornstr.55, 52056 Aachen, Germany
{renzel,klamma,kravcik}@dbis.rwth-aachen.de
2Knowledge Technologies Institute, Graz University of Technology
Inffeldgasse 13, 8010 Graz, Austria
alexander.nussbaumer@tugraz.at
Abstract. Self-Regulated Learning (SRL) and related meta-cognitive
learning competences help to increase learning progress. However, facili-
tating the acquisition of such competences with learning technologies is
challenging. Training requires an individualized approach and the right
balance between the learner’s freedom and guidance. To support SRL,
we applied personalisation and adaptive technologies in the development
of an Open Source toolkit for Responsive Open Learning Environments
(ROLE). In this paper we present a conceptual foundation for the opera-
tionalization of self-regulated learning in personal learning environments
as a cyclic process model. Furthermore, we present results of a long-term
usage data analysis of the ROLE Sandbox, an open and free Web-based
hosting environment for personal learning environments. In particular,
we trace self-regulated learning activities in three years of productive
operation. We conclude our findings with guidelines for self-regulated
learning in personal learning environments.
Keywords: self-regulated learning, personal learning environments, long-
term evaluation, guidelines
1 Introduction
Self-Regulated Learning (SRL) has become a very active research field [17, 2, 15]
with contributions from various disciplines, including pedagogy, psychology and
neuroscience. The cyclic phase structure of the SRL process as well as meta-
cognitive, motivational and behavioural aspects have been in the focus of educa-
tional research. Our main interest is the use of Personal Learning Environments
(PLE) [13, 16] to support SRL processes. PLE support learners in defining their
own learning goals and managing their learning contents, tools, and peers with
a high degree of autonomy. Pedagogical scaffolding can be provided by suitable
recommenders in such community-based adaptive systems [8].
We argue that SRL processes in reality are embedded in many learning set-
tings ranging from formal to informal settings, even within formal educational
2 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
institutions. We also argue that in many learning situations the learner is not
ready for autonomously performed learning processes. Current research under-
estimates the complexity stemming from differences in technology and learning
culture as well from the real learner needs.
We have performed extensive research in realistic learning settings ranging
from learning in higher education organizations in different countries, in work-
place learning and in self-organized social software-supported learning commu-
nities. We found that besides the collection of Web 2.0 tools, learning tools must
be integrated into existing learning platforms like organizational or institutional
Learning Management Systems (LMS). We also found that learner support must
range from total freedom to detailed control of the learning process, contents,
tools, and peers. These dimensions have been underestimated in research so far
and led to coining the term Responsive Open Learning Environments (ROLE)
for a new class of SRL-enabled personal learning environments.
This paper makes the following contributions. First, we present our con-
ceptual self-regulated learning process model with an emphasis on its opera-
tionalization in PLE. Second, based on usage data sampled from over three
years of operation, we analyzed activity in the ROLE Sandbox, a PLE man-
agement platform including SRL support. The goal was to trace self-regulated
learning over a long time in naturalistic informal learning settings as supple-
ment to rather short-termed controlled laboratory studies. Our results suggest
that only a small fraction of ROLE Sandbox users were or have become self-
regulated learners. However, these few self-regulated learners are PLE power
users in terms of serendipity and repeated frequent use, thus suggesting that
the included SRL tools provide appropriate, however yet improvable support for
different SRL phases. Third, resulting from our observations, we derive a set of
design guidelines for better support of self-regulated learning in PLE.
In Section 2 we discuss related work on self-regulated learning and personal
learning environments. In Section 3 we introduce our SRL Process Model and
its operationalization in widget-based responsive open learning environments. In
Section 4 we present the results of our analysis on SRL activity in the ROLE
Sandbox, based on a long-term collection of usage data. In Section 5 we conclude
five key requirements for PLE management platforms to achieve support for self-
regulated learning .
2 Related Work
In the last decades, much research has been conducted in the field of SRL, which
overlaps with different disciplines, including pedagogy, psychology, neuroscience,
and technology enhanced learning. In particular from a psychological point of
view, self-regulated learning is a complex field of research that combines moti-
vational, cognitive and personality theories. Components of SRL are cognition,
meta-cognition,motivation,affects, and volition [10], accompanied by six key
processes essential for self-regulated learning [3]. These are goal setting,self-
monitoring,self-evaluation,task strategies,help seeking, and time management.
Tracing SRL in Responsive Open Learning Environments 3
A cyclic approach to model SRL has been given by Zimmerman [17]. SRL is a
process of meta-cognitive activities consisting of three phases, namely the fore-
thought phase, the performance phase, and the self-reflection phase. According to
this model, learning performance and behavior consist of both cognitive activities
(e.g. attaining new knowledge) and meta-cognitive activities for controlling the
learning process. Aviram et al. [1] extended this model towards a self-regulated
personalised learning (SRPL) approach by adding a self-profile enabling learners
to indicate own preferences.
PLE [13] empower learners to control and arrange their own learning tool
sets, contents, and processes [6]. They support individuals to combine own suit-
able learning environments from a global ecosystem of generic Web 2.0 services,
e.g. blogs, links, wikis, social software and RSS feeds. The goal of widget-based
PLE platforms is to technically empower learners without strong technical back-
grounds to realize such combinations on their own with steep learning curves.
In such platforms, learners select multiple widgets from an existing ecosystem
(e.g. widget store) and combine them on a desktop surface metaphor. Thereby,
widgets are conceived as small-scale Web applications clearly focused on specific
simple tasks and activities. However, most existing widget platforms and Web
2.0 tools do not meet the complex needs of self-regulated learners in PLE with
respect to planning, learning, and reflection activities. We addressed these short-
comings from conceptual, methodological and technical perspectives with the
help of the ROLE SDK, a development environment and management platform
for PLE with built-in conceptual and technical support for self-regulated learn-
ing. Due to the comparably short existence of widget-based PLE, little research
has been conducted to observe and analyze self-regulated learning processes in
naturalistic PLE settings over longer periods.
3 Self-Regulated Learning in Widget-Based PLE
From a user (i.e. learner or teacher) perspective, the central concept of the
ROLE PLE management platform is the space, which also provides the user
interface for the learner. A space serves as container context for widgets, i.e.
small learning tools usually consisting of a Web front-end and a Web-service in
the backend. Examples of such widgets are search tools for learning material or
tools for planning learning activities. The user can select widgets from a ROLE
widget store [4] and add them to a space. The ROLE implementation provides
several core features of the space concept on top of plain PLE management.
First, it provides a dynamic storage of space and user resources where context
and learner-specific information can be stored. Second, it enables both widgets
and users in the same space to interact with each other, thus enabling a seamless
orchestration of learning tools and real-time communication and collaboration
among learners. Third, the platform is augmented with rich logging facilities,
thus enabling learning analytics [14]. Altogether, a ROLE space can be defined
as a bundle of widgets that includes storage for user and context information
4 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
and widget data. Spaces can also be pre-configured and shared with others, thus
allowing rich re-use of learning environments across learning contexts.
From a conceptual point of view, ROLE spaces are the contexts in which we
model self-regulated learning as a cyclic process. Our SRL process model [9] is
based on Zimmerman’s original SRL conception [17] and the SRPL approach [1].
Our contribution is its adaptation for operationalization in PLE in two ways.
First, we introduced an additional phase primarily related to the creation of the
own learning environment (preparation phase). Second, the phases are consti-
tuted not only on the meta-cognitive level, but also on the cognitive level. These
adaptations are necessary to consider active PLE creation and management as
part of the SRL process. Our learner-centric SRL process model thus consists of
four cyclic phases (planning,preparing,learning, and reflecting). Each of these
phases is associated with concrete learning strategies, in turn established by
learning activities on a finer granularity level. Figure 3 depicts our four-phase
process model including the related learning strategies. In order to operationalize
our model, we defined a taxonomy of learning strategies and techniques assigned
to the learning phases. In this way learning phases are described with clearly de-
fined learning activities, thus establishing a connection between the pedagogical
constructs and concrete learning tools. Instead of directly assigning pedagogi-
cal constructs to widgets, widget functionalities are used as mediator construct.
This mediator approach has the advantage that widget creators simply describe
widget functionality without the need to describe pedagogical purpose. By con-
trast, pedagogical experts can make the assignment of learning techniques with
functionalities without knowing which widgets are available.
Fig. 1. The ROLE Self-Regulated Learning Process Model
Tracing SRL in Responsive Open Learning Environments 5
4 Tracing SRL in the ROLE Sandbox
In 2012, we launched the ROLE Sandbox as a Web deployment of the ROLE
SDK with widget and widget-bundle developers being the intended target group.
Equipped with usage data collection, cleaning, and enrichment [14], ROLE Sand-
box supports the observation of arbitrary human interaction with the system’s
APIs. Entries in our data set include origin IP address (who), timestamp (when),
request URL and operation (what), along with context enrichments. In partic-
ular, the data set provides logs of PLE management operations such as create
space,join/leave space,add/remove widget to/from space, including context in-
formation on widget categories from the ROLE Widget Store [12, 4]. Apart from
typical repeating patterns of widget development and testing, we found emergent
patterns of system interaction indicating learning activity wrt. our operational-
ization of SRL in PLE (cf. Section 3), thus motivating deeper analysis.
We collected our data set over more than three years of operation (2012 -
2015). For this work, we selected the first 1.5 year sample (03/2012 - 09/2013)
including 1.72 million API requests from >3900 IP addresses in >600 cities and
>80 countries. Figure 2 provides an overview of spatio-temporal platform usage
distribution. The map on top indicates geospatial usage distribution limited to
IPs having accessed the system >10 times. The bottom timeline chart shows
temporal usage distribution in terms of request frequency (blue/red graphs) and
data transfer (yellow/green graphs).
Dots on the map aggregate amounts of distinct IP addresses resolved to the
same geo-coordinates, encoded with different dot symbols and colors. High num-
bers of IP addresses with same geo-location hint to usage in larger institutions,
Fig. 2. Spatio-temporal distribution of ROLE Sandbox use from 2012/03 - 2013/09
6 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
while low numbers indicate private use by individuals. Thus, we find that the sys-
tem was used by both individuals and institutions with varying size and varying
intensity. Usage mainly concentrates on European countries, but also includes
large institutions in both China and the US.
From our data sample we find, that users created 974 spaces in total, where
682 (70%) can be considered active spaces, i.e. spaces loaded frequently. About
1324 (33.5%) different users interacted with the ROLE Sandbox in different ways.
178 (4.5%) users created new spaces, 251 (6.4%) joined spaces, 320 (8.1%) added
widgets to spaces. 1231 (31.2%) users loaded spaces designed by others. These
statistics confirm the usual observation in social software systems, that only small
fractions of users become active in terms of designing learning environments
on their own, while most other users only benefit from learning environments
previously created by others. According to our SRL process model extension,
self-regulated learning includes the design of personal learning environments as
well as refinements after repeated self-reflection and evaluation. Thus, we can
assume that only a small amount of ROLE Sandbox users are self-regulated
learners in that sense.
Furthermore, we analyzed the widgets used in the ROLE Sandbox with re-
gard to SRL. In total, users employed 634 distinct widgets in personal or collab-
orative learning environments. In order to support self-regulated learning, the
ROLE consortium designed and implemented a set of 15 SRL widgets for dif-
ferent purposes such as recommendation, self-assessment, self-monitoring, self-
evaluation, and SRL-style work. In our data we found the use of 40 (6.3%)
SRL-related widgets, most of them being further customized deployments of the
Fig. 3. SRL-related and non-SRL widget categories
Tracing SRL in Responsive Open Learning Environments 7
15 original SRL widgets. Furthermore, another 40 (6.3%) widgets were assigned
to one or more of the categories Search & Get Recommendation,Plan & Or-
ganize,Communicate & Collaborate,Create & Modify,Train & Test,Explore
& View Content and Reflect & Evaluate, which relate to typical functionality
groups assigned to the four phases of our SRL process model. Figure 3 provides
an overview of SRL-related (red dots) vs. non-SRL-related widgets (grey dots)
assigned to the categories (blue dots). Widget node size encodes how often the
particular widget was added to a space. Category node size encodes the number
of widgets assigned to the particular category. Most widgets are not assigned to
any category, including the most influential SRL widgets. Most of the assigned
widgets are only associated with exactly one category, thus indicating their clear
purpose for one of the SRL learning phases. We also find that most of the widgets
assigned to categories were used more frequently than those not assigned to any
category. This difference is explained by category-based widget recommendations
issued to space designers. We conclude that category metadata is helpful for a
better overview of the available widget ecosystem and ultimately for improved
guidance and tool selection. The majority of unassigned widgets should thus be
assigned post-hoc to make them accessible for targeted recommendations.
In order to examine the relation widget added by user we constructed a bi-
partite graph of users and widgets. We thereby distinguished SRL widgets vs.
non-SRL widgets and SRL widget adders (i.e. users who added at least one
SRL widget to any space) vs. non-SRL widget adders. The resulting graph (not
shown due to space restrictions) exhibits 22 connected components with one
giant component. Most of the widgets are used by multiple users, and most users
added more than one widget. Users in the smaller components mostly represent
widget developers working on widget prototypes never used by others. Notably,
all SRL widgets and SRL widget adders were part of the giant component. 63
(1.5%) users added SRL widgets to spaces, 29 (0.7%) more than one. 123 (3.1%)
users actually used SRL widgets in space contexts, 34 (0.8%) more than one. In
general, we find a correlation between the number of widget add operations and
the number of distinct widgets added by a user. This correlation is significantly
stronger for SRL-widget adders. At the same time, we find that SRL widget
adders are significantly more active in designing own PLE.
We furthermore analyzed our data from the space perspective with the goal
to investigate significant differences between SRL spaces, i.e. spaces in which at
least one SRL widget was added or loaded, and non-SRL spaces. In total, we
found 138 (20.1%) SRL spaces. In a first step we aggregated widget add frequen-
cies, distinct widget categories, IP addresses and widgets per space. Regarding
widget add frequency per space, we clearly observed distribution differences for
SRL and non-SRL spaces. Non-SRL spaces span a wider range of widget add
frequencies. However, a majority of spaces exhibits very few widget add opera-
tions. In contrast, SRL spaces span a closer range, but tend to better distribute
widget add frequencies in this range. Regarding the number of distinct categories
per space, we again found significant differences. SRL spaces tend to cover more
distinct categories to higher extents and less often exhibit widget constellations
8 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
not assigned to any category. Since categories more or less directly map to the
different phases of our SRL model, we can conclude, that SRL spaces tend to
cover more learning phases than non-SRL spaces. Again, categories seem to fulfill
their role as guidance support.
Regarding the number of distinct IP addresses per space, we find that SRL
spaces involve less learner collaboration than non-SRL spaces. The number of
SRL spaces with only one learner involved is significantly higher than for non-
SRL spaces. The maximum numbers of distinct IP addresses per space also
significantly differ with 25 for non-SRL spaces vs. 13 for SRL spaces. Regarding
the number of distinct widgets, we find that non-SRL spaces concentrate on lower
numbers, while SRL spaces exhibit a larger range with a better distribution.
Thus, we can conclude that more exploration of potentially valuable widgets for
learning happens in SRL spaces than in non-SRL spaces, again attributable to
the SRL recommendation tools we developed.
Finally, we examined widget add operations from the category perspective
to receive an overview which categories showed to be most influential in SRL
spaces and in spaces in general. Again, we found significant differences, as de-
picted in Figure 4. First, a significantly lower percentage of widgets added to
spaces belonged to no specific category for SRL spaces (58.8%) in comparison
to non-SRL spaces (64.8%). Furthermore, we find differences regarding the im-
portance of categories for SRL and non-SRL spaces. While non-SRL spaces put
the strongest focus on the category Collaborate & Communicate, SRL spaces are
stronger in support for SRL-relevant categories such as Plan & Organize (13.0%
vs. 8.7%) and Reflect & Evaluate (4.7% vs. 2.6%). In other categories, differences
between SRL and non-SRL spaces were insignificant.
In total, we found significant differences in terms of PLE management be-
haviour between SRL-related entities (users and spaces). Category metadata as
part of our conceptual SRL process model was helpful for guidance and recom-
mendation. It should be noted that our analysis cannot prove that SRL took
place after all. The data merely captures PLE management operations, and not
concrete learning activities. However, we consider PLE management operations
as integral part of SRL, in particular in planning and exploration phases. We
thus see our analysis as relevant evidence that SRL happened in the ROLE
Sandbox.
Fig. 4. Categories of widgets added to SRL spaces vs. all spaces
Tracing SRL in Responsive Open Learning Environments 9
5 Conclusions
From our observations, we conclude five key aspects of self-regulated learning in
PLE (cf. [7]): personalisation,guidance & freedom,meta-cognition & awareness,
motivation and collaboration & good practice sharing.
Learners should be empowered to personalize and adapt their PLE to own
needs and preferred learning techniques, content, tools, services, peers, and
communities. The widget space concept provides a context for the creation
of own learning environments by adding widgets relevant to respective learn-
ing goals. However, personalisation requires an understanding of how to create
pedagogically-enabled widget spaces.
The degree of guidance and freedom is important regarding optimal learner
support. While proficient learners prefer freedom and unobtrusiveness, novice
learners need guidance. Therefore, different levels and types of guidance are
needed for learners with different SRL skills. Guidance should never be prescrip-
tive, but leave the decision to the learners. Initial guidance helps in assembling
or finding widget bundles to build initial PLE. In-learning-process guidance in-
cludes support for performing learning activities, using specific widgets, finding
content, and developing awareness of their own learning process. In ROLE, such
guidance is realized with widgets for personal (i.e. teachers, peers) or algorithmic
recommendations, based on usage data and category metadata.
Meta-cognition is a kind of self-monitoring, self-observation and self-regulation
related to cognitive and information processing. It is stimulated by engaging
learners with the key processes of the self-regulated learning process model.
To support meta-cognitive processes, we recommend widgets or widget bundles
designed to perform meta-cognitive activities (e.g. reflection or progress visu-
alization) and to give learners feedback about their own learning process, thus
stimulating awareness. Such feedback requires support for learning analytics.
For stimulation of intrinsic motivation, PLE must enable learners to develop
autonomy, competence, and relatedness. Motivation is a consequence of the right
balance between guidance and freedom, thus being an implicit feature of our
framework. However, initial extrinsic motivation facilitated by teachers or peers
is advisable in any case.
Collaborative learning comprises extra activities generated by interaction
among peers [5]. These collaborative activities trigger additional cognitive mech-
anisms and appear more frequently in collaborative learning situations than in
individual learning [11]. ROLE spaces include a range of ready-to-use collabora-
tion and communication features, including facilities to share and re-use personal
learning environments for arbitrary learner constellations and learning contexts.
ROLE Sandbox will continue to operate and receive improvements in an
Open Source manner to even better enable self-regulated learning and analytics
thereof in naturalistic informal settings over even longer evaluation periods.
Acknowledgements. This research was supported by the European Com-
mission in the 7th Framework Programme project Learning Layers, grant no.
318209.
10 D. Renzel, R. Klamma, M. Kravcik, A. Nussbaumer
References
1. Aviram, A., Ronen, Y., Somekh, S., Winer, A., Sarid, A.: Self-Regulated Person-
alized Learning (SRPL): Developing iClass’s Pedagogical Model. elearning Papers
9(9), 1–17 (2008)
2. Cassidy, S.: Self-regulated learning in higher education: identifying key component
processes. Studies in Higher Education 36(8), 989–1000 (2011)
3. Dabbagh, N., Kitsantas, A.: Supporting Self-Regulation in Student-Centered Web-
Based Learning Environments. International Journal on E-Learning 3(1), 40–47
(2004)
4. Dahrendorf, D., Dikke, D., Faltin, N.: Sharing Personal Learning Environments
for Widget Based Systems using a Widget Marketplace. In: Pedro, L., Santos, C.,
Almeida, S. (eds.) The PLE Conference 2012, Proceedings. pp. 17:1–17:7 (2012)
5. Dillenbourg, P.: Introduction: What do you mean by ’collaborative learning’? In:
Dillenbourg, P. (ed.) Collaborative-learning: Cognitive and Computational Ap-
proaches, pp. 1–15. Elsevier, Oxford (1999)
6. Downes, S.: Learning networks in practice. Emerging Technologies for Learning 2,
19–27 (2007)
7. Efklides, A.: The role of meta-cognitive experiences in the learning process. Psi-
cothema 21(1), 76–82 (2009)
8. Farzan, R., Brusilovsky, P.: Social Navigation Support in a Course Recommenda-
tion System. In: Wade, V.P., Ashman, H., Smyth, B. (eds.) Adaptive Hypermedia
and Adaptive Web-Based Systems, LNCS, vol. 4018, pp. 91–100. Springer, Berlin,
Heidelberg (2006)
9. Fruhmann, K., Nussbaumer, A., Albert, D.: A Psycho-Pedagogical Framework for
Self-Regulated Learning in a Responsive Open Learning Environment. In: Ham-
bach, S., Martens, A., Tavangarian, D., Urban, B. (eds.) 3rd International Confer-
ence eLearning Baltics Science. Fraunhofer Verlag, Rostock, Germany (2010)
10. Kitsantas, A.: Test Preparation and Performance: A Self-Regulatory Analysis. The
Journal of Experimental Education 70(2), 101–113 (2002)
11. McConnell, D.: Implementing Computer Supported Cooperative Learning. Kogan
Page Limited, London, UK, 2nd edn. (2000)
12. Nussbaumer, A., Berthold, M., Dahrendorf, D., Schmitz, H.C., Kravcik, M., Albert,
D.: A Mashup Recommender for Creating Personal Learning Environments. In:
Popescu, E., Li, Q., Klamma, R., Leung, H., Specht, M. (eds.) Advances in Web-
based Learning. LNCS, vol. 7558, pp. 79–88. Springer, Berlin Heidelberg (2012)
13. Olivier, B., Liber, O.: Lifelong Learning: The Need for Portable Personal Learning
Environments and Supporting Interoperability Standards. Tech. rep., The JISC
Centre for Educational Technology Interoperability Standards, Bolton (2001)
14. Renzel, D., Klamma, R.: From Micro to Macro: Analyzing Activity in the ROLE
Sandbox. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) The Third
ACM International Conference on Learning Analytics. pp. 250–254. ACM (2013)
15. Sabourin, J., Mott, B., Lester, J.: Utilizing Dynamic Bayes Nets to Improve Early
Prediction Models of Self-Regulated Learning. In: Carberry, S., Weibelzahl, S.,
Micarelli, A., Semeraro, G. (eds.) User Modeling, Adaptation, and Personalization,
LNCS, vol. 7899, pp. 228–241. Springer, Berlin, Heidelberg (2013)
16. van Harmelen, M.: Personal Learning Environments. In: Kinshuk, Koper, R., Kom-
mers, P., Kirschner, P.A., Sampson, D.G., Didderen, W. (eds.) 6th IEEE Interna-
tional Conference on Advanced Learning Technologies. pp. 815–816. IEEE (2006)
17. Zimmerman, B.J.: Becoming a Self-Regulated Learner: An Overview. Theory Into
Practice 41(2), 64–70 (2002)
