Beyond ePortfolios: Creating, Exploiting, and Archiving Activity Traces, Learning Outcomes, and Learning Analytics as Personal Shareable Online Spaces

Full text

Beyond ePortfolios
Creating, Exploiting, and Archiving Activity Traces, Learning Outcomes,
and Learning Analytics as Personal Shareable Online Spaces
D. Gillet, M.J. Rodríguez-Triana, A. Holzer,
A. Vozniuk, J.C. Farah
School of Engineering
Swiss Federal Institute of Technology (EPFL)
Lausanne, Switzerland
denis.gillet@epfl.ch
R. Matsuba
Research Center for Instructional Systems
Kumamoto University
Kumamoto, Japan
matsuba@kumamoto-u.ac.jp
Abstract — The concept of an ePortfolio was introduced in
1999 as an electronic learning record and further developed to
overcome the limitation of learning management systems lacking
persistent storage for learning artifacts. Nowadays, educational
platforms have evolved towards personal learning environments
and social media platforms, enabling the creation, exploitation
and archiving of activity traces, learning outcomes, and learning
analytics thanks to built-in export and sharing features. This
paper presents the design and implementation of archiving and
sharing services in Graasp, a general-purpose learning platform.
These features enable implementing Graasp as an ePortfolio
platform, allowing students to archive learning artifacts as
evidence of competences and activity traces for analytics-driven
self-assessment. Additionally, we discuss user requirements for
designing such general ePortfolio services elicited from the
analysis of functionalities of the Mahara popular ePortfolio
platform and from a participatory design session with expert
users of Graasp.
Keywords—Engineering Education; Learning Environment;
Social Media; ePortfolio; Personal Learning
I. INTRODUCTION
Historically, the storage of learning artifacts on learning
management systems and their access by students was
bounded in time [2, 3], usually by the duration of the class.
After a class was finished, artifacts were deleted or student
access was revoked. To address this issue and support long-
term access to learning artifacts by the students, ePortfolios
were conceived [1]. Nowadays, educational platforms have
evolved towards personal learning environments (PLEs) and
social media platforms [4,10], enabling the creation,
exploitation and archiving of activity traces, learning
outcomes (such as digital content, data, or discussions), and
learning analytics [5] thanks to built-in export and sharing
features. Such features enable students to store their digital
learning artifacts together with the resources provided by the
teacher(s) as archive files, PDF documents, eBooks, or
shareable online spaces that can be considered as rich open
educational resources [6] and evidence of goal achievement
and competence acquisition.
This paper describes the design and implementation of
versatile archiving and sharing services for self-assessment
and competence management dedicated to science and
engineering education, as well as to teacher professional
development. Such services are implemented in Graasp1, an
educational social media platform initially designed as a
solution to create and exploit personal learning environments
and open educational resources, but later extended to match
features required by ePortfolios.
Section II presents the technological context in which the
design of general purpose educational services and platforms
are developed. Section III details the methodological approach
implemented to carry out participatory design with key digital
education stakeholders and elicit user requirements. Section
IV provides a comparative study between a general purpose
educational platform and a widely used ePortfolio system.
Section V details the outcome of a participatory design session
carried out with advanced users of educational platforms to
understand their needs in terms of archiving features and
ePortfolio services. Section VI summarizes the requirements
elicited in the previous sections and Section VII discusses the
implementation of the core required features in Graasp. In
Section VIII, we describe a use case based on the current
version of Graasp exploited to support both learning and
competence management in an instructional design class,
before concluding in Section IX.
II. TECHNOLOGICAL CONTEXT
Graasp is an open access educational social media
platform developed in the framework of three successive
European initiatives dedicated to online personal and inquiry
learning [7,10]. In addition to enabling the creation of personal
learning spaces supporting individual and group work, it
allows the creation and exploitation of inquiry learning spaces
(ILS), as well as their free publication and distribution through
the Go-Lab repository2.
Fig. 1 shows a student view of a sample ILS called My
ILS. The ILS tabs contain activities to be carried out as part of
inquiry learning phases and provide the necessary online
resources, such as documents, embedded web pages, YouTube
1 Graasp: http://graasp.eu
2 Go-Lab repository (Golabz): http://golabz.eu
2017 7th World Engineering Education Forum (WEEF)
43
978-1-5386-1523-2/17/$31.00 ©2017 IEEE

videos, web apps, and virtual or remote labs. The dashboard
tab can integrate personalized learning analytics apps, such as
one displaying a student’s progress (Fig. 2) compared to other
peers in the class [9].
Fig. 1. An inquiry learning space called My ILS with tabs providing
resources to carry out various inquiry learning activities and
integrating a learning analytics dashboard. The selected tab is the
Orientation phase that includes an embedded YouTube video and a
web app enabling the student to add personal documents through
drag-and-drop (cloud icon).
Students do not need to sign up or sign in to exploit an
ILS. Once they have received the ILS’s secret URL to the
student view from their teacher, they can access it by
providing an individual nickname. This schema preserves
students privacy while enabling students to store and identify
their learning outcomes and activity traces.
Fig. 2. The Dashboard tab of the ILS introduced in Fig. 1 integrates a web
app showing how long students have spent in the various inquiry
learning phases and a web app enabling the students to self-assess
their current progress level.
III. METHODOLOGICAL APPROACH
For the elicitation of requirements for an ePortfolio
system, we followed two complementary approaches. First,
we carried out a comparative study between Graasp and
Mahara — a popular tool for ePortfolio management [8]. The
purpose of this study was to understand how the current
capabilities of Graasp compare with Mahara and to identify
directions for improvement within Graasp. Additionally, we
organized a participatory design session with 24 teachers in
July 2017. These teachers were proficient users of Graasp and
the Go-Lab repository. By involving them in the design
process, our goal was to learn how useful they considered
ePortfolio services and archiving features in Graasp.
Furthermore, we aimed to gather feedback on what data
should be included in these ePortfolios and archives, as well as
their preferred export formats.
IV. COMPARATIVE STUDY
In this section, we discuss our study comparing Graasp and
Mahara based on the minimal set of features required by an
ePortfolio system as listed in [11]. These features are the
following:
● digital collection of artifacts and reflections;
● representation of an individual’s learning and
achievements;
● set of items to be shared with others.
We selected Mahara as a representative example of
ePortfolios because of its large and increasing worldwide user
base. This adoption is due to its easy installation, good
operability, and the fact that it is customizable with plug-in
modules. In Table 1, we present a summary of the comparison.
TABLE I. COMPARISON OF FEATURES OFFERED BY THE GRAASP
AND MAHARA PLATFORMS.
Activities
Platforms
Learner
Instructor
Graasp
Mahara
Goal
Management
Profile
Resume, Profile,
Plan,
SmartEvidence
Connection /
Sharing
Summative
Evaluation
Discussions &
Documents
Collection, Page,
SmartEvidence
Reflection†
/ Selection
Documents,
Files
Page, Note
Selection /
Reflection‡
Formative
Evaluation
Documents,
Files
Page, Note
Collection
Documents,
Files, Links,
Apps
File, Journal, Note,
Images and Video,
External Content
† Retrospective reflection for integration of knowledge.
‡ Immediate reflection for formative evaluation of learning.
The two platforms integrate the required set of features
listed above. Nevertheless, there are implementation
differences regarding the collection of artifacts. Mahara
includes various ways to collect learning artifacts and
especially common multimedia documents. On the other hand,
and in addition to collecting multimedia documents, Graasp
enables the aggregation of external web apps and embedded
web pages. The SmartEvidence feature of Mahara is a
competency-visualizing tool in which the competencies
acquired by students through their learning activities are
2017 7th World Engineering Education Forum (WEEF)
44

associated with pieces of evidence. There is currently no such
a feature in Graasp. On the Graasp side, the main advantage is
the flexibility to personalize the platform through apps or
embedded pages that can be easily integrated. These
integrations enable the display of learning achievements
acquired in other platforms. As nowadays students can use a
multitude of learning platforms online, Graasp’s ability to
integrate external resources allows it to become the personal
repository for a broad learning ecosystem, where students can
compile knowledge acquired in both formal and informal
learning settings and environments.
V. PARTICIPATORY DESIGN
We organized a participatory design session with 24 expert
teachers in July 2017. In this session, teachers reflected on
how they envisioned the archiving of ILSs and ePortfolio
services to be integrated into Graasp.
Archiving for teachers. Out of 24 teachers, 21 (87.5%)
considered that keeping an archive was relevant for their own
practice, 1 (4.16%) was not sure, and 2 (8.33%) did not
answer. By decreasing number of votes, the main reasons
motivating teachers to keep an archive of their ILSs were (1)
assessment purposes (as evidence of the student’s work and
their progress), (2) compiling and having access to the
material generated by the teachers, (3) re-usability (including
reviewing and redesigning), (4) sharing with other peers, (5)
self-reflection, and (6) research.
Regarding the content that such archive should include,
teachers considered that it should contain evidence not only
about the content (24, 100%) but also about the activity (23,
95.83%). Table 2 shows the teachers’ preferences in terms of
archiving solutions. It should be noted that several solutions
could be chosen by the teachers at the same time. The main
options were to keep the archives in Graasp (83.33%), in their
own computer (62.5%) or as a PDF (62.5%). Regarding the
certification, most of the teachers (23, 95.83%) were interested
in getting a certification from the platform showing that the
ILSs were produced by them.
Student ePortfolio. Out of 24 teachers, 21 (87.5%) stated
that it would be useful for students to add their ILSs to their
ePortfolios, and 3 (12.5%) were not sure. In order to better
address the student needs, we asked the teachers what kind of
information should be included in an ePortfolio. On this
regard, 21 teachers (87.5%) agreed on the need for combining
evidence about the learning outcomes as well as the learning
process. Besides, other data sources were considered as
relevant in an ePorfolio, e.g., the student self-assessment or
the feedback provided by peers or teachers. Indeed, 22
teachers (91.67%) were willing to certify the authenticity of
the work done by the students (the other 2 were not sure).
In terms of how ILSs could be exported, the preferences
diverge from the ones chosen for teacher archiving. As it is
shown in Table 2, the most voted options were PDF files (15
teachers, 62.5%), in Graasp (14 teachers, 58.33%), and
ePub/eBook (10 teachers, 41.67%). Interestingly, as for
teacher archiving, despite the PDF format appearing among
the main options, it is not aligned by default with the
possibility of including raw evidence about the learning
process. Nevertheless, this data could be aggregated and
analyzed by ad-hoc analytics apps, which can also provide
some guidelines for interpretation and reflection.
TABLE II. TEACHER INTEREST IN THE DIFFERENT ARCHIVING
AND EPORTFOLIO STRATEGIES.
Archiving / ePortfolio Strategies
For Teachers
For Students
Printed Copy (Paper-Based)
4 (16.66%)
5 (20.83%)
PDF File
15 (62.50%)
15 (62.50%)
ePub / eBook File
7 (29.16%)
10 (41.67%)
In Graasp
20 (83.33%)
14 (58.33%)
In User's Computer
15 (62.50%)
4 (16.66%)
In Another Server (e.g., In User’s
School Network)
5 (20.83%)
7 (29.16%)
Google Drive
1 (4.16%)
1 (4.16%)
VI. ELICITED REQUIREMENTS
Based on the comparative study and the participatory
design presented in Sections IV and V, respectively, we
formulate two additional requirements for ePortfolio systems:
● storage of activity traces together with artifacts;
● proper management of user identities.
Activity traces storage. Most of the ePortfolio systems
focus on collecting artifacts representing the final product of a
learner’s work. In addition, it has an important role as a
workplace of learning. We argue that for analytical and
reflection reasons, the activities performed while working on
the artifacts are also important. Having access to the activities
as part of an ePortfolio can enable using learning analytics and
personalization tools tailored to the learning style of a
particular student. For instance, based on the traces stored as
part of an ePortfolio, the system can recommend learning
activities with short duration matching previous successful
experiences of the learner. Moreover, such activity traces can
serve as a data source for teacher dashboards assisting the
teacher in the examination of an ePortfolio for a particular
student.
A challenge arises when persisting traces of interactions
that happened as part of collaborative activities. One example
could be a collaborative report writing (e.g., in a Google Doc
or a wiki page), where storing activity traces of a single user
(e.g., characters written by the user) will provide an
incomplete picture regarding the activities that led to the
creation of the artifact. On the other hand, storing traces of all
users participating in collaborative activities can raise privacy
concerns related to the ownership of the traces. An ePortfolio
system should have a clear policy and inform users how their
traces are managed.
User identity management. Collection and storage of
activity traces are strongly coupled with user identity
management. As an example, when students interact in Graasp
in the student view (such as the one shown in Fig. 1), they use
nicknames to represent their identity. Such nicknames aim to
protect user privacy during the class’ online activities. At the
2017 7th World Engineering Education Forum (WEEF)
45

same time, when exporting artifacts and traces as part of an
ePortfolio, it is necessary for the learner to make a link with a
real identity to be able to present the artifacts as proof of their
work. Such identity linking mechanisms are required to be
implemented by ePortfolio systems, especially given the fact
that currently learning activities often happen across several
learning environments, each having different user identities.
The following additional challenges must be also taken
into account when implementing ePortfolio features:
● Compliance: How can we guarantee compliance
with legal frameworks such as the EU General Data
Protection Regulation3? Can underage students create
accounts on open educational platforms?
● Access: Who (teachers, parents, hosting institutions,
etc.) has temporary or permanent access to the
content created by students in the context of a class?
Who manages access permissions?
● Ownership: How do we store the learning outcomes
of collaborative learning activities? How are rights
transferred when students reach adulthood?
● Interoperability: How do we retrieve and store
online content and apps integrated in ILSs while
complying with their various license schemes?
● Identification: How can we extend the anonymous
exploitation scheme based on contextual nicknames
to support sustainable access for users with
permanent credentials (independent from institutional
identity)?
● Durability: How do we guarantee the durability of
information?
VII. IMPLEMENTATION OF EXPORT AND ARCHIVING
FEATURES
To enable reflection and lifelong competence management
in a digital society where the lifespan of cloud or legacy
platforms is unpredictable, the most important feature to
enable ePortfolio services is to ensure easy export and
archiving in standard formats. This section describes the
envisioned design and seamless integration of such features in
Graasp towards its extension for exploitation also as an
ePortfolio. For teachers, being able to export and archive the
outcomes and traces of activities with their students is a way
to reflect on and improve their teaching practices and
professional achievements. For students, being able to export
and archive the outcomes and traces of their activities is a way
to support self-assessment of their learning practices and
enable competence management.
The current export feature available in Graasp relies on
printing the full content of ILSs as a PNG image or a PDF file
(see in Fig. 1 the small printer icon located next to the name of
the ILS on the top). We are currently implementing two
further export features: Graasp Copy and Zip Archive.
Graasp Copy. This functionality will enable students to
obtain their own personal copy of the ILS in Graasp. This will
require students to provide Graasp credentials so that Graasp
can verify their identity. Fig. 3 shows the Graasp copy of an
3 http://www.eugdpr.org
ILS as it has been created by the teacher (with subspaces
depicted as folders and exploited to collect and aggregate
cloud resources supporting the various phases). Note that in
the teacher space, the Vault subspace contains the learning
outcome of all students. The individual student copies should
only contain personal production and traces. The personal
collection of ILSs stored in Graasp can be considered as an
ePortfolio, since it is persistent and can be freely shared by its
owner(s) for temporary or permanent consultation or
exploitation. It should be emphasized that the copy of the
space should include not only content such as documents and
images, but also, discussions, activity traces and analytics.
These are exported and kept for later use by the students, even
in the case the teacher(s) decide to delete the original space.
Fig. 2 shows an example of a dashboard of the previously-
introduced ILS integrating two learning analytics apps.
Zip Archive. Finally, as it is already the case for registered
Graasp users, it will be possible for students to archive an ILS
in a zip format directly from the student view without the need
to provide credentials. In the zip, spaces become folders and
all content and descriptions are saved as files. Possibly, this
folder can be uploaded to Graasp, where it recreates a fully
working space. It should be noted that the same restrictions
regarding personal traces in Graasp Copy apply here.
Fig. 3. The editable and sharable view of the My ILS inquiry learning space
as created by the teacher and archived by the students.
VIII. USE CASE
As shown above, Graasp is a learning environment that
enables students not only to achieve their goals in formal
and/or informal learning, but also to manage their learning
outcomes and traces. Hence, Kumamoto University has
designed a higher education course for graduate students in
instructional design aiming at supporting the development of
ePortfolio literacy by using a learning platform integrating
ePortfolio services. It will be extended later for undergraduate
students for the development of more general digital literacy.
The scenario of the graduate course and the way the Graasp
platform will be exploited are detailed below.
The course will be organized in 6 modules of one day each.
Students will be requested to reflect on their own learning
activities. In this task, students will collect learning outcomes
and describe their learning experience.
To guide the activity, students will first be asked to
2017 7th World Engineering Education Forum (WEEF)
46

summarize what they have learned, and provide adequate
artifacts or articles constituting evidence of their learning,
justifying the decision and highlighting the relevance of the
evidence provided. This simultaneously provides scaffolding
for reflection and connection. As it is shown in Table 1,
students can include documents, files, links, and apps in
Graasp. We expect links or apps to be particularly useful,
because students can easily aggregate outcomes available as
artifacts on a learning management system or on Internet
cloud services.
The reflection phase is about figuring out own learning
outcomes through self-discovery. During the activity the
students need to carefully consider the significance and the
importance of their own learning experiences using selected
data and notes accumulated in the previous phase. The
reflection assignments will include showing the student
outcomes (as a narrative story) in a context of this learning
experience with evidence, and explaining how and why the
student identified and/or produced artifacts as evidence.
Students will try to explain key learning dimensions, such
as what they learn, how they learn, difficulties, and
insufficient understanding of their learning experiences. Then
they will be asked to consider what knowledge is acquired in
which learning context or through which modality.
The continuing phase of reflection and connection is part of
social learning. Hence, the students will be asked to provide a
story (narrative) about their achievements in the perspective of
collaborative learning. The reflection and connection
assignments will entail publishing the learning achievements
(narrative) in the perspective of collaborative learning (class,
institution, company, etc.), and providing actions and
approaches necessary to achieve the student's future goals.
Learning how to clearly convey their own understanding to
others is an important task for students supported by reflecting
on their own achievements. This also supports the decision-
making process for defining learning goals, strategies and
actions. The Graasp personalized structure with spaces and
subspaces dedicated to selected activities, as well as the
associated aggregated resources, can be seen as scaffolds to
support this process and the acquisition of the associated
competencies.
The course described here will be adapted and offered in
different contexts, including science education at school and
teacher training at university. The latter is about documenting
teaching activities, exchanging opinions, and sharing best
practices with colleagues using ePortfolio services.
IX. CONCLUSIONS
This paper discusses how a general purpose educational
platform, i.e. Graasp, can be exploited and possibly extended
to offer ePortfolio services. We have presented archiving and
sharing services in Graasp with the aim of supporting student
self-assessment and competence management, as well as for
teacher professional development. Offering a general-purpose
educational platform including ePortfolio features enables
learners to use a single platform for both their learning
activities and the exploitation of their outcomes and traces for
self-reflection and competence management.
The implementation of these services has highlighted a
number of data management challenges regarding compliance,
access, ownership, interoperability, identification, and
durability. In future work, we aim to present solutions
addressing these challenges, including the exploitation of
blockchain technologies for the storage and the verification of
learning outcomes, activity traces and other elements stored in
ePortfolios.
ACKNOWLEDGMENTS
This work was partially funded by the European Union in
the context of the Go-Lab Integrated Project and the Next-Lab
Innovation Action. The latter has received funding from the
European Union’s Horizon 2020 Research and Innovation
Program under Grant Agreement no. 731685. This publication
reflects only the authors’ view and the European Commission
is not responsible for any use that may be made of the
information it contains.
REFERENCES
[1] K. Barker, “The Electronic Learning Record: Assessment and
Management of Skills and Knowledge”, FuturEd, Literacy BC and the
National Literacy Secretariat, Vancouver, Canada, 1999.
[2] J.A. Macías, “Enhancing Project-Based Learning in Software
Engineering Lab Teaching Through an E-Portfolio Approach”, IEEE
Transactions on Education, Vol. 55, No. 4, 2012, pp. 502-507.
[3] R. Matsuba, S.-I., M. Miyazaki, T. Kita, J. Nemoto, K. Suzuki, H.
Nakano, “An Implementation of a Learning Portfolio”, Proceedings of
the 10th International ePortfolio and Identity Conference (ePIC), 2012,
pp. 186-190.
[4] D. Gillet, A. Vozniuk, M.J. Rodríguez-Triana, and A. Holzer, “Agile,
Versatile, and Comprehensive Social Media Platform for Creating,
Sharing, Exploiting, and Archiving Personal Learning Spaces, Artifacts,
and Traces”, Proceedings of the World Engineering Education Forum,
2016, pp. 1-12.
[5] B. Schwendimann, M.J. Rodríguez-Triana, A. Vozniuk, L.P. Prieto, M.
Boroujeni, A. Holzer, D. Gillet, and P. Dillenbourg, “Perceiving
learning at a glance: A systematic literature review of learning
dashboard research”, IEEE Transactions on Learning Technologies,
Vol. 10, No. 1, 2017, pp. 30-41.
[6] M.J. Rodríguez-Triana, and al., “Rich Open Educational Resources for
Personal and Inquiry Learning Agile Creation, Sharing and Reuse in
Educational Social Media Platforms”, Proceedings of the International
Conference on Web & Open Access to Learning, 2014, pp. 1-6.
[7] T. de Jong, S. Sotiriou, and D. Gillet, “Innovations in STEM education:
The Go-Lab federation of online labs”, J. Smart Learning Environments,
Vol. 1, No. 3, 2014, pp. 1-16.
[8] Brown, M., Anderson, B., Simpson, M., & Suddaby, G. Showcasing
Mahara: A new open source eportfolio. Proceedings of ASCILITE
Singapore, 2007, 82-84.
[9] A. Vozniuk, M. J. Rodríguez-Triana, A. C. Holzer, S. Govaerts and D.
Sandoz et al. Contextual Learning Analytics Apps to Create Awareness
in Blended Inquiry Learning. Proceedings of the 14th International
Conference on Information Technology Based Higher Education and
Training, 2015, pp. 1-5.
[10] A. Vozniuk, D. Gillet (Dir.). “Enhancing Social Media Platforms for
Educational and Humanitarian Knowledge Sharing: Analytics, Privacy,
Discovery, and Delivery Aspects.” PhD Thesis n° 7495, EPFL,
Lausanne, 2017.
[11] T.P. Light, H.L. Chen, J.C. Ittelson. Documenting learning with
ePortfolios: A guide for college instructors. John Wiley & Sons, 2012.
2017 7th World Engineering Education Forum (WEEF)
47