Content uploaded by Jalal Nouri
Author content
All content in this area was uploaded by Jalal Nouri on Nov 29, 2019
Content may be subject to copyright.
Content uploaded by Jalal Nouri
Author content
All content in this area was uploaded by Jalal Nouri on Aug 21, 2019
Content may be subject to copyright.
Content uploaded by Mohammed Saqr
Author content
All content in this area was uploaded by Mohammed Saqr on Jul 28, 2019
Content may be subject to copyright.
Paper—Efforts in Europe for Data-Driven Improvement of Education
Efforts in Europe for Data-Driven
Improvement of Education
A Review of Learning Analytics Research in Seven Countries
https://doi.org/10.3991/ijai.v1i1.11053
Jalal Nouri (*)
Stockholm University, Stockholm, Sweden
jalal@dsv.su.se
Martin Ebner
Graz University of Technology, Styria, Austria
Dirk Ifenthaler
Universität Mannheim, Mannheim, Germany
Mohammed Saqr
University of Eastern Finland, Joensuu & Kuopio, Finland
Jonna Malmberg
University of Oulu, Oulu, Finland
Mohammad Khalil
University of Bergen, Bergen, Norway
Jesper Bruun
University of Copenhagen, Scandinavia, Denmark
Olga Viberg
Royal Institute of Technology, Stockholm, Sweden
Miguel Ángel Conde González
Universidad de León, Leon, Spain
Zacharoula Papamitsiou
Norwegian University of Science and Technology, Trondheim, Norway
Ulf Dalvad Berthelsen
Aarhus University, Aarhus C, Denmark
Abstract—Information and communication technologies are increasingly
mediating learning and teaching practices as well as how educational institu-
tions are handling their administrative work. As such, students and teachers are
leaving large amounts of digital footprints and traces in various educational
apps and learning management platforms, and educational administrators regis-
8
http://www.i-jai.org
To cite: Nouri, J., Ebner, M., Ifenthaler, D., Saqr, M., Malmberg, J., Khalil, M., Viberg, O., Bruun, J., Conde-Gonzalez, M., Papamitsiou, Z. & Berthelsen, U. (2019).
Efforts in Europe for Data-Driven Improvement of Education – A review of learning analytics research in seven countries.
International Journal of Learning Analytics and Artificial Intelligence for Education. 1(1). https://doi.org/10.3991/ijai.v1i1.11053
Paper—Efforts in Europe for Data-Driven Improvement of Education
ter various processes and outcomes in digital administrative systems. It is
against such a background we in recent years have seen the emergence of the
fast-growing and multi-disciplinary field of learning analytics. In this paper, we
examine the research efforts that have been conducted in the field of learning
analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and Sweden.
More specifically, we report on developed national policies, infrastructures and
competence centers, as well as major research projects and developed research
strands within the selected countries. The main conclusions of this paper are
that the work of researchers around Europe has not led to national adoption or
European level strategies for learning analytics. Furthermore, most countries
have not established national policies for learners’ data or guidelines that gov-
ern the ethical usage of data in research or education. We also conclude, that
learning analytics research on pre-university level to high extent have been
overlooked. In the same vein, learning analytics has not received enough focus
form national and European national bodies. Such funding is necessary for tak-
ing steps towards data-driven development of education.
Keywords—Learning analytics, Europe, data-driven improvement, education
1 Introduction
Over the past decades, the world has undergone a transformation process, which
many consider to be as important as the Industrial Revolution once. In this post-
industrial society, also called the information and knowledge society, information
technology plays a crucial role. It permeates and transforms how we work, study,
relate to information and knowledge and how we spend our free time. As a conse-
quence of this digitization, huge quantities of data, i.e. big data, is generated that re-
flects our activities. Therefore, in many fields, such as business or medicine, we have
witnessed how essential the use of analytics has become to process generated big data
in order to develop data-driven insights into people’s activities for the optimization of
processes and outputs.
Today, the educational systems around the world are also undergoing major digital
transformations. Information and communication technologies are increasingly medi-
ating learning and teaching practices as well as how educational institutions are han-
dling their administrative work. As such, students and teachers are leaving large
amounts of digital footprints and traces in various educational apps and learning man-
agement platforms, and educational administrators register various processes and
outcomes in digital administrative systems.
It is against such a background we in recent years have seen the emergence of the
fast-growing and multi-disciplinary field of learning analytics. The field, which origi-
nates from disciplines such as “business intelligence, web analytics, educational data
mining and recommender systems” [1, p. 1) attempts to exploit data generated in edu-
cational settings “for purposes of understanding and optimizing learning and the
environments in which it occurs” [2, p. 34].
iJAI ‒ Vol. 1, No. 1, 2019
9
Paper—Efforts in Europe for Data-Driven Improvement of Education
Although the field of learning analytics is still in its infancy, seen from an interna-
tional perspective, it has already produced innovative educational research that
demonstrate the utility of learning analytics on the micro level (understanding and
developing learning and teaching), on the meso level (understanding and developing
single educational organizations), and on the macro level (understanding and develop-
ing on a national level or across educational organizations) [3, 4, 5].
From a European standpoint, the potentials of learning analytics were recognized
early on. Already in 2013, only two years after the official birth of the field of learn-
ing analytics, the European Commission emphasized that learning analytics can con-
tribute to “develop new solutions for better personalised learning, by allowing teach-
ers to have a more accurate and up-to-date follow up of each learner. Through learn-
ing analytics, new and more learner-centred teaching methods can emerge since the
evolution of learners who use ICT regularly can be closely monitored.” [6, p.5). In
another more recent report the European Commission Working Group on Digital
Skills and Competences (ET2020) once again pointed to the potential of learning
analytics to “contribute to the quality of teaching and learning and the modernization
of educational systems in Europe.” [7, p.2]. Moreover, ET2020 urged for capacity
building in the field and collaborative research projects. And indeed, research in
learning analytics is growing in Europe, especially from countries such as Spain,
United Kingdom, Germany, Netherlands, and Austria. Furthermore, several European
countries, such as Norway, Denmark, and Netherlands, are developing nationwide
learning analytics strategies that includes infrastructure, competence centers, and
national policies.
In this paper, we examine the research efforts that have been conducted in the field
of learning analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and
Sweden. More specifically, we report on developed national policies, infrastructures
and competence centers, as well as major research projects and developed research
strands within the selected countries.
2 European Research
2.1 Austria
The research efforts in Austria has been started already back in 2011
1
. Based on
personal meetings between George Siemens, Erik Duval and Martin Ebner at the ED-
Media conference in Lisbon, Portugal, the idea of a conference on Big Data for Learn-
ing was shared to a large community - today, well known as International Conference
Learning Analytics and Knowledge, shortly LAK. The second was done in Vancou-
ver, Canada in April 2012
2
. At this conference, a first research work from Austria was
presented and discussed - just a simple multiplication trainer for schoolchildren aged
8-12 years [8].
1
https://www.aace.org/review/ed-media-2011-lisbon-portugal-final-call-for-participation-april-12/ (last
visited June 2019)
2
https://solaresearch.org/events/lak/#lak12 (last visited in June 2019)
10
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
The research team followed the idea to collect and gather all calculations done by
the children and to give feedback to the learners as well as the teachers. Today the
application holds more than 1.000.000 calculations and we know very precisely how
the learning of the multiplication table is happening described in several publications
[9] [10] [11].
In addition, the research team worked on more applications for school children - an
addition, a subtraction [12], a division and a multi-digit trainer [13]. As follow up,
Graz University of Technology contributed to a first European project about German
spelling acquisition [14]. The project aims to offer children an online editor for writ-
ing short essay. In the background beside a typical spell-checking dictionary an intel-
ligent one was implemented. This developed one holds words, written in all possible
and false forms, categorized in different groups. The Learning Analytics part analyses
each text and provide feedback to learners and teachers divided to the defined catego-
ries.
Beside this project in secondary education in 2014 first research has been done in
higher and adult education. The University of Graz as well as the University of Tech-
nology of Graz founded in 2014 the first and till now online MOOC platform in Aus-
tria, called iMooX. Due to the fact that MOOCs are addressing a huge amount of
learner’s data driven investigations seems to be a logical step firstly described in de-
tail in [15]. Different studies pointed out how Learning Analytics can help to identify
different kind of learners [16], how students remain in MOOCs [17], how gamifica-
tion elements assist the learning process [18] and even how new didactical approach-
es, called Inverse Blended Learning, are introduced [19].
Another joint project on European level between KU Leuven, University Notting-
ham, TU Delft and TU Graz called STELA (“Successful Transition from secondary to
higher Education using Learning Analytics”) aimed to assist students during their
transition phase from secondary to higher education [20]. The outcome of the project
provided a general framework for building students’ dashboards [21] (Leitner & Eb-
ner, 2016) and different prototypes at each single university.
Finally, there are also some work done on a policy level for national issues. Due to
the fact that in Austria the data protection law is rather strong, it is from high im-
portance to think about how Learning Analytics can be integrated on an institutional
level. One first research work was about the de-identification of data [22] and general
challenges to overcome if Learning Analytics will be introduced in Higher education
institutions [23].
Currently a white paper on Learning Analytics for Higher Education is elaborated
under the lead of the nationwide association of new media for teaching and learning.
Finally, the ministry of education, science and research announced to give financial
support for Learning Analytics applications in the next years.
2.2 Denmark
Learning Analytics as a field in Denmark seems rather as disconnected islands than
as a connected whole. While research efforts within the scope of LA as defined in this
article have been done for some time, research has until recently been conducted at
iJAI ‒ Vol. 1, No. 1, 2019
11
Paper—Efforts in Europe for Data-Driven Improvement of Education
separate universities and university colleges without much interaction and collabora-
tion. Furthermore, the two overall aims for LA, understanding and optimizing seem to
be a source of divisions in the Danish landscape. Some researchers focus on under-
standing from an educational standpoint [24], others on optimizing from a computer
science standpoint [25]. In addition to this division, a third division seems to be be-
tween econometrics and educational research, where econometrics as a field is more
interested in the effects of intervention and use educational test-data as outcome vari-
ables [26].
Also, worth mentioning, though not as such LA research, is a strand of research
critically monitoring the consequences of the digital transformation of the Danish
educational system, among other things, focusing on the consequences of integrating
analytics into the ecosystem of public primary education.
Apart from divisions in terms of research focus, the divisions between educational
research and econometrics is evidenced by public debates [27] in the wake of public
reports on Danish national tests [28]. We speculate that the origin of this divide is
determined by the broad field of the researcher; computer scientists engaging in LA
are more likely to focus on optimizing, while educational researchers are more likely
to link LA data to e.g. constructivist theories of learning. Econometrics seems to be
narrowly focused on establishing causal relationships using linear models. Thus, we
see a gap between (at least) three traditions: a computer scientist tradition, an econo-
metrics tradition, and an educational tradition. Here, we provide examples to illustrate
the LA landscape in Denmark and then present current or very recent efforts to begin
bridging the gap.
The Danish Center for Big Data Analytics Driven Innovation (DABAI), was estab-
lished in 2016, with learning analytics as one of the goals. Within the field, DABAI is
to pursue optimization of e-learning personalization, student behavior modeling, pre-
dicting student performance, similarity among quizzes, authorship verification, and
curriculum trainer [25]. With regards to student behavior modeling, researchers have
created a model of student drop-outs in Danish upper secondary school.
Another avenue of research in play is a design-based approach. Here, educational
researchers design online and blended learning materials for science courses while at
the same time monitoring clickstreams, videotaping lessons, and audiotaping student
discussions for joint multimodal analyses [29]. On such project is the Virtual Neu-
trons for Teaching project (eneutrons.org), in which students learn neutron scattering
via online textbooks and quizzes [30] [31]. One of the outcomes of the projects is a
novel method for analyzing online student behaviors using clickstream data [32].
Two large ongoing projects, both funded by Innovation Fund Denmark (Innova-
tionsfonden), are also worth mentioning. The first, Game-based Leaning in the 21st
Century (GBL21), is a large collaborative project aimed at developing design thinking
skills through game-based learning (https://gbl21.aau.dk). As a part of GBL21 educa-
tional researchers from DPU, Aarhus University and Aalborg University are develop-
ing an online tool for assessing different aspects of design thinking skills. The other
project, Automatically Tracking Early Stage Literacy Skills (ATEL), involves re-
searchers from DPU, Aarhus University and Technical University of Denmark (DTU)
12
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
collaborating on developing analytics tools for tracking early stage literacy develop-
ment (
Furthermore, in 2010, Denmark implemented national tests in grades 2-4 as well as
6-8 in primary school. The tests are obligatory and target different subjects in differ-
ent grades (https://www.uvm.dk/folkeskolen/elevplaner-nationale-test--trivselsmaaling-og-spr
ogproever/nationale-test/klassetrin-fag-og-profilomraader). The tests are adaptive, meaning
that each student will be presented with test items during the test-period (usually 1
hour in-class) and that the difficulty of test item i+1 is dependent on the student an-
swer to test item i. The system collects data on the students, which is used to monitor,
single students, classes, and schools. While in-depth analyses are possible, research
seems to have focused mainly on macro-scale variables, such as whether there is a
positive or negative effect of the national tests.
We have argued that in Denmark, a gap between LA as undertaken by computer
scientists, econometrics researchers, and educational researchers exists. We believe
that in order to bridge this gap, computer scientists will need to learn to operationalize
current educational theories and results, while educational researchers need to utilize
and interpret results from currently used computer algorithms. The gap between edu-
cational researchers and econometrics researchers seems to be rooted in substantially
different aims and methods for the two fields. Bridging all of these gaps will require
extensive cooperation and probably compromises.
Despite the lack of coherence and collaboration in the Danish LA research com-
munity, there are, however, efforts to connect to the larger Nordic cross-disciplinary
community of LA researchers, e.g. by hosting the Nordic LASI 2018 at Aalborg Uni-
versity and organizing Learning Analytics Research Symposium (LARS), which was
held at University of Copenhagen in November 2018 (https://www.ind.ku.dk/lars2018 )
2.3 Finland
Research on learning analytics started early in Finland with a focus on the social
and collaborative aspects of learning. Tervakari and colleagues investigated the “TUT
circle” which was an online social media enhanced learning platform at Tampere
University of Technology. They reported on the utility of visualization of students’
interactions, and researched the learning analytics potentials of the platform. The
group later contributed with research on content analytics, social media analytics and
teacher tools [33]. Other aspects of learning analytics followed such as predicting
students’ performance in programming courses and using machine learning methods
to predict students’ need for assistance [34]. A notable body of research comes from
Järvelä and colleagues on self-regulated and collaborative learning, who investigated
social shared regulation, interactions and engagement with collaborative learning and
the temporal sequence of regulatory processes [35] [36]. The group are also working
on multimodal physiological data as well as dispositional learning analytics [37] [38].
As the field is gaining recognition, many Finnish universities are now embracing the
concept and research is increasingly reported from most Finnish universities.
Despite the development of techniques and methods to model and predict human
learning, the field still lacks ability to connect the powers of learning sciences and
iJAI ‒ Vol. 1, No. 1, 2019
13
Paper—Efforts in Europe for Data-Driven Improvement of Education
learning analytics in effective way to understand the complexity of learning alone or
together, reveal hidden human mental processes and model and trace trigger moments
and critical patterns of learning processes [37]. The emerging need to combine the
power of learning sciences and learning analytics is also recognized by Finish Science
Academy, which funds high-quality scientific research and strengthens the position of
science and research. For example, in spring 2019 Academy of Finland launched a
project call “Digital Humanities” which reflects the emerging need to develop new
methodologies. Especially Digital Humanities program emphasizes new ways to ad-
dress novel methods and techniques in which digital technology and state-of-the-art
computational science methods are used for collecting, managing and analyzing data
in humanities and social sciences research as well as for modelling humanities and
social science phenomena. Thus, the emphasis lies not only in collecting "big data",
but also "small" (deep, rich) data, since so far, the potential of many overarching con-
ceptual and methodological questions remain unexplored and under-theorized.
The large body of empirical and theoretical advances in the field of self-regulated
learning (SRL) [37] has indisputable evidence that such skills improve learning with
students of all ages. However, there has been much less understanding how learning
analytics are grounded in the literature on self-regulated learning and how self-
regulated learning is supported. This is much due the fact, that SRL processes (i.e.
cognitive, metacognitive, motivational and emotional) are invisible for naked eye and
therefore difficult to capture [37]. Fortunately, due the technological and methodolog-
ical advancements in the field, there is potential to transform these mental processes at
least to some extent in a visible form to provide learning analytics for teacher and
students.
During the past years, there has been increasing interest to collect and analyze mul-
timodal data (i.e. log data, physiological data, situated self-reports) to better capture
the mental processes of human learning [37]. For example, [39] applied multimodal
data (e.g. physiological data, facial expression data and video data) evidencing that it
is possible to make situational characteristic involving to the regulated learning pro-
cess visible. Facial expression recognition has potential to reveal valence of emotions
during collaborative learning. Visible interactions recorded from the video data has
potential to reveal type of interaction, but also instances when students engage for
self-regulated learning.
2.4 Germany
In 2016, Ifenthaler and Schumacher [40] report that research on learning analytics
in Germany is scarce and that there are only a few projects focusing on the implemen-
tation of learning analytics systems. In 2019, several research projects are being fund-
ed by the German Federal Ministry of Education and Research focusing on technolo-
gy integration and analytics in educational organizations [41]. For example, the aim of
the project ‘Utilizing Learning Analytics for Study Success’ is to conduct a systematic
review and construct a set of policies for German higher education institutions to
adopt learning analytics capabilities into their existing learning environments. Precise-
ly, the goals of the project are a) first to build a systematic review of empirical evi-
14
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
dence demonstrating how learning analytics have been successful in facilitating stu-
dent success in continuation and completion of their university courses both nationally
and internationally, and forming the basis for aim b) to make policy recommendations
for the German higher education sector in order to accept and implement such systems
within institutions.
It became evident from the integrative review that robust empirical findings on a
large scale to support the effectiveness of learning analytics actually retaining students
onto courses are still lacking [42]. Therefore, it is imperative to leverage existing
learning theory, psychological methods and connecting them to advances of learning
analytics research for designing (quasi-)experimental studies including theoretical
frameworks and sound empirical methodologies. The project findings of the interview
study indicate that more work on ethical and privacy guidelines supporting a wider
adoption of learning analytics systems is needed [42] as well as work towards a stand-
ardized learning analytics system which can be integrated into any learning environ-
ment providing reliable at-risk student prediction, prevention and intervention strate-
gies [43]. In particular, personalized learning environments are increasingly demand-
ed and valued in education institutions to create a tailored learning package optimized
for each individual learner based on their personal profile which could contain infor-
mation such as their geo-social demographic backgrounds, their previous qualifica-
tions, how they engaged in the recruitment journey, their activities on social media
and websites, as well as tracking information on their searches [44].
Additional findings document issues with organizational readiness (Ifenthaler,
2017). For example, a standard infrastructure of educational institutions includes a
student management system, a learning management system and a course manage-
ment system. However, these systems are deeply embedded into the organization’s
infrastructure and often are not designed to reveal data for analytics [45]. For access-
ing the necessary data, various connections to the organizations’ legacy systems have
to be established which are able to access the students’ profiles, to capture the actual
learning processes and get access to curricula data. As these legacy systems are often
based on various technologies, each connection has to be implemented as an individu-
al project which is labor and cost intensive. Besides the technological challenges, staff
capabilities are also changing when implementing learning analytics systems. Not
only new staff roles but also further development of existing staff is required for suc-
cessful implementation of learning analytics systems [46].
2.5 Norway
Although the collection, interpretation, and visualization of multimodal data has
been extremely challenging for researchers, recent technological developments and
data science, and AI advancements have boosted the growth of non-invasive high-
frequency multimodal-data collections.
Learners’ traces are generated during their interaction with technologies, such in-
teraction is often complex but offers opportunities for collecting rich and multimodal
data [47] [48], MultiModal Learning Analytics (MMLA), as the literature refers to
them. In order to unfold the benefits of MMLA, the Learner-Computer Interaction
iJAI ‒ Vol. 1, No. 1, 2019
15
Paper—Efforts in Europe for Data-Driven Improvement of Education
(LCI) lab at the Norwegian University of Science and Technology (NTNU) focuses
on overcoming the difficulties in gathering and making sense of MMLA. In other
words, we attempt to identify, how insights generated during learner-computer inter-
action help us to design future learning environments and improve the learning expe-
rience.
For many years, the design of learning technologies has been utilizing click-
streams and keystrokes as the primary data source for modelling and predicting learn-
ing behavior. In recent work at NTNU researchers set out to quantify what, if any,
advantages do physiological sensing techniques provide for the design of learning
technologies [47] in a lab context with 251 game sessions and 17 users focusing on
skill development (i.e., user's ability to master complex tasks).
Furthermore, when dealing with data channels in multiple modes and modalities, a
major issue lies in determining combinations of the multimodal data channels that are
necessary for one to make valid and reliable inferences regarding the temporally un-
folding learning processes, and selecting the algorithms and analytical tools to use.
Recent research at NTNU has proposed a novel approach, called “grey-box” ap-
proach, that bridges the hypothesis/literature-driven (measurements/feature selection)
“white-box” approach with the computation-driven (feature fusion) “black-box” ap-
proach [49]. The authors aimed to extend current methodological paradigms in under-
standing effortful behavior and learning performance in adaptive learning conditions
with new, cutting-edge, interdisciplinary work on building pipelines for educational
data, using innovative tools and techniques
Another research dimension explored at NTNU with regard to multimodal learn-
ing analytics is the modelling of learner behavior, by taking advantage of the inherent
temporality in the physiological data. The Generalized Auto-Regressive Conditional
Heteroskedasticity (GARCH) method was applied with learners’ physiological time-
series data to model their behavior, and make suggestions about how the models can
be further utilized to provide proactive feedback to learners [49].
Moreover, investigating and explaining the patterns of learners’ engagement in
adaptive learning conditions is a core issue towards improving the quality of personal-
ized learning services. The research group at NTNU bridged complexity theory with
multimodal data in order to capture specific patterns of engagement that foretell and
explain learners’ level of performance on adaptive learning procedures [50].
In a different context, the joint collaboration between the Centre for the Science of
Learning & Technology (SLATE) from University of Bergen in Norway and Erasmus
University Rotterdam from the Netherlands gave birth to interesting research studies
in MOOCs. The first one was on exploring self-regulated learning in Coursera plat-
form by [51]. Wong et al [51] employed sequence pattern mining to identify self-
regulated learning studying strategies to a group in MOOCs where student was of-
fered self-reflection and monitoring intervention. One more contribution between
SLATE and the Dutch university is a book chapter titled “Educational Theories and
Learning Analytics: From Data to Knowledge” by [52]. The chapter aimed at discuss-
ing how learning theories and learning analytics are important components of educa-
tional research. In addition, the chapter suggests that more experimental studies are
needed for applied learning analytics in general and in Europe more specifically.
16
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
In Norway as well, there has been research on ethical aspects of learning analytics.
For instance [53] carried out a research study at the Open University in the United
Kingdom in which they examined student’s behavior in higher education and their
attitudes to privacy. The authors plan to carry out the same research study in Norway
next year together with South Africa. Furthermore, the study by [53] looked at the
three levels of consent in MOOCs, micro, meso, and macro. Based on reviewing the
policies of the biggest four MOOC providers, the paper proposes a need for greater
transparency around the implications of users granting consent at the point of registra-
tion.
Another interesting project brought from Norway are those related to the medical
and health sector. SLATE from the University of Bergen is involved in a consortium
project called OERBioMed. Biomedicine seeks to explain physiological processes at
the molecular and individual level. Such information is essential for the understanding
of disease progression and for the development of new treatments and therapies. This
current and future medical research relies on the existence of people with expertise in
the biomedical field. To raise the quality of teaching, training and learning within the
field of biomedicine, new and innovative approaches are required. To this end, this
project deals with open-access and online courses to increase the bioethical
knowledge and awareness in the biomedical community. SLATE is involved in this
project by providing help and support to launch massive open online courses in col-
laboration with all the partners from the Nordic countries. SLATE also provide learn-
ing analytics and statistical services to the partners.
2.6 Spain
Data driven education has been a hot topic in Spain in the last 10 years. In fact,
there are several relevant works in the field of Learning Analytics, Visual Analytics,
Educational Data Mining, Multimodal Analytics, etc. However, there are some issues
in the Spanish landscape related with Data Driven Education, issues that are also
common in other countries. These are:
• Data driven education is a relatively new research field, and therefore the quantity
and variety of topics of interest is high
• The current level of global or campus-wide application of learning analytics in
companies and public and private organizations is low
• The fragmentation of research groups, and the difficulty they have to reuse and
replicate research designs, results and outcomes of others
• The focus of disciplines such as learning analytics has been mostly technical and it
is necessary a multidisciplinary perspective that involves also profiles such as edu-
cators, psychologist or sociologist
• It is also desirable to engage different organizations in the use of disciplines such
as Learning Analytics (companies, public administrators, non-higher education in-
stitutions, etc.)
iJAI ‒ Vol. 1, No. 1, 2019
17
Paper—Efforts in Europe for Data-Driven Improvement of Education
• The low number of professionals in this field, with the emergence of the new data
scientists this is close to be solved, but it is still needed a very specific profile in the
educational field
• The knowledge that research groups have about themselves, ongoing projects,
institutions and organizations interested in Data Driven Education is limited.
In order address these problems and centralize the efforts carried out related to
these topics SNOLA was defined. SNOLA (Spanish Network Of Learning Analytics,
http://snola.es), informally created in 2013, emerges as a network composed of the
main national researchers in the LA field, comprising 10 researchers from 9 Spanish
Research Groups. SNOLA primarily has a technical approach, but it also integrates
other educational visions and disciplines that give the network a wider scope. The
main objectives of SNOLA are:
• The promotion of collaboration among the participants in the Network, as well as
with other interested parties (companies and public and private educational institu-
tions) and other European and international LA collaboration networks
• The diffusion and organization of LA initiatives at a national scale
• Making resources to effectively integrate LA in educational processes available to
the public; and 4) provide researchers and professionals adequate training to face
and give answer to the new challenges of Digital Society risen by the incorporation
of ICT in education. From December of 2015 to June of 2018, SNOLA was grant-
ed by the Spanish Government as Thematic Network which help the network to
support different events and activities.
At the same time that SNOLA was defined several projects have been developed
with Spanish institutions related with Data Driven Education. Some are international
projects (mostly European) such as: LACE Project (http://www.laceproject.eu/),
SHEILA Project (https://sheilaproject.eu/), Make World (https://makeworld.eu), Go-Lab
(https://www.golabz.eu/) or LALA Project (https://www.lalaproject.org/); some other are
granted by the Spanish Government through national calls such as the projects EEE or
RESET; or by Regional Governments such as eMadrid. These projects deal with dif-
ferent issues and in some of them Data Driven Education is the main topic and in
other it is employed to support the results, processes or products developed. Some of
topics explored could be the teamwork assessment, intelligent tutors that make sug-
gestions depending on learning evidences or the use of Learning Analytics in CSCL.
Regarding the scientific events it should be noted that exist several relevant initia-
tives:
The Learning Analytics Summer Institute Spain, linked to the SOLAR intiative. It
is a conference supported by SNOLA with several editions. First Spanish Edition was
hold in 2013 at Granada (http://grinugr.org/noticias-de-eventos/lasi-spain/), 2014 at Ma-
drid (http://lasimadrid2014.scc.uned.es/), 2015 (https://blogs.deusto.es/lasi2015Bilbao/) and
2016 (https://lasi16.snola.es/) at Bilbao, 2017 at Madrid (https://lasi17.snola.es/) and 2018
at León (https://lasi18.snola.es/). This year will take place in Vigo. It began as an event
for discussion between researchers in the field of Learning Analytics and also as a
18
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
contact point with experts. However now it includes also sessions with scientific pa-
pers presentations, networking sessions, companies’ sessions, etc.
The Learning Analytics Track included in the TEEM Conference, an international
scientific conference defined in Spain in 2013. The track is leaded by Spanish re-
searchers but includes works from people all around the world. It began in 2013 and
since then this track has taken place every year. During the track several scientific
papers are briefly presented and discussed with experts. Some of this track editions
have been associated to special issues in journals such as Computers in Human Be-
havior [2] or the International Journal of Engineering Education [3]. Several topics
have been discussed during the track, some of the most significant could be: Predic-
tion of students success or failure based in their learning and interaction evidences;
tools to improve learning process, Learning Analytics and Mobile devices, Ethics
about Learning Analytics, Visual Learning Analytics, Academic Analytics, Multi-
modal Learning Analytics, Social Network Analytics, Competence Assessment
through Learning Analytics, Discussion about the quantity and quality of data to make
decisions in educational contexts, Application of Learning Analytics tools and tech-
niques, Personalization of Learning by using Learning Analytics, etc. More infor-
mation about this track and the research woks included in it can be found here [54-
59].
Other seminars such as LAIKA, SIIE 2016, WPLA at ECTEL, LATCEE (in
Educon) were also leaded by Spanish research groups and deals with similar topics as
the previous ones.
It should be noted that in conferences such as LASI – Spain companies were in-
volved. In this way it was possible to know what were they doing about Learning
Analytics and what they require from the academy. Companies such IBM, Euskaltel,
Sun Edison, Brambles or Eticas Consulting participate in several LASI Conference
and enrich the perspective about Data Driven Education in Spain.
2.7 Sweden
Even though the field of learning analytics is an evolving field of both research and
practice [4], there have already been some relevant efforts in terms of its development
in a Swedish context. These attempts are currently expanding in higher educational
settings. For instance, a number of research directions have been explored with a
learning analytics approach by a research group at Stockholm University, focusing on
aspects such as problem-based learning [60], teacher education [61], collaborative
learning [62], self-regulated behavior [63], prediction of student performance in
blended learning and in flipped classroom settings [64], and prediction of perfor-
mance and completion of master- and bachelor thesis [65].
Other examples of research conducted by Swedish researchers include the explora-
tion of multimodal learning analytics [66], and prediction of students’ mastery of
skills [67].
The efforts in K-12 education are in particular relevant to an ongoing digitalization
of the Swedish education system. One of the recent developments in this regard in-
clude: i) the Swedish government decision for digitalization of the schools, with a
iJAI ‒ Vol. 1, No. 1, 2019
19
Paper—Efforts in Europe for Data-Driven Improvement of Education
supplement that presents a national strategy for this (and ii) National action plan for
digitization of the Swedish school system. Both of these documents highlight a need
for a strategic and systematic data collection for increased opportunities for follow-up
of the school system's digitalization, with the main purpose to increase the availability
of comparable data that makes it possible to study the connection between methods
and results. However, these endeavors are still in their infancy. Thus, the present re-
port illustrates the state of the art that largely relates to adult and particularly, higher
education.
We are only aware of one project at the K-12 level, which focused on 21st century
skills and collaborative problem-solving of students in secondary school [68]. In this
project, which was funded by Swedish Association of Local Authorities and Regions
(SKL), researchers aimed to assess collaborative problem-solving skills in technolo-
gy-enhanced environments. The project included students from five different schools
in the Stockholm area. In this particular study, the researchers exploited multimodal
data (video and log data) in order to identify and predict students collaborative prob-
lem-solving skills.
Swedish Educational Data: Data-Driven Innovation for World Learning Educa-
tion is one of the recent development projects (with no particular educational level
focused) that aims at establishing Swedish Educational Data as a support organization
for data-driven innovation for education (2017-2019). The project is funded by Vin-
nova, Sweden’s innovation agency and led by KTH Royal Institute of Technology. It
includes both public and private actors from the education industry. They interact
actively to increase data usage for education. This is important since fragmentation
can cause each part to develop their own analytical methods and their own data man-
agement when they instead may be applicable across the entire field. The project’s
results will be released at the end of 2019.
In another recent research and development project funded by IFOUS (2017-2020),
Programming in school subjects (“Programmering i ämnesundervisningen), which do
not have an explicit focus on learning analytics, researchers are currently exploring
how digital data generated in K-12 classrooms can be used to develop teaching prac-
tices and the identification of students computational thinking skills, based on prelim-
inary findings reported in [69].
In general, albeit some research has been conducted in Sweden, so far learning ana-
lytics research has not been funded by the larger research agencies.
3 Discussion
Today, in the era of big data and analytics, researcher as well as educational stake-
holders are calling for data-driven development of education. Consequently, aiming to
capitalize on the rewarding applications of big data in different fields, researchers are
hard at work building the field of data-driven education and research (through the
field of learning analytics). In Europe, learning analytics has been embraced by re-
searchers since the early days, contributions span all venues of research in the field,
such as collaborative learning, visualization of learners’ interactions, learning dash-
20
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
boards, dispositional learning analytics, self-regulated learning and multimodal learn-
ing analytics. Researchers have also explored all kinds of data from single course
digital traces to large scale academic analytics. Efforts are increasingly organized to
tackle new problems, establish collaborative research groups, set up learning analytics
focused scientific events, and build capacity in the interdisciplinary field.
Lately, Some European projects have been launched, examples are the STELA pro-
ject between KU Leuven, University of Nottingham, TU Delft and TU Graz (Leitner
& Ebner, 2016), The LACE project (The Learning Analytics Community Exchange),
SHEILA Project and the collaboration between (SLATE) at the University of Bergen
and Erasmus University Rotterdam. Nonetheless, collaboration and funding on the
European level are still relatively scarce. On the national level, funding of learning
analytics projects is just taking off. In Austria, the ministry of education will support
learning analytics research in the next years. In Denmark, The ATEL project was
funded by Innovation Fund Denmark (Innovationsfonden) which will track the early
stage literacy development through learning analytics. In Finland, the Finish Science
Academy launched the project “Digital Humanities” to use data for analyzing and
modelling humanities and social sciences. In Germany, the Federal Ministry of Edu-
cation and Research have funded some projects such as ‘Utilizing Learning Analytics
for Study Success’ (Mao et al., 2019). In Norway and Spain several projects are start-
ing with the help of national funding agencies. However, in Sweden, the funded pro-
jects are still very few.
Taking together, the previous examples for funding are way behind the expected in
a time where learning is in the center of public and political attention. Let alone the
accelerating successes of using big data across many disciplines. The rich and diverse
potentials of data-driven applications are reflected in the heterogeneous nature of
reported research from different research groups. Although such diversity and breadth
of applications have helped emphasize the worth of using data to improve education,
it has also emphasized a need for organizing efforts. Fragmentation, division and
paucity of collaborative projects seem to be prevailing. Nevertheless, a number of
collaborative groups are emerging. Examples include The Learning Analytics re-
search group at Stockholm University, The Danish Center for Big Data Analytics
Driven Innovation (DABAI) and the Spanish Network of Learning Analytics. Relat-
edly, learning analytics scientific events have been organized, such as the Nordic-
LASI.
As the field of learning analytics is relatively new, researchers around the world are
working to tackle the emerging challenges, such as proving the value of using data-
driven decision making, aligning the field with learning sciences, collecting useful
data while securing the privacy and agency of learners. European researchers are no
exception, they are facing the same challenges as well as their own challenges. most
important is that the large interest and work of researchers and groups around Europe
has not led to national adoption or European level strategies relative the ubiquitous
adoption of technology in education. Most countries have not established national
policies for learners’ data or guidelines that govern the ethical usage of data in re-
search or education. We also conclude, that learning analytics research on pre-
university level to high extent have been overlooked. In the same vein, learning ana-
iJAI ‒ Vol. 1, No. 1, 2019
21
Paper—Efforts in Europe for Data-Driven Improvement of Education
lytics has not received enough focus form national and European national bodies.
Such funding is necessary for taking steps towards data-driven development of educa-
tion.
4 References
[1] Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. Internat-
ional Journal of Technology Enhanced Learning, 4(5/6), 304-317. https://doi.org/
10.1504/ijtel.2012.051816
[2] Long, P. D., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and educa-
tion. EDUCAUSE Review, 46(5), 31–40.
[3] Ifenthaler, D., Mah, D.-K., & Yau, J. Y.-K. (2019). Utilising learning analytics for study
success. Reflections on current empirical findings. In D. Ifenthaler, J. Y.-K. Yau, & D.-K.
Mah (Eds.), Utilizing learning analytics to support study success (pp. 27–36). New York,
NY: Springer. https://doi.org/10.1007/978-3-319-64792-0_2
[4] Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of le-
arning analytics in higher education. Computers in Human Behavior. https://doi.org/
10.1016/j.chb.2018.07.027
[5] Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mi-
ning in practice: A systematic literature review of empirical evidence. Journal of Educat-
ional Technology & Society, 17(4), 49-64.
[6] Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., & Rienties,
B. (2016). Research Evidence on the Use of Learning Analytics. Tech. Rep.
[7] ET2020 (2016). Learning Analytics – Key messages. report, European Commission. Avai-
lable online at https://ec.europa.eu/education/sites/education/files/2016-pla-learning- ana-
lytics_en.pdf.
[8] Schön, M., Ebner, M., Kothmeier, G. (2012) It's Just About Learning the Multiplication
Table, In Proceedings of the 2nd International Conference on Learning Analytics and
Knowledge (LAK '12), Simon Buckingham Shum, Dragan Gasevic, and Rebecca Ferguson
(Eds.). ACM, New York, NY, USA, 73-81. https://doi.org/10.1145/2330601.2330624
[9] Ebner, M., Schön, M. (2013) Why Learning Analytics in Primary Education Mat-
ters!,Bulletin of the Technical Committee on Learning Technology, Karagiannidis, C. &
Graf, S (Ed.), Volume 15, Issue 2, April 2013, p. 14-17
[10] Taraghi, B., Ebner, M., Saranti, A., Schön, M. (2014) On Using Markov Chain to Evidence
the Learning Structures and Difficulty Levels of One Digit Multiplication, In: Proceedings
of the Fourth International Conference on Learning Analytics And Knowledge, ACM, New
York, p. 68-72. https://doi.org/10.1145/2567574.2567614
[11] Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative
Learning Analytics Application. In J. Johnston (Ed.), Proceedings of EdMedia: World
Conference on Educational Media and Technology 2017 (pp. 810-820). Association for the
Advancement of Computing in Education (AACE).
[12] Ebner, M., Schön, M., Neuhold, B. (2014). Learning Analytics in basic math education –
first results from the field, eLearning Papers, 36, pp. 24-27
[13] Ebner, M. & Pronegg, M. (2015). Use of Learning Analytics Applications in Mathematics
with Elementary Learners. International Journal of Academic Research in Education, 1(2),
26-39. https://doi.org/10.17985/ijare.37247
[14] Edtstadler, K., Ebner, M., Ebner, M. (2015) Improved German Spelling Acquisition
through Learning Analytics. eLearning Papers, 45, pp. 17-28
22
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
[15] Khalil, M., Ebner, M. (2016). What is Learning Analytics about? A Survey of Different
Methods Used in 2013-2015. Conference proceeding of the 8th e-Learning Excellence
Conference, 2016. Dubai, UAE. pp. 294-304
[16] Khalil, M., Ebner, M. (2016) Clustering patterns of engagement in Massive Open Online
Courses (MOOCs): the use of learning analytics to reveal student categories. Journal of
Computing in Higher Education. Volume 29. pp. 114-132. https://doi.org/10.1007/s12528-
016-9126-9
[17] Khalil, M. & Ebner, M. (2016). Learning Analytics in MOOCs: Can Data Improve Stu-
dents Retention and Learning?. In Proceedings of EdMedia: World Conference on Educa-
tional Media and Technology 2016 (pp. 569-576). Association for the Advancement of
Computing in Education (AACE)
[18] Khalil, M., Ebner, M., & Admiraal, W. (2017). How can Gamification Improve MOOC
Students Engagement?. In proceedings of the European Conference on Game Based
Learning, Graz, Austria, (pp. 819-828)
[19] Ebner, M., Khalil, M., Schön, S., Gütl, C., Aschemann, B., Frei, W., Röthler, D. (2017)
How Inverse Blended Learning Can Turn Up Learning with MOOCs? In: Proceedings of
the International Conference MOOC-MAKER 2017. Antigua Guatemala, Guatemala, No-
vember 16-17, 2017, pp. 21-30
[20] De Laet, T, Broos, T.,van Staalduinen, J.-P., Leitner, P., Ebner, M. (2016) Successful tran-
sition from secondary to higher education using learning analytics. 44th SEFI Conference,
Tampere, Finland
[21] Leitner P., Ebner M. (2017) Development of a Dashboard for Learning Analytics in Higher
Education. In: Zaphiris P., Ioannou A. (eds) Learning and Collabo- ration Technologies.
Technology in Education. LCT 2017. Lecture Notes in Computer Science, vol 10296. pp.
293-301 Springer, Cham. https://doi.org/10.1007/978-3-319-58515-4_23
[22] Khalil, M. & Ebner, M. (2016d) De-Identification in Learning Analytics. Journal of Learn-
ing Analytics. 3(1). pp. 129 – 138. https://doi.org/10.18608/jla.2016.31.8
[23] Leitnerr P., Ebner M., Ebner M. (2019) Learning Analytics Challenges to Overcome in
Higher Education Institutions. In: Ifenthaler D., Mah DK., Yau JK. (eds) Utilizing Learn-
ing Analytics to Support Study Success. Springer, Cham. https://doi.org/10.1007/978-3-
319-64792-0_6
[24] Bruun, J., Ray, P. J., & Udby, L. (2019). Network analyses of student engagement with on-
line textbook problems. arXiv preprint arXiv:1903.11390.
[25] Alstrup, S., Hansen, C., Hansen, C., Hjuler, N., Lorenzen, S., & Pham, N. (2017). DABAI:
A data driven project for e-Learning in Denmark. In P. Peres, & A. Mesquita (Eds.),
ECEL17 - Proceedings of the 16th European Conference on e-Learning (pp. 18-24). Aca-
demic Conferences and Publishing International.
[26] Beuchert, L., Humlum, M. K., Nielsen, H. S., & Smith, N. (2018). The short-term effects
of school consolidation on student achievement: Evidence of disruption?. Economics of
Education Review, 65, 31-47. https://doi.org/10.1016/j.econedurev.2018.05.004
[27] Ringgaard, A. (2019, April 4). Forskning baseret på nationale test kan være smittet med
fejl. Retrieved from https://videnskab.dk/kultur-samfund/forskning-baseret-paa-nationale-
test-kan-vaere-smittet-med-fejl. https://doi.org/10.4220/sykepleienf.2008.0078
[28] Bundsgaard, J. & Kreiner S. (2019). Undersøgelse af de nationale tests måleegenskaber.
2nd edition. Copenhagen: Aarhus Universitet, Danmarks institut for Pædagogik og Uddan-
nelse. Retrieved from: http://edu.au.dk/fileadmin/edu/Udgivelser/Rapporter/Ebog_-
_Nationale_tests_maaleegenskaber.pdf . https://doi.org/10.7146/aul.319.217
[29] Spikol, D., Bruun, J., & Udby, L. Exploring Physics Education in the Classroom and the
Laboratory with Multimodal Learning Analytics. and provide timely personalised feedback
iJAI ‒ Vol. 1, No. 1, 2019
23
Paper—Efforts in Europe for Data-Driven Improvement of Education
to students (Page 10) 6. Khalil, M: The Unbearable Lightness of Consent: An Eye on
Learning Analytics and MOOCs (Page 13) 7. Minard, M. S: Learning Analytics from a
Student Perspective (Page 15), 21. https://doi.org/10.1145/2883851.2883920
[30] Udby, L., Jensen, P., Bruun, J., Willendrup, P., Schober, H., Neuhaus, J., . & Lefmann, K.
(2013). E-learning neutron scattering. Neutron News, 24(1), 18-23. https://doi.org/10.1080/
10448632.2013.751795
[31] Overgaard, J. H., Bruun, J., May, M., & Udby, L. (2017). Virtual neutron scattering exper-
iments-Training and preparing students for large-scale facility experiments. Tidsskriftet
Læring og Medier (LOM), 9(16). https://doi.org/10.7146/lom.v9i16.24231
[32] Bruun, J., Ray, P. J., & Udby, L. (2019). Network analyses of student engagement with on-
line textbook problems. arXiv preprint arXiv:1903.11390.
[33] Kuosa, K., Koro, J., Tervakari, A., Paukkeri, J., & Kailanto, M. (2014). Content analysis
and visualizations: Tools for a social media-enhanced learning environment. In Proceed-
ings of 2014 International Conference on Interactive Collaborative Learning, ICL 2014
(pp. 361–367). https://doi.org/10.1109/icl.2014.7017799
[34] Ahadi, A., Lister, R., Haapala, H., & Vihavainen, A. (2015). Exploring Machine Learning
Methods to Automatically Identify Students in Need of Assistance (pp. 121–130). Univer-
sity of Technology, Sydney, Australia. https://doi.org/10.1145/2787622.2787717
[35] Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., & Sobocinski, M. (2016). How do
types of interaction and phases of self-regulated learning set a stage for collaborative en-
gagement? Learning and Instruction, 43, 39–51. https://doi.org/10.1016/j.learninstruc.
2016.01.005
[36] Järvelä, S., Kirschner, P. A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., … Järve-
noja, H. (2014). Enhancing socially shared regulation in collaborative learning groups: de-
signing for CSCL regulation tools. Educational Technology Research and Development,
63(1), 125–142. https://doi.org/10.1007/s11423-014-9358-1
[37] Järvelä, S., Malmberg, J., Haataja, E., Sobocinski, M., & Kirschner, P. A. (2019). What
multimodal data can tell us about the students’ regulation of their learning process? Learn-
ing and Instruction. https://doi.org/10.1016/j.learninstruc.2019.04.004
[38] Järvenoja, H., Järvelä, S., Törmänen, T., Näykki, P., Malmberg, J., Kurki, K., … Isohätälä,
J. (2018). Capturing motivation and emotion regulation during a learning process. Front-
line Learning Research, 6(3), 85–104. https://doi.org/10.14786/flr.v6i3.369
[39] Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2019). Going
beyond what is visible: What multichannel data can reveal about interaction in the context
of collaborative learning?. Computers in Human Behavior, 96, 235-245. https://doi.org/
10.1016/j.chb.2018.06.030
[40] Ifenthaler, D., & Schumacher, C. (2016a). Learning Analytics im Hochschulkontext. WiSt
- Wirtschaftswissenschaftliches Studium, 4, 172–177. https://doi.org/10.15358/0340-1650-
2016-4-176
[41] Mao, J., Ifenthaler, D., Fujimoto, T., Garavaglia, A., & Rossi, P. G. (2019). National poli-
cies and educational technology: a synopsis of trends and perspectives from five countries.
TechTrends, 63(3), 284–293. https://doi.org/10.1007/s11528-019-00396-0
[42] Ifenthaler, D., & Schumacher, C. (2019). Releasing personal information within learning
analytics systems. In D. G. Sampson, J. M. Spector, D. Ifenthaler, P. Isaias, & S. Sergis
(Eds.), Learning technologies for transforming teaching, learning and assessment at large
scale (pp. 3–18). New York, NY: Springer. https://doi.org/10.1007/978-3-030-15130-0_1
[43] Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not
promote one size fits all: The effects of instructional conditions in predicting academic
24
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
success. Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.
10.002
[44] Schumacher, C., & Ifenthaler, D. (2018a). Features students really expect from learning
analytics. Computers in Human Behavior, 78, 397–407. https://doi.org/10.1016/j.chb.2017.
06.030
[45] Klasen, D., & Ifenthaler, D. (2019). Implementing learning analytics into existing higher
education legacy systems. In D. Ifenthaler, J. Y.-K. Yau, & D.-K. Mah (Eds.), Utilizing
learning analytics to support study success (pp. 61–72). New York, NY: Springer.
https://doi.org/10.1007/978-3-319-64792-0_4
[46] Ifenthaler, D. (2017). Are higher education institutions prepared for learning analytics?
TechTrends, 61(4), 366–371. https://doi.org/10.1007/s11528-016-0154-0
[47] Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V., Velloso, E. (2019). Multi-
modaldata as a means to understand the learning experience. International Journal ofIn-
formation Management, 48, 108–119 https://doi.org/10.1016/j.ijinfomgt.2019.02.003
[48] Giannakos, M., Sharma, K., Martinez-Maldonado, R., Dillenbourg, P., & Rogers, Y.
(2018). Learner-computer interaction. In Proceedings of the 10th Nordic Conference on
Human-Computer Interaction (pp. 968-971). ACM. https://doi.org/10.1145/3240167.324
0259
[49] Sharma, K., Papamitsiou, Z., Giannakos, M.N. (2019). Building Pipelines for Educa-
tionalData: Using AI and Multimodal Analytics to Explain Learning in Adaptive Self-
Assessment. British Journal of Educational Technology (to appear)
[50] Papamitsiou, Z., Pappas, I.O., Sharma, K., Giannakos, M.N. (2019). Utilizing multimodal-
data through an fsQCA approach to explain engagement in adaptive learning. IEEE Trans-
actions on Learning Technology (to appear)
[51] Wong, J., Khalil, M., Baars, M., de Koning, B. B., & Paas, F. (2019a). Exploring sequenc-
es of learner activities in relation to self-regulated learning in a massive open online
course. Computers & Education, 103595. https://doi.org/10.1016/j.compedu.2019.1
03595
[52] Wong, J., Baars, M., de Koning, B. B., van der Zee, T., Davis, D., Khalil, M., . & Paas, F.
(2019b). Educational Theories and Learning Analytics: From Data to Knowledge. In Uti-
lizing Learning Analytics to Support Study Success (pp. 3-25). Springer, Cham.
https://doi.org/10.1007/978-3-319-64792-0_1
[53] Slade, S., Prinsloo, P., & Khalil, M. (2019). Learning analytics at the intersections of stu-
dent trust, disclosure and benefit. In Proceedings of the 9th International Conference on
Learning Analytics & Knowledge (pp. 235-244). ACM. https://doi.org/10.1145/3303772.
3303796
[54] Hernández-García, Á., and Conde, M.Á.: ‘Learning analytics: The good, the bad (and the
ugly)’. Proc. of the 6th International Conference on Technological Ecosystems for Enhan-
cing Multiculturality, Salamanca, Spain2018, pp. 290-293. https://doi.org/10.1145/
3284179.3284229
[55] Conde, M.Á., and Hernández-García, Á.: ‘Learning analytics: Expanding the frontier’.
Proc. of the 5th International Conference on Technological Ecosystems for Enhancing
Multiculturality, Cádiz, Spain2017 pp. 1-5. https://doi.org/10.1145/3144826.3145386
[56] Hernández-García, Á., and Conde, M.Á.: ‘Learning analytics: needs and opportunities’.
Proc. of the 4th International Conference on Technological Ecosystems for Enhancing
Multiculturality, Salamanca, Spain, 2016, pp.309-312. https://doi.org/10.1145/3012430.3
012533
[57] Conde, M.Á., Hernández-García, Á., and Oliveira, A.: ‘Endless horizons?: addressing
current concerns about learning analytics’. Proc. of the 3rd International Conference on
iJAI ‒ Vol. 1, No. 1, 2019
25
Paper—Efforts in Europe for Data-Driven Improvement of Education
Technological Ecosystems for Enhancing Multiculturality, Porto, Portugal, 2015, pp. 259-
262. https://doi.org/10.1145/2808580.2808619
[58] Hernández-García, Á., and Conde-González, M.Á.: ‘Dealing with complexity: educational
data and tools for learning analytics’. Proc. of the 2nd International Conference on Techno-
logical Ecosystems for Enhancing Multiculturality, Salamanca, Spain, 2014, pp. 263-268.
https://doi.org/10.1145/2669711.2669909
[59] Conde, M.A., and Hernández-García, A.: ‘A promised land for educational decision-
making?: present and future of learning analytics’. Proc. of the 1st International Confe-
rence on Technological Ecosystem for Enhancing Multiculturality, Salamanca, Spain,
2013 pp. 239-243. https://doi.org/10.1145/2536536.2536573
[60] Saqr, M., Fors, U., Nouri, J. (2018). Using Social Network Analysis to Understand Online
Problem Based Learning and Predict Performance. PLoS One. https://doi.org/10.1371/
journal.pone.0203590
[61] Karunaratne T., and Byungura J. (2017). Using log data of virtual learning environments to
examine the effectiveness of online learning for teacher education in Rwanda, In Procee-
dings of the 2017 International conference IST-Africa, 31 May – 02 June 2017, Windhoek,
Namibia. https://doi.org/10.23919/istafrica.2017.8102410
[62] Saqr, M., Fors, U., Tedre, M. & Nouri, J. (2018b). How Social Network Analysis Can Be
Used to Monitor Online Collaborative Learning and Guide an Informed Intervention. PLoS
One. https://doi.org/10.1371/journal.pone.0194777
[63] Saqr, M., Nouri, J. Fors, U. (2018). Time to focus on the temporal dimension of learning.
A learning analytics study of the temporal patterns of students’ interactions and self-
regulation. International Journal of Technology Enhanced Learning. https://doi.org/10.15
04/ijtel.2019.10020597
[64] Nouri, J., Saqr, M & Fors, U. (2019a). Predicting performance of students in a flipped
classroom using machine learning: towards automated data-driven formative feedback.
10th International Conference on Education, Training and Informatics (ICETI 2019).
[65] Nouri, J., Larsson, K., & Saqr, M. (2019). Identifying factors for master thesis completion
and non-completion through learning analytics and machine learning. In proceeding of
14th European Conference on Technology Enhanced Learning (EC-TEL).
[66] Spikol, D., Bruun, J., & Udby, L. Exploring Physics Education in the Classroom and the
Laboratory with Multimodal Learning Analytics and provide timely personalised feedback
to students. LASI 2018.
[67] Bälter, O., Zimmaro, D., & Thille, C. (2018). Estimating the minimum number of op-
portunities needed for all students to achieve predicted mastery. Smart Learning Environ-
ments, 5(1), 15. https://doi.org/10.1186/s40561-018-0064-z
[68] Nouri, J. Åkerfeldt, A., Fors, U., Selander, S. (2017). Assessing collaborative problem
solving skills in technology-enhanced learning environments – the PISA framework and
modes of communication. International Journal of Emerging Technologies in Learning.
[S.l.], v. 12, n. 04, p. pp. 163-174. https://doi.org/10.3991/ijet.v12i04.6737
[69] Nouri, J., Zhang, L., Mannila, L.& Norén, E. (2019). Development of computational think-
ing, digital competence and 21st century skills when learning programming in K-9. Educa-
tion Inquiry. https://doi.org/10.1080/20004508.2019.1627844
26
http://www.i-jai.org
Paper—Efforts in Europe for Data-Driven Improvement of Education
5 Authors
Jalal Nouri is with Stockholm University, Sweden
Martin Ebner is with Graz University of Technology, Austria
Dirk Ifenthaler is with Universität Mannheim, Germany.
Mohammed Saqr is with University of Eastern Finland, Finland.
Jonna Malmberg is with University of Oulu, Finland.
Mohammad Khalil is with University of Bergen, Norway.
Jesper Bruun is with University of Copenhagen, Denmark.
Olga Viberg is with Royal Institute of Technology, Sweden.
Miguel Ángel Conde González is with Universidad de León, Spain.
Zacharoula Papamitsiou is with Norwegian University of Science and Technolo-
gy, Norway.
Ulf Dalvad Berthelsen is with Aarhus University, Denmark.
Article submitted 2019-06-18. Resubmitted 2019-07-24. Final acceptance 2019-07-26. Final version
published as submitted by the authors.
iJAI ‒ Vol. 1, No. 1, 2019
27
