ArticlePDF Available


Information and communication technologies are increasingly mediating learning and teaching practices as well as how educational institutions are handling their administrative work. As such, students and teachers are leaving large amounts of digital footprints and traces in various educational apps and learning management platforms, and educational administrators register various processes and outcomes in digital administrative systems. It is against such a background we in recent years have seen the emergence of the fast-growing and multi-disciplinary field of learning analytics. In this paper, we examine the research efforts that have been conducted in the field of learning analytics in Austria, Finland, Norway, Germany, Spain, and Sweden. More specifically, we report on developed national policies, infrastructures and competence centers, as well as major research projects and developed research strands within the selected countries. The main conclusions of this paper are that the work of researchers around Europe has not led to national adoption or European level strategies for learning analytics. Furthermore, most countries have not established national policies for learners’ data or guidelines that govern the ethical usage of data in research or education. We also conclude that learning analytics research on the pre-university level to a high extent have been overlooked. In the same vein, learning analytics has not received enough focus form national and European national bodies. Such funding is necessary for taking steps towards data-driven development of education.
PaperEfforts in Europe for Data-Driven Improvement of Education
Efforts in Europe for Data-Driven
Improvement of Education
A Review of Learning Analytics Research in Seven Countries
Jalal Nouri (*)
Stockholm University, Stockholm, Sweden
Martin Ebner
Graz University of Technology, Styria, Austria
Dirk Ifenthaler
Universität Mannheim, Mannheim, Germany
Mohammed Saqr
University of Eastern Finland, Joensuu & Kuopio, Finland
Jonna Malmberg
University of Oulu, Oulu, Finland
Mohammad Khalil
University of Bergen, Bergen, Norway
Jesper Bruun
University of Copenhagen, Scandinavia, Denmark
Olga Viberg
Royal Institute of Technology, Stockholm, Sweden
Miguel Ángel Conde González
Universidad de León, Leon, Spain
Zacharoula Papamitsiou
Norwegian University of Science and Technology, Trondheim, Norway
Ulf Dalvad Berthelsen
Aarhus University, Aarhus C, Denmark
AbstractInformation and communication technologies are increasingly
mediating learning and teaching practices as well as how educational institu-
tions are handling their administrative work. As such, students and teachers are
leaving large amounts of digital footprints and traces in various educational
apps and learning management platforms, and educational administrators regis-
To cite: Nouri, J., Ebner, M., Ifenthaler, D., Saqr, M., Malmberg, J., Khalil, M., Viberg, O., Bruun, J., Conde-Gonzalez, M., Papamitsiou, Z. & Berthelsen, U. (2019).
Efforts in Europe for Data-Driven Improvement of Education – A review of learning analytics research in seven countries.
International Journal of Learning Analytics and Artificial Intelligence for Education. 1(1).
PaperEfforts in Europe for Data-Driven Improvement of Education
ter various processes and outcomes in digital administrative systems. It is
against such a background we in recent years have seen the emergence of the
fast-growing and multi-disciplinary field of learning analytics. In this paper, we
examine the research efforts that have been conducted in the field of learning
analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and Sweden.
More specifically, we report on developed national policies, infrastructures and
competence centers, as well as major research projects and developed research
strands within the selected countries. The main conclusions of this paper are
that the work of researchers around Europe has not led to national adoption or
European level strategies for learning analytics. Furthermore, most countries
have not established national policies for learners’ data or guidelines that gov-
ern the ethical usage of data in research or education. We also conclude, that
learning analytics research on pre-university level to high extent have been
overlooked. In the same vein, learning analytics has not received enough focus
form national and European national bodies. Such funding is necessary for tak-
ing steps towards data-driven development of education.
KeywordsLearning analytics, Europe, data-driven improvement, education
1 Introduction
Over the past decades, the world has undergone a transformation process, which
many consider to be as important as the Industrial Revolution once. In this post-
industrial society, also called the information and knowledge society, information
technology plays a crucial role. It permeates and transforms how we work, study,
relate to information and knowledge and how we spend our free time. As a conse-
quence of this digitization, huge quantities of data, i.e. big data, is generated that re-
flects our activities. Therefore, in many fields, such as business or medicine, we have
witnessed how essential the use of analytics has become to process generated big data
in order to develop data-driven insights into people’s activities for the optimization of
processes and outputs.
Today, the educational systems around the world are also undergoing major digital
transformations. Information and communication technologies are increasingly medi-
ating learning and teaching practices as well as how educational institutions are han-
dling their administrative work. As such, students and teachers are leaving large
amounts of digital footprints and traces in various educational apps and learning man-
agement platforms, and educational administrators register various processes and
outcomes in digital administrative systems.
It is against such a background we in recent years have seen the emergence of the
fast-growing and multi-disciplinary field of learning analytics. The field, which origi-
nates from disciplines such as business intelligence, web analytics, educational data
mining and recommender systems[1, p. 1) attempts to exploit data generated in edu-
cational settings “for purposes of understanding and optimizing learning and the
environments in which it occurs[2, p. 34].
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
Although the field of learning analytics is still in its infancy, seen from an interna-
tional perspective, it has already produced innovative educational research that
demonstrate the utility of learning analytics on the micro level (understanding and
developing learning and teaching), on the meso level (understanding and developing
single educational organizations), and on the macro level (understanding and develop-
ing on a national level or across educational organizations) [3, 4, 5].
From a European standpoint, the potentials of learning analytics were recognized
early on. Already in 2013, only two years after the official birth of the field of learn-
ing analytics, the European Commission emphasized that learning analytics can con-
tribute to develop new solutions for better personalised learning, by allowing teach-
ers to have a more accurate and up-to-date follow up of each learner. Through learn-
ing analytics, new and more learner-centred teaching methods can emerge since the
evolution of learners who use ICT regularly can be closely monitored.” [6, p.5). In
another more recent report the European Commission Working Group on Digital
Skills and Competences (ET2020) once again pointed to the potential of learning
analytics to “contribute to the quality of teaching and learning and the modernization
of educational systems in Europe.” [7, p.2]. Moreover, ET2020 urged for capacity
building in the field and collaborative research projects. And indeed, research in
learning analytics is growing in Europe, especially from countries such as Spain,
United Kingdom, Germany, Netherlands, and Austria. Furthermore, several European
countries, such as Norway, Denmark, and Netherlands, are developing nationwide
learning analytics strategies that includes infrastructure, competence centers, and
national policies.
In this paper, we examine the research efforts that have been conducted in the field
of learning analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and
Sweden. More specifically, we report on developed national policies, infrastructures
and competence centers, as well as major research projects and developed research
strands within the selected countries.
2 European Research
2.1 Austria
The research efforts in Austria has been started already back in 2011
. Based on
personal meetings between George Siemens, Erik Duval and Martin Ebner at the ED-
Media conference in Lisbon, Portugal, the idea of a conference on Big Data for Learn-
ing was shared to a large community - today, well known as International Conference
Learning Analytics and Knowledge, shortly LAK. The second was done in Vancou-
ver, Canada in April 2012
. At this conference, a first research work from Austria was
presented and discussed - just a simple multiplication trainer for schoolchildren aged
8-12 years [8].
1 (last
visited June 2019)
2 (last visited in June 2019)
PaperEfforts in Europe for Data-Driven Improvement of Education
The research team followed the idea to collect and gather all calculations done by
the children and to give feedback to the learners as well as the teachers. Today the
application holds more than 1.000.000 calculations and we know very precisely how
the learning of the multiplication table is happening described in several publications
[9] [10] [11].
In addition, the research team worked on more applications for school children - an
addition, a subtraction [12], a division and a multi-digit trainer [13]. As follow up,
Graz University of Technology contributed to a first European project about German
spelling acquisition [14]. The project aims to offer children an online editor for writ-
ing short essay. In the background beside a typical spell-checking dictionary an intel-
ligent one was implemented. This developed one holds words, written in all possible
and false forms, categorized in different groups. The Learning Analytics part analyses
each text and provide feedback to learners and teachers divided to the defined catego-
Beside this project in secondary education in 2014 first research has been done in
higher and adult education. The University of Graz as well as the University of Tech-
nology of Graz founded in 2014 the first and till now online MOOC platform in Aus-
tria, called iMooX. Due to the fact that MOOCs are addressing a huge amount of
learner’s data driven investigations seems to be a logical step firstly described in de-
tail in [15]. Different studies pointed out how Learning Analytics can help to identify
different kind of learners [16], how students remain in MOOCs [17], how gamifica-
tion elements assist the learning process [18] and even how new didactical approach-
es, called Inverse Blended Learning, are introduced [19].
Another joint project on European level between KU Leuven, University Notting-
ham, TU Delft and TU Graz called STELA (“Successful Transition from secondary to
higher Education using Learning Analytics”) aimed to assist students during their
transition phase from secondary to higher education [20]. The outcome of the project
provided a general framework for building students’ dashboards [21] (Leitner & Eb-
ner, 2016) and different prototypes at each single university.
Finally, there are also some work done on a policy level for national issues. Due to
the fact that in Austria the data protection law is rather strong, it is from high im-
portance to think about how Learning Analytics can be integrated on an institutional
level. One first research work was about the de-identification of data [22] and general
challenges to overcome if Learning Analytics will be introduced in Higher education
institutions [23].
Currently a white paper on Learning Analytics for Higher Education is elaborated
under the lead of the nationwide association of new media for teaching and learning.
Finally, the ministry of education, science and research announced to give financial
support for Learning Analytics applications in the next years.
2.2 Denmark
Learning Analytics as a field in Denmark seems rather as disconnected islands than
as a connected whole. While research efforts within the scope of LA as defined in this
article have been done for some time, research has until recently been conducted at
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
separate universities and university colleges without much interaction and collabora-
tion. Furthermore, the two overall aims for LA, understanding and optimizing seem to
be a source of divisions in the Danish landscape. Some researchers focus on under-
standing from an educational standpoint [24], others on optimizing from a computer
science standpoint [25]. In addition to this division, a third division seems to be be-
tween econometrics and educational research, where econometrics as a field is more
interested in the effects of intervention and use educational test-data as outcome vari-
ables [26].
Also, worth mentioning, though not as such LA research, is a strand of research
critically monitoring the consequences of the digital transformation of the Danish
educational system, among other things, focusing on the consequences of integrating
analytics into the ecosystem of public primary education.
Apart from divisions in terms of research focus, the divisions between educational
research and econometrics is evidenced by public debates [27] in the wake of public
reports on Danish national tests [28]. We speculate that the origin of this divide is
determined by the broad field of the researcher; computer scientists engaging in LA
are more likely to focus on optimizing, while educational researchers are more likely
to link LA data to e.g. constructivist theories of learning. Econometrics seems to be
narrowly focused on establishing causal relationships using linear models. Thus, we
see a gap between (at least) three traditions: a computer scientist tradition, an econo-
metrics tradition, and an educational tradition. Here, we provide examples to illustrate
the LA landscape in Denmark and then present current or very recent efforts to begin
bridging the gap.
The Danish Center for Big Data Analytics Driven Innovation (DABAI), was estab-
lished in 2016, with learning analytics as one of the goals. Within the field, DABAI is
to pursue optimization of e-learning personalization, student behavior modeling, pre-
dicting student performance, similarity among quizzes, authorship verification, and
curriculum trainer [25]. With regards to student behavior modeling, researchers have
created a model of student drop-outs in Danish upper secondary school.
Another avenue of research in play is a design-based approach. Here, educational
researchers design online and blended learning materials for science courses while at
the same time monitoring clickstreams, videotaping lessons, and audiotaping student
discussions for joint multimodal analyses [29]. On such project is the Virtual Neu-
trons for Teaching project (, in which students learn neutron scattering
via online textbooks and quizzes [30] [31]. One of the outcomes of the projects is a
novel method for analyzing online student behaviors using clickstream data [32].
Two large ongoing projects, both funded by Innovation Fund Denmark (Innova-
tionsfonden), are also worth mentioning. The first, Game-based Leaning in the 21st
Century (GBL21), is a large collaborative project aimed at developing design thinking
skills through game-based learning ( As a part of GBL21 educa-
tional researchers from DPU, Aarhus University and Aalborg University are develop-
ing an online tool for assessing different aspects of design thinking skills. The other
project, Automatically Tracking Early Stage Literacy Skills (ATEL), involves re-
searchers from DPU, Aarhus University and Technical University of Denmark (DTU)
PaperEfforts in Europe for Data-Driven Improvement of Education
collaborating on developing analytics tools for tracking early stage literacy develop-
ment (
Furthermore, in 2010, Denmark implemented national tests in grades 2-4 as well as
6-8 in primary school. The tests are obligatory and target different subjects in differ-
ent grades (
ogproever/nationale-test/klassetrin-fag-og-profilomraader). The tests are adaptive, meaning
that each student will be presented with test items during the test-period (usually 1
hour in-class) and that the difficulty of test item i+1 is dependent on the student an-
swer to test item i. The system collects data on the students, which is used to monitor,
single students, classes, and schools. While in-depth analyses are possible, research
seems to have focused mainly on macro-scale variables, such as whether there is a
positive or negative effect of the national tests.
We have argued that in Denmark, a gap between LA as undertaken by computer
scientists, econometrics researchers, and educational researchers exists. We believe
that in order to bridge this gap, computer scientists will need to learn to operationalize
current educational theories and results, while educational researchers need to utilize
and interpret results from currently used computer algorithms. The gap between edu-
cational researchers and econometrics researchers seems to be rooted in substantially
different aims and methods for the two fields. Bridging all of these gaps will require
extensive cooperation and probably compromises.
Despite the lack of coherence and collaboration in the Danish LA research com-
munity, there are, however, efforts to connect to the larger Nordic cross-disciplinary
community of LA researchers, e.g. by hosting the Nordic LASI 2018 at Aalborg Uni-
versity and organizing Learning Analytics Research Symposium (LARS), which was
held at University of Copenhagen in November 2018 ( )
2.3 Finland
Research on learning analytics started early in Finland with a focus on the social
and collaborative aspects of learning. Tervakari and colleagues investigated the “TUT
circle” which was an online social media enhanced learning platform at Tampere
University of Technology. They reported on the utility of visualization of students’
interactions, and researched the learning analytics potentials of the platform. The
group later contributed with research on content analytics, social media analytics and
teacher tools [33]. Other aspects of learning analytics followed such as predicting
students’ performance in programming courses and using machine learning methods
to predict students’ need for assistance [34]. A notable body of research comes from
Järvelä and colleagues on self-regulated and collaborative learning, who investigated
social shared regulation, interactions and engagement with collaborative learning and
the temporal sequence of regulatory processes [35] [36]. The group are also working
on multimodal physiological data as well as dispositional learning analytics [37] [38].
As the field is gaining recognition, many Finnish universities are now embracing the
concept and research is increasingly reported from most Finnish universities.
Despite the development of techniques and methods to model and predict human
learning, the field still lacks ability to connect the powers of learning sciences and
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
learning analytics in effective way to understand the complexity of learning alone or
together, reveal hidden human mental processes and model and trace trigger moments
and critical patterns of learning processes [37]. The emerging need to combine the
power of learning sciences and learning analytics is also recognized by Finish Science
Academy, which funds high-quality scientific research and strengthens the position of
science and research. For example, in spring 2019 Academy of Finland launched a
project call “Digital Humanities” which reflects the emerging need to develop new
methodologies. Especially Digital Humanities program emphasizes new ways to ad-
dress novel methods and techniques in which digital technology and state-of-the-art
computational science methods are used for collecting, managing and analyzing data
in humanities and social sciences research as well as for modelling humanities and
social science phenomena. Thus, the emphasis lies not only in collecting "big data",
but also "small" (deep, rich) data, since so far, the potential of many overarching con-
ceptual and methodological questions remain unexplored and under-theorized.
The large body of empirical and theoretical advances in the field of self-regulated
learning (SRL) [37] has indisputable evidence that such skills improve learning with
students of all ages. However, there has been much less understanding how learning
analytics are grounded in the literature on self-regulated learning and how self-
regulated learning is supported. This is much due the fact, that SRL processes (i.e.
cognitive, metacognitive, motivational and emotional) are invisible for naked eye and
therefore difficult to capture [37]. Fortunately, due the technological and methodolog-
ical advancements in the field, there is potential to transform these mental processes at
least to some extent in a visible form to provide learning analytics for teacher and
During the past years, there has been increasing interest to collect and analyze mul-
timodal data (i.e. log data, physiological data, situated self-reports) to better capture
the mental processes of human learning [37]. For example, [39] applied multimodal
data (e.g. physiological data, facial expression data and video data) evidencing that it
is possible to make situational characteristic involving to the regulated learning pro-
cess visible. Facial expression recognition has potential to reveal valence of emotions
during collaborative learning. Visible interactions recorded from the video data has
potential to reveal type of interaction, but also instances when students engage for
self-regulated learning.
2.4 Germany
In 2016, Ifenthaler and Schumacher [40] report that research on learning analytics
in Germany is scarce and that there are only a few projects focusing on the implemen-
tation of learning analytics systems. In 2019, several research projects are being fund-
ed by the German Federal Ministry of Education and Research focusing on technolo-
gy integration and analytics in educational organizations [41]. For example, the aim of
the project ‘Utilizing Learning Analytics for Study Success’ is to conduct a systematic
review and construct a set of policies for German higher education institutions to
adopt learning analytics capabilities into their existing learning environments. Precise-
ly, the goals of the project are a) first to build a systematic review of empirical evi-
PaperEfforts in Europe for Data-Driven Improvement of Education
dence demonstrating how learning analytics have been successful in facilitating stu-
dent success in continuation and completion of their university courses both nationally
and internationally, and forming the basis for aim b) to make policy recommendations
for the German higher education sector in order to accept and implement such systems
within institutions.
It became evident from the integrative review that robust empirical findings on a
large scale to support the effectiveness of learning analytics actually retaining students
onto courses are still lacking [42]. Therefore, it is imperative to leverage existing
learning theory, psychological methods and connecting them to advances of learning
analytics research for designing (quasi-)experimental studies including theoretical
frameworks and sound empirical methodologies. The project findings of the interview
study indicate that more work on ethical and privacy guidelines supporting a wider
adoption of learning analytics systems is needed [42] as well as work towards a stand-
ardized learning analytics system which can be integrated into any learning environ-
ment providing reliable at-risk student prediction, prevention and intervention strate-
gies [43]. In particular, personalized learning environments are increasingly demand-
ed and valued in education institutions to create a tailored learning package optimized
for each individual learner based on their personal profile which could contain infor-
mation such as their geo-social demographic backgrounds, their previous qualifica-
tions, how they engaged in the recruitment journey, their activities on social media
and websites, as well as tracking information on their searches [44].
Additional findings document issues with organizational readiness (Ifenthaler,
2017). For example, a standard infrastructure of educational institutions includes a
student management system, a learning management system and a course manage-
ment system. However, these systems are deeply embedded into the organization’s
infrastructure and often are not designed to reveal data for analytics [45]. For access-
ing the necessary data, various connections to the organizations’ legacy systems have
to be established which are able to access the students’ profiles, to capture the actual
learning processes and get access to curricula data. As these legacy systems are often
based on various technologies, each connection has to be implemented as an individu-
al project which is labor and cost intensive. Besides the technological challenges, staff
capabilities are also changing when implementing learning analytics systems. Not
only new staff roles but also further development of existing staff is required for suc-
cessful implementation of learning analytics systems [46].
2.5 Norway
Although the collection, interpretation, and visualization of multimodal data has
been extremely challenging for researchers, recent technological developments and
data science, and AI advancements have boosted the growth of non-invasive high-
frequency multimodal-data collections.
Learners’ traces are generated during their interaction with technologies, such in-
teraction is often complex but offers opportunities for collecting rich and multimodal
data [47] [48], MultiModal Learning Analytics (MMLA), as the literature refers to
them. In order to unfold the benefits of MMLA, the Learner-Computer Interaction
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
(LCI) lab at the Norwegian University of Science and Technology (NTNU) focuses
on overcoming the difficulties in gathering and making sense of MMLA. In other
words, we attempt to identify, how insights generated during learner-computer inter-
action help us to design future learning environments and improve the learning expe-
For many years, the design of learning technologies has been utilizing click-
streams and keystrokes as the primary data source for modelling and predicting learn-
ing behavior. In recent work at NTNU researchers set out to quantify what, if any,
advantages do physiological sensing techniques provide for the design of learning
technologies [47] in a lab context with 251 game sessions and 17 users focusing on
skill development (i.e., user's ability to master complex tasks).
Furthermore, when dealing with data channels in multiple modes and modalities, a
major issue lies in determining combinations of the multimodal data channels that are
necessary for one to make valid and reliable inferences regarding the temporally un-
folding learning processes, and selecting the algorithms and analytical tools to use.
Recent research at NTNU has proposed a novel approach, called “grey-box” ap-
proach, that bridges the hypothesis/literature-driven (measurements/feature selection)
“white-box” approach with the computation-driven (feature fusion) “black-box” ap-
proach [49]. The authors aimed to extend current methodological paradigms in under-
standing effortful behavior and learning performance in adaptive learning conditions
with new, cutting-edge, interdisciplinary work on building pipelines for educational
data, using innovative tools and techniques
Another research dimension explored at NTNU with regard to multimodal learn-
ing analytics is the modelling of learner behavior, by taking advantage of the inherent
temporality in the physiological data. The Generalized Auto-Regressive Conditional
Heteroskedasticity (GARCH) method was applied with learners’ physiological time-
series data to model their behavior, and make suggestions about how the models can
be further utilized to provide proactive feedback to learners [49].
Moreover, investigating and explaining the patterns of learners’ engagement in
adaptive learning conditions is a core issue towards improving the quality of personal-
ized learning services. The research group at NTNU bridged complexity theory with
multimodal data in order to capture specific patterns of engagement that foretell and
explain learners’ level of performance on adaptive learning procedures [50].
In a different context, the joint collaboration between the Centre for the Science of
Learning & Technology (SLATE) from University of Bergen in Norway and Erasmus
University Rotterdam from the Netherlands gave birth to interesting research studies
in MOOCs. The first one was on exploring self-regulated learning in Coursera plat-
form by [51]. Wong et al [51] employed sequence pattern mining to identify self-
regulated learning studying strategies to a group in MOOCs where student was of-
fered self-reflection and monitoring intervention. One more contribution between
SLATE and the Dutch university is a book chapter titled Educational Theories and
Learning Analytics: From Data to Knowledge” by [52]. The chapter aimed at discuss-
ing how learning theories and learning analytics are important components of educa-
tional research. In addition, the chapter suggests that more experimental studies are
needed for applied learning analytics in general and in Europe more specifically.
PaperEfforts in Europe for Data-Driven Improvement of Education
In Norway as well, there has been research on ethical aspects of learning analytics.
For instance [53] carried out a research study at the Open University in the United
Kingdom in which they examined student’s behavior in higher education and their
attitudes to privacy. The authors plan to carry out the same research study in Norway
next year together with South Africa. Furthermore, the study by [53] looked at the
three levels of consent in MOOCs, micro, meso, and macro. Based on reviewing the
policies of the biggest four MOOC providers, the paper proposes a need for greater
transparency around the implications of users granting consent at the point of registra-
Another interesting project brought from Norway are those related to the medical
and health sector. SLATE from the University of Bergen is involved in a consortium
project called OERBioMed. Biomedicine seeks to explain physiological processes at
the molecular and individual level. Such information is essential for the understanding
of disease progression and for the development of new treatments and therapies. This
current and future medical research relies on the existence of people with expertise in
the biomedical field. To raise the quality of teaching, training and learning within the
field of biomedicine, new and innovative approaches are required. To this end, this
project deals with open-access and online courses to increase the bioethical
knowledge and awareness in the biomedical community. SLATE is involved in this
project by providing help and support to launch massive open online courses in col-
laboration with all the partners from the Nordic countries. SLATE also provide learn-
ing analytics and statistical services to the partners.
2.6 Spain
Data driven education has been a hot topic in Spain in the last 10 years. In fact,
there are several relevant works in the field of Learning Analytics, Visual Analytics,
Educational Data Mining, Multimodal Analytics, etc. However, there are some issues
in the Spanish landscape related with Data Driven Education, issues that are also
common in other countries. These are:
Data driven education is a relatively new research field, and therefore the quantity
and variety of topics of interest is high
The current level of global or campus-wide application of learning analytics in
companies and public and private organizations is low
The fragmentation of research groups, and the difficulty they have to reuse and
replicate research designs, results and outcomes of others
The focus of disciplines such as learning analytics has been mostly technical and it
is necessary a multidisciplinary perspective that involves also profiles such as edu-
cators, psychologist or sociologist
It is also desirable to engage different organizations in the use of disciplines such
as Learning Analytics (companies, public administrators, non-higher education in-
stitutions, etc.)
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
The low number of professionals in this field, with the emergence of the new data
scientists this is close to be solved, but it is still needed a very specific profile in the
educational field
The knowledge that research groups have about themselves, ongoing projects,
institutions and organizations interested in Data Driven Education is limited.
In order address these problems and centralize the efforts carried out related to
these topics SNOLA was defined. SNOLA (Spanish Network Of Learning Analytics,, informally created in 2013, emerges as a network composed of the
main national researchers in the LA field, comprising 10 researchers from 9 Spanish
Research Groups. SNOLA primarily has a technical approach, but it also integrates
other educational visions and disciplines that give the network a wider scope. The
main objectives of SNOLA are:
The promotion of collaboration among the participants in the Network, as well as
with other interested parties (companies and public and private educational institu-
tions) and other European and international LA collaboration networks
The diffusion and organization of LA initiatives at a national scale
Making resources to effectively integrate LA in educational processes available to
the public; and 4) provide researchers and professionals adequate training to face
and give answer to the new challenges of Digital Society risen by the incorporation
of ICT in education. From December of 2015 to June of 2018, SNOLA was grant-
ed by the Spanish Government as Thematic Network which help the network to
support different events and activities.
At the same time that SNOLA was defined several projects have been developed
with Spanish institutions related with Data Driven Education. Some are international
projects (mostly European) such as: LACE Project (,
SHEILA Project (, Make World (, Go-Lab
( or LALA Project (; some other are
granted by the Spanish Government through national calls such as the projects EEE or
RESET; or by Regional Governments such as eMadrid. These projects deal with dif-
ferent issues and in some of them Data Driven Education is the main topic and in
other it is employed to support the results, processes or products developed. Some of
topics explored could be the teamwork assessment, intelligent tutors that make sug-
gestions depending on learning evidences or the use of Learning Analytics in CSCL.
Regarding the scientific events it should be noted that exist several relevant initia-
The Learning Analytics Summer Institute Spain, linked to the SOLAR intiative. It
is a conference supported by SNOLA with several editions. First Spanish Edition was
hold in 2013 at Granada (, 2014 at Ma-
drid (, 2015 ( and
2016 ( at Bilbao, 2017 at Madrid ( and 2018
at León ( This year will take place in Vigo. It began as an event
for discussion between researchers in the field of Learning Analytics and also as a
PaperEfforts in Europe for Data-Driven Improvement of Education
contact point with experts. However now it includes also sessions with scientific pa-
pers presentations, networking sessions, companies’ sessions, etc.
The Learning Analytics Track included in the TEEM Conference, an international
scientific conference defined in Spain in 2013. The track is leaded by Spanish re-
searchers but includes works from people all around the world. It began in 2013 and
since then this track has taken place every year. During the track several scientific
papers are briefly presented and discussed with experts. Some of this track editions
have been associated to special issues in journals such as Computers in Human Be-
havior [2] or the International Journal of Engineering Education [3]. Several topics
have been discussed during the track, some of the most significant could be: Predic-
tion of students success or failure based in their learning and interaction evidences;
tools to improve learning process, Learning Analytics and Mobile devices, Ethics
about Learning Analytics, Visual Learning Analytics, Academic Analytics, Multi-
modal Learning Analytics, Social Network Analytics, Competence Assessment
through Learning Analytics, Discussion about the quantity and quality of data to make
decisions in educational contexts, Application of Learning Analytics tools and tech-
niques, Personalization of Learning by using Learning Analytics, etc. More infor-
mation about this track and the research woks included in it can be found here [54-
Other seminars such as LAIKA, SIIE 2016, WPLA at ECTEL, LATCEE (in
Educon) were also leaded by Spanish research groups and deals with similar topics as
the previous ones.
It should be noted that in conferences such as LASI Spain companies were in-
volved. In this way it was possible to know what were they doing about Learning
Analytics and what they require from the academy. Companies such IBM, Euskaltel,
Sun Edison, Brambles or Eticas Consulting participate in several LASI Conference
and enrich the perspective about Data Driven Education in Spain.
2.7 Sweden
Even though the field of learning analytics is an evolving field of both research and
practice [4], there have already been some relevant efforts in terms of its development
in a Swedish context. These attempts are currently expanding in higher educational
settings. For instance, a number of research directions have been explored with a
learning analytics approach by a research group at Stockholm University, focusing on
aspects such as problem-based learning [60], teacher education [61], collaborative
learning [62], self-regulated behavior [63], prediction of student performance in
blended learning and in flipped classroom settings [64], and prediction of perfor-
mance and completion of master- and bachelor thesis [65].
Other examples of research conducted by Swedish researchers include the explora-
tion of multimodal learning analytics [66], and prediction of students’ mastery of
skills [67].
The efforts in K-12 education are in particular relevant to an ongoing digitalization
of the Swedish education system. One of the recent developments in this regard in-
clude: i) the Swedish government decision for digitalization of the schools, with a
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
supplement that presents a national strategy for this (and ii) National action plan for
digitization of the Swedish school system. Both of these documents highlight a need
for a strategic and systematic data collection for increased opportunities for follow-up
of the school system's digitalization, with the main purpose to increase the availability
of comparable data that makes it possible to study the connection between methods
and results. However, these endeavors are still in their infancy. Thus, the present re-
port illustrates the state of the art that largely relates to adult and particularly, higher
We are only aware of one project at the K-12 level, which focused on 21st century
skills and collaborative problem-solving of students in secondary school [68]. In this
project, which was funded by Swedish Association of Local Authorities and Regions
(SKL), researchers aimed to assess collaborative problem-solving skills in technolo-
gy-enhanced environments. The project included students from five different schools
in the Stockholm area. In this particular study, the researchers exploited multimodal
data (video and log data) in order to identify and predict students collaborative prob-
lem-solving skills.
Swedish Educational Data: Data-Driven Innovation for World Learning Educa-
tion is one of the recent development projects (with no particular educational level
focused) that aims at establishing Swedish Educational Data as a support organization
for data-driven innovation for education (2017-2019). The project is funded by Vin-
nova, Sweden’s innovation agency and led by KTH Royal Institute of Technology. It
includes both public and private actors from the education industry. They interact
actively to increase data usage for education. This is important since fragmentation
can cause each part to develop their own analytical methods and their own data man-
agement when they instead may be applicable across the entire field. The project’s
results will be released at the end of 2019.
In another recent research and development project funded by IFOUS (2017-2020),
Programming in school subjects (“Programmering i ämnesundervisningen), which do
not have an explicit focus on learning analytics, researchers are currently exploring
how digital data generated in K-12 classrooms can be used to develop teaching prac-
tices and the identification of students computational thinking skills, based on prelim-
inary findings reported in [69].
In general, albeit some research has been conducted in Sweden, so far learning ana-
lytics research has not been funded by the larger research agencies.
3 Discussion
Today, in the era of big data and analytics, researcher as well as educational stake-
holders are calling for data-driven development of education. Consequently, aiming to
capitalize on the rewarding applications of big data in different fields, researchers are
hard at work building the field of data-driven education and research (through the
field of learning analytics). In Europe, learning analytics has been embraced by re-
searchers since the early days, contributions span all venues of research in the field,
such as collaborative learning, visualization of learners’ interactions, learning dash-
PaperEfforts in Europe for Data-Driven Improvement of Education
boards, dispositional learning analytics, self-regulated learning and multimodal learn-
ing analytics. Researchers have also explored all kinds of data from single course
digital traces to large scale academic analytics. Efforts are increasingly organized to
tackle new problems, establish collaborative research groups, set up learning analytics
focused scientific events, and build capacity in the interdisciplinary field.
Lately, Some European projects have been launched, examples are the STELA pro-
ject between KU Leuven, University of Nottingham, TU Delft and TU Graz (Leitner
& Ebner, 2016), The LACE project (The Learning Analytics Community Exchange),
SHEILA Project and the collaboration between (SLATE) at the University of Bergen
and Erasmus University Rotterdam. Nonetheless, collaboration and funding on the
European level are still relatively scarce. On the national level, funding of learning
analytics projects is just taking off. In Austria, the ministry of education will support
learning analytics research in the next years. In Denmark, The ATEL project was
funded by Innovation Fund Denmark (Innovationsfonden) which will track the early
stage literacy development through learning analytics. In Finland, the Finish Science
Academy launched the project “Digital Humanities” to use data for analyzing and
modelling humanities and social sciences. In Germany, the Federal Ministry of Edu-
cation and Research have funded some projects such as ‘Utilizing Learning Analytics
for Study Success’ (Mao et al., 2019). In Norway and Spain several projects are start-
ing with the help of national funding agencies. However, in Sweden, the funded pro-
jects are still very few.
Taking together, the previous examples for funding are way behind the expected in
a time where learning is in the center of public and political attention. Let alone the
accelerating successes of using big data across many disciplines. The rich and diverse
potentials of data-driven applications are reflected in the heterogeneous nature of
reported research from different research groups. Although such diversity and breadth
of applications have helped emphasize the worth of using data to improve education,
it has also emphasized a need for organizing efforts. Fragmentation, division and
paucity of collaborative projects seem to be prevailing. Nevertheless, a number of
collaborative groups are emerging. Examples include The Learning Analytics re-
search group at Stockholm University, The Danish Center for Big Data Analytics
Driven Innovation (DABAI) and the Spanish Network of Learning Analytics. Relat-
edly, learning analytics scientific events have been organized, such as the Nordic-
As the field of learning analytics is relatively new, researchers around the world are
working to tackle the emerging challenges, such as proving the value of using data-
driven decision making, aligning the field with learning sciences, collecting useful
data while securing the privacy and agency of learners. European researchers are no
exception, they are facing the same challenges as well as their own challenges. most
important is that the large interest and work of researchers and groups around Europe
has not led to national adoption or European level strategies relative the ubiquitous
adoption of technology in education. Most countries have not established national
policies for learners’ data or guidelines that govern the ethical usage of data in re-
search or education. We also conclude, that learning analytics research on pre-
university level to high extent have been overlooked. In the same vein, learning ana-
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
lytics has not received enough focus form national and European national bodies.
Such funding is necessary for taking steps towards data-driven development of educa-
4 References
[1] Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. Internat-
ional Journal of Technology Enhanced Learning, 4(5/6), 304-317.
[2] Long, P. D., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and educa-
tion. EDUCAUSE Review, 46(5), 31–40.
[3] Ifenthaler, D., Mah, D.-K., & Yau, J. Y.-K. (2019). Utilising learning analytics for study
success. Reflections on current empirical findings. In D. Ifenthaler, J. Y.-K. Yau, & D.-K.
Mah (Eds.), Utilizing learning analytics to support study success (pp. 2736). New York,
NY: Springer.
[4] Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of le-
arning analytics in higher education. Computers in Human Behavior.
[5] Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mi-
ning in practice: A systematic literature review of empirical evidence. Journal of Educat-
ional Technology & Society, 17(4), 49-64.
[6] Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., & Rienties,
B. (2016). Research Evidence on the Use of Learning Analytics. Tech. Rep.
[7] ET2020 (2016). Learning Analytics Key messages. report, European Commission. Avai-
lable online at ana-
[8] Schön, M., Ebner, M., Kothmeier, G. (2012) It's Just About Learning the Multiplication
Table, In Proceedings of the 2nd International Conference on Learning Analytics and
Knowledge (LAK '12), Simon Buckingham Shum, Dragan Gasevic, and Rebecca Ferguson
(Eds.). ACM, New York, NY, USA, 73-81.
[9] Ebner, M., Schön, M. (2013) Why Learning Analytics in Primary Education Mat-
ters!,Bulletin of the Technical Committee on Learning Technology, Karagiannidis, C. &
Graf, S (Ed.), Volume 15, Issue 2, April 2013, p. 14-17
[10] Taraghi, B., Ebner, M., Saranti, A., Schön, M. (2014) On Using Markov Chain to Evidence
the Learning Structures and Difficulty Levels of One Digit Multiplication, In: Proceedings
of the Fourth International Conference on Learning Analytics And Knowledge, ACM, New
York, p. 68-72.
[11] Kraja, E., Taraghi, B. & Ebner, M. (2017). The Multiplication Table as an innovative
Learning Analytics Application. In J. Johnston (Ed.), Proceedings of EdMedia: World
Conference on Educational Media and Technology 2017 (pp. 810-820). Association for the
Advancement of Computing in Education (AACE).
[12] Ebner, M., Schön, M., Neuhold, B. (2014). Learning Analytics in basic math education
first results from the field, eLearning Papers, 36, pp. 24-27
[13] Ebner, M. & Pronegg, M. (2015). Use of Learning Analytics Applications in Mathematics
with Elementary Learners. International Journal of Academic Research in Education, 1(2),
[14] Edtstadler, K., Ebner, M., Ebner, M. (2015) Improved German Spelling Acquisition
through Learning Analytics. eLearning Papers, 45, pp. 17-28
PaperEfforts in Europe for Data-Driven Improvement of Education
[15] Khalil, M., Ebner, M. (2016). What is Learning Analytics about? A Survey of Different
Methods Used in 2013-2015. Conference proceeding of the 8th e-Learning Excellence
Conference, 2016. Dubai, UAE. pp. 294-304
[16] Khalil, M., Ebner, M. (2016) Clustering patterns of engagement in Massive Open Online
Courses (MOOCs): the use of learning analytics to reveal student categories. Journal of
Computing in Higher Education. Volume 29. pp. 114-132.
[17] Khalil, M. & Ebner, M. (2016). Learning Analytics in MOOCs: Can Data Improve Stu-
dents Retention and Learning?. In Proceedings of EdMedia: World Conference on Educa-
tional Media and Technology 2016 (pp. 569-576). Association for the Advancement of
Computing in Education (AACE)
[18] Khalil, M., Ebner, M., & Admiraal, W. (2017). How can Gamification Improve MOOC
Students Engagement?. In proceedings of the European Conference on Game Based
Learning, Graz, Austria, (pp. 819-828)
[19] Ebner, M., Khalil, M., Schön, S., Gütl, C., Aschemann, B., Frei, W., Röthler, D. (2017)
How Inverse Blended Learning Can Turn Up Learning with MOOCs? In: Proceedings of
the International Conference MOOC-MAKER 2017. Antigua Guatemala, Guatemala, No-
vember 16-17, 2017, pp. 21-30
[20] De Laet, T, Broos, T.,van Staalduinen, J.-P., Leitner, P., Ebner, M. (2016) Successful tran-
sition from secondary to higher education using learning analytics. 44th SEFI Conference,
Tampere, Finland
[21] Leitner P., Ebner M. (2017) Development of a Dashboard for Learning Analytics in Higher
Education. In: Zaphiris P., Ioannou A. (eds) Learning and Collabo- ration Technologies.
Technology in Education. LCT 2017. Lecture Notes in Computer Science, vol 10296. pp.
293-301 Springer, Cham.
[22] Khalil, M. & Ebner, M. (2016d) De-Identification in Learning Analytics. Journal of Learn-
ing Analytics. 3(1). pp. 129 138.
[23] Leitnerr P., Ebner M., Ebner M. (2019) Learning Analytics Challenges to Overcome in
Higher Education Institutions. In: Ifenthaler D., Mah DK., Yau JK. (eds) Utilizing Learn-
ing Analytics to Support Study Success. Springer, Cham.
[24] Bruun, J., Ray, P. J., & Udby, L. (2019). Network analyses of student engagement with on-
line textbook problems. arXiv preprint arXiv:1903.11390.
[25] Alstrup, S., Hansen, C., Hansen, C., Hjuler, N., Lorenzen, S., & Pham, N. (2017). DABAI:
A data driven project for e-Learning in Denmark. In P. Peres, & A. Mesquita (Eds.),
ECEL17 - Proceedings of the 16th European Conference on e-Learning (pp. 18-24). Aca-
demic Conferences and Publishing International.
[26] Beuchert, L., Humlum, M. K., Nielsen, H. S., & Smith, N. (2018). The short-term effects
of school consolidation on student achievement: Evidence of disruption?. Economics of
Education Review, 65, 31-47.
[27] Ringgaard, A. (2019, April 4). Forskning baseret på nationale test kan være smittet med
fejl. Retrieved from
[28] Bundsgaard, J. & Kreiner S. (2019). Undersøgelse af de nationale tests måleegenskaber.
2nd edition. Copenhagen: Aarhus Universitet, Danmarks institut for Pædagogik og Uddan-
nelse. Retrieved from:
_Nationale_tests_maaleegenskaber.pdf .
[29] Spikol, D., Bruun, J., & Udby, L. Exploring Physics Education in the Classroom and the
Laboratory with Multimodal Learning Analytics. and provide timely personalised feedback
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
to students (Page 10) 6. Khalil, M: The Unbearable Lightness of Consent: An Eye on
Learning Analytics and MOOCs (Page 13) 7. Minard, M. S: Learning Analytics from a
Student Perspective (Page 15), 21.
[30] Udby, L., Jensen, P., Bruun, J., Willendrup, P., Schober, H., Neuhaus, J., . & Lefmann, K.
(2013). E-learning neutron scattering. Neutron News, 24(1), 18-23.
[31] Overgaard, J. H., Bruun, J., May, M., & Udby, L. (2017). Virtual neutron scattering exper-
iments-Training and preparing students for large-scale facility experiments. Tidsskriftet
Læring og Medier (LOM), 9(16).
[32] Bruun, J., Ray, P. J., & Udby, L. (2019). Network analyses of student engagement with on-
line textbook problems. arXiv preprint arXiv:1903.11390.
[33] Kuosa, K., Koro, J., Tervakari, A., Paukkeri, J., & Kailanto, M. (2014). Content analysis
and visualizations: Tools for a social media-enhanced learning environment. In Proceed-
ings of 2014 International Conference on Interactive Collaborative Learning, ICL 2014
(pp. 361367).
[34] Ahadi, A., Lister, R., Haapala, H., & Vihavainen, A. (2015). Exploring Machine Learning
Methods to Automatically Identify Students in Need of Assistance (pp. 121130). Univer-
sity of Technology, Sydney, Australia.
[35] Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., & Sobocinski, M. (2016). How do
types of interaction and phases of self-regulated learning set a stage for collaborative en-
gagement? Learning and Instruction, 43, 3951.
[36] Järvelä, S., Kirschner, P. A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., … Järve-
noja, H. (2014). Enhancing socially shared regulation in collaborative learning groups: de-
signing for CSCL regulation tools. Educational Technology Research and Development,
63(1), 125142.
[37] Järvelä, S., Malmberg, J., Haataja, E., Sobocinski, M., & Kirschner, P. A. (2019). What
multimodal data can tell us about the students’ regulation of their learning process? Learn-
ing and Instruction.
[38] Järvenoja, H., Järvelä, S., Törmänen, T., Näykki, P., Malmberg, J., Kurki, K., Isohätälä,
J. (2018). Capturing motivation and emotion regulation during a learning process. Front-
line Learning Research, 6(3), 85104.
[39] Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2019). Going
beyond what is visible: What multichannel data can reveal about interaction in the context
of collaborative learning?. Computers in Human Behavior, 96, 235-245.
[40] Ifenthaler, D., & Schumacher, C. (2016a). Learning Analytics im Hochschulkontext. WiSt
- Wirtschaftswissenschaftliches Studium, 4, 172177.
[41] Mao, J., Ifenthaler, D., Fujimoto, T., Garavaglia, A., & Rossi, P. G. (2019). National poli-
cies and educational technology: a synopsis of trends and perspectives from five countries.
TechTrends, 63(3), 284293.
[42] Ifenthaler, D., & Schumacher, C. (2019). Releasing personal information within learning
analytics systems. In D. G. Sampson, J. M. Spector, D. Ifenthaler, P. Isaias, & S. Sergis
(Eds.), Learning technologies for transforming teaching, learning and assessment at large
scale (pp. 318). New York, NY: Springer.
[43] Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not
promote one size fits all: The effects of instructional conditions in predicting academic
PaperEfforts in Europe for Data-Driven Improvement of Education
success. Internet and Higher Education, 28, 6884.
[44] Schumacher, C., & Ifenthaler, D. (2018a). Features students really expect from learning
analytics. Computers in Human Behavior, 78, 397407.
[45] Klasen, D., & Ifenthaler, D. (2019). Implementing learning analytics into existing higher
education legacy systems. In D. Ifenthaler, J. Y.-K. Yau, & D.-K. Mah (Eds.), Utilizing
learning analytics to support study success (pp. 6172). New York, NY: Springer.
[46] Ifenthaler, D. (2017). Are higher education institutions prepared for learning analytics?
TechTrends, 61(4), 366371.
[47] Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V., Velloso, E. (2019). Multi-
modaldata as a means to understand the learning experience. International Journal ofIn-
formation Management, 48, 108119
[48] Giannakos, M., Sharma, K., Martinez-Maldonado, R., Dillenbourg, P., & Rogers, Y.
(2018). Learner-computer interaction. In Proceedings of the 10th Nordic Conference on
Human-Computer Interaction (pp. 968-971). ACM.
[49] Sharma, K., Papamitsiou, Z., Giannakos, M.N. (2019). Building Pipelines for Educa-
tionalData: Using AI and Multimodal Analytics to Explain Learning in Adaptive Self-
Assessment. British Journal of Educational Technology (to appear)
[50] Papamitsiou, Z., Pappas, I.O., Sharma, K., Giannakos, M.N. (2019). Utilizing multimodal-
data through an fsQCA approach to explain engagement in adaptive learning. IEEE Trans-
actions on Learning Technology (to appear)
[51] Wong, J., Khalil, M., Baars, M., de Koning, B. B., & Paas, F. (2019a). Exploring sequenc-
es of learner activities in relation to self-regulated learning in a massive open online
course. Computers & Education, 103595.
[52] Wong, J., Baars, M., de Koning, B. B., van der Zee, T., Davis, D., Khalil, M., . & Paas, F.
(2019b). Educational Theories and Learning Analytics: From Data to Knowledge. In Uti-
lizing Learning Analytics to Support Study Success (pp. 3-25). Springer, Cham.
[53] Slade, S., Prinsloo, P., & Khalil, M. (2019). Learning analytics at the intersections of stu-
dent trust, disclosure and benefit. In Proceedings of the 9th International Conference on
Learning Analytics & Knowledge (pp. 235-244). ACM.
[54] Hernández-García, Á., and Conde, M.Á.: ‘Learning analytics: The good, the bad (and the
ugly)’. Proc. of the 6th International Conference on Technological Ecosystems for Enhan-
cing Multiculturality, Salamanca, Spain2018, pp. 290-293.
[55] Conde, M.Á., and Hernández-García, Á.: ‘Learning analytics: Expanding the frontier’.
Proc. of the 5th International Conference on Technological Ecosystems for Enhancing
Multiculturality, Cádiz, Spain2017 pp. 1-5.
[56] Hernández-García, Á., and Conde, M.Á.: ‘Learning analytics: needs and opportunities’.
Proc. of the 4th International Conference on Technological Ecosystems for Enhancing
Multiculturality, Salamanca, Spain, 2016, pp.309-312.
[57] Conde, M.Á., Hernández-García, Á., and Oliveira, A.: ‘Endless horizons?: addressing
current concerns about learning analytics’. Proc. of the 3rd International Conference on
iJAI Vol. 1, No. 1, 2019
PaperEfforts in Europe for Data-Driven Improvement of Education
Technological Ecosystems for Enhancing Multiculturality, Porto, Portugal, 2015, pp. 259-
[58] Hernández-García, Á., and Conde-González, M.Á.: ‘Dealing with complexity: educational
data and tools for learning analytics’. Proc. of the 2nd International Conference on Techno-
logical Ecosystems for Enhancing Multiculturality, Salamanca, Spain, 2014, pp. 263-268.
[59] Conde, M.A., and Hernández-García, A.: ‘A promised land for educational decision-
making?: present and future of learning analytics’. Proc. of the 1st International Confe-
rence on Technological Ecosystem for Enhancing Multiculturality, Salamanca, Spain,
2013 pp. 239-243.
[60] Saqr, M., Fors, U., Nouri, J. (2018). Using Social Network Analysis to Understand Online
Problem Based Learning and Predict Performance. PLoS One.
[61] Karunaratne T., and Byungura J. (2017). Using log data of virtual learning environments to
examine the effectiveness of online learning for teacher education in Rwanda, In Procee-
dings of the 2017 International conference IST-Africa, 31 May 02 June 2017, Windhoek,
[62] Saqr, M., Fors, U., Tedre, M. & Nouri, J. (2018b). How Social Network Analysis Can Be
Used to Monitor Online Collaborative Learning and Guide an Informed Intervention. PLoS
[63] Saqr, M., Nouri, J. Fors, U. (2018). Time to focus on the temporal dimension of learning.
A learning analytics study of the temporal patterns of students’ interactions and self-
regulation. International Journal of Technology Enhanced Learning.
[64] Nouri, J., Saqr, M & Fors, U. (2019a). Predicting performance of students in a flipped
classroom using machine learning: towards automated data-driven formative feedback.
10th International Conference on Education, Training and Informatics (ICETI 2019).
[65] Nouri, J., Larsson, K., & Saqr, M. (2019). Identifying factors for master thesis completion
and non-completion through learning analytics and machine learning. In proceeding of
14th European Conference on Technology Enhanced Learning (EC-TEL).
[66] Spikol, D., Bruun, J., & Udby, L. Exploring Physics Education in the Classroom and the
Laboratory with Multimodal Learning Analytics and provide timely personalised feedback
to students. LASI 2018.
[67] Bälter, O., Zimmaro, D., & Thille, C. (2018). Estimating the minimum number of op-
portunities needed for all students to achieve predicted mastery. Smart Learning Environ-
ments, 5(1), 15.
[68] Nouri, J. Åkerfeldt, A., Fors, U., Selander, S. (2017). Assessing collaborative problem
solving skills in technology-enhanced learning environments the PISA framework and
modes of communication. International Journal of Emerging Technologies in Learning.
[S.l.], v. 12, n. 04, p. pp. 163-174.
[69] Nouri, J., Zhang, L., Mannila, L.& Norén, E. (2019). Development of computational think-
ing, digital competence and 21st century skills when learning programming in K-9. Educa-
tion Inquiry.
PaperEfforts in Europe for Data-Driven Improvement of Education
5 Authors
Jalal Nouri is with Stockholm University, Sweden
Martin Ebner is with Graz University of Technology, Austria
Dirk Ifenthaler is with Universität Mannheim, Germany.
Mohammed Saqr is with University of Eastern Finland, Finland.
Jonna Malmberg is with University of Oulu, Finland.
Mohammad Khalil is with University of Bergen, Norway.
Jesper Bruun is with University of Copenhagen, Denmark.
Olga Viberg is with Royal Institute of Technology, Sweden.
Miguel Ángel Conde González is with Universidad de León, Spain.
Zacharoula Papamitsiou is with Norwegian University of Science and Technolo-
gy, Norway.
Ulf Dalvad Berthelsen is with Aarhus University, Denmark.
Article submitted 2019-06-18. Resubmitted 2019-07-24. Final acceptance 2019-07-26. Final version
published as submitted by the authors.
iJAI Vol. 1, No. 1, 2019
... [15] (Vayndorf-Sysoeva, Subocheva, 2021)), развитие массовых онлайн-курсов (Денг Р., Бенкендорф П. [16] (Deng, Benckendorff , Gannaway, 2019)), обсуждение проблем развития цифровой экономики и ее влияние на образование -Кузьминов Я.И. [17] (Kuzminov, 2018), процессы глобализации и увеличение активности пользователей на онлайн-курсах [18] (Dhawal Shah, 2021), анализ демографических показателей и анализ трудовых ресурсов, вовлеченных в процессы цифровизации [19]; 4) исследование возможностей использования интернет-маркетинга для продвижения бренда университета, оценка эффективности рекламной кампании, аналитика обучения, использование Big Data, выделение уровней микро, мезо, макро, разработка систем индикаторов: Ифенталер Д. [20] (Ifenthaler, 2017), Джонсон Р. [21], определение общеевропейских трендов в аналитике образования Ноури Дж. [22] (Nouri Jalal et al., 2019), определение направлений продвижения бренда университета [23] , тенденции развития рынка EdTech в России (работы Тимченко В.В. [24,25] (Timchenko, Trapitsin, Apevalova, 202;) и за рубежом [26] (Laufer et al., 2021). ...
... Исследование академической мобильности может использовать широкий спектр методов, в данном случае использовался системный анализ информации на основе статистики Росстата [27] и ФНИСЦ РАН [3] для того, чтобы объективно оценить сложившуюся ситуацию с разных сторон: использовались модели экстраполяции и интерполяции, основывающиеся на анализе уже сделанных прогнозов, а также маркетинговый анализ потенциальных рынков, основанный на анализе Big Data поисковых запросов пользователей на русском и английском языках, отражающие интерес за рубежом потенциальных абитуриентов к разным странам. Анализ данных в сфере образования позволяет выделить микро-, мезо-, макроуровень в структуре данных [22] (Nouri Jalal et al., 2019). В данном исследовании акцент сделан на мезо-и макроуровне, что позволить уточнить прирост (сокращение) студентов из различных регионов. ...
Full-text available
The training of foreign students is an opportunity for Russian universities to increase their incomes, compensate for the demographic crisis, expand the contingent of students, diversify risks in their work and improve their image in the educational services market. In order to obtain a more effective result in this direction, the development of methods for optimizing advertising and admission campaigns is required. The article contains recommendations on development of a strategy for attracting foreign students, popularizing the brand of the university. This process includes the following stages: determining the purpose of the promotion program, searching and characterizing the target audience, compiling a set of communication channels used, an action plan, a forecast of the effectiveness of the program, and a study of internet traffi c indicators for universities in the region. Marketing and strategic planning tools, including SWOT analysis, allow to identify the strengths and weaknesses of regional Russian universities in attracting foreign students and develop methods to improve this situation. The article will be of interest to employees working in the reception campaign, marketers and content managers.
... Baek and Doleck (2020) conducted a bibliometric analysis of AI in education including learners of all ages. From examining articles across 2015-2020, the findings show that scholars also focused their efforts on examining specific AI tools, such as learning analytics (Nouri et al. 2019), adaptive learning systems (Kabudi et al. 2021), and intelligent tutoring systems (Nye 2015). Other scholars conducted systematic reviews on tools in specific countries. ...
... Other scholars conducted systematic reviews on tools in specific countries. For example, Nouri et al. (2019) examined learning analytics and how they were used in seven countries: Austria, Denmark, Finland, Norway, Germany, Spain, and Sweden. ...
Full-text available
Artificial intelligence (AI) is heralded as a technology holding many unique affordances to K-12 teaching and learning. The purpose of this study was to examine how AI has been used to support teaching and learning in the K-12 context. Specifically examining the affordances for K-12 educators and students. A thematic systematic review methodology was used with PRISMA principles to examine peer review journal articles from 2010 to 2020. This thematic systematic review revealed four themes of how AI was being used by educators to support student learning: Student Monitoring, Group Management, Automated Grading, and Data-Driven Decisions. The Group Management theme included three sub-themes as AI specifically supported educators with group formation, group moderation, and group facilitation. In the examination of affordances for students, three overarching themes emerged. AI was used for AI tutors, to extend student thinking and Just-for-You-Learning which provided a bespoke learning experience built on students’ strengths and weaknesses, preferences, and interests.
... In a broader educational context, data from computerized formative assessments can also be used for establishing a learning analytics (LA) framework where teachers can implement data-informed (or data-driven) decision making and optimize student learning in real time [3][4][5]. The primary goal of LA is to "exploit data generated in educational settings for purposes of optimizing learning and the environments in which it occurs" [6]. Researchers often build predictive LA models based on historical student data to help teachers make predictions about future educational outcomes (e.g., students' final course grades or course failure) and take appropriate actions [7]. ...
... Then, the residual d was added to the predicted value,ŷ d . Third, the Theil-Sen estimation method was performed to obtain the intercept, slope,ŷ, and in Equation (6). Lastly, we repeated the same simulation procedure for a total number of 1000 samples (i.e., 1000 simulated students). ...
Full-text available
The use of computerized formative assessments in K–12 classrooms has yielded valuable data that can be utilized by learning analytics (LA) systems to produce actionable insights for teachers and other school-based professionals. For example, LA systems utilizing computerized formative assessments can be used for monitoring students’ progress in reading and identifying struggling readers. Using such LA systems, teachers can also determine whether progress is adequate as the student works towards their instructional goal. However, due to the lack of guidelines on the timing, number, and frequency of computerized formative assessments, teachers often follow a one-size-fits-all approach by testing all students together on pre-determined dates. This approach leads to a rigid test scheduling that ignores the pace at which students improve their reading skills. In some cases, the consequence is testing that yields little to no useful data, while increasing the amount of instructional time that students miss. In this study, we propose an intelligent recommender system (IRS) based on Dijkstra’s shortest path algorithm that can produce an optimal assessment schedule for each student based on their reading progress throughout the school year. We demonstrated the feasibility of the IRS using real data from a large sample of students in grade two (n = 668,324) and grade four (n = 727,147) who participated in a series of computerized reading assessments. Also, we conducted a Monte Carlo simulation study to evaluate the performance of the IRS in the presence of unusual growth trajectories in reading (e.g., negative growth, no growth, and plateau). Our results showed that the IRS could reduce the number of test administrations required at both grade levels by eliminating test administrations in which students’ reading growth did not change substantially. In addition, the simulation results indicated that the IRS could yield robust results with meaningful recommendations under relatively extreme growth trajectories. Implications for the use of recommender systems in K–12 education and recommendations for future research are discussed.
... For the field of AI in education, a split picture emerges in the scientific literature. Meta-analyses of the opportunities of AI in relation to personalised learning tend to show a minor pedagogical effect (Nouri et al., 2019). However, at the same time, numerous publications emphasise potential opportunities, but in some cases do not conduct any evaluations and, therefore, do not provide any evidence. ...
Full-text available
The Inclusive Digital Education (IDE) activity aims to thoroughly examine new priorities and demands in relation to inclusive digital education and blended learning. This report gives an overall picture of inclusive digital education and of issues remaining to be tackled. It describes the policy context for inclusive digital education and considers issues of vulnerability and inclusion. It reviews relevant research literature from 2016–2021 and examines thematic trends in related implementation projects and conferences. Experts in the field have validated the report’s findings. This report is part of a package of materials from the IDE activity, consisting of the following: Methodology paper, detailing the methodology chosen to analyse the topic Project examples, collating a selection of Erasmus+ projects dealing with specific issues related to inclusive digital education (forthcoming) Policy brief, detailing issues not yet sufficiently addressed in the field of inclusive digital education (forthcoming). Full text and accompanying documents available at:
... er the student interacts, connects or exploits the resources made available to him. These traces give clues about the learning strategies of students who, according to the logic of Learning Analytics, would be automatically stored on a computer server before they could be processed by algorithms capable of analyzing and establishing their profiles (Nouri. Jalal et al. 2019). Each student's online activities can be recorded and used for learning analytics, including access to online management systems, access to course resources, searching for information on the online library, taking online exams, writing homework or conducting hypothetical exchanges with colleagues, provided they are ethical and confident ...
Conference Paper
Full-text available
Learning Analytics is an emerging discipline in the field of learning. In this sense, the objective of this work is to focus on learning analytics and their application to e-learning. Thus, to present the main interaction indicators. Once this topic is addressed, many questions arise: How can we define learning analytics? What is an indicator and what are its types? Our work aims to determine a new classification of indicators. The results of this study are complemented by the conclusions of a precise literature review.
... Studies focusing on developing the possibilities of learning analytics have predominately been oriented towards higher education, and there is still a lack of studies focusing on kindergarten to 12th grade (K-12) education (Nouri et al., 2019;Misiejuk & Wasson, 2017;Vieira et al., 2018). Within K-12, studies have reported that teachers feel unprepared to engage with data that involve different information on student learning. ...
... Fostering students' HOTS and SRL is not simple (Koh et al., 2012;Nouri et al., 2019;Yen & Halili, 2015). Therefore, some scholars adopt learning analytics (LA) to assess to what extent students deploy specific strategies during the learning process (Tabuenca et al., 2015;van Horne et al., 2017;Yamada et al., 2016Yamada et al., , 2017You, 2016). ...
Full-text available
Self-regulated learning (SRL) and higher-order thinking skills (HOTS) are associated with academic achievement, but fostering these skills is not easy. Scholars have suggested an alternative way to scaffold these important skills through learning analytics (LA). This paper presents a formative evaluation of a course-level LA implementation through the lens of self-regulated learning (SRL) and higher-order thinking skills (HOTS). We explored the changes in students' SRL, HOTS, and perceptions at the end of the course term. Results indicate an increase in some elements of SRL and HOTS, and positive student perceptions. Discussion on implications and opportunities for informing future teaching strategies and course design reiteration are included.
... Im viel rezipierten Horizon Report, der jährlich aktuelle und prognostizierte Trends technologiegestützten Lehrens und Lernens vorstellt, wird Learning Analytics regelmäßig als bedeutsamer Entwicklungsbereich aufgeführt (zum Beispiel jüngst Brown et al. 2020). Im Gegensatz etwa zu den USA, Kanada, Australien oder China erhielt dieses noch junge Feld in Europa -von Ausnahmen abgesehen, zum Beispiel in den Niederlanden -bislang eher wenig Aufmerksamkeit (Nouri et al. 2019). ...
Full-text available
Der digitale Wandel führt zu einer Zunahme des Datenvolumens und neuen Möglichkeiten der Datennutzung. Auch an Hochschulen befördert der voranschreitende Technologieeinsatz in Studium und Lehre die als „Datafication“ beschriebene Entwicklung. Hieraus erwachsen einerseits Anforderungen bezüglich des kompetenten Umgangs mit Daten (Data Literacy) und insbesondere des Datenschutzes, denen sich Hochschulen schon heute nicht entziehen können, andererseits aber auch neue Potenziale der gezielten Datennutzung. Unter „Learning Analytics“ werden seit etwa einer Dekade Möglichkeiten der Datenauswertung entwickelt und diskutiert, die ein tieferes Verständnis und eine Optimierung von Lernumgebungen und -prozessen in Aussicht stellen und sowohl Dozierende und Studierende als auch hochschulische Leitungspersonen und -gremien adressieren. Ungeachtet der technischen Dimension der Datenverarbeitung ist die Implementierung von Learning Analytics vor allem auch organisational anspruchsvoll. Zugleich korrespondieren mit Szenarien hochschulischer Datennutzung begründete Einwände und ethische Bedenken. Im Beitrag wird zunächst in das noch junge Feld Learning Analytics eingeführt und anschließend ein Orientierungsrahmen für Hochschulen vorgelegt, der aus institutioneller Perspektive 25 Handlungsanforderungen für einen sensiblen, potenzialorientierten und organisationsbewussten Umgang mit lehr- und lernbezogenen Daten beschreibt.
Conference Paper
Full-text available
Development of actionable learning analytics with a user-centric approach to development to improve actionability, suitability, acceptability and usage in higher education. Processes, results and lessons learnt.
Full-text available
Lernplattformen, Lernsoftware, E-Assessments, Campusmanagementsysteme, Online- Evaluationen – der Einsatz von Informationstechnologien in Studium und Lehre nahm seit der Jahrtausendwende massiv zu und dürfte aufgrund der flächendeckenden Verfügbarkeit von internetfähigen mobilen Endgeräten weiter steigen. Hierbei werden große Mengen an Daten generiert, deren Auswertung u. a. Einblicke in Lehr- und Lernprozesse in Aussicht stellt. Unter Learning Analytics firmieren seit ca. einer Dekade primär im Hochschulkontext Aktivitäten, die im Wege der Datenanalyse das Lernen gezielt zu unterstützen und Lernumgebungen zu optimieren beanspruchen. Der Beitrag führt in das Feld ein und illustriert anhand ausgewählter Beispiele ebenso Potenziale von Learning Analytics wie damit korrespondierende Bedenken und Limitationen.
Full-text available
Investigating and explaining the patterns of learners' engagement in adaptive learning conditions is a core issue towards improving the quality of personalized learning services. This study collects learner data from multiple sources during an adaptive learning activity, and employs a fuzzy set qualitative comparative analysis (fsQCA) approach to shed light to learners’ engagement patterns, with respect to their learning performance. Specifically, this study measures and codes learners' engagement by fusing and compiling clickstreams (e.g., response time), physiological data (e.g., eye-tracking, EEG, electrodermal activity) and survey data (e.g., goal-orientation) to determine what configurations of those data explain when learners can attain high or medium/low learning performance. For the evaluation of the approach, an empirical study with 32 undergraduates was conducted. The analysis revealed six configurations that explain learners' high performance and three that explain learners' medium/low performance, driven by engagement measures coming from the multimodal data. Since fsQCA explains the outcome of interest itself, rather than its variance, these findings advance our understanding on the combined effect of the multiple indicators of engagement on learners’ performance. Limitations and potential implications of the findings are also discussed.
Full-text available
Most work in the design of learning technology uses click-streams as their primary data source for modelling & predicting learning behaviour. In this paper we set out to quantify what, if any, advantages do physiological sensing techniques provide for the design of learning technologies. We conducted a lab study with 251 game sessions and 17 users focusing on skill development (i.e., user's ability to master complex tasks). We collected click-stream data, as well as eye-tracking, electroencephalography (EEG), video, and wristband data during the experiment. Our analysis shows that traditional click-stream models achieve 39% error rate in predicting learning performance (and 18% when we perform feature selection), while for fused multimodal the error drops up to 6%. Our work highlights the limitations of standalone click-stream models, and quantifies the expected benefits of using a variety of multimodal data coming from physiological sensing. Our findings help shape the future of learning technology research by pointing out the substantial benefits of physiological sensing.
Full-text available
Teachers around the world have started teaching programming at the K-9 level, some due to the formal introduction of programming in the national curriculum, others without such pressure and on their own initiative. In this study, we attempted to understand which skills – both CT-related and general – are developed in the process. To do so, we interviewed 19 Swedish teachers who had been teaching programming for a couple of years on their own initiative. The teachers were selected based on their experience in teaching programming. Our thematic analysis of these interviews shed light on what skills teachers perceive pupils develop when programming. This led us to identify three themes related to CT skills and five themes related to general skills. The CT skills identified corresponded well with and were thus thematically structured according to the dimensions of CT proposed in the framework of Brennan and Resnick (2012), namely computational concepts, computational practices and computational perspectives. In addition to the CT skills, our thematic analysis also resulted in the identification of general skills related to digital competency and 21st century skills, namely cognitive skills and attitudes, language skills, collaborative skills and attitudes and creative problem-solving skills and attitudes.
Full-text available
Self-regulated learning (SRL) refers to how learners steer their own learning. Supporting SRL has been shown to enhance the use of SRL strategies and learning performance in computer-based learning environments. However, little is known about supporting SRL in Massive Open Online Courses (MOOCs). In this study, weekly SRL prompts were embedded as videos in a MOOC. We employed a sequential pattern mining algorithm, Sequential Pattern Discovery using Equivalence classes (cSPADE), on gathered log data to explore whether differences exist between learners who viewed the SRL-prompt videos and those who did not. Results showed that SRL-prompt viewers interacted with more course activities and completed these activities in a more similar sequential pattern than non SRL-prompt viewers. Also, SRL-prompt viewers tended to follow the course structure, which has been identified as a behavioral characteristic of students who scored higher on SRL (i.e., comprehensive learners) in previous research. Based on the results, implications for supporting SRL in MOOCs are discussed.
Conference Paper
Full-text available
Evidence suggests that individuals are often willing to exchange personal data for (real or perceived) benefits. Such an exchange may be impacted by their trust in a particular context and their (real or perceived) control over their data. Students remain concerned about the scope and detail of surveillance of their learning behavior, their privacy, their control over what data are collected, the purpose of the collection, and the implications of any analysis. Questions arise as to the extent to which students are aware of the benefits and risks inherent in the exchange of their data, and whether they are willing to exchange personal data for more effective and supported learning experiences. This study reports on the views of entry level students at the Open University (OU) in 2018. The primary aim is to explore differences between stated attitudes to privacy and their online behaviors, and whether these same attitudes extend to their university's uses of their (personal) data. The analysis indicates, inter alia, that there is no obvious relationship between how often students are online or their awareness of/concerns about privacy issues in online contexts and what they actually do to protect themselves. Significantly though, the findings indicate that students overwhelmingly have an inherent trust in their university to use their data appropriately and ethically. Based on the findings, we outline a number of issues for consideration by higher education institutions, such as the need for transparency (of purpose and scope), the provision of some element of student control, and an acknowledgment of the exchange value of information in the nexus of the privacy calculus.
Full-text available
While a large number of scientific publications explain the development of prototypes or the implementation of case studies in detail, descriptions of the challenges and proper solutions when implementing learning analytics initiatives are rare. In this chapter, we provide a practical tool that can be used to identify risks and challenges that arise when implementing learning analytics (LA) initiatives and discuss how to approach these to find acceptable solutions. In this way, implementers are given the opportunity to handle challenges early on and avoid being surprised at a critical moment in the project, which will save time, resources, and effort. We are aware that all aspects needed to successfully carry out learning analytics initiatives are co-dependent. Nonetheless, we identified and categorized the criteria necessary for implementing successful learning analytics initiatives. We conclude this chapter with an overview of the challenges faced and possible approaches that can be taken to facilitate the successful implementation of learning analytics.
Besides the well-documented benefits of learning analytics, serious concerns and challenges are associated with the application of these data-driven systems. Most notably, empirical evidence regarding privacy issues such as for learning analytics is next to nothing. The purpose of this study was to investigate if students are prepared to release any personal data in order to inform learning analytics systems. A total of 330 university students participated in an exploratory study confronting them with learning analytics systems and associated issues of control of data and sharing of information. Findings indicate that sharing of data for educational purposes is correlated to study-related constructs, usage of Internet, awareness of control over data, and expected benefits from a learning analytics system. Based on the relationship between the willingness to release personal data for learning analytics systems and various constructs closely related to individual characteristics of students, it is concluded that students need to be equally involved when implementing learning analytics systems at higher education institutions.
There have been continued efforts in exploring how educational technology impacts human learning and performance. Through a synopsis of the trends and perspectives on educational technology in five countries by using the STEEP (social, technological, economic, environmental and political) framework, the authors discuss the direct influence of national policies on educational technology implementation and research, the constraints of the local STEEP elements of culture on the adoption of and research on innovation and change, and the strategies and reforms that different countries have adopted to prepare the next generation for the constantly changing, globalized twenty-first century. Limitations and future research following this work are discussed.
Learning analytics have become a well-considered aspect of modern digital learning environments. One opportunity of learning analytics is the use of learning process data enabling lecturers to analyse students’ learning progression as well as to identify obstacles and risks. With this analytics knowledge, lecturers may want to scaffold students’ learning activities to improve the learning progress and overcome obstacles or risks. Prompts are known to be a possible solution for such scaffolding mechanics. However, implementing prompts into existing legacy systems in learning environments with high data privacy concerns is quite a challenge. This research shows how a prompting application has been implemented into an existing university environment by adding a plug-in to the local digital learning platform which injects user-centric prompts to specific objects within their digital learning environment. The prompts are dynamically loaded from a separate learning analytics application which also collects the students’ learning trails and progress. The system is evaluated in two units in the fall semester 2017 with more than 400 students altogether. The system collects up to two thousand student events per day. An in-depth empirical investigation on how various prompts influence students’ learning behaviours and outcomes is currently conducted.