Content uploaded by Gerti Pishtari
Author content
All content in this area was uploaded by Gerti Pishtari on Mar 24, 2020
Content may be subject to copyright.
An Overview of Learning Design and Analytics
in Mobile and Ubiquitous Learning
Gerti Pishtari, Mar´ıa Jes´us Rodr´ıguez-Triana, Edna Milena
Sarmiento-M´arquez, Jaanus Terasmaa, K¨ulli Kori, Mihkel Kangur, Oleksandr
Cherednychenko, Terje V¨aljataga, and Liisa Puusepp
Tallinn University, Narva maantee 25, 10120 Tallinn, Estonia
{gpishtar,mjrt,msm,pootsman,kyllikor,mihkelk,olcher,terjev,liisa}@tlu.ee
Abstract. Mobile and ubiquitous learning models have been widely
adopted in technology-enhanced learning (TEL) practices. Apart from
potential benefits, these models introduce additional complexity in de-
signing, monitoring and evaluating learning activities, as learning hap-
pens across different spaces. In recent years, literature on learning de-
sign (LD) and learning analytics (LA) has started to address these issues.
This paper presents a systematic review on how LD and LA communities
understand mobile and ubiquitous learning, as well as their respective
contributions in these fields. The search included seven main academic
databases in TEL, resulting in 1722 papers, from which 54 papers were
included in the final analysis. Results point out the lack of common def-
initions for mobile and ubiquitous learning, raises research trends and
(unexploited) synergies between between LD and LA communities, and
identifies areas that require further attention from these communities.
Keywords: Mobile learning ·Ubiquitous learning ·Learning Design ·
Learning Analytics ·Systematic literature review
1 Introduction
There has been a growing research interest in mobile learning (m-learning) and
ubiquitous learning (u-learning). Nevertheless, there is no consensus yet in the
academic and professional communities about their meanings [19]. Indeed, both
terms are often used interchangeably in literature [3]. Nevertheless, authors seem
to agree on what m-learning and u-learning promote, highlighting among other
aspects: accessibility, interactivity, self-regulated and situational learning, conti-
nuity and connectivity among contexts [19, 3]. Despite these potential benefits,
m-learning/u-learning pose additional complexity for designing, monitoring and
evaluating learning scenarios. For example, in these settings, learning usually
happens across multiple spaces. Designing in these situations typically involves
the usage of different authoring tools, specific for each particular space (see for
instance Smartzoos, for the design of learning activities in geo-located physical
spaces [18], or PyramidApp in a digital space [9]), as well as requires gather-
ing data from different spaces in order to achieve a global view of the learning
process [14].
2 Pishtari et al.
In the last 15 years, the communities of LD and LA have contributed with
various proposals to address these issues in m-learning/u-learning. In general,
the community of LD has been focused on the importance of facilitating practi-
tioners in sharing, modifying and reusing their pedagogical plans. On the other
hand, research in LA has aimed to investigate techniques of handling learners’
data to support decision making of different actors involved at different stages
in the learning process [16]. There is also a growing community of researchers
interested in the synergy between LD and LA. [2,10]. Lockyer et al. [8] claim
that research in LD should take advantage and harness the results of the field of
LA. The state of the art in LA for LD is still in its early days but preliminary
research has shown the potential and has shown both the potential and chal-
lenges of aligning LA and LD. [13]. Looking at mobile and ubiquitous learning
contexts, few LD and LA research studies are found. Indeed, in a systematic
review about research in LA for LD, Mangaroska and Giannakos [10] identified
only 1 work (out of 43 reviewed), connected to m-learning/u-learning. While,
existing systematic reviews in m-learning/u-learning have focused on: research
trends [20], identifying open research issues [5], specific educational settings (e.g.,
m-learning in higher education [17]), or specific learning models such as collab-
orative learning [1]. Some of these reviews reflect on LD aspects (e.g., [1]), while
none of them has focused on the role of LA in m-learning/u-learning.
Thus, in order to better understand the role that the LD and LA communities
may play in m-learning/u-learning, and how they could enrich each other, we
carry out a systematic review. Concretely, our research questions are:
– RQ1: What are the definitions and aspects of m-learning/u-learning which
have been considered by the LD and LA communities?
– RQ2: How were the learning contexts where LD and/or LA supported m-
learning/u-learning?
– RQ3: How have LD and LA supported m-learning/u-learning (type of con-
tributions)?
2 Methodology
To carry out the systematic review we followed the guidelines proposed by
Kitchenham and Chartrs [6]. We used seven main databases in TEL: ACM Dig-
ital Library, AISEL, IEEE XPLORE, SpringerLink, ScienceDirect, Scopus and
Wiley. In addition, Google Scholar was included in order to detect potentially
relevant grey literature. To perform the search, we broke down the question into
the learning settings (m-learning, or u-learning) and the research field where
the proposal was framed (LA, or LD). The resulting search string was (”learn-
ing design” OR ”learning analytics”) AND (”mobile learning” OR ”ubiquitous
learning”). Using this query, we obtained 1622 papers. In addition, we added
the first 100 results from Google Scholar. The search was conducted on April,
4th 2019 and since then new papers might have appeared. To standardize the
search (since different databases have different filtering criteria), we accepted
papers where the query was satisfied in the core parts of the it (title, abstract,
Title Suppressed Due to Excessive Length 3
or keywords), resulting in 209 papers. Furthermore, each paper was reviewed by
at least two researchers, discarding those that were out of scope, not in English,
or less than 4-pages long. Doubtful papers were discussed among all researchers,
resulting in 54 papers1which were considered for the in-depth review.
3 Results
This section provides an overview of existing LD and LA proposals in m-learning
and u-learning. We analyzed the definitions of m-learning and u-learning that
were covered in the proposals (subsection 3.1), the learning contexts where these
proposals were framed (3.2), and the type of contributions (3.3). Table 1 provides
a summary of the main results per research question.
Table 1. Main findings per research topic.
Research topic Findings
Core aspects of
m-learning/u-
learning which
have been
considered by the
LD and LA
communities
m-learning
learning with mobile technologies (15)
learning anytime anywhere (13)
user mobility (7)
context-aware learning (5)
learning across spaces (2)
u-learning
learning anytime, anywhere (7)
context-aware learning (6)
learning with ubiquitous computing technologies (4)
learning across spaces (3)
Learning
contexts where
LD and/or LA
supported
m-learning/u-
learning
scenarios formal: university (24) + K-12 (8)
informal: open to all users (8) + children (1)
spaces across physical and digital spaces (39) + several digital
spaces (4) + singe digital space (2) + several physical
spaces (1)
physical spaces: indoor (11) + outdoor (7) + both (22)
target users teachers (37)
learners (22)
LD and LA
contributions for
m-learning/u-
learning
LD theoretical: models (17), guidelines (6) and frameworks
(6)
practical: tools used mainly before (10) and during (11)
learning activities
LA theoretical: data analysis (8), models (8), and guide-
lines/good practices (8)
practical: tools used during (16) and after (10) learning
activities
Out of 54 reviewed papers, 28 (51.9%) were about LD, 23 (42.6%) about
LA, and 3 (5.6%) referred both LD and LA. Thirty (55.5%) papers referred
m-learning, 15 (27.7%) to u-learning, while 9 (16.7%) referred to both terms.
1Reviewed papers: hhttps://gitlab.com/gertipishtari/list-of-papers
4 Pishtari et al.
3.1 Definitions of m-learning and u-learning
In order to answer RQ1, we analyzed thematically the definitions (separately for
m-learning and u-learning) that were used by the communities of LD and LA,
in order to identify common core aspects. We started by identifying the parts
of the definitions that referred to specific aspects of m-learning/u-learning (e.g.,
learning with mobile technologies). These were later on clustered together and
coded with a specific keyword, or phrase. Similar keywords were further grouped
together to form the core aspects.
From papers related to m-learning, 14 provided their own definition, 13 re-
ferred to other authors, while 19 took the definition for granted. It should be
emphasized that papers that were referring to other authors, mentioned in total
more than 10 different publications. Despite the multitude of definitions, several
core aspects related to the definition of m-learning that were mentioned across
the papers surged from the thematic analysis, such as: learning with mobile
technologies (15); learning anytime anywhere (13); and user mobility (7), and
context-aware learning (5). In the case of u-learning, 6 papers provided their own
definition, 6 referred to other authors, while 15 took the definition for granted.
The core aspects that were mentioned in this case included: learning anytime,
anywhere (7); context-aware learning (6); and learning with ubiquitous comput-
ing technologies (4). Core aspects of m-learning and u-learning do not change
significantly, when we filter the results based on LD, or LA contributions.
The communities of LD and LA attribute similar aspects to both terms of
m-learning and u-learning, such as learning anytime anywhere, or context-aware
learning. In relation to this, one of the cited papers from u-learning, Hwang et al.
[4], while discussing the similarities between m-learning and u-learning, proposes
the term context-aware u-learning to distinguish u-learning from m-learning. As
it can be seen from the results, m-learning tends to be more technocratic with
attributes such as mobile technologies and user mobility, but these attributes
mainly appear in older publications and they can be explained with the focus
on technology that m-learning had in the beginning. Since then m-learning has
undergone a transformation and it is not seen anymore as exclusively related to
learning with mobile technologies, or user mobility. In fact Traxler [19], the most
cited author from the m-learning papers under review (although not massively
cited), analyses the evolution of the definition and of m-learning from its techno-
cratic beginnings, into a more mature moment when the research field was trying
to understand the meaning of m-learning in an age that is characterized by a
fast evolution of technologies and when the focus passed from the technology to
the learners and the learning process.
3.2 Learning context
To answer RQ2, we grouped the papers based on learning scenarios, educational
levels, spaces, and target users.
Learning scenarios. From 54 papers, 26 (48.2%) targeted formal learning,
8 (14.8%) informal learning (together with non-formal learning), 2 (3.7%) both
Title Suppressed Due to Excessive Length 5
formal and informal learning, while 18 (33.3%) did not specify it. From the papers
addressing formal learning, 24 were about university settings and 8 about K-12.
In the case of papers addressing informal learning, 4 focused on university level,
3 papers were open to all users, 1 was explicitly for children, and 2 did not specify
it. Both communities of LD and LA have principally focused on formal learning
with 13 and 16 papers, respectively. It should be also noticed that most of the
cases where the type of learning went unspecified belong to LD, concretely 14
papers.
Learning spaces. We grouped papers based on the typology of the space
where learning occurred. The most common typology of learning spaces was
across physical and digital spaces (39), followed by in several digital spaces (4),
in a singe digital space (2), several physical spaces (1), and not specified (8).
From the papers that described a learning activity, 11 papers referred to learn-
ing activities that happened indoor (most of them in a classroom), 7 papers
referred to learning activities outdoor (e.g., in thematic parks, or in the city),
while in 22 papers learning activities happened both indoor and outdoor. In
general, the community of LD has focused more on indoor learning activities
(5) and activities in both settings (16), while papers related to LA had an even
distribution between indoor (6), outdoor (7), and in both settings (8). As an
example, Mu˜noz-Crist´obal et al. [15] propose GLUEPS-AR, a system that helps
teachers to deploy and enact LDs across physical and digital spaces, both in-
door and outdoor. In another paper, Melero et al. [11] present QuestINsitu, a
tool that supports the design and monitoring of geo-localized learning activities
outdoor, where the learning activity happens across digital and physical spaces.
Lkhagvasuren et al. [7] propose the Learning Log Analytics Dashboard (L2D),
which tracks, analyzes and visualizes data about learning activities that happen
in a digital space that supports language learning.
Target users. Various target users were mentioned in the papers such as
teachers, instructional designers, learners, researchers and developers. When con-
sidering both communities of LD and LA, teachers in 37 out of 54 papers (68.5%)
and learners in 22 (40.7%) were the main target users. The community of LD
has focused mostly on teachers (25) and instructional designers (15), while the
community of LA has focused more on learners (15) and teachers (15).
Traditionally, the community of LA has focused on analyzing students’ data
to inform teachers. On the contrary, in m-learning and u-learning there is also
a large focus on supporting directly learners. This facet can be related to the
self-regulated nature of learning in m-learning/u-learning, where the role of the
student is central. Thus, we could expect contributions moving their focus from
teachers to learners in the coming years.
3.3 Types of contributions
To answer RQ3 we grouped the contributions based on the research field (LD,
LA) and the type of contribution (theoretical, practical). Theoretical contribu-
tions were further clustered into architectures, theoretical models, indicators,
frameworks, data analysis, guidelines/good practices. Practical contributions
6 Pishtari et al.
were grouped into functionalities that were expected to be used before, dur-
ing, or after the learning activity. We also labeled the papers under review based
on the purpose of the LD/LA functionalities that they described (e.g., support
the design of learning activities, provide personalized feedback, etc.).
The revised papers were evenly distributed among LD (28), LA (23), while
3 papers referred to both LD and LA. Papers mentioning LD were found to
include more theoretical (25) than practical contributions (11), while LA papers
had a balanced distribution between theoretical contributions (19) and practical
ones (17). The 3 papers related to both LD and LA offered practical contribu-
tions. Theoretical contributions in LD have been mostly models (17), guidelines
(6) and frameworks (6), while theoretical contributions in LA have been mostly
data analysis (8), models (8), and guidelines/good practices (8). Practical con-
tributions from the community of LD (e.g., QuestInSitu [11]) were used mainly
during (11) and before the learning activity (10). In the case of LA papers, prac-
tical contributions (e.g., SCROLL [7]) were expected to be used during (16) and
after learning activities (10).
LD functionalities were mainly aimed to support the design of learning activ-
ities (29), while LA functionalities focused on providing personalized feedback
(17), and supporting the reflection about the learning activities (10). Papers in-
cluding contributions in LA for LD also emphasized other aspects such as raising
awareness about LD practices and support the evidence-based decision making.
As it can be seen from the results, there is a low number of practical imple-
mentations benefiting from the synergies between both LD and LA. From these,
two papers were LA for LD [11,14], while third case used to both LD and LA
independently, without aligning them [12]. It should be mentioned that there
were no contributions on LD for LA.
4 Conclusion and future work
Despite the lack of widely accepted definitions for both m-learning and u-learning,
this review shows that both communities emphasize similar core aspects of m-
learning and u-learning. The communities of LD and LA attribute similar shared
aspects to both terms. M-learning tends to be more technocratic, but these
technical attributes are found mainly in older publications, when the field of
m-learning was focusing more on the technological aspect.
Regarding the learning context, a significant finding is about target users. LA
papers in m-learning/u-learning emphasize the importance of informing learners,
which can be related to the self-regulated nature of learning in these settings.
This could imply that more LA effort could be devoted to further support self-
regulated learners in m-learning/u-learning. In a similar way as in other reviews
about LD and LA, most of the papers have been focused on university settings.
This is to be expected due to the fact that university environments are easier to
access by researchers. Therefore, there is still a need to explore the benefits of LD
and LA in m-learning/u-learning in K-12 contexts and in non-formal settings.
Title Suppressed Due to Excessive Length 7
While responding to the RQ3, about the type of contributions, we identified
possible points of synergies between the communities of LD and LA that may
lead to a joined research agenda in m-learning/u-learning. We noticed a low
number of papers that include aspects from both LD and LA (3). Two of these
papers were about LA for LD, while none referred to LD for LA. There is a low
number of implementations benefiting from the synergies between LD and LA.
These two communities can complement each other. As identified from the anal-
ysis, LD can support participants before the learning activity, while LA can do
it during and after the activity. Moreover, as mentioned in recent literature [10,
2], both communities can further collaborate and close the loop. Specifically, LD
can guide and give a context to data analysis, by making them more meaningful
for involved stakeholders, while LA can inform design decisions and support the
process evaluating LDs (as emphasized also from the papers under review that
had a contribution related to LA for LD).
Relevant limitations of the current review have to do with the keywords that
constitute the query and the method that was used to filter the paper (respec-
tively searching with the query in the title, abstract and keywords) . Important
contributions that did not comply with the search criteria might have been left
out of the review. Also, other learning contexts such as seamless learning, or
hybrid learning, as well as related terminology for LD (e.g., scripting), or LA
(e.g., educational data mining) were not included in the query and could have
left out complementary contributions to the list of works under review.
Future work will aim to extend further the review by: analyzing in detail
aspects about the learning context; identifying important aspects that need to
be designed or monitored in m-learning/u-learning; evaluating the maturity of
the contributions; identifying the main challenges that should be addressed by
the communities of LD and LA; as well as identifying potential synergies of both
communities in these learning contexts.
Acknowledgments
This research has been partially funded by the European Union in the context of
CEITER (Horizon 2020 Research and Innovation Programme, grant agreement
no. 669074).
References
1. Fu, Q.K., Hwang, G.J.: Trends in mobile technology-supported collaborative learn-
ing: A systematic review of journal publications from 2007 to 2016. Computers &
Education 119, 129–143 (2018)
2. Hern´andez-Leo, D., Martinez-Maldonado, R., Pardo, A., Mu˜noz-Crist´obal, J.A.,
Rodr´ıguez-Triana, M.J.: Analytics for learning design: A layered framework and
tools. British Journal of Educational Technology 50(1), 139–152 (2019)
3. Hwang, G.J., Tsai, C.C.: Research trends in mobile and ubiquitous learning: A
review of publications in selected journals from 2001 to 2010. British Journal of
Educational Technology 42(4), E65–E70 (2011)
8 Pishtari et al.
4. Hwang, G.J., Tsai, C.C., Yang, S.J.: Criteria, strategies and research issues of
context-aware ubiquitous learning. Journal of Educational Technology & Society
11(2), 81–91 (2008)
5. Hwang, G.J., Wu, P.H.: Applications, impacts and trends of mobile technology-
enhanced learning: a review of 2008–2012 publications in selected ssci journals.
International Journal of Mobile Learning and Organisation 8(2), 83–95 (2014)
6. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature re-
views in software engineering (2007)
7. Lkhagvasuren, E., Matsuura, K., Mouri, K., Ogata, H.: Dashboard for analyzing
ubiquitous learning log. International Journal of Distance Education Technologies
(IJDET) 14(3), 1–20 (2016)
8. Lockyer, L., Heathcote, E., Dawson, S.: Informing pedagogical action: Aligning
learning analytics with learning design. American Behavioral Scientist 57(10),
1439–1459 (2013)
9. Manathunga, K., Hern´andez-Leo, D.: Authoring and enactment of mobile pyramid-
based collaborative learning activities. British Journal of Educational Technology
49(2), 262–275 (2018)
10. Mangaroska, K., Giannakos, M.N.: Learning analytics for learning design: A sys-
tematic literature review of analytics-driven design to enhance learning. IEEE
Transactions on Learning Technologies (2018)
11. Melero, J., Hern´andez-Leo, D., Sun, J., Santos, P., Blat, J.: How was the activity?
a visualization support for a case of location-based learning design. British Journal
of Educational Technology 46(2), 317–329 (2015)
12. Mikroyannidis, A., G´omez-Goiri, A., Smith, A., Domingue, J.: Pt anywhere: a mo-
bile environment for practical learning of network engineering. Interactive Learning
Environments pp. 1–15 (2018)
13. Mor, Y., Ferguson, R., Wasson, B.: Learning design, teacher inquiry into student
learning and learning analytics: A call for action. British Journal of Educational
Technology 46(2), 221–229 (2015)
14. Mu˜noz-Crist´obal, J.A., Rodr´ıguez-Triana, M.J., Gallego-Lema, V., Arribas-
Cubero, H.F., Asensio-P´erez, J.I., Mart´ınez-Mon´es, A.: Monitoring for aware-
ness and reflection in ubiquitous learning environments. International Journal of
Human–Computer Interaction 34(2), 146–165 (2018)
15. Mu˜noz-Crist´obal, J.A., Mart´ınez-Mon´es, A., Asensio-P´erez, J.I., Villagr´a-Sobrino,
S., Hoyos-Torio, J.E., Dimitriadis, Y.A.: City ads: Embedding virtual worlds and
augmented reality in everyday educational practice. J. UCS 20(12), 1670–1689
(2014)
16. Persico, D., Pozzi, F.: Informing learning design with learning analytics to improve
teacher inquiry. British Journal of Educational Technology 46(2), 230–248 (2015)
17. Pimmer, C., Mateescu, M., Gr¨ohbiel, U.: Mobile and ubiquitous learning in higher
education settings. a systematic review of empirical studies. Computers in Human
Behavior 63, 490–501 (2016)
18. Pishtari, G., V¨aljataga, T., Tammets, P., Savitski, P., Rodr´ıguez-Triana, M.J., Ley,
T.: Smartzoos: modular open educational resources for location-based games. In:
European Conference on Technology Enhanced Learning. pp. 513–516. Springer
(2017)
19. Traxler, J.: Learning in a mobile age. International Journal of Mobile and Blended
Learning (IJMBL) 1(1), 1–12 (2009)
20. Wu, W.H., Wu, Y.C.J., Chen, C.Y., Kao, H.Y., Lin, C.H., Huang, S.H.: Review
of trends from mobile learning studies: A meta-analysis. Computers & Education
59(2), 817–827 (2012)
