Content uploaded by Sølvi Lilllejord
Author content
All content in this area was uploaded by Sølvi Lilllejord on Aug 16, 2018
Content may be subject to copyright.
LEARNING AND TEACHING WITH TECHNOLOGY
IN HIGHER EDUCATION
SØLVI LILLEJORD, KRISTIN BØRTE, KATRINE NESJE AND ERIK RUUD
KNOWLEDGE CENTRE FOR EDUCATION
|
1
KNOWLEDGE CENTRE FOR EDUCATION
VISITING ADDRESS: Drammensveien 288, 0283 Oslo
POSTAL ADDRESS: P.O. BOX 564, NO- 1327 Lysaker
ISBN: 978-82-12-03703-8
REFERENCE NO: KSU 2/2018
PUBLISHED: June 2018
PHOTO: Shuerstock
TITLE: Learning and teaching with technology in higher
educaon – a systemac review
REFERENCE: Lillejord S., Børte K., Nesje K. & Ruud E.
(2018). Learning and teaching with technology in
higher educaon – a systemac review.
Oslo: Knowledge Centre for Educaon,
www.kunnskapssenter.no
In collaboraon with SLATE (Centre for the Science of
Learning & Technology) at the University of Bergen
FUNDED BY: this report is funded by the Norwegian
Ministry of Educaon and Research
© 2018 Knowledge Centre for Educaon, The
Research Council of Norway, Oslo. It is permied to
quote this report for research use or other
non-commercial purposes – provided that the
representaon is accurate, that no rights are aected
and that the report is cited correctly. Any other use
requires wrien permission.
KNOWLEDGE CENTRE FOR EDUCATION
|
1
CONTENT
Summary ..............................................................................................................2
..................................................................................................................................................................................... 5
1.1 Policy iniaves ....................................................................................................................................................................... 6
1.1.1 Student learning and the need for technological competence ..........................................7
1.1.2 Suggesons for improvement...............................................................................................................................7
1.2 Status and challenges ........................................................................................................................................................ 8
1.3 Outline of the review ......................................................................................................................................................... 9
2 Method ..............................................................................................................................................................................................10
2.1 Searching and sorng .....................................................................................................................................................10
2.2 Preparaon for synthesis ............................................................................................................................................12
.................................................................................................................15
3.1 Instuonal level: Decision making ................................................................................................................16
3.1.1 Learning analycs, learning design and MOOCs ............................................................................17
3.2 Learning and teaching across contexts .........................................................................................................25
3.2.1 Lecture capture ................................................................................................................................................................26
3.2.2 Mobile learning ................................................................................................................................................................29
3.2.3 Hybrid learning contexts .........................................................................................................................................32
3.3 Emerging educaonal technologies and innovave learning .................................................34
3.3.1 Augmented Reality .......................................................................................................................................................35
3.3.2 Games and interacve response systems ..............................................................................................38
3.3.3 Pedagogical implicaons of technology use .......................................................................................43
3.4 Collaborave learning ....................................................................................................................................................45
3.5 Barriers to technology use and innovave teaching .......................................................................50
................................................54
.............................................................................................................................58
.........................................................................................................60
....................................................................................62
.......................................................63
2
|
KNOWLEDGE CENTRE FOR EDUCATION |
This systemac review was commissioned by the
Norwegian Ministry of Educaon and Research and
answers the following research queson: How can
teaching with technology support student acve
learning in higher educaon? The systemac review
was conducted in collaboraon with SLATE (Centre for
the Science of Learning & Technology) and has
explored how technology is inuencing educaonal
pracces in higher educaon instuons.
The systemac review has 5 chapters. Chapter 1,
Introducon, presents strategies and policy iniaves
for digitalisaon of Norwegian higher educaon. As a
result of an increasingly diverse student populaon
and the expected exponenal growth of demand for
educaon provision, higher educaon instuons
currently face major changes. The Norwegian Ministry
of Educaon and Research has recently taken several
iniaves to promote technology use in higher
educaon instuons, both on infrastructure, and
related to teaching and learning. The eCampus-
programme was iniated to provide accessible and
robust ICT soluons and to support the pedagogical
use of technology. In 2013, the MOOCs commission
was appointed to invesgate opportunies and
challenges arising from the emergence of Massive
Open Online Courses and similar oers. The
commission reported a series of recommendaons,
including a targeted fund, the development of a
naonal MOOC plaorm, digital competence
development for teachers, and increased use of open
educaonal resources.
A systemac mapping of the eects of ICT on learning
outcome1 showed that it is how digital tools are
implemented and used pedagogically that maer for
students’ learning outcome, not the technology itself.
1 Morgan, K., Morgan, M., Johansson, L. & Ruud, E. (2016) A systemac
mapping of the eects of ICT on learning outcomes. Oslo. Knowledge
Centre for Educaon. www.kunnskapssenter.no
This nding is conrmed in two recent reports from
NIFU 2, 3. Having found that students self-organise a
scaolding peer support system to compensate for
insucient interacon with teachers, a study of the
rst internaonal MOOC developed at the University
of Oslo, concludes that new pedagogical pracces
appears to be in the making for online learning. This
indicates that digital technologies must be integrated
into course designs and their use facilitated by
teachers4 because it is not the digital technologies per
se that solve teaching and learning challenges.
The Status report on Norwegian higher educaon5
showed that higher educaon instuons are not
fully exploing the possibilies in digital technology.
Norwegian students reported that they only to a
small degree experienced pedagogical use of digital
technology in their educaon. This problem is not
exclusive to Norway. The EU Commission6 argues that
member states should be supported in developing
naonal frameworks and infrastructure for integrang
new modes of learning and teaching across the higher
educaon system. Across OECD-countries, the
expectaon is that digital technologies and pedagogy
should be integral to higher educaon instuons’
strategies for teaching and learning, and in parallel, a
competency framework for teachers’ digital skills
must be developed.
2 Damşa, C., de Lange, T., Elken, M., Esterhazy, R., Fossland, T., Frølich, N.,
... & Stensaker, B. (2015). Quality in Norwegian higher educaon: A
review of research on aspects aecng student learning. 2015: 24
3 Nerland, M., & Prøitz, T. S. (2018). Pathways to quality in higher
educaon: Case studies of educaonal pracces in eight courses. NIFU
report 2018:3
4 Henderson, M., Selwyn, N., & Aston, R. (2017). What works and why?
Student percepons of ‘useful’digital technology in university teaching
and learning. Studies in Higher Educaon, 42(8), 1567-1579.
5 Tilstandsrapport for høyere utdanning 2018 hps://www.regjeringen.
no/no/dokumenter/lstandsrapport-for-hoyere-utdanning-2018/
id2600317/
6 European Commission (2014) Report to the EU Commission on New
modes of learning and teaching in higher educaon hp://ec.europa.
eu/dgs/educaon_culture/repository/educaon/library/reports/
modernisaon-universies_en.pdf
KNOWLEDGE CENTRE FOR EDUCATION
|
3
Chapter 2 describes the systemac review method.
Electronic searches for studies published between
2012 and 2018 were conducted in seven databases
September 2017 and January 2018. Addional
supplementary and hand searches were conducted,
and the process yielded 6526 hits. Due to the large
number of papers, text mining technology was used
to assist the idencaon of relevant studies. Aer
the rst stage of relevance assessment, 71 studies
with potenal relevance for the systemac review
were idened and read in full text. 35 studies with
high or medium quality and relevance are included in
the systemac review. A congurave synthesis
suitable for analysing ndings from heterogeneous
studies has been conducted.
Chapter 3 presents the 35 included studies, in ve
subchapters. 3.1: Instuonal level and decision
making, presents ve studies with ndings of
parcular relevance for higher educaon leaders and
administrators. These studies cover themes such as
learning analycs (LA), learning design and MOOCs
and provide informaon about big data, knowledge
ulisaon, evaluaon and big-scale iniaves that
require leaders’ aenon, funding and instuon
wide training and support to reach the potenals
inherent in new technologies. The studies show the
need for instuons to establish systems for
connuous learning, where data gathered is
systemacally transformed into acon-relevant
knowledge that can be used to design learning
environments beer adapted to students’ individual
and social needs. Successful learning designs support
student acve learning by allowing them to
communicate, produce, experiment, interact and
engage in varied forms of assessment. Learning
Analycs has the potenal to support this work
through providing useful big and small data.
In 3.2: Learning and teaching across contexts, ten
studies with relevance for department heads,
lecturers and students are presented. An underlying
assumpon in the studies is that teaching can no
longer be the sole responsibility of individual
teachers. Having invesgated the potenal
educaonal benets of a combinaon of capture
technologies (recorded lectures) and a variety of
tradional classroom pracces across digital and
physical learning contexts, studies report inconsistent
ndings. While researchers perceive capture
technologies as a potenally producve learning
design, research cannot establish posive outcomes.
A behaviourist learning paradigm, where instrucon is
perceived as content delivery, seems to dominate
higher educaon teaching pracces, even when
teachers use capture technologies. Researchers
report that both teachers and students are challenged
when learning happens across formats. Blended and
hybrid learning requires increased me commitment
from teachers, and students are expected to develop
skills in goal seng, monitoring, me management
and self-evaluaon, in addion to a range of self-
regulaon strategies. In the studies included in this
category, the need for instuonal and technical
support for sta is a major issue.
In 3.3: Emerging educaonal technologies and
innovave learning, ten studies invesgate the
potenal of emerging technologies and what is
required of instuons in terms of facilies,
organisaon and sta development for these
innovaons to impact the instuons’ teaching
pracce. It is argued that instuons must develop
policies for how they want to educate young
technology users. Augmented Reality is a promising
emerging technology with educaonal potenal as it
projects digital materials onto real-world objects,
enhances and expands students’ learning experiences
and facilitates collaboraon and student acve
learning. The included studies show that emerging
technologies, such as games, must be goal directed,
compeve, and designed within a framework of
choices and feedback to enable teachers and students
to monitor learning progress. Playing and designing
games can contribute to acve, engaging, and
authenc educaonal experiences. Introducing new
technology does not, in itself, guarantee innovave
pracces in higher educaon instuons. Instead of
taking the opportunity to introduce student acve
teaching methods, sta tends to adapt new
technologies to tradional pracce. The dichotomy
digital/non-digital should not overshadow the fact
that pedagogical quality is the most important issue
in both face-to-face and technology supported
educaonal provision.
In 3.4: Collaborave learning, ve studies are
presented. There are indicaons in the research that
when students work in groups, responsibility tends to
be dispersed. This highlights the need for learning
designs that support collaboraon and acvate each
student. Students in higher educaon are expected to
learn to argue. In academically producve talk (APT),
students build on prior knowledge and connect their
4
|
KNOWLEDGE CENTRE FOR EDUCATION |
contribuons to domain concepts to support their
claims and arguments. Encouraging students to make
their knowledge sources explicit is considered vital in
academic environments. Studies also nd that
student collaboraon happens more spontaneously in
apps designed for social media use than in more
formal learning technologies. Depending on the
design, Wikis are perceived as a favourable tool to
support collaborave learning. A review of research
on telecollaboraon reveals tradional online
pracces with email dominang the communicaon.
Researchers also ask why academics don’t recognise
their own responsibility for professional development
in the area of technology use in teaching, but expect
external iniaves.
In 3.5: Barriers to technology use and innovave
teaching, ve studies are presented. The studies show
that there are signicant barriers to technology use in
higher educaon instuons. One paradox idened
is that academics appear not to be using a scholarly
approach when implemenng technology in
educaon. Research indicates that pedagogy is a
more fundamental barrier to innovave teaching in
higher educaon than technology use. Therefore, the
conclusion in all ve studies is the obvious need to
ensure that the focus of sta development programs
in higher educaon is on instructors’ percepon of
teaching rst, and then on technology. Knowing how
to use technology is important, but not sucient, if
the instuonal goal is student acve learning.
Chapter 4 presents the congurave synthesis. The
included studies reveal a consistent paern: while
researchers assume the transforming potenal of
technology, studies nd few examples of sustainable
innovave teaching pracces in higher educaon. The
overall picture is that tradional ideas about how
students learn sll dominate and that instead of
challenging the tradion, technological devices are
adapted to the tradion. Technology is a tool with the
potenal to transform teaching and learning, facilitate
collaboraon and communicaon across contexts,
and support student acve learning. However, this
potenal is not realized unless teachers and sta use
technology in a pedagogically appropriate manner.
Researchers suggest that teachers abandon a
behaviourisc perspecve on learning and adopt a
socio-cultural, construcvist approach. This requires
that instuons priorise professional development.
Instuons should take the iniave to develop
scholarly teachers who are research-informed, inquire
into their own professional learning opportunies,
and disseminate their ndings. The status of teaching
must be heightened, the knowledge base for teaching
strengthened and an infrastructure developed for
connuous inquiry into quesons of importance for
pedagogy and didaccs.
Chapter 5 concludes and lists knowledge gaps in the
research on the use of technology in higher educaon
idened in this review.
KNOWLEDGE CENTRE FOR EDUCATION
|
5
1 INTRODUCTION
This systemac review is commissioned by the
Norwegian Ministry of Educaon and Research and
conducted in collaboraon with SLATE (Centre for the
Science of Learning & Technology)7. It answers the
following research queson:
How can teaching with technology support
student acve learning in higher educaon?
Digitalisaon inuences and challenges how
educaon is organised and administered. The
worldwide demand for higher educaon provision is
expected to grow exponenally, and over the next 10
years, e-learning is projected to grow een-fold,
accounng for 30% of all educaonal provision8. The
compeon between higher educaon instuons
increases when well-reputed instuons, such as
Harvard, Stanford and the Massachuses Instute of
Technology (MIT), provide free MOOCs. At the same
me, this opens for new opportunies9. The
Norwegian Government expect leaders and managers
in higher educaon to focus both on how technology
can contribute to a more ecient and robust sector,
and how it can be used to renew pracces and
enhance educaonal quality.
7 The Norwegian Knowledge Centre parcularly thanks Professor Barbara
Wasson for valuable input at seminars, and comments on dras.
PhD-candidate Kamila Misiejuk (SLATE) has read arcles and contributed
to seminars. Professor Konrad Morgan has read and commented on
dras, read arcles and parcipated in seminars. Researcher Tamara
Kalandadze has read arcles and contributed at the early stages of the
review.
8 European Commission (2014). Report to the EU Commission on New
modes of learning and teaching in higher educaon hp://ec.europa.
eu/dgs/educaon_culture/repository/educaon/library/reports/
modernisaon-universies_en.pdf
9 Meld. St. 18 (2014-2015). Konsentrasjon for kvalitet — Strukturreform i
universitets- og høyskolesektoren
hps://www.regjeringen.no/no/dokumenter/meld.-st.-18-2014-2015/
id2402377/
Following up the White Paper Culture for Quality in
Higher Educaon10, the Norwegian Ministry of
Educaon and Research has developed a strategy for
digitalisaon of higher educaon (2017-2021)11. As
digitalisaon and new plaorms take a more
prominent place in the sector, Informaon and
Communicaon Technology (ICT)-soluons impact the
quality of educaon and research. The use of learning
analycs to understand students’ learning paerns
and improve learning processes, is sll in its infancy12,
but is expected to assist instuons in reaching the
goal of improving student learning, broadly facilitate
study opons, and support outstanding research. The
interacve use of technology for knowledge
development must be elevated to a strategic level at
higher educaon instuons and integrated into all
academic and administrave acvies. How
technology is developed and used must therefore be
an integral part of naonal and instuonal
strategies.
The Norwegian higher educaon sector is at the
forefront of co-operaon on digital soluons, with
eecve infrastructure soluons and joint services for
administrave tasks, educaon, and research.
Nevertheless, there is signicant potenal for quality
improvement by exploing exisng and new ICT
soluons, and these aims are outlined for data and
infrastructure, students and teachers:
10 Meld. St. 16 (2016–2017). Kultur for kvalitet i høyere utdanning
hps://www.regjeringen.no/no/dokumenter/meld.-st.-16-20162017/
id2536007/
11 hps://www.regjeringen.no/no/dokumenter/digitaliseringsstrategi-for-
universitets--og-hoyskolesektoren---/id2571085/
12 The MOOC Commiee’s proposal to establish an environment for
research-based knowledge development, development work, and
knowledge-sharing related to learning analysis was followed up through
the establishment of the Centre for the Science of Learning &
Technology (SLATE) in 2016 by the Norwegian Ministry of Educaon and
Research with the University of Bergen as the host instuon.
6
|
KNOWLEDGE CENTRE FOR EDUCATION |
: Data is stored once
and made available from a single source. Data is
retrievable, available, interoperable, and reusable in
accordance with the FAIR principles. Infrastructure is
exible and facilitates mobility and development.
Cohesive governance and management of
informaon security are fundamental to digitalisaon
and strategic eorts.
: Students have access to a modern
and exible learning environment that facilitates
individual and collaborave learning. They parcipate
in an academic community where technology is
integrated in acve and varied methods for teaching
and assessment, and provide students with advanced
academic and digital qualicaons. When parcipang
in research projects (research-based teaching),
students learn principles and pracces of research.
: Teachers have high levels of digital
and pedagogical skills, incenves for the development
of their own teaching, access to support services and
collegial communies. They are familiar with a wide
range of applicaons, digital tools and services that
support teaching, from planning, through interacon
with students and colleagues, to the follow-up and
evaluaon of students at individual and group level.
Based on documented results, teachers can be
remunerated or given me to further innovate their
pedagogical pracce.
1.1 POLICY INITIATIVES
In recent years, the Norwegian Ministry for Educaon
and Research has taken several iniaves related to
digitalisaon in higher educaon instuons; both on
quesons of technology and infrastructure, as well as
changes in teaching and student acve learning.
In White Paper no. 18 (2012-2013) Long-term
perspecves – knowledge provides opportunity13, the
Government calls for a strengthened eort regarding
high-quality higher educaon, free access to learning
resources along with relevant competence and skills
development, by establishing the ve-year and NOK
70 million eCampus program. ICT-supported exible
educaon ensures equal access to higher educaon,
and instuons are expected to cooperate on the
13 Meld. St. 18 (2012–2013), Lange linjer – kunnskap gir muligheter
hps://www.regjeringen.no/no/dokumenter/meld-st-18-20122013/
id716040/
exible use of professional resources and
technological soluons. Digital learning resources can
lower the thresholds to higher educaon, by
facilitang access, independent of geography, age and
other factors. When evaluang the eCampus
program,14 NIFU15 found that the program has
succeeded in providing accessible and robust ICT
soluons and have promoted the use of ICT based
tools. However, the use of ICT tools varies across
dierent instuons.
In June 2013, a Commission16 was appointed by the
Norwegian Government to invesgate the
opportunies and challenges arising from the
emergence of Massive Open Online Courses (MOOCs)
and similar oers. The Commission should map the
development of MOOCs and provide
recommendaons on how Norwegian authories and
instuons should relate to technological
developments. The report showed that MOOCs were
not central to the strategic planning of Norwegian
universies and colleges and not perceived as tools
for pedagogical development. A tradional,
instrucon-based model for online educaon seemed
to be the most widely used. The Norwegian
Commission on MOOCs reported a series of
recommendaons including a targeted fund, the
development of a naonal MOOC plaorm, digital
competence development for teachers, and more use
of open educaonal resources. Studies17 on
digitalisaon at Norwegian higher educaon
instuons indicate that digital innovaons are not
necessarily anchored in instuonal strategies, but
driven by individual enthusiasts. Studies also indicate
that newly trained teachers lack the sucient digital
skills18, also conrmed by the MOOC Commiee19.
Several instuons have developed MOOCs with
support from the Norwegian Agency for Digital
Learning in Higher Educaon. New digital assessment
14 Tømte, C., Aanstad, S., og Løver, N. (2016) Evaluering av eCampus-
programmet, NIFU rapport 2016:44
15 Nordic Instute for Studies in Innovaon, Research and Educaon
16 NOU 2014: 5 MOOC l Norge. Nye digitale læringsformer i høyere
utdanning
17 Norwegian Agency for Digital Learning in Higher Educaon, Digital
lstand 2014, which follows on from corresponding surveys from 2008
and 2011.
18 cf. Norwegian Ministry of Educaon and Research’s digitalisaon
strategy for basic educaon (2017-2021)
19 NOU 2014:5 MOOC for Norway. New digital learning methods in higher
educaon.
KNOWLEDGE CENTRE FOR EDUCATION
|
7
methods are being developed20, and exams are
digitalised.
A study of how Naonal Governments and instuons
shape the development of MOOCs nds ve central
movaons for adopng MOOCs in Norwegian higher
educaon: 1) strengthen the quality,2) increase
access, 3) recruit students and promote Haigher
Educaon Instuons, 4) increase cooperaon, and 5)
reduce costs21. A study of the rst internaonal
MOOC developed at the University of Oslo nds that
students self-organize and establish a scaolding peer
support system to compensate for insucient
interacon with teachers. The study concludes that
new pedagogical pracces appears to only be in the
making for online learning 22.
1.1.1
White Paper no. 16 (2016-2017) Culture for Quality in
Higher Educaon highlights student learning and
teaching23. One objecve is that all students should
experience smulang and varied learning and
assessment methods where digital opportunies are
exploited. The White Paper further states that
technological tools can help students get the best
possible educaon and feedback, also in large student
groups. Educaon should be based on knowledge of
how students are best educated and developed. While
nine out of ten students report that digital tools are
important in their daily student life, only half believe
that the tools help them learn beer. There are many
indicaons that learning management systems are
more successful in managing learning than supporng
the pracce of learning, as instuons do not priorize
implemenng digital tools in curricula, subject
descripons and work requirements. There are many
high quality open learning resources available online.
Student response systems can be a way of engaging
the students. Flipped classroom, where students
prepare for the lecture in advance, allows the teacher
20 Both the Norwegian Agency for Digital Learning in Higher Educaon and
SLATE are central to these development eorts.
21 Tømte, C. E., Fevolden, A. M., & Aanstad, S. (2017). Massive, Open,
Online, and Naonal? A Study of How Naonal Governments and
Instuons Shape the Development of MOOCs. The Internaonal
Review of Research in Open and Distributed Learning, 18(5).
22 Singh, A. B., & Mørch, A. I. (2018). An Analysis of Parcipants’
Experiences from the First Internaonal MOOC Oered at the University
of Oslo. Nordic Journal of Digital Literacy, 13(01), 40-64.
23 Meld. St. 16 (2016–2017)- Kultur for kvalitet i høyere utdanning
hps://www.regjeringen.no/no/dokumenter/meld.-st.-16-20162017/
id2536007/
to spend me discussing with the students. Video
recording of lectures and/or podcasts give students
possibilies for repeons. Digital learning combined
with more tradional classroom learning (blended
learning) appear to be eecvely enhancing learning.
The long-term plan for research and higher
educaon24 shows that digitalisaon also closes the
gap between educaon and working life by allowing
students to work more acvely with the subject
maer. By allowing each student to choose when he
or she wants to focus on the study material, it opens
for collaboraon between instuons, as well as with
the business community, trade and industry. However,
as emphasised in a report from the EU commission25,
students are unique, and so is the way they learn.
Teaching tools used in universies and colleges should
therefore cater for individual learning, with the
student at the centre. Digital media can facilitate
more acve, problem-based learning which has been
demonstrated to encourage greater student
engagement and improved learning outcomes. Some
learn beer with the help of interacve media with
images, graphics, videos and audio as incorporated
elements. Technology can combine these for a
personalised learning experience, based on individual
strengths.
The EU-report further stresses that teaching sta
must be equipped with the necessary skills and
knowledge to allow them to fully ulise the range of
new teaching tools. New technologies and associated
pedagogies require a very dierent skill-set from
more convenonal teaching. Academic sta are not
all technology experts, and many have had lile or no
pedagogical training. If they are to deliver quality
teaching with technology, they need specic training,
guidance and support.
1.1.2
Digital technologies and pedagogy should be an
integral element of higher educaon instuons’
strategies for teaching and learning, and in parallel, a
competency framework for higher educaon
teachers’ digital skills must be developed. The EU
24 Meld. St. 7 (2014-2015) Long -term plan for research and higher
educaon 2015-2024
25 European Commission (2014) Report to the EU Commission on New
modes of learning and teaching in higher educaon hp://ec.europa.
eu/dgs/educaon_culture/repository/educaon/library/reports/
modernisaon-universies_en.pdf
8
|
KNOWLEDGE CENTRE FOR EDUCATION |
Commission26 argue that member states should be
supported in developing naonal frameworks and
infrastructure for integrang new modes of learning
and teaching across the higher educaon system.
Legal frameworks that allow higher educaon
instuons to collect and analyse learning data must
be developed at naonal level. The full and informed
consent of students is a requirement and the data
should only be used for educaonal purposes. Online
plaorms should inform users about their privacy and
data protecon policy and individuals should always
be allowed to anonymise their data.
The importance of research leadership in the
development of outstanding research is
acknowledged, and the same principle applies for
outstanding educaonal achievements. The Long-
term plan for research and higher educaon 2015-
202427 emphasises closer collaboraon between
research- and educaon environments. Developing
clusters for internaonal, cross-disciplinary
cooperaon, combining educaon, research and
innovaon, will increase the relevance of the studies
and can contribute to making academic work more
engaging for the students.
1.2 STATUS AND CHALLENGES
When presented in May 2018, the Status report on
Norwegian higher educaon28 showed that higher
educaon instuons are not fully exploing the
possibilies inherent in digital technology. While 76 %
of students reported that digital tools provide
exibility and freedom and are important for their
studies29, these tools were infrequently or not used.
Moreover, 42 % of Norwegian students reported that
they only to a small degree experienced pedagogical
use of digital technology in their educaon. When
teachers use digital tools, less than 50 % of the
students report that the use supports student acve
learning. How digital tools are used for assessment
purposes diers immensely. A forthcoming arcle
26 European Commission (2014) Report to the EU Commission on New
modes of learning and teaching in higher educaon hp://ec.europa.
eu/dgs/educaon_culture/repository/educaon/library/reports/
modernisaon-universies_en.pdf
27 Meld. St. 7 (2014-2015) Long -term plan for research and higher
educaon 2015-2024
28 Tilstandsrapport for høyere utdanning 2018 hps://www.regjeringen.
no/no/dokumenter/lstandsrapport-for-hoyere-utdanning-2018/
id2600317/
29 NOKUT`s Studiebarometer shows student`s percepons about quality of
their study program, hp://www.studiebarometeret.no/en/
from the expert group at Norgesuniversitetet on
digital assessment30 nds that the lack of competence
is a huge challenge when using digital tools for
assessment purposes. There is too lile knowledge
about alternaves to the tradional school exam, but
also lile understanding of how digital tools can be
used in assessment.
A report on ICT in teacher educaon31 focuses upon
how teachers learn to teach by using digital tools. The
report nds that the development of professional
digital competence is weakly anchored in the
management and leadership of teacher educaon
instuons and most instuons lack an integrated
approach for competence development. Moreover,
the competence amongst the academic sta varies,
and the development of teacher students’ digital
competence are oen dependent upon enthusiasts.
This is not sustainable, and will aect teacher
student’s possibilies to make pedagogical use of ICT
when they become teachers themselves.
A systemac mapping of the eects of ICT on learning
outcome32 showed that ICT has an impact on learning
outcome when technology is implemented as a
planned part of a comprehensive teaching
environment with clear goals, teaching plans,
teaching materials, supporng technical resources,
teacher training and development. Hence, it is how
digital tools are being implemented and pedagogically
used that maer for students’ learning outcome, not
the technology itself. This nding is later conrmed in
two reports33 34. It is not the digital technologies per
se that solve teaching and learning challenges. Digital
technologies must be carefully integrated into course
designs and their use must be facilitated by
teachers35.
30 hps://norgesuniversitetet.no/ekspertgruppe/digital-vurdering
31 Tømte, C., Kårstein, A., & Olsen, D. S. (2013). IKT i lærerutdanningen: På
vei mot profesjonsfaglig digital kompetanse?. NIFU report 20/2013
32 Morgan, K., Morgan, M., Johansson, L. & Ruud, E. (2016) A systemac
mapping of the eects of ICt on learning outcomes. Oslo. Knowledge
Centre for Educaon. www.kunnskapssenter.no
33 Damşa, C., de Lange, T., Elken, M., Esterhazy, R., Fossland, T., Frølich, N.,
... & Stensaker, B. (2015). Quality in Norwegian higher educaon: A
review of research on aspects aecng student learning. 2015: 24
34 Nerland, M., & Prøitz, T. S. (2018). Pathways to quality in higher
educaon: Case studies of educaonal pracces in eight courses. NIFU
report 2018:3
35 Henderson, M., Selwyn, N., & Aston, R. (2017). What works and why?
Student percepons of ‘useful’digital technology in university teaching
and learning. Studies in Higher Educaon, 42(8), 1567-1579.
KNOWLEDGE CENTRE FOR EDUCATION
|
9
The introducon has shown that the challenges when
it comes to ulizing the potenal of technology and
digitalisaon in educaon are related to leadership,
infrastructure, and competence. The systemac
review has analysed and synthesised 35 arcles about
pedagogical use of technology and innovave learning
and teaching in higher educaon, and concludes with
prerequisites for how teaching with technology can
support student acve learning.
1.3 OUTLINE OF THE REVIEW
The systemac review is outlined as follows: Chapter
2 presents the systemac review method, literature
search, sorng, quality and relevance assessment of
the arcles included in the systemac review. Chapter
3 presents the 35 included arcles, organised in ve
subchapters: 3.1 Instuonal level: Decision making,
3.2 Learning and teaching across contexts, 3.3
Emerging educaonal technologies and innovave
learning, 3.4 Collaborave learning, 3.5 Barriers to
technology use and innovave teaching. Secons 3.4
and 3.5 highlight themes that cross through all the
studies. In Chapter 4 the studies are synthesised, and
chapter 5 concludes, gives recommendaons and
shows knowledge gaps.
10
|
KNOWLEDGE CENTRE FOR EDUCATION |
2
A key characterisc of systemac reviews is
transparency and the presence of an explicit method
that describes and determines their conduct36. This
systemac review takes the form of a rapid review37,
performed to synthesize qualitave and quantave
studies as well as literature reviews and systemac
reviews. The rapid review method is a developing
format that may be perceived as a compromise
between what is expected from a systemac review,
and policy-makers’ need for evidence to be available
in a shorter me than the 1-2 years it typically takes
to conduct a full systemac review38. Rapid reviews
have been dened as brief, readable, and usable
responses to guide decision making, typically
completed within 6 months39. While they dier in
format, the similarity of rapid reviews lies in their
close relaonship with the end-user to meet decision-
making needs in an idened meframe. Rapid
reviews are systemac and transparent, and follow
the same quality- and relevance assessment
procedures as systemac reviews, but make
limitaons to nish the work in a shorter me span.
Typical limitaons are: searching fewer databases;
liming the use of grey literature; narrowing the
36 Gough, D., Oliver, S. Thomas, J. (2017). An introducon to systemac
reviews. London: Sage Ltd.
37 Khangura, S., Konnuy, K. Cushman, R., Grimshaw, J. and Moher, D.
(2012): Evidence summaries and the evoluon of a rapid review
approach, Systemac Reviews, 1-10.
Featherstone, R. M., Michelle, D. M., Guise, J-M., Mitchell, M.D.,
Paynter, R. A., Robinson, K. A., Umscheid, C. A., and Hartling, L. (2015):
Advancing knowledge of rapid reviews: An analysis of results,
conclusions and recommendaons from published review arcles
examining rapid reviews. Systemac reviews 4:50.
38 Thomas, J., Newman, M. and Oliver, S. (2013): Rapid evidence
assessment of research to inform social policy: taking stock and moving
forward, Evidence & Policy, 9 (1), 5-27
39 Andradas, E., Blasco, J. A., Valenn, B., López-Pedraza, M. J., & Gracia, F.
J. (2008). Dening products for a new health technology assessment
agency in Madrid, Spain: a survey of decision makers. Internaonal
Journal of Technology Assessment in Health Care, 24(1), 60-69.
scope; restricng the type of studies included etc40. In
this systemac review, the following limitaons are
made 1) only studies published in peer-reviewed
journals are included; 2) systemac searches are
limited to studies published aer 1. January 2012;
and 3) language is limited to arcles published in
English, Norwegian, Swedish or Danish.
The systemac review answers this research queson:
How can teaching with technology support
student acve learning in higher educaon?
2.1 SEARCHING AND SORTING
Having idened concepts that are central to the
research on digitalisaon of higher educaon, a search
string with search words was developed and several
trial searches conducted in electronic databases. Main
electronic searches were conducted 25.09.17 and
28.01.18 in seven databases: Educaon Collecon,
Applied Social Sciences Index and Abstracts (ASSIA),
Internaonal Bibliography of the Social Sciences (IBSS),
Educaon Database, Educaon Resources Informaon
Center (ERIC), Psycinfo and Scopus. The searches were
conducted with free text and themac words in tle
and abstract, and resulted in 6513 hits. Appendix 1
shows the search string with the Scopus syntax. In
addion, a hand search was conducted 14th and 15th
December and supplementary searches 12.12.17;
02.01.18 and 07.02.18. The included arcles cover the
publicaon period 2012 to 2018.
40 Hartling, L., Guise, J. M., Kato, E., Anderson, J., Aronson, N., Belinson, S.,
... & Mitchell, M. (2015). EPC methods: an exploraon of methods and
context for the producon of rapid reviews. Research White Paper. .
(Prepared by the Scienc Resource Center under Contract No.
290-2012-00004-C.) AHRQ Publicaon No. 15-EHC008-EF. Rockville, MD:
Agency for Healthcare Research and Quality hps://www.ncbi.nlm.nih.
gov/books/NBK274092/pdf/Bookshelf_NBK274092.pdf
KNOWLEDGE CENTRE FOR EDUCATION
|
11
Title and abstract of all the hits from the
literature searches were imported to the
soware EPPI-reviewer 4, developed for
systemac reviewing by the EPPI-centre at
the University College, London42.
Preparing the data for synthesis requires a
three-stage process, following pre-dened
criteria. At the rst stage, arcles are read
and assessed on tle and abstract. At the
second stage, arcles are read in full-text.
At the third stage, data is extracted from
the arcles, described and prepared for
synthesis. Figure 1 illustrates the two rst
stages of the sorng process in this
systemac review:
Stage 1
Table 1 provides an overview of the pre-determined
inclusion criteria used in the sorng process.
Table 1. Inclusion criteria
INCLUSION CRITERIA EXPLANATION
1. Theme The study must address innovave use of ICT, how technology inuences
teaching and/or promotes student acve learning.
Higher educaon.
The arcle must be published in a peer-reviewed journal.
The arcle must be published in English, Norwegian, Swedish or Danish.
Include arcles with above average rangs.
Include arcles with above average rangs.
41 Gough, D., Oliver, S. Thomas, J. (2017). An introducon to systemac reviews. London: Sage Ltd.
Electronic searches: 6513
Hand search: 13
Relevance assessment based
on tle and abstract
Excluded
6455
Step 1
Quality and relevance
assessment based on full text
Arcles included in the systemac
review
Excluded
36
Step 2
6526
35
71
Figure 1. Flow diagram
12
|
KNOWLEDGE CENTRE FOR EDUCATION |
Due to the large number of publicaons idened in
the database searches, text mining technology
integrated in the EPPI-Reviewer 4 soware, called
machine learning42, was used to expedite the
idencaon of relevant research. Machine learning is
an iterave process by which the machine learns from
the researchers which arcles should be included or
excluded. The machine makes connuous relevance
calculaons and sorts the data so that the most relevant
arcles are added rst in the screening process. Aer
screening a limited number of arcles, most of the
relevant arcles are idened. This technology makes it
possible to screen large amounts of data in less me43.
The inclusion criteria number 5. Citaon index and
number 6. Scimago Journal Rank Indicator were
applied the following way: The total number of
citaons for each arcle was idened in Google
Scholar, and the number of citaons per year
calculated, not counng the publicaon year. Having
calculated the annual average number of citaons for
all arcles; arcles with above average rangs were
included. This ensures that arcles have high quality
and relevance within their eld of research.
42 Thomas, J., & O'Mara-Eves, A. (2011). How can we nd relevant
research more quickly? In: NCRM Methods News. UK: NCRM; 2011. p. 3.
43 O'Mara-Eves, A., Kelly, M. P., & Thomas, J. (2014). Pinpoinng needles in
giant haystacks: use of text mining to reduce impraccal screening
workload in extremely large scoping reviews. Research Synthesis
Methods, 5(1), 31-49.
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S.
(2015). Using text mining for study idencaon in systemac reviews: a
systemac review of current approaches. Systemac reviews, 4(1), 5
Wallace, B. C., Trikalinos, T. A., Lau, J., Brodley, C., & Schmid, C. H.
(2010). Semi-automated screening of biomedical citaons for systemac
reviews. BMC bioinformacs, 11(1), 55.
As arcles normally have few citaons the rst year(s)
of publicaon, arcles published in 2017 and 2018
were assessed based on the Scimago Journal Rank
indicator (SJR indicator), a measure of scienc
inuence of scholarly journals that accounts for both
the number of citaons and the presge of the
journals cing the arcle. The 2016 SJR indicator was
obtained from the Scopus tle list index. Only arcles
published in journals with above average ranking
were included.
Aer the relevance assessment on stage one based
on tle and abstract, 71 arcles with potenal
relevance for the systemac review were idened.
Stage 2:
At the second stage, the 71 arcles with potenal
relevance were read in full text. Two researchers
assessed, independently, the studies’ quality and
relevance for the review. Table 2 gives an overview of
the quality criteria used. The studies are scored high,
medium or low. Aer the second step, 35 arcles
remained, and are included in the systemac review.
2.2 PREPARATION FOR SYNTHESIS
To synthesize the included arcles an overview of the
data material is needed to facilitate data extracon.
First a mapping is conducted. The mapping show that
the arcles are from 14 dierent countries and
published between 2012 and 2018. Table 3 show the
mapping on country based upon the rst author`s
aliaon.
Table 2. Criteria for assessing quality
CRITERIA FOR QUALITY ASSESSMENT ASSESSMENT VALUE
•
•
•
•
•
•
High: Explicit and detailed descripon of method,
data collecon, analysis and results; the
interpretaons/analysis are clearly supported by
the ndings.
Medium: Sasfactory descripon of method, data
collecon, analysis and results; the interpretaons/
analysis are parally supported by the ndings.
Low: Weak descripon of method, data collecon,
analysis and results; interpretaons/analysis have
lile support in the ndings.
KNOWLEDGE CENTRE FOR EDUCATION
|
13
Table 3. Mapping of country
COUNTRY NUMBER OF STUDIES
5
Canada 1
1
1
2
1
Korea 1
1
1
3
1
2
UK 8
USA 6
TOTAL 35
The mapping further shows that 4 studies have used
quantave methods, 10 have used qualitave
methods, 10 studies are based on both quantave
and qualitave methods, 2 papers are theorecal and
7 papers are reviews (3 systemac reviews and 4
literature reviews). 2 papers have used mixed
methods. 20 studies are scored with high quality,
15 with medium quality and none with low quality.
Appendix 2 shows method used and quality of the
arcles.
Having mapped the papers on theme, the included arcles were categorised as follows:
CATEGORY ARTICLES
Avella et al. (2016); Rienes & Toetenel (2016); Lee, Morrone
& Siering (2018); Maringe & Sing (2014); Toven-Lindsey et al.
(2015).
•
•
•
Wion (2017); Al-Nashash & Gunn (2013); Hung, Kinshuk &
Chen (2018); Dennen & Hao (2014); Pimmer, Mateescu &
Gröhbiel (2016); Cochrane (2014); Mesh (2016); Wanner &
Palmer (2015); Blau & Shamir-Inbal (2017); Ali et al. (2017).
•
•
•
Wang (2017a); Blanco-Fernandez et al. (2014); Lameras et al.
(2017); Vlachopoulos & Maki (2017); Edmonds & Smith
(2017); Wang (2017b); jones & Benne (2017); Barak (2017);
Ng'Ambi (2013); Van Es et al. (2016).
Tegos et al. (2016); Akiyama & Cunningham (2018); Newland
& Byles (2014); Rambe & Bere (2013); Zheng et al. (2015).
Amemado (2014); Kirkwood & Price (2013); Shelton (2017);
Sinclair & Aho (2018); Walker, Jenkins & Voce (2017).
14
|
KNOWLEDGE CENTRE FOR EDUCATION |
A congurave synthesis
Once the arcles are categorised, data is extracted
and each arcle is briey summarised. The goal is to
elicit the meaning of the study, an idiomac
translaon44. The brief summaries make it possible to
analyse and synthesize the studies to idenfy
common paerns across the data. Synthesis is an
analyc acvity that generates new knowledge and
understanding in response to the review`s research
queson, and a synthesis is normally more than
simply the sum of its parts45. A congurave synthesis
aims to nd similaries between heterogenous
studies, even when they use dierent concepts to
describe similar events46, which is the case in this
systemac review. Translaon is central to
congurave synthesis, and the ambion is to
contribute to claricaon, theory development, and
conceptual innovaon. The synthesis results in a
narrave that answers the research queson by
idenfying transcending paerns in the included
44 Noblit, G.W. & Hare, R.D. (1988) Meta-ethnography: Synthesizing
qualitave studies. Sage`s university paper series on Qualitave
research methods volume 11, California: Sage publicaons
45 Gough, D., Oliver, S., & Thomas, J. (2017). An introducon to systemac
reviews, p.182, London: Sage.
46 Etymologically, congure means to piece together parts to form an
overall picture.
studies47. The goal is not simply to list the ndings,
but to interpret ndings from each study in a way that
contributes to new knowledge. Data sources in
systemac reviews are the included studies, and the
synthesising process aims at translang the studies
into each other48 or make them talk to each other49 to
generate insights that transcend each study’s
contribuon.
Based on analysis of the brief summaries, two
transcending paerns were idened across the
studies: 1) From content delivery to student acve
learning and 2) Professional development of sta. To
analyse the paerns in depth, all the arcles were
uploaded to NVivo Pro 11, and coded accordingly.
Data extracts concerning student acve learning,
collaboraon and professional development and
training were analysed in depth, before the studies
were synthesised.
47 Popay, J., Roberts, H., Sowden, A., Pecrew, M., Arai, L., Rodgers, M., &
Duy, S. (2006). Guidance on the conduct of narrave synthesis in
systemac reviews. A product from the ESRC methods programme.
Version, 1.
48 Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing
qualitave studies (Vol. 11). Sage.
49 Gough, D., Oliver, S., & Thomas, J. (2012). An introducon to systemac
reviews, p. 188, London: Sage.
KNOWLEDGE CENTRE FOR EDUCATION
|
15
3
Chapter 3 presents the 35 included arcles. Figure 2,
below, shows how the chapter is organised into ve
subchapters (3.1. – 3.5).
Chapter 3 Overview
3.4. Collaborave learning
3.5. Barriers to technology use
3.1. Instuonal level:
Decision making
Learning analycs
3.2. Learning and teaching
across contexts
Lecture capture
3.3. Emerging technologies
Agumented Reality
MOOCs Blended learning
Flipped learning
Learning desing
Pedagogical implicaons
GamesMobile learning
Figure 2. Overview of chapter 3
In 3.1.: Instuonal level: Decision making, studies
with relevance for policymakers and higher educaon
leaders and administrators are presented. These
studies cover themes such as learning analycs (LA),
learning design and MOOCs and provide informaon
about big data, knowledge ulisaon, evaluaon and
big-scale iniaves that require leaders’ aenon,
funding and instuon-wide training and support if
they are to reach the potenals inherent in new
technologies. Learning analycs is a vast and rapidly
growing research eld with the potenal to generate
informaon instuons can use when designing
learning. Designing producve learning environments
is, however, a very complex task that cannot solely be
the responsibility of individual sta members.
Instuons must develop policies that state how they
want students to learn, iniate and lead change
processes and follow up with data analysis, training
and support.
Subchapter 3.2.: Learning and teaching across
contexts, presents studies where the underlying
assumpon is that teaching no longer can be the sole
responsibility of individual teachers. To gain status,
teaching must be a more knowledge-informed acvity
with work processes beer aligned with those
16
|
KNOWLEDGE CENTRE FOR EDUCATION |
academics use when they engage in research. Data
gathered through learning analycs can be used to
design innovave learning environments where
students and teachers collaborate to reach the broad
spectre of learning goals. The studies presented here
have researched potenal educaonal benets of
combining digital and physical learning environments
and focused on characteriscs of learning designs that
may enhance student learning. The studies cover
themes such as lecture capture, mobile learning,
blended and ipped learning.
The potenal of educaonal benets is even more
strongly emphasised in sub chapter 3.3.: Emerging
educaonal technologies and innovave learning,
where the presented studies show promising
emerging technologies and what is required of
instuons, facilies, leaders and sta for these
innovaons to be an integral part of the instuons’
teaching pracce.
The two last subchapters, 3.4. and 3.5., are visualised
as crossing themes because all the included arcles
stress the educaonal benet of collaborave
learning and most studies nd barriers to innovave
teaching. In 3.4.: Collaborave learning, collaborave
learning approaches in online learning and teaching
are presented, for instance how conversaonal agents
may promote academically producve interacons,
modalies and pracces in telecollaboraon, what
promotes and hinders collaborave technology use in
higher educaon and social learning pracces with
apps and wikis.
In 3.5.: Barriers to technology use and innovave
teaching, studies nd barriers to technology use in
higher educaon instuons, and argue that these
barriers may also explain why teaching in higher
educaon instuons largely remains prescripve and
teacher-centered, even when the intenon is a
student-acve approach to learning.
3.1
While new technologies open the way for new
possibilies, they also bring praccal, nancial and
ethical issues that go beyond the responsibility of
individual sta members, teams or departments. This
rst chapter therefore presents ve studies that have
invesgated quesons related to digitalisaon of
higher educaon with implicaons for the
instuonal level, i. e. top level strategists, managers
and administrators, faculty, and/or department
leadership.
Studies show that for implementaon to succeed,
leaders must develop policies and guidelines, make
funding available and provide the necessary training
and competence development for sta and students.
The rst three studies give an overview of the
emerging eld of learning analycs and how learning
and teaching can be designed, based on systemac
analysis and ulisaon of big data. The fourth study
describes developing trends in higher educaon and
the last study describes challenges encountered when
developing, running and renewing MOOCs.
AUTHORS COUNTRY HAVE INVESTIGATED METHODS USED
USA Learning analycs Systemac review
UK Learning design Mulple regression models
USA Pedagogy, space, technology Convergent parallel mixed
methods design, triangulaon
(interview, surveys, syllabi)
South
Africa
Development trends in HE Theorecal
USA Pedagogical tools used in
MOOCs
Qualitave mul-case study
analysis
KNOWLEDGE CENTRE FOR EDUCATION
|
17
3.1.1
and MOOCs
The advancement of technology has provided the
opportunity to track and store students’ online
learning acvies as big data sets. The purpose of
learning analycs (LA) in such a context is to tailor
educaonal opportunies to individual learners’
needs and abilies, such as providing adapted
feedback and mely instruconal content. While
there is no universally agreed denion of learning
analycs, it refers to acvies such as the
measurement, collecon, analysis and reporng of
data about learners and their context, with the
purpose to understand and opmise learning and the
environment in which it occurs50. There is a growing
interest in how instuonal data can be used to
understand academic retenon, for instance to
idenfy students’ paern of behaviour in online
educaon to improve students’ learning, gure out
how teaching can be more engaging and increase
retenon rates.
Learning analycs is a mul-disciplinary approach
based on data processing, technology-learning
enhancement, educaonal data mining, and
visualisaon51, more specically the process of
systemacally collecng and analysing large data sets
from online courses, with the purpose to improve
learning processes52. LA can help learners and
educators make construcve decisions and more
eecvely perform their tasks. Analycs refers to the
scienc process that examines data, presents paths
to make decisions and formulates conclusions53.
Examples of concepts frequently used in this research
eld, and their meaning, is presented here:
50 hp://www.laceproject.eu/faqs/learning-analycs/
51 Scheel, M., Drachsler, H., Stoyanov, S., & Specht, M. (2014). Quality
indicators for learning analycs. Journal of Educaonal Technology &
Society, 17(4), 117.
52 Brown, M. (2012). Learning analycs: Moving from concept to pracce.
EDUCAUSE Learning iniave. hps://library.educause.edu/
resources/2012/7/learning-analycs-moving-from-concept-to-pracce
53 Picciano, A. G. (2012). The evoluon of big data and learning analycs in
American higher educaon. Journal of Asynchronous Learning Networks,
16(3), 9-20.
CONCEPT MEANING OF CONCEPT
Big Data The capability of storing large
quanes of data over an
extended period and down to
the parcular transacon.
Data analycs The scienc process that
examines data to formulate
conclusions and to present
paths to make decisions.
Educaonal data
mining
Academic
analycs
Learning analycs
Data mining uses algorithms
to solve educaonal issues
and develop new
computaonal data analysis
methods. Academic analycs
is an applicaon of business
intelligence methods and
tools to performance and
decision-making in the
educaonal instuons.
Learning analycs tries to
improve student learning and
learning environments
through methods such as
predicve analysis, clustering,
and relaonship mining.
Learning analycs integrates and uses analysis
techniques such as data mining, data visualisaon,
machine learning, social network analysis, semancs,
arcial intelligence and e-learning. Social network
analysis (SNA) analyses relaonships between
learners as well as between learners and instructors
to idenfy when students are engaged or
disconnected. Visual data analysis includes highly
advanced computaonal methods and graphics to
expose paerns and trends in large, complex
datasets54. Other methods are predicaon, clustering,
relaonship mining and discovery with models.
Researchers currently argue that LA should take a
social turn as most research aims at predicng
individual performance. They fear that simple LA
metrics (e.g. number of clicks, number of downloads)
may hamper the advancement of LA research and
argue that “simple” LA metrics provide limited insight
54 Examples are Gapminder, IBM Many Eyes, FlowingData and Visualizaon
community.
18
|
KNOWLEDGE CENTRE FOR EDUCATION |
into the complexity of learning dynamics55 and the
relaonal nature of teaching and learning. While
clicking behaviour explains around 10 % of variaon
in academic performance; movaon, emoons and
learners’ acvies account for 50 % of the variaon.
conducted a systemac review
with the ambion to answer three quesons: What
does the research on learning analycs say about
methods used in LA; what does it say about benets
of using LA, and what does it say about challenges
encountered when using LA? A systemac search,
with the explicit goal to nd empirical studies,
generated 112 arcles. Among these, 10 addressed
methods, 16 focused on benets and 18 on
challenges. The next secon presents and summarises
how the included arcles answer the three review
quesons:
methods
Learning analycs begins with leaders who are
commied to decision-making based on instuonal
data. This commitment must be reected in the hiring
of administrave sta, skilled at data analysis, and
training sta in understanding the potenal and
proper ethical conduct of data-driven decision-
making. Five stages of data capturing are idened56:
1) reporng the data paern and trends; 2) predicng
a model based on the data; 3) acng by using an
intervenon based on the model to 4) improve
learning and teaching and, 5) rening the developed
model. Researchers suggest a macro-level process
perceiving the LA process as a ow of informaon in
the system, from the students to the stakeholders
within the framework of a hierarchy or a cycle57,
where researchers collect data from the students,
process the data into metrics, use the results to
perform an intervenon, and collect addional data
for the next iterave cycle.
55 Tempelaar, D. T., Rienes, B., & Giesbers, B. (2015). In search for the
most informave data for feedback generaon: Learning Analycs in a
data-rich context. Computers in Human Behavior, 47, 157-167.
56 Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic
analycs: A new tool for a new era. EDUCAUSE review, 42(4), 40.
57 Clow, D. (2012). The learning analycs cycle: closing the loop eecvely.
In Proceedings of the 2nd internaonal conference on learning analycs
and knowledge (pp. 134-138). ACM.
Clow, D. (2013). An overview of learning analycs. Teaching in Higher
Educaon, 18(6), 683-695.
benets
Avella et al. (2016) found that careful analysis of big
data may help stakeholders to elicit useful
informaon that can benet educaonal instuons,
students, instructors, and researchers. The benets
are listed and exemplied below:
STAKEHOLDER
BENEFITS REPORTED
BY RESEARCH
EXAMPLES
Targeted course
oering
By examining trends,
instuons can predict
graduate numbers for
long-term planning
Curriculum
development
Analysing big data, educators
can determine weaknesses in
student learning and
comprehension and use this
for improvement purposes
Students' learning
process, learning
outcomes and
behaviour
Data analysis helps educators
understand the students'
learning experience
Personalised
learning
LA allows for real-me
recepon, review and
incorporaon of data, and
real-me feedback to
students
Improved instructor
performance
Data analysis can idenfy
areas in need of
improvement by the
instructor to facilitate
enhanced instructor-student
interacons
Post-educaonal
employment
opportunies
Using big data can help
stakeholders beer assess
student learning programs
for vocaonal compability
Improved research
in the eld of
educaon
Researchers can more easily
share informaon and
collaborate, idenfy gaps and
accumulate knowledge
KNOWLEDGE CENTRE FOR EDUCATION
|
19
challenges
Avella et al. (2016) found the following learning analycs challenges:
AREAS OF CHALLENGE EXAMPLES
Data tracking Monitoring via Learning Management Systems, Plaorms (Moodle, Canvas, EPIC,
Blackboard); informaon about student log-in, involvement, how engaging the
curriculum presented is, which areas that cause confusion
Data collecon Availability of resources, viable social plaorm, dicules in sharing proprietary
informaon, compeon between bidders instead of teamwork
Data evaluaon and
analysis
For LA to help instructors, data must be delivered mely and accurately.
Technical challenges, errors may occur when manually conducng data analysis
Connecon with learning
sciences
To opmise learning requires understanding how to support knowledge
development, connecng cognion, metacognion, and pedagogy
Learning environment
opmisaon
Individual and social learning analycs, beer understanding of the learning
context. Research focusing on LA and pedagogy is sll in the early stages
Emerging technologies Learning analycs develops as new technologies emerge.
Ethical concerns, legal and
privacy issues
Privacy consideraons such as consent, data accuracy, how to respect privacy,
maintain anonymity, opng out of data gathering. Data interpretaon,
ownership, sharing, who owns aggregate data. Four guiding principles: 1) Clear
communicaon; 2) Care; 3) Consent and 4) Complaint.
The review revealed that LA is an interdisciplinary
eld that selects and uses methods and analysis
techniques from other disciplines to achieve the goal
of improving educaon. Mechanisms must provide
transparency, data controls by students, informaon
security, and accountability safeguards. The research
eld of Learning Analycs also stresses the ethical
implicaon of data collecon and use58 and DELICATE
is one suggested framework:
Decide on the purpose of learning
analycs for your instuon.
Dene the scope of data collecon and
usage.
Explain how you operate within the legal
frameworks, refer to the essenal legislaon.
Talk to stakeholders and give assurances
about the data distribuon and use.
Seek consent through clear consent
quesons.
De-idenfy individuals as much as
possible
Monitor who has access to data,
58 DELICATE, developed within the LACE-project hp://www.laceproject.
eu/ethics-privacy/
especially in areas with high sta turn-over.
Make sure externals provide
highest data security standards.
used mulple regression
models when linking 151 modules and 111.256
students with student behaviour, sasfacon and
performance at the Open University (OU), UK. The OU
has used learner feedback to improve students’
learning experience and learning designs for 30 years,
and academic retenon ranges between 34,46% and
100%, with an average of 69,35%. Learning design
(LD) is described as a methodology for enabling
teachers/ designers to make more informed decisions
in how they go about designing learning acvies and
intervenons, which are pedagogically informed and
make eecve use of appropriate resources and
technologies59.
The study aims to gure out to what extent learning
design decisions made by teachers predict student
engagement, sasfacon and academic performance.
Virtual Learning Environment (VLE) data was collected
59 Conole, G. (2012). Designing for learning in an open world. Dordrecht:
Springer, p. 121.
20
|
KNOWLEDGE CENTRE FOR EDUCATION |
per module: a) average me spent on VLE per week;
b) average me spent per session on VLE.
Learning design is process based and follows a
collaborave design approach in which praconers
make informed design decisions with a pedagogical
focus. Five categories describe opons available for
teachers to create an interacve, social learning
environment where acvies are 1) Communicave;
2) Producve; 3) Experimental; 4) Interacve; and 5)
Assessed.
This is an overview of OULDI60 learning design acvies:
LABEL TYPE OF ACTIVITY EXAMPLE
Assimilave Aending to informaon Read, watch, listen, think about, access
Finding/handling
informaon
Searching for and processing List, analyse, collate, plot, nd, discover,
access, use, gather
Communicaon Discussing module content with
at least one other person
Communicate, debate, discuss, argue,
share, report
Producve Acvely construcng an artefact Create, build, contribute design, construct,
Experienal Apply learning in real-world
seng
Pracce, apply, experience, mimic, explore,
invesgate
Interacve/adapve Apply learning in simulated
seng
Explore, experiment, trial, improve, model,
simulate
Assessment All forms Write, present, report, demonstrate,
crique
60 Open University Learning Design Iniave (OULDI)
KNOWLEDGE CENTRE FOR EDUCATION
|
21
The dependent variable was academic retenon (the
number of learners who completed and passed the
module relave to the number of learners who
registered for each module). Analycs data included
the level of the course, the discipline, year of
implementaon, size of class or module. All data were
collected on an aggregate, module level. Learning
design (LD) data was merged with virtual learning
environment (VLE) and learner retenon data based
upon module ID and year of implementaon.
Posive correlaons were found between nding
informaon and communicaon, and between
producve and experienal outcomes. Total workload
was posively related to communicaon and
experienal and negavely to assessment, indicang
that teachers dedicated relavely more me for
learning acvies and less for assessment.
The study nds that learning design acvies strongly
inuenced academic retenon, with the relave
amount of communicaon acvies and me spent
on communicaon as primary predictors, controlling
for instuonal and disciplinary factors. As the focus
in online learning tends to be on designing for
cognion rather than social learning acvies, this is
an important nding. A second nding is that learner
sasfacon was strongly inuenced by learning
design, while learner sasfacon and retenon was
not. This may indicate that learning at mes can be
hard and dicult, and not always a pleasant
experience. Universies must consider how they can
balance designing learning acvies that stretch
students to their maximum ability, while keeping
students happy.
Rienes & Toetenel (2016) conclude that learning
design had a signicant and substanal impact on
learner experience. Communicaon seemed to be a
key lever for retenon in blended and online distance
educaon at the OU. Modules with more assimilave
and fewer inquiry and discovery-based learning
acvies were perceived to lead to beer learner
experiences. Separate analysis indicated that
assimilave acvies signicantly and posively
predicted learner sasfacon. To enhance academic
retenon, a way forward may be appropriate,
well-designed communicaon tasks that align with
the learning objecves of the course.
invesgated
instruconal components and class acvies that
support acve learning in a collaborave learning
studio (CLS) with 29 students, and how spaal and
technological features reect design and
implementaon processes. Acve learning is used
about instruconal approaches that acvely engage
students in the learning process through
collaboraon, cooperaon and discussions, rather
than having them passively receive informaon from
their instructors. Data were collected through
interviews with faculty (semi-structured) and students
(focus-group), surveys (faculty and students) and
syllabi for courses taught in the CLS.
TECHNOLOGIES
Designed for acve learning
Small-group acvies
Movable chairs and monitors in U-shaped student tables
Video wall, control panels, push capabilies,
projector screens, student monitors, instructor
desktop, wireless microphone, document cameras,
student desktops and push-to-talk microphone on
student tables.
The lecture was an essenal component in most
courses, used to frame learning content for students,
communicate main ideas before and aer group
acvies and to invite guest lectures. Students
generally found the collaborave learning space
helpful in their learning (n=25), but some students felt
it hindered learning (n= 7). Most students (23 of 29)
favored group acvies and 11 reported lectures as
least favorable.
Four collaborave learning paerns were revealed: 1)
lecture – group acvies – class-wide discussion (5
courses); 2) lecture – group acvies almost daily (3
courses); 3) lecture – group acvies once in a while (1
course); 4) group acvies – class-wide discussion (1
course). Students and faculty rated group acvies as
working best, either computer-based, non-computer
based group discussions, paper-based or physical
group acvies. Class wide discussion typically started
with a presentaon of group work, followed by
22
|
KNOWLEDGE CENTRE FOR EDUCATION |
instructor and student comments and was a central
component of collaborave learning, as it allowed
students to reect on their own group acvies and
connect group work to the course content.
The combinaon of lectures and discussion
emphasises the importance of exibility in classroom
design. Training of how to use the in-room
technologies is valued, and the faculty interviewees
said they needed me to explore the technologies to
know how to implement new pedagogical approaches
and beer ulise the room features. Also, mely
technical assistance was important.
idenfy four drivers in Higher
Educaon: 1) Massicaon; 2) Mobility; 3)
Markesaon; and 4) Stagnang sta numbers. In a
theorecal arcle they address issues of large,
demographically diverse university classes, dened as
“any class where the number of students pose both
perceived and real challenges in the delivery of
quality and equal learning opportunies to all
students in that classroom” (p.763). Four pedagogical
principles underpin the equity dimension. A)
Increased student parcipaon and engagement
requires teachers to provide prior readings, allowing
students to summarise their thoughts on the topic
before the session, create buzz-groups etc.; B)
Increased curricula access requires teachers to ensure
that students have access to teaching material; C)
Increased sta intercultural understanding requires
teachers to engage students in discussions on how
they may benet from the course; D) Increased
opportunies for deep learning for all requires
teachers to inspire students through crical
engagement with texts and the applicaon of
conceptual ideas in designing research quesons and
empirical invesgaons. Maringe & Sing (2014) also
idenfy four quality measures of crical importance
for a quality learning experience in HE: 1) Connuous
monitoring of student sasfacon; 2) Increased
opportunies to achieve; 3) Diversicaon of
assessment and 4) The potenal of MOOCs.
have
invesgated frequently used pedagogical tools in 24
MOOCs and provide a brief history of MOOCs before
presenng their study.
A MOOC is a model for educaon delivery typically
dened as “massive, with theorecally no limit to
enrolment; open, for anyone to parcipate, usually at
no cost; online, with learning acvies taking place
over the web; and a course, structured around a set
of learning goals in a dened area of study”61. The
term massive open online course (MOOC), was rst
used in 2008, to describe a course on learning theory
taught by George Siemens and Stephen Downes at
the University of Manitoba62. The original ambion
was to create an open, collaborave online learning
community centred around “the acve engagement
of several hundred to several thousand students who
self-organise their parcipaon according to learning
goals, prior knowledge and skills, and common
interests”63. Since 2012, when private companies
including Coursera and Udacity were established, the
goals of the MOOC movement have shied to
encompass the massicaon of exisng courses and
potenal for revenue generaon. Empirical research
on teaching strategies and learning outcomes
associated with MOOCs is limited.
Although there is signicant variaon in pedagogical
approaches, most courses sll ulise tradional
classroom methods (lectures, group discussions and
mulple-choice assessment). Research nds that
students are more sased with online courses that
include higher levels of interacon and reecon64
and a major challenge for MOOC instructors has been
opportunies for interacon and engagement
between students and the instructor as MOOCs oen
rely on automated instruconal tools and compleon
rates have been extremely low65.
The inial pedagogical model of MOOCs focused on
incorporang high levels of learner control, oering
synchronous, or real-me, sessions with the facilitator
and other speakers, providing a digital artefact that
summarised course acvies (i.e. parcipant blogs,
61 Educause (2013). Seven things you should know about MOOCs II.
Educause learning iniave
(Retrieved from hp://net.educause.edu/ir/library/pdf/ELI7097.pdf).
62 Parry, M. (2010, August 29). Online, bigger classes may be beer
classes. The chronicle of higher educaon Retrieved from hp://
chronicle.com/arcle/Open-Teaching-Whenthe/124170.
63 McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC
model for digital pracce. Retrieved from. hps://oerknowledgecloud.
org/sites/oerknowledgecloud.org/les/MOOC_Final_0.pdf
64 Arbaugh, J. B. (2000). How classroom environment and student
engagement aect learning in Internet-based MBA courses. Business
Communicaon Quarterly, 63(4), 9–26
Vonderwell, S., Liang, X., & Alderman, K. (2007). Asynchronous
discussions and assessment in online learning. Journal of Research on
Technology in Educaon, 39(3), 309–328.
65 Parr, C. (2013, May 10). Not staying the course. Inside Higher Ed
(Retrieved from hp://www.insidehighered.com/news/2013/05/10/
new-study-low-mooc-compleon-rate).
KNOWLEDGE CENTRE FOR EDUCATION
|
23
posts, online discussion), developing dynamic social
systems as a means of parcipant organizaon and
collaboraon66. Students are assessed automacally,
by their peers, or engage in self-assessment. MOOCs
require that parcipants be self-directed and have a
level of crical literacy adequate to navigate the
course and engage in the learning community67. While
more experienced and independent students may
thrive in this environment, many parcipants struggle
with the lack of structure and instruconal support
inherent in courses68.
MOOCs are expected to be disrupve, and transform
higher educaon by creang a ‘revoluon’. Yet, at
present, the major providers are developing open
online courses that mimic tradional face-to-face
courses with a focus on measurable learning
outcomes, which may se creavity among
instructors and developers.
66 McAuley et al. (2010) op.cit.
67 Kop, R. (2011). The challenges to connecvist learning on open online
networks: Learning experiences during a massive open online course.
The Internaonal Review of Research in Open and Distance Learning,
12(3), 19–38.
68 Kop, R., Fournier, H., & Mak, J. S. F. (2011). A pedagogy of abundance or
a pedagogy to support human beings? Parcipant support on massive
open online courses. The Internaonal Review of Research in Open and
Distance Learning, 12(7), 74–93.
In their study, Toven-Lindsey, Rhoads and Lozano
(2015) invesgated the range of pedagogical tools
used in 24 MOOCs from public and private
universies, private companies, and not-for-prot
enterprises, covering several topics and disciplines
(social sciences, humanies and STEM) and consider
the extent to which these courses provide students
with high-quality, collaborave learning experiences.
The study answered the following research quesons:
1. What instruconal tools and pedagogical pracces
are being ulised in MOOCs?
2. How are new digital and networked technologies
impacng the delivery of MOOCs?
3. To what extent are MOOCs able to provide a space
for crical inquiry and acve student engagement
in the learning process?
Data was collected by reviewing the curriculum,
content and various instruconal elements of the
online courses. The Teaching Approach Framework69
was used to idenfy and categorise the pedagogical
tools, and pedagogical approaches idened were
grouped in four categories – objecvist-individual,
objecvist-group, construcvist-individual and
construcvist-group:
69 Arbaugh, J., & Benbunan-Fich, R. (2006). An invesgaon of epistemolo-
gical and social dimensions of teaching in online learning environments.
The Academy of Management Learning and Educaon, 5(4), 435–447.
EPISTEMOLOGICAL
DIMENSION
sources
SOCIAL DIMENSION INDIVIDUAL GROUP INDIVIDUAL GROUP
PEDAGOGICAL
APPROACH
• Video
recordings
• Computer
graphics
• Text-based
lessons and
assignments
• Discussion
board
• Assignments/
exams
submied to
deadlines
• Open-ended,
short-response
quesons in
assignments and
quizzes
• External
resources;
websites, open
access textbooks,
reports, online
labs, simulaons
• Peer-reviewed
wring
assignments
• Group acvies
or debates on the
discussion board
• Live video
conferencing with
the instructor
24
|
KNOWLEDGE CENTRE FOR EDUCATION |
1. All 24 MOOCs
had an objecvist individual approach; 18 used
text-based lessons and readings, illustraons,
simulaons, and review quesons to encourage
engagement; 22 used video recordings
(PowerPoints with voiceover instrucon,
recordings of the instructor speaking directly into
the camera, an animated whiteboard, recordings
from a tradional classroom seng, full animaon
or use of an avatar)70.
2. The objecvist-group
is based on a one-way transmission of content
from the instructor, requiring students to
collaborate on group assignments, and was
common in MOOCs with a specied start and end
date. 11 MOOCs used a pre-determined meline
for instrucon and an online discussion board to
encourage student interacon. Students generally
moved through the material at the same me,
accessed informaon weekly and submied
assignments/exams by specic deadlines.
3. Eight MOOCs
used open-ended, short-response quesons in
assignments and quizzes. Students could compare
their response to a computer-generated answer
key provided by the instructor, but were
encouraged to ulize external resources, including
websites, textbooks, reports, and online labs and
simulaons. In six MOOCs students were
encouraged to engage with the material and
reect on learning in their context.
4. encourages
collaboraon and crical inquiry among
parcipants. While none of the MOOCs in this
study ulized this approach for the majority of
course acvies, one third of the courses
incorporated a construcvist-group acvity in
some way, including peer-reviewed wring
assignments, group acvies or debates on the
discussion board, and live video conferencing with
the instructor.
Five courses were based on open-ended quesons
and required wrien responses that were reviewed
by fellow students. Students earned points for
parcipaon more than substance, and course
discussion boards showed mixed reviews of the
eecveness of the peer-review process.
70 Example from the open Yale course in Toven-Lindsey, Rhoads & Lozano,
(2015, p. 6)
While peer-reviewed wring assignments can be a
highly useful tool, students in MOOCs complete these
acvies independently and with limited opportunity
for collaboraon. Even discussion boards do not
necessarily encourage group collaboraon and
learning since students generally just respond to
quesons posted and do not engage in a dialogue on
the topic71.
An objecvist-individual approach would be
appropriate if the goal is to increase eciency by
making instrucon scalable to an unlimited audience.
Transfer of knowledge from expert to novice is,
however, insucient If the goal is to use technology
to enhance instruconal quality and provide
meaningful learning opportunies. Only in a few of
the MOOCs, and with mixed results, did instructors
use the boards to post discussion topics, requiring
students to comment, or iniang group acvies.
The dominance of the objecvist approach raises
quesons about the kind of knowledge that is valued
in open online educaon.
Even though the objecvist-individual teaching
approach was prevalent, nearly half of the courses
incorporated at least one instruconal tool that
encouraged parcipants to acvely link curriculum to
real world sengs, or interact with fellow learners.
Compared to courses in other elds, MOOCs in the
hard sciences were less likely to incorporate
construcvist teaching approaches.
If MOOCs are to achieve the revoluonary potenal
ancipated, the focus should be on creang a
community of learners and give students an
opportunity to deepen their understanding through
collaborave learning.
has idened
internaonal trends in higher educaon such as
massicaon, diversity, mobility, personalisaon and
stagnang sta numbers. These trends emphasise
that instuons must establish systems for
connuous learning, where data gathered is
systemacally transformed into acon-relevant
informaon that can be used to design learning
environments beer adapted to students’ individual
and social needs. Learning Analycs has the potenal
to provide useful big and small data for this work.
71 Example criminal law, Toven-Lindsey, Rhoads & Lozano (2015, p. 8)
KNOWLEDGE CENTRE FOR EDUCATION
|
25
Central to learning designs adapted to student acve
learning is possibilies to invesgate, communicate,
produce, experiment, interact, and parcipate in
varied forms of assessment.
3.2 LEARNING AND TEACHING ACROSS
CONTEXTS
This chapter presents ten studies researching the
potenal educaonal benets of a combinaon of
recorded lectures and a variety of tradional
classroom pracces across digital and physical
learning contexts. The researchers are interested in
which learning designs or characteriscs of designs
may enhance student learning. The more overarching
term used for this category of studies is capture
technologies, and the specic labels used are lecture
capture, mobile learning and ipped learning. First,
three studies on lecture capture, webcast lectures
and interacve video lecture are presented; then
three studies on mobile learning, and nally four on
blended and ipped learning designs.
AUTHOR COUNTRY HAVE INVESTIGATED METHODS USED
UK Lecture capture Pilot – evaluated by a
survey
Emirates Students benets and drawbacks of
using webcast lectures
Survey, focus groups
interview and stascs
Taiwan Interacve video lecture Experiment – between
subjects design
USA A framework for mobile learning Authors' own case
descripons
Switzerland Mobile learning Systemac review
New Zealand Web 2.0 Parcipatory acon
research
Italy The use of blended learning in
university based language courses
Descripve study
including comparave
data on student
performance
Australia Students and teachers' percepons
of a ipped classroom course
including exible assessment
Surveys and focus group
interviews
Israel Students percepons of teaching
and learning processes in a ipped
learning course
Qualitave analysis of
students' wrien
reecons
Korea Development of a learning plaorm
for ipped learning
Descripon of a learning
plaorm
26
|
KNOWLEDGE CENTRE FOR EDUCATION |
3.2.1
Before presenng the three studies on lecture
capture (Wion, 2017; Al-Nashash & Gunn, 2013;
Hung et al., 2018) a brief background descripon and
introducon to concepts is given.
According to Wion (2017), capture technologies are
commonly referred to as Lecture Capture, and
typically used to record lectures. It refers to a
combinaon of soware and hardware that will
record any combinaon of audio, video, presentaon
slides etc. that can be viewed online, at any me,
from any place and on any device. These terms are
used in the capture technology research:
TERMS USED EXAMPLE OF USAGE
The system used to create and distribute recorded and live streamed video
content.
The capture system plus all devices associated with the capture process,
including computers, cameras, microphones and mobile devices.
Any learning content created and distributed using the capture system (e.g.,
recorded lectures).
Pre-recorded informaon viewed by students in advance, providing an
opportunity for group-work.
Pre-recorded demonstraons of acvies viewed by students in advance
(laboratory exercises etc).
Addional learning materials (e.g., short clips) created ad-hoc to enhance
standard curriculum.
The lecturer anonymizes students' quesons and records a response for the
whole group.
Content is captured o-campus, such as eldwork or examples from the
workplace.
Most published studies on capture technologies have
focused on the use and impact of recorded lectures,
linking lecture capture with student sasfacon. The
research shows that students adapt their use of the
available captured content depending on their
individual learning needs and that student learning
increases when sta deliberately incorporate
captured material into their overall educaonal
approach72. While ipping the classroom can improve
student performance73, it does not always make
students more sased74. Lile or no research has
shown posive impact on student aainment and a
72 Marchand, J., Pearson, M., & Albon, S. (2014). Student and faculty
member perspecves on lecture capture in pharmacy educaon.
American Journal of Pharmaceucal Educaon, 78(4), Arcle 74, 1-7.
73 Baepler, P., Walker, J., & Driessen, M. (2014). It's not about seat me:
blending, ipping and eciency in acve learning classrooms.
Computers & Educaon 78, 227-236.
74 Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping
the classroom to improve student performance and sasfacon. Journal
of Nursing Educaon, 52(10), 597-599
few studies report detrimental impact on academic
performance resulng from the availability of
recorded lectures75.
explores the use of capture
technologies at the University of Wolverhampton,
where an award- winning science center (the Rosalind
Franklin Building) was designed with no tradional
teaching spaces (no classrooms, no lecture theatre,
no podiums, or projectors). Flipped classroom
pedagogy inuenced the design with the vision to
facilitate acve parcipaon and all informaon
delivery by video. Pre-recorded demonstraons allow
students to prepare, reect, and review before their
laboratory sessions.
75 Johnston, A., Massa, H., & Burne, T. (2013). Digital learning recording: a
cauonary tale. Nurse Educaon in Pracce, 13(1), 40-47.
KNOWLEDGE CENTRE FOR EDUCATION
|
27
A small-scale pilot collected just over 100 hours of
content with over 1000 hours of viewing by students,
and the usage gures revealed a large variance (2-4
hours of viewing for each hour recorded) in the
amount of captured content viewed by students
between dierent subject areas.
To evaluate the learning acvies, a survey was
distributed among 650 students (111 responded). All
respondents wanted the university to connue with
capture technologies and found all types of captured
content helpful to their learning, with the pre-
recorded demonstraons of praccal science the most
popular type of content. Student responses indicate
that they value exibility and playback control
provided by captured materials, as this enhances
concentraon, improves understanding and increases
condence in their own learning. This conicted with
the analycs, which idened supplementary
materials as the most viewed type of content.
Academic sta (13 of 62 responded) said they would
like to make more use of the technology in the future.
Main barriers to greater engagement was workload
and lack of available me to capture new materials,
but respondents agreed that captured content would
ulmately save me. This new way of working
required a shi in their focus during face-to-face
sessions. Rather than concentrang on the how to of
scienc techniques they were able to facilitate
deeper learning by focusing on why. The evaluaon of
the pilot indicates that purposeful use of capture
technologies leads to greater engagement with the
types of captured content, which is likely to have a
posive impact on student aainment.
invesgated the use of
webcast lectures among 40 students in two electrical
engineering classes at a university in the United Arab
Emirates. Every lecture was captured by the interacve
eBeam whiteboard technology, consisng of a standard
whiteboard, a data projector, a desktop, the eBeam
edge transceiver and a stylus pen. The pen movement
is transferred to the computer and sound recorded
directly. The main disadvantage of the system is the
inability to video record the instructor while lecturing.
Survey data (n= 38), focus group interview (n= 4) and
stascs revealed that 37 out of 38 students either
strongly agreed or agreed that the videos helped
them understand the course material, and 34 of 38
thought having access to the video would raise their
course grade. Most of the students regarded the
video lectures, where they were freed from taking
notes, as an addional learning tool, not as a
replacement for the lecture. Data from the surveys
and course management system reports indicate that
students regularly view the course video contents.
Peaks were observed prior to midterm exams. Even
though the students did not see the lecturer, they sll
28
|
KNOWLEDGE CENTRE FOR EDUCATION |
found the lecture capture helpful. Main drawback was
associated with technical dicules.
developed an
embodied interacve video lecture (EIVL) with
collecve intelligence and natural user interface (NUI)
technology, and evaluated the eects of the video
lecture on learners’ comprehension, retenon and
cognive load of the learning content.
Collecve intelligence (CI) draws on interacve
learning acvies and can generate valuable
informaon for improving learning design. It
aggregates interacons undertaken by groups of
students who reect, argue and debate in discussion
forums, where knowledge grows over me and is
considered a useful educaonal resource as it helps
students comprehend the learning content from the
perspecve of many online students, who discuss as
they watch the video lectures.
Interacve learning acvies (ILAs) provide the
learner content interacon through storyboard
development, spoken scripts, pedagogical designs and
creaon of mulmedia content and typically entail
clicking on a mouse and typing with a keyboard. A
new type of human-computer interface, natural user
interface (NUI), allows students to directly interact
with the learning content through the moon-sensing
funconality of Kinect sensor instead of the mouse
and keyboard opons. The implementaon of EIVL is
guided by six scaolding funcons: recruitment,
reducon in the degree of freedom, direcon
maintenance, marking crical features, frustraon
control and demonstraon76.
The content is generally recorded when instructors give
on-site presentaons and usually includes instructor`s
voice, lecture slides, visual aids, mulmedia materials,
the lecturing environment and interacon with the
on-site audience. In this study the CI content is
categorised into four types; extended reading,
reecon, hands-on pracce, and discussion. Each type
of the CI content reects dierent levels of diculty. To
provide learner ILAs with construcve support, six
types of interacve learning acvies based on the six
scaolding funcons are delivered (see table 4 below)
76 Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem
solving. Journal of child psychology and psychiatry, 17(2), 89-100.
Table 4. Design of interacve learning acvies for embodied interacve video lectures (Hung et al., 2018 p. 120)
INTERACTIVE
LEARNING
ACTIVITY
SCAFFOLDING
FUNCTION
DESCRIPTION
Recruitment An instructor encourages the learner with a prologue, or an
audience express movaon for the content. Then, the learner
makes a simple response to the instructor or audience.
Reducon in degrees of
freedom
An audience acvely asks the learner a queson for reecon,
and the learner has 30 s to think about it. Then the audience
provides a thought related to the queson.
Direcon maintenance The learner performs an exercise or a simple simulaon related
to the content with the guidance of an instructor
Marking crical
features
An instructor provides crucial learning concepts to the learner,
allowing the learner to strengthen the impression on the
learning concept.
Demonstraon An instructor provides an example, ideal case, or soluon to
interpret a learning concept, and the learner can have a beer
understanding of the learning concept.
Frustraon control The learner can ask an instructor for assistance from a set of
selected quesons and receive a corresponding answer when
being in trouble with a learning concept.
KNOWLEDGE CENTRE FOR EDUCATION
|
29
The study design includes embodied interacve video
lecture (EIVL) with NUI technologies (experimental
group A), non-embodied interacve video lecture,
using the mouse to control the learning process
(experiment group B) and convenonal video lecture,
learning content without interacvity (control group
C). 90 students from a university in Taiwan were
randomly assigned to the three groups.
A pre-test was adopted to measure students’
language ability and prior knowledge. Two post-tests,
to measure learning outcomes, took place
immediately aer the video lecture, and seven days
later. To understand how interacve technologies
inuence learners, a cognive load quesonnaire was
used as the self-reported measurement with 9-point
Likert scale including two constructs (mental eort
and perceived diculty).
The pre-test showed no signicant dierences among
the three groups for parcipants’ prior knowledge.
The post-test showed a signicant dierence for
parcipants’ comprehension depending on video
lecture types. While the experimental group A
outperformed control group C and experimental
group B outperformed control group C, no signicant
dierences were found between the two
experimental groups. The delayed tests showed the
same paern as the post-test. There were no
signicant dierences between the three groups for
parcipants’ overall cognive load.
The post-test shows that EIVL signicantly
outperformed non-embodied IVL and convenonal VL
in comprehension, but no signicant improvement
was found between EIVL and non-embodied IVL.
Findings suggest that embodied interacve video
lecture provide learners with more learning cues and
thus can help them improve their comprehension and
benet retenon.
has shown inconsistent ndings
in the research on capture technologies. While
researchers perceive capture technologies as a
potenally producve learning design, research
cannot establish posive outcomes. Students perceive
captured content (webcasts, video lectures) as tools
that improve their understanding of the course
material. They believe that simply having access can
raise their course grade and argue that all types of
captured content are helpful to their learning.
Analycs reveal, on the other hand, that most viewed
type of content is supplementary material. This
nding may indicate that students use captured
material to broaden their understanding of the topic
instead of taking the opportunity to deepen it.
3.2.2
Three arcles report from studies on mobile learning,
and are presented here. Mobile learning has been
dened as the processes of coming to know through
conversaons across mulple contexts among people
and personal interacve technologies77. Mobile
devices are considered cultural tools transforming
socio-cultural pracces and structures in all spheres
of life78, and the educaonal use of digital mobile
technology is at the core of research labelled mobile
and ubiquitous learning.
present the M-COPE
framework for mobile learning in higher educaon,
created to support academics who use devices and
apps in their teaching. The framework was developed
to visualise the systemac interplay of components in
the mobile learning context, and to facilitate sound
decision making at each step of the design process in
both formal and informal mobile learning acvies.
Key mobile-specic consideraons were extracted,
reviewed across cases and grouped by topic.
Framework validaon occurred via a literature review
and an expert review panel.
M-COPE focuses on design of mobile learning
acvies, both instructor-facilitated in formal or
informal sengs, and learner-iniated. Instructors are
expected to consider ve crical areas: Aordances of
mobility, Condions, Outcomes, Pedagogy and Ethics.
The framework is exible; readily integrated with
established instruconal design process models, and
connuously prompts instructors to consider learning
needs and constraints79. The model shows the
M-Cope framework integrated with the generic
instruconal design model Addie (analysis, design,
development, implementaon and evaluaon):
77 Sharples, M., Taylor, J., & Vavoula, G. (2007). A theory of learning for the
mobile age. In R. Andrews, & C. Haythornthwaite (Eds.), The handbook
of e-learning research. London: Sage.
78 Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile Learning: Structures,
agency, pracces. New York, Dordrecht, Heidelberg, London: Springer.
79 The ADDIE model-Analysis, Design, Development, Implementaon and
Evaluaon - is considered the generic or baseline instruconal systems
design (ISD) model. The model promotes a focus on ve key processes
during the larger process of instruconal design.
30
|
KNOWLEDGE CENTRE FOR EDUCATION |
Desi gn phase
Developm ent pha se
Mobile: How can MD enable learning interactions that are not
otherwise possible in a specific environment/content area?
Conditions: Learners prior experience with MD and ML
Outcomes: Does le arning objectives relate to MD?
Pedagogy: Why will mobile activity support learning in a specific
con text?
Ethics: Voluntary participants, internet safety, digital footprints?
Mobile: Role of MD, elements of MD nece ssary to sup port LA,
exisiting mobile to ols used in the LA?
Conditions: N ew or ad opted m obile resources? One devi ce for
each learner? Learners own devices? Where and when does the
LA take place?
Out come s: How will MD enable learners t o meet learning
objectives?
Pedagogy: How does MD support instructional methods?
Eth ics: If «Bring your own device»-model is used, might anyone
be left out?
Mobile: Is the desired mobile functionality possible?
Conditions: Functioning of mobi le resources across pla tforms /
device types?
Out co me s: Is activit y aligned with learnin g outcomes?
Pedagogy: Conflicts between desired methods and mobile
func tion ali ty?
Eth ics: User friendliness and security of MD.
Imp leme ntat ion
phase
Evaluation phase
Analys is phas e
Mobile: MD prepared for lessons?
Conditions: Internett connection sufficient?
Out co me s: Are learning support necessary to ensure learning
outcomes met?
Pedagogy: Fa cilitation of mobile intera ctions. Tea cher ac tions
necessary to ensure learning.
Eth ics: Digital footprints from learn ers? Lear ner access to
technology needed? Learners negative emotions related to ML.
Mobile: Evaluat ion data collected automatically via MD? Did the
MD and apps function well?
Conditions: How did MD support learning? Learner attitudes?
Out co me s: Anticipated outcomes achieved? Unanticipated
outcomes?
Peda gogy: Did M D enhance the learning experience?
Eth ics: Learner s emotion s related to use of MD. Treatment of
data generated fr om ML a ctivity.
Add ie-model M-Cope promting questions for each phase
M-Cope key
elements Overarching questions to consider
Mobile
What value does using a mobile device add to
the lear ning context?
Condition
Learner preparedness, environmental
suita bility , time and d isru ption.
Outco me
Ensure t hat mobile use supp orts rathe r than
detracts from meeting the objectives of an
instruc tional desi gn. Uninten ded outcomes
important to consider.
Pedago gy
Specific instructional approaches may vary;
acti ve presentat ion, discus sion, expe riences,
probl em solving or sim ulations. T he pedagogica l
method sho uld be selecte d once the con ditions
and outcomes are known.
Ethi cs
Owners hip of educat ional produc ts (includin g
archive d c onversat ions and s ocial medi a
contr ibutions), occurring and the
appr opriateness of po tentially ub iqui tous and
non-stop engagement. Device ownership.
Challe nges concern ing digital foot prints and
stored data about learner performance.
MD: Mobile devices; ML: Mobile learning; LA: Learning acvity
The M-COPE framework supports careful
consideraon of the condions for learning, desired
outcomes and pedagogical approach related to
mobile technologies and potenal ethical issues that
arise in a mobile learning context.
conducted a systemac review
of 36 papers published between 2000 and 2013
invesgang mobile and ubiquitous learning designs.
They idened a variety of educaonal designs, with
instruconism as the most prevalent (22 studies),
followed by construconist learning (13) and situated
acon (12). A hybrid of situated, construconist and
collaborave designs characterised 6 studies.
Instruconism is rooted in behaviourism, teacher
driven, prescripve and focuses on the organisaon
of instrucon80. Technology use means, in this
tradion, using computers to instruct learners or
having computers do the instrucon. Three themes
were detected in this category: 1) Ad-hoc and
post-hoc transmission of lectures (e. g. Podcasts); 2)
Supplementary study materials (provided to students’
mobile devices) and 3) Acvaon and formave
assessment (aempts to acvate students during or
aer lectures). Studies showed, however, that
students tended to postpone reading the items they
received on their mobile devices and that podcasts
were infrequently used. Studies on acvaon and
formave assessment showed mixed results. In
80 Laurillard, D. (2009). The pedagogical challenges to collaborave
technologies. Internaonal Journal of Computer-Supported Collabora-
ve Learning, 4, 5-20.
KNOWLEDGE CENTRE FOR EDUCATION
|
31
general, the instruconal design was grounded in rote
learning and most studies measured the acquision
of simple items. Studies did not measure higher-level
learning goals, such as deeper understanding,
sense-making, or the applicaon of knowledge to new
situaons. Posive knowledge gains were frequently
explained by repeon.
Situated acon designs facilitate inquiry-based
learning and problem solving. Compared to the more
teacher-guided instruconist design, situated acon
learning happen in authenc real-life situaons.
Poorly structured learning environments were
supported by mobile devices, providing spaal,
sequenal, and cognive scaolds adapted to the
learners’ specic context, such as nursing and medical
students using PDAs to access tools that facilitated
informed decision-making in their work context.
Studies with a situated acon and scaolding design
had inconclusive results regarding learning outcome
and conceptual understanding.
Construconist learning design is centred on the idea
that learning is a sense- or meaning-making process of
knowledge construcon and co-construcon. Learning
is a process of making something that makes sense in
the real life of the learners (real objects or virtual
enes)81. Studies included in the review found that
the mulmodal and communicaon capabilies of
mobile devices support the construcon, co-
construcon, and sharing of knowledge in the form of
linguisc representaons (wrien and recorded
speech), and visual representaons. Photographs
taken with mobile devices, is menoned as a valuable
feature of this learning design.
Pimmer et al. (2016) found the hybrid studies to be
the most convincing as they integrated situated and
construconist approaches, and connected them to
the students’ experiences in more formal learning
environments. Assignments aimed to develop
mulmodal representaons in situated, real-life
learning environments enhanced the students’
situated awareness, made them observe and reect
more consciously on their experiences; and connect
their observaons with concepts from formal
educaon. Studies that involve hybridisaon provided
81 Papert, S., & Harel, I. (1991). Situang construconism. In I. Harel, & S.
Papert (Eds.), Construconism: Research reports and essays, 1985-1990.
Norwood, N.J: Ablex Publishing Corporaon.
convincing arguments for what is viewed as the core
of mobile learning: the facilitaon of learning across
mulple contexts.
Mobile learning can expand curricula by connecng
learning in and outside higher educaon
environments. For this to succeed, educators must
develop extended learning designs that link dierent
pedagogical strategies.
The review concludes that the hybridisaon of
situated, collaborave, and construconist
approaches via mobile devices can create new and
unprecedented educaonal opportunies by
connecng knowledge from formal learning sengs
with informal learning pracces. These educaonal
experiences then facilitate reecon and discussion in
the classroom. The ndings conrm previous reviews
in which most studies of mobile and ubiquitous
learning showed posive eects. As many mobile
learning projects take instruconist approaches and
many studies reveal that the tradional behavioural
learning paradigm sll dominates, the widely
expressed expectaon that mobile learning will
transform higher educaon is unlikely to be fullled.
presents ndings from a longitudinal
study invesgang the potenal of mobile web 2.0
tools to facilitate social construcvist learning82 across
mulple learning contexts. Parcipatory acon
research was used to invesgate mobile learning
(mlearning) projects from 2006 to 2011, aiming at
pedagogical transformaon. Data involved pre-project
surveys, reecve blogs and eportofolios, followed by
post project surveys and focus groups.
The project goal was to facilitate student-directed or
negoated learning. Learning acvies and
assessments were redesigned to facilitate student-
generated content published in web 2.0 porolios,
with accounts created by each student who invited
peers and lecturers into the collaborave spaces.
Four general pedagogical frameworks guided the
design and implementaon of the research:
Communies of Pracce, the Conversaonal
Framework of Pracce, Learner-Generated Contexts,
82 Social construcvist learning postulates that we learn most eecvely
by being acvely involved in knowledge construcon in groups with
guidance from more knowledgeable peers (Cochrane, 2014).
32
|
KNOWLEDGE CENTRE FOR EDUCATION |
and Authenc Learning. Data analysis of parcipant
feedback, surveys, focus groups, and journals (blogs)
from the mobile learning projects idened six
pedagogical success factors crucial to enabling
signicant pedagogical change within a course:
1. How technology is pedagogically integrated into
the course and assessment
2. Lecturer modelling of the pedagogical use
of the tools
3. Creang a supporve learning community
4. Appropriate choice of mobile devices and
web 2.0 social soware
5. Technological and pedagogical support
6. Sustained interacons facilitate ontological shis,
both for lecturers and students
Crossing these crical success factors is the sixth
factor sustained interacon facilitang ontological
shis83. Cochrane (2014) idened these shis as
necessary for signicant pedagogical change: (1)
Reconceptualising the role of the teacher (from
content deliverer to facilitator of authenc
experience), (2) Reconceptualising the role of the
learner (from passive recipient to acve co-
constructor of knowledge), and (3) A radical
conceptual shi in how we understand the
aordances of mobile social media to augment
tradional physical learning spaces and interacon.
Having compared previously idened success
factors, the key contribuons to mobile web 2.0
crical success factors idened by Cochrane (2014)
include:
1. The need for technological and pedagogical
support for matching the unique aordances of
mobile web 2.0 with social construcvist learning
paradigms.
2. The explicit scaolding of the ontological shis in
pedagogical transformaon via a structured and
sustained intenonal community of pracce
model over a signicant period.
Cochrane (2014) concludes that the Communies of
Pracce model for supporng the mobile web 2.0
projects has led to the development of collaborave
83 An ontological shi involves either a reassignment of understanding
from one ontological category to another (radical conceptual change) or
within a category (conceptual change) (Cochrane, 2014).
partnerships, resulng in increased student
engagement, deeper pedagogical reecon, and
pracce-based research outputs.
has shown that for mobile
learning to succeed, educators must create new and
extended learning designs that link dierent
pedagogical strategies. Mobile learning design must
take into consideraon expected outcome, context,
desired pedagogy, ethics, and mobile-specic
aordances. Important factors for sustained pedagogy
in mobile learning are integraon, support, interacve
use, and appropriate choice of tools. Sll, a
behaviourist learning paradigm, where instrucon is
perceived as content delivery, seems to dominate
higher educaon teaching pracces, even in mobile
learning environments.
3.2.3
Four of the included studies invesgated learning
across contexts. Flipped classroom is a form of
blended learning that can be dened as an
educaonal technique consisng of: (1) acve
face-to-face classroom learning, most of the me in
groups, and (2) online digital technologies and
well-designed self-regulated, technology-assisted
learning outside the classroom. In the ipped learning
approach, direct instrucon is delivered outside of
the classroom, through digital tools, PowerPoint-
presentaons, videos of pre-recorded lectures and
text, while class me is used for peer collaboraon
and instructor guidance.
describes the 10-year development of
blended learning for English language classes at the
University of Siena Language Centre. 21st century
learning requires that students also learn so skills,
such as intercultural communicaon, presentaon
skills, and teamwork, in addion to acquiring
language competence. The study employed a
curriculum-based approach, where course content is
developed through input from students and
educators.
Over the years, the blended-learning design changed
signicantly. Courses were taken not only by
university students but also adults, either in the form
of self-study or blended learning. Thanks to a needs
analysis, gradually more online acvies were
integrated into the course design, such as forum
discussions, wikis, videos, and online assessment.
Aer two years of using a digital workplace created
KNOWLEDGE CENTRE FOR EDUCATION
|
33
for collaborave knowledge building, a Moodle
learning environment was adopted. The number of
students grew from around 1,000 in 2005 to over
3,000 in 2015.
Data collected over one academic year (2014-2015)
suggests that blended learning methods can be as
eecve, or even more, than tradional methods.
However, blended learning is more eecve when
special aenon is given to its course design. This
conclusion was based on the comparison of blended
learning and face-to-face learning groups on student
retenon (percentage of students who registered, but
never aended a class) and academic performance
(percentage of students that passed the exam).
report from a ipped
learning approach in an advanced undergraduate
course where exible assessment was introduced
along with more choices and individualised
submission dates for 109 students. Flexible
assessment involved more learning-oriented
assessment; assessment as and for, not just of
learning. Students and teachers evaluated the
approach through surveys and focus group interviews.
The study found that students want personalised
learning with exible assessment, not only in online
acvies, but also through interacve, collaborave,
well-structured learning acvies in face-to-face
environments. This conrms previous research stang
that students prefer a blended learning approach to
fully online learning. Wanner and Palmer (2015) argue
that how teachers design learning is crucial. Flexible
and ipped learning requires instuonal support and
commied teachers, both in the process of designing,
implemenng and running a ipped learning course.
Blended learning challenges both teachers and
students. Students should be self-movated, well
organised and independent, which is oen unfamiliar
for those used to tradional teaching. The study nds
that students were concerned about technical issues,
self-movaon, remembering to do course tasks, as
well as addional workload and potenal lack of
direcon. This adds to prior research showing the
importance of encouraging student control over the
learning process.
Longer face-to-face sessions in small group acvies,
set up by the teacher for interacve, collaborave
learning benets student engagement and learning
experiences. When they had completed the learning
modules, students felt beer prepared for classroom
acvies.
There is limited evidence that ipped classroom and
personalised learning leads to beer grades and
learning outcomes. In addion, there is lile research
on what level of control is benecial for students, and
at which level of exibility higher educaon courses
are eecve in improving student engagement,
experience, and learning outcomes.
explore the core
elements of a ipped learning design with self-
regulated learning. The course builds on the holisc
ipped classroom model84 connecng the physical
classroom and online synchronous and asynchronous
environments that students can access from home or
from mobile devices. This model shis the focus from
lectures to learning, emphasising which acvies
students should complete, and how acvies should
be delivered in class or at home.
The study involved 36 students who aended the
course “Technologies and Learning Systems” at the
Open University in Israel, largely based on teamwork,
but also face-to-face, asynchronous and synchronous
lessons. Students were required to learn independently
or in small groups, while both me and place were
exible. The course website contained course readings
and videos, guidelines for assignments, schedule,
forums, links to collaborave documents, recorded
lectures, recordings of synchronous lessons,
presentaon les, and learning outcomes shared by
the students. Course content was open for eding,
allowing students to share their insights and link to
new informaon. Students discussed, in groups of
three, various study topics through online discussion
forums. Discussions were summed up in collaborave
documents and students assessed their own and peers’
performance, following evaluaon criteria developed
for each course assignment.
The tradional ipped learning model uses
technology at home as a channel for transming
informaon to students, while in the classroom it
applies a construcvist pedagogy without technology.
The re-designed ipped learning model highlights the
84 Chen, Y., Wang, Y., & Chen, N. S. (2014). Is ip enough? Or should we
use the ipped model instead? Computers & Educaon, 79, 16-27.
34
|
KNOWLEDGE CENTRE FOR EDUCATION |
important role of digital tools in class acvies.
Technology was used to support learning on all levels
including remembering, understanding, applying,
analysing, evaluang, and creang (cf. Blooms
taxonomy). Presentaon apps enabled technology-
enhanced collaborave learning acvies in- and
out-of-class. While e-accessibility of the learning
content and acve learning by individual students has
become a common pracce in higher educaon, Blau
& Shamir-Inbal (2017) argues that co-creaon of the
course content by students and co-creaon of
learning outcomes by virtual teams of students
remain rare, even though these acvies benet
students’ learning.
For ipped pedagogy to be successful, students must
acquire strategies for self-regulated learning, co-
regulaon, and shared regulaon. The re-designed
model emphasises technology enhanced embedded
assessment, were students develop self-regulaon
strategies as they co-create course content and
individual reecons are combined with peer
feedback. Based on study ndings, a re-designed
model of the holisc ipped classroom is suggested,
that considers the added value of technologies in
promong higher order thinking skills during both in-
and out-of-class learning. Five core competences for
successful learning in digital environments were
idened: communicaon, collaboraon, crical
thinking, complex problem solving, and creavity.
presents the architecture of the
Internet-of-Things Flip Learning Plaorm (IoTFLiP).
and the Interacve Case-Based Flip Learning Tool
(ICBFLT), a tool that is already used in various medical
applicaons. It provides students with virtual cases to
solve and a working scenario for case-based learning
by connecng devices in a network. The IoTFLIP was
developed as an extension to ICBFLT to improve
teaching and learning in medical educaon by
working with real paent cases.
Pedagogical approaches used in this study are ipped
learning (FL) and case-based learning (CBL), a form of
small group learning, where students try to solve a
case based on authenc data before learning the
theory. CBL can be implemented in both clinical and
non-clinical courses, and was successfully used as a
basis of the medical curriculum at the University of
Missouri. FL refers to a way to organise a course,
where students aend face-to-face lectures, but some
parts of the material are accessible online. IoTFLiP
comprises a local block with four layers (Data
Percepon, Aggregaon and Preprocessing, Local
Security, and Access Technologies) and a cloud block
with four layers (Cloud Security, Presentaon,
Applicaon and Service, and Business Layer).
ICBFLT provides virtual cases for students to solve.
The eight step working scenario is developed with
medical experts, who interview paents to collect the
data. The interview data is complemented by data
collected from wearable devices. On this basis the
expert builds scenarios for students to solve and get
feedback from the expert. The main conclusion in this
study is that there is a potenal for a successful
implementaon of the plaorm.
The study idened three main research gaps: the
need of combining CBL with FL, the potenal of using
IoT technology in medical educaon, and the
potenal of supporng CBL with IoT.
Researchers have
reported that both teachers and students are
challenged when learning happens across contexts;
face-to-face and technology enabled. Students are
expected to develop a range of self-regulaon
strategies (goal seng, monitoring, me
management and self-evaluaon). Blended and
hybrid learning requires increased me commitment
from lecturers. A major issue in the studies is the
need for instuonal and technical support for sta.
Research also shows that students appreciate the
possibilies that hybrid learning formats oer and
that blended learning is at least equal to tradional
face-to-face teaching and learning in achieving
student learning outcomes.
3.3 EMERGING EDUCATIONAL TECHNOLOGIES
AND INNOVATIVE LEARNING
Ten of the included studies address quesons of
innovave learning pracces, methods and devices,
including Augmented Reality, games, interacve
response systems, cloud pedagogy, virtual teaching
methodology and pedagogical implicaons of
emerging technologies.
KNOWLEDGE CENTRE FOR EDUCATION
|
35
AUTHOR COUNTRY HAVE INVESTIGATED METHODS USED
Taiwan Augmented Reality Quasi-experimental design
Spain Augmented Reality Design study
UK The design and use of serious
games
Evidence-based review and
synthesis
Cyprus Games and simulaons Systemac literature review
Australia Mobile learning games Observaons, surveys,
focus groups, game
analycs
Taiwan Interacve response systems,
Kahoot
Controlled experiment
Australia The digital/non-digital binary Theorecal
Israel Cloud pedagogy, web 2.0
technology
Sequenal explanatory
mixed methods design
South-Africa Emerging technologies Survey and interviews
Australia Cytopathology whole slide images
and adapve tutorials
Randomized crossover trial
3.3.1
Two arcles (Wang, 2017a and Blanco-Fernandez et
al., 2014) have focused on Augmented Reality (AR).
Before presenng the studies, a brief background is
provided85. Augmented Reality refers to technologies
that project digital materials onto real world objects86;
allow for interacon with 2D or 3D virtual objects
integrated in a real-world environment87, and enable
the addion of missing informaon in real life by
adding virtual objects to real scenes88.
85 From Jamali, S. S., Shiratuddin, M. F., & Wong, K. W. (2013). A review of
augmented reality (AR) and mobile-augmented reality (mAR)
technology: Learning in terary educaon. Internaonal Journal of
Learning in Higher Educaon, 20(2), 37-54.
86 Cuendet, S., Bonnard, Q., Do-Lenh, S., & Dillenbourg, P. (2013).
Designing augmented reality for the classroom. Computers & Educaon,
68, 557-569.
87 Dunleavy, M., Dede, C., & Mitchell, R. (2009). Aordances and
limitaons of immersive parcipatory augmented reality simulaons for
teaching and learning. Journal of science Educaon and Technology,
18(1), 7-22.
88 El Sayed, N. A., Zayed, H. H., & Sharawy, M. I. (2011). ARSC: Augmented
reality student card. Computers & Educaon, 56(4), 1045-1061.
The term Augmented Reality was coined by Caudell
and Mizell in 199289. An AR system allows for
seamlessly combining or supplemenng real world
objects with virtual objects or superimposed
informaon. As a result, virtual objects seem to
coexist in the same space with the real world and can
be applied to seeing, hearing, touching, and
smelling90. Augmented Reality research has matured
to a level that applicaons can now be found in both
mobile and non-mobile devices, and research nds
that AR increases student movaon in the learning
process91.
89 Caudell, T. P., & Mizell, D. W. (1992, January). Augmented reality: An
applicaon of heads-up display technology to manual manufacturing
processes. In System Sciences, 1992. Proceedings of the Twenty-Fih
Hawaii Internaonal Conference on (Vol. 2, pp. 659-669). IEEE.
90 Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B.
(2001). Recent advances in augmented reality. IEEE computer graphics
and applicaons, 21(6), 34-47.
91 Bacca, J., Baldiris, S., Fabregat, R., & Graf, S. (2014). Augmented reality
trends in educaon: a systemac review of research and applicaons.
Journal of Educaonal Technology & Society, 17(4), 133.
36
|
KNOWLEDGE CENTRE FOR EDUCATION |
The cinematographer Morton Heilig developed the
idea of the experience of mul-sensory
immersiveness in 1950. He intended to immerse
viewers with on-Screen acvies by incorporang all
the senses of the story into a viewer’s real-world
experience. In 1968, Ivan Sutherland invented the rst
Virtual Reality (VR) device – a head mounted display,
The Sword of Damocles. Two years later he developed
the rst AR interface design system using an opcal
see-through HMD. The rst system which allowed
users to interact with virtual objects in a real-me
applicaon was an arcial laboratory called the
Videoplace, developed in 1985. Mobile AR is a rapidly
emerging research area and includes GPS tracking,
user studies, visualisaon, and collaborave
applicaons. As a display technology m-AR could
replace the HMD, binoculars, helmets, etc. There is a
considerable amount of research published about
Augmented Reality (AR) applicaons in educaonal
contexts, but the eld is sll in its infancy; the
potenal of AR is being explored92 and we are only
beginning to understand characteriscs of eecve
instruconal designs for this emerging technology.
has integrated Augmented Reality (AR)
techniques into a digital video course to invesgate
dierent learning eects for students using online-
and AR-based blended learning strategies. In a
quasi-experiment, 103 students from two classes
were divided into one experimental group (N= 59)
92 Chen, C. M., & Tsai, Y. N. (2012). Interacve augmented reality system
for enhancing library instrucon in elementary schools. Computers &
Educaon, 59(2), 638-652.
and one control group (N=54). The instructor
designed and taught three learning topics over three
weeks. The topics taught rst and third (i.e., capon
and subtles, and special video eects) followed the
original teaching method. The instructor rst used
PowerPoint, followed by a step-by-step soware
demonstraon, and students then pracced. During
the second topic, the teacher adopted PowerPoint for
lecturing but integrated the AR-based contents for the
experiment group and online-based contents for the
control group to support their soware pracce.
Quantave and qualitave data were collected
through quesonnaires, grades of weekly learning
works, weekly learning diaries, self-evaluaon scores,
and video recordings of students’ engagement during
the pracce sessions. Students uploaded their weekly
work to the learning plaorm and the instructor
evaluated the quality of their work and registered if
the work was handed in on me.
Results showed an increase for both groups in the
percentage of students handing in their work on me
from week 1 to week 2 (when the AR-based and
online-based blended learning strategies were
adopted). Aer week three, when the learning
supports were removed, the average grade of
students in the experiment groups was slightly lower
than those of the control group. The results also show
that the AR-based blended learning environment
enhanced the students’ learning movaon. The
online-based blended learning environment was
useful for learning, but it did not prove to be helpful
for sustaining learning movaon.
KNOWLEDGE CENTRE FOR EDUCATION
|
37
The students were posive about using the learning
supports, and learning supports facilitated course
discussion. Discussions diered in the two groups; the
experiment group had lively discussions, with
students exchanging experiences of how to succeed.
Students in the experiment group had beer learning
interacons. The lecturer found the atmosphere in
the classroom vivid, the room was vibrant with
learning discussions, students moved around to check
the learning progress with peers for further learning.
In contrast, the control group was quiet, with the
students concentrang on the online contents to
complete the assigned work.
Some students in the experiment group felt busy and
unfocused when they had to pay aenon to the
teacher’s instrucon and AR-based contents. Some
had problems using the AR-based contents due to
Internet connecons, screen size of the devices, and
limited aordances for AR interacon on mobile
phones. The students preferred blended learning, and
thus gave the online-based contents more posive
feedback than the AR-based content. These ndings
support previous research arguing that the use of
various learning media might not result in signicant
dierences in educaonal outcomes, but that AR
facilitates collaborave learning and peer discussions
beer than computer-based environments.
presented REENACT, a
project exploing Augmented Reality (AR)
technologies to improve the understanding of
historical events with the aid of tacle mobile devices,
repositories of mulmedia contents, an advanced
technological facility, and a remote expert.
REENACT is organized in three stages and allow
parcipants to live the event from inside as
reenactors, and from the outside, as historians. The
study reports from a case where parcipants were
invited to relive the Bale of Thermopylae (480 BC).
Due to the re-enactment and the brainstorming
driven by the expert, the parcipants said they had
gained new perspecves on the Bale of
Thermopylae.
STAGES ACTIVITIES
Involving groups of people in the re-enactment of bales. They can physically
move around in a room, playing the acons dened for a given role by a script
of the historic event and interact with the other parcipants inside the game.
The parcipants analyse what happened in a projecon room. Having
experienced the bale, with a paral vision, they now learn to watch things
from outside, and see how their recreaon compares to the real historic events.
The expert drives a collecve brainstorming about the consequences of the
conict in the short, medium and long terms.
The data analysis revealed the following benets for
the parcipants:
• Museum educators can invite parcipants to new
types of collecve experiences, supplemenng the
experse and knowledge provided by experts.
• Museum visitors can enjoy new edutainment
aimed at improved understanding of historic
events, relying on social networking funconalies
and Augmented Reality capabilies.
• Experts can collaborate with museum educators in
new pedagogical sengs.
• Content creators/providers can nd an addional
outlet for the mulmedia contents they produce,
which can provide historically-meaningful
explanaons to situaons arisen during the
re-enactments and to arguments raised in the
debates.
Even though parcipants in the re-enactment of the
Bale of Thermopylae were pleased with the
experience, they usually asked for more videos and
3D views of the dierent locaons of the game.
3.3.1 Research nds to be a
promising emerging technology with educaonal
potenal as it projects digital materials onto real-
world objects, thereby allowing user interacon with
virtual objects. AR enhances and expands students’
learning experience as it facilitates collaboraon,
38
|
KNOWLEDGE CENTRE FOR EDUCATION |
inspires and movates students and supports
student-acve learning. Empirical research that
conrms the manifestaons of these expectaons is,
however, scarce.
3.3.2
This secon presents four arcles. Two reviews have
examined the use of games in higher educaon; the
design and use of serious games and the design,
integraon, and impact of games and simulaons.
One arcle reports from a project on mobile learning
games and one arcle from a study of the interacve
response system Kahoot!
reviewed research on design
and use of serious games (SG) in higher educaon,
asking: 1) how is the use of games for teaching and
learning conceptualised, theorised, modelled and
researched? 2) what are the essenal features of SGs
in higher educaon, and 3) how do learning aributes
match game elements as a means to opmise SG
design and students’ learning experiences? Included
in the review are 165 papers reporng conceptual
and empirical evidence on how university teachers
may plan, design and implement learning aributes
and game mechanics.
Serious games design is a relavely new discipline
that couples learning design with game mechanics
and logic. Designs for serious games involve creang
learning acvies that use the whole game or a
gaming element (e.g., leader boards, virtual
currencies, in-game hints) aiming at transforming the
student`s learning experience. Serious games have
been dened as: a mental contest, played with a
computer according to certain rules, that uses
entertainment to further government or corporate
training and educaon93. SGs are appropriate for
93 Zyda, M. (2005). From visual simulaon to virtual reality to games.
Computer, 38(9), 25-32.
educaonal purposes as they discern learning theory,
teaching and learning approaches, assessment and
feedback. Some dierenate between entertaining
and serious games, with SGs as more complex
artefacts.
To link the entertainment aspect with learning
features, two conceptual dimensions are suggested
that allow students to expand their knowledge
beyond the intended learning outcome set out by the
teacher: movaon (e.g., playing the same level more
than once) and aenon (introduce new content
along with in-game learning acvies).
In games, tasks and acvies are used synonymously,
as tasks assigned by the teacher are transformed into
student learning acvies. Outputs of some acvies
are used as inputs to others, resulng in game ows
that can be adapted while playing and learning.
Learning acvies encompass mental elements (e.g.,
to explore gravity by vising virtual planets), game
elements (e.g., a scoring mechanism) and physical
elements (e.g., a scienc tool). The evidence
whether or not SGs enhance student learning
experiences is, however, inconclusive.
Meaningful feedback encourages students to reect
on misconcepons and transfer learning to new
contexts. In games, the most common representaon
of feedback is through 1) progress bars, 2) in-game
hints, 3) scoring, 4) achievements, 5) experience
points, 6) virtual currencies, 7) prompts, 8)
assessment tools, and 9) dashboards. Feedback is
dened as a response to a learner`s performance
against criteria of quality; and feedback progress
indicators (FPI) show the current posion of a student
within a larger acvity94. The SCAMP framework95 is
used for reviewing progress:
94 Gaved, M., Kukulska-Agnes, H., Jones, A., Scanlon, E., Dunwell, I.,
Lameras, P., et al. (2013). Creang coherent incidental learning journeys
on mobile devices through feedback and progress indicators. Paper
presented at the 12th World Conference on Mobile and Contextual
Learning College of the North Atlanc, Doha, Qatar.
95 Jones, A., Gaved, M., Kukulska-Hulme, A., Scanlon, E., Pearson, C.,
Lameras, P., ... & Jones, J. (2014). Creang coherent incidental learning
journeys on smartphones using feedback and progress indicators.
Internaonal Journal of Mobile and Blended Learning, 6(4), 75-92.
KNOWLEDGE CENTRE FOR EDUCATION
|
39
Embedded in game mechanics that indicate learning acvity from students'
interacons with: Non- Player Characteriscs (NPCs), peers or teachers involved
in playing simultaneously.
E.g., formave feedback provided by the system, focusing on accuracy of
understanding and correcng misconcepons.
Atudes and moods, feelings and emoons (e.g., game gis such as extra
characters, apparels and objects for enhancing movaon).
Aims to trigger students' curiosity to start playing the game and maintain
student`s curiosity, aenon and involvement by balancing fun (game
mechanics) with learning elements to achieve engagement.
Captures students' increased competence towards mastery: The performance of
in-game learning tasks and the transfer of the knowledge gained to realisc
contexts.
It could be assumed that game design inuences how
teachers act. Teachers must support and guide
students who fail to see how to proceed to the next
level by acvely explaining rules, objecves, and
learning outcomes, and provide game-play direcons
or observe student`s acons during the game.
Teachers must, however, be aware of, and be
responsive to potenal frustraon of students who
struggle with complex or ill-dened game acvies.
Games are structured through emergence and
progression. Emergence is a game structure, specied
as a small number of rules that combine large
numbers of game variaons for which the players
must design strategies to handle. Progression is
where the player must perform a predened set of
acons to complete the game. The game designer has
control over the sequence of events, and games with
strong storytelling features are dominant as
progression games. It is generally assumed that
games should be goal directed, compeve, and
designed within a framework of choices and feedback
to enable teachers and students to monitor progress
towards the goal. Goals should be communicated by
game aributes such as a score mechanism or a
puzzle to resolve, which adds a compeve
dimension to the design.
A classicaon is developed as a research instrument,
providing guidance and support, and may be used by
game praconers or game science researchers who
intend to plan, design, and develop a serious game or
a SGs authoring environment for delivering a
parcular topic or lesson at any scale. This
classicaon is shown in Table 5, below:
40
|
KNOWLEDGE CENTRE FOR EDUCATION |
Table 5. Linking learning, game aributes, outcomes, feedback and teacher acvies (Lameras et al. 2017, p. 987)
LEARNING ATTRIBUTE GAME ATTRIBUTES OUTCOMES FEEDBACK/
ASSESSMENT
TEACHER
ACTIVITY
(Lecture, memorising concepts,
labelling diagrams and concepts,
exampling, incomplete
statements, lecture summary,
listening)
Task descripon;
choices, content
descripon,
challenge
repeon, scoring
Remembering Progress; aect
Summave
Designer/
evaluator
(web-quest, exercise solving,
carrying out scienc
experiments, reecon,
simulaons, modelling, role
playing, inquiry – pose quesons,
determining evidence, analysing
evidence, formulang evidence,
connect explanaons to
knowledge)
Game journal,
missions,
objecve cards,
storytelling,
dialogues,
puzzles, branch
tasks, research
points, study
requirements,
game levels
Understanding,
analysing
Movaonal;
progress, aect
formave and/or
summave
Player,
facilitator,
designer,
movator,
evaluator
(Brainstorming, group projects,
group web-quests, rank and
report, group of students posing
quesons to each other, group
simulaons, pair-problem solving,
group data gathering, group data
analysis, group reecon)
Role-playing,
community
collaboraon, epic
meaning,
bonuses, contest,
scoring, mers,
coins, inventories,
leader boards,
communal
discovery, game
levels
Applying
analysis,
evaluang,
creang
Movaonal,
social formave
and/or
summave
Player,
facilitator,
movator
(Guided discussions – topic
provided by teacher, open
discussions – topics provided by
students, choices: data on events
and several choices for students
to make comments, debates –
jusfying explanaons)
Nested dialogues,
NPC interacon,
in-game chats;
game levels,
research track,
maps, progress
trees
Evaluang,
understanding,
analysis
Movaonal,
aecve, social
formave
Movator,
evaluator,
facilitator
Most reviewed papers showed that the integraon of
learning elements into the design of a game creates
misconcepons, discrepancies, and uncertainty in
terms of how learning acvies, feedback, and
assessment may be used. How teachers should guide
the learning of gaming students is fuzzy and unclear.
To link teacher acvies to the game elements and
students’ learning experience is imperave to the
advancements of the eld and it is central that
teachers interact with students who construct
in-game learning experiences. How feedback is
designed and realised in the game play is key for the
learning experience and outcome.
idened seven
types of games: acon games, adventure games,
KNOWLEDGE CENTRE FOR EDUCATION
|
41
ghng games, role-playing games, simulaons,
sports games, and strategy games. The systemac
literature review examined the design, integraon,
and impact of games and simulaons in higher
educaon with the goal to nd the best pracces and
build a framework that can help educators to include
games in their own pracce to support their
pedagogical approach and teaching objecves.
123 papers were included in the review. The two most
popular genres of games were virtual/online games/
simulaon (88%) and simulaon games (42%). The
subject with the largest number of studies was
Business Management and Markeng, followed by
Biology/Health, and Computer Science. The impact of
games and simulaons was divided into three groups:
cognive outcomes, behavioural outcomes, and
aecve outcomes. Findings were compared to the
synthesized results from previous literature reviews
and meta-analyses.
Findings indicate posive impact of games and
simulaons on cognive learning outcomes including
knowledge acquision, conceptual applicaon,
content understanding and acon-directed learning. It
is, however, noted that learners’ posive outcomes
are dependent on what teachers do, such as seng
achievable learning goals, interacng with students,
promong knowledge, supporng, facilitang, and
movang them to construct new game-based
knowledge.
The review’s main ndings, divided by type of
learning outcome, is summed up here:
TYPE OF LEARNING
OUTCOME
FINDINGS LIMITATIONS
Games and
simulaons can support acon-directed
learning and deepen the understanding of
theorecal concepts.
: Games and simulaons can
help learners develop complex cognive skills,
such as problem-solving, decision-making, and
crical thinking.
There is mixed evidence on the
performance improving eect of
games and simulaons compared
to other methods. Even though
teachers could also benet from
integrang games and simulaons
in their teaching, there seems to be
a disconnect between games and
curriculum, which highlights the
important role of the faculty in
technology.
There seems to be an overall posive inuence
of games and simulaons on collaborave
learning and interacon, with a conrmed
posive eect on behavioural outcomes, such
as the development of social, emoonal, and
collaborave skills; helping students build
strong relaonship with peers; collaborate and
work in groups more eciently; become
organized; adapt to new tasks; resolve
emerging conicts.
It is more benecial to play
individually than in groups.
Collaborave playing was seen as a
distracon to achieving learning
objecves.
Games gave students fewer
opportunies to interact with other
learners and the teacher.
Most studies found that games and
simulaons had a posive eect on learners’
movaon and engagement. Aecve
outcomes include movaonal and
engagement outcomes, emoonal
development, sasfacon, self-assessment,
atude, emoon, and self-ecacy.
Excepons show that games and
simulaons are no more movang
than other learning methods.
Signicant nancial barriers (design
and development of games and
simulaons) must be taken into
consideraon.
42
|
KNOWLEDGE CENTRE FOR EDUCATION |
invesgated students’
educaonal benets of playing locaon-based mobile
learning games (LBMLG) on engagement, movaon
and learning, and how the design of LBMLG promoted
their educaonal experiences. Two approaches were
used: learning by playing a LBMLG (Study 1) and
learning by designing a LBMLG (Study 2). In the rst
approach, games were developed for undergraduate
courses, in four discipline areas, introduced during
lectures, and played by students during a tutorial, as a
self-guided acvity or eld excursion. In the second
approach, students designed and developed their
own prototype games to explore pedagogical
strategies in personalised learning. Students were
observed as they played and designed games. Online
surveys, focus groups, and game analycs were used
to understand player behaviour, sasfacon rates,
engagement, and the impact on learning outcomes.
Data was collected over a period of 3 years.
Findings suggest that playing LBMLG enhanced
students’ educaonal experiences. They enjoyed the
authencity of real world learning (85%) and
considered the game a fun way to learn (85%). They
also agreed that the LBMLG helped them to learn
more (67%), movated them to do research (54%), or
gave opportunies to pracce dierent skills (61%).
Most parcipants agreed that designing and
developing a mobile game was engaging (84 %)
cooperave (84%) and a fun way to learn (76%). Most
of them asserted that developing a game gave them
opportunity to pracce dierent skills (84%) and
implement their own ideas (84%).
The study concluded that both playing and designing
LBMLGs can provide benets by delivering acve,
engaging, and authenc educaonal experiences,
which enhance the opportunies to interact with
locaons, online content, and with each other.
Designing LBMLGs oers students an opportunity to
develop research skills (e.g., managing, operang, and
applying ICT) as they conceptualise, develop, and
implement their own ideas.
conducted a quasi-experimental study
in a course using an IRS (Interacve Response System)
developed by Kahoot! from NTNU96. Kahoot! allows
the instructor to create quizzes, discussions, and
surveys and can be used by any device with a web
browser. A quiz is projected on a canvas or screen in a
96 Norwegian University of Technology and Science
classroom, and students can join the quiz with their
personal devices. Kahoot! uses mulple choice
quesons, answered in real-me with the parcipants
compeng to achieve the highest score. Given it is
correct, the fastest answer collects the most points,
and as soon as every parcipant has submied a
response, scores appear on the screen. The IRS is
intended to interacvely engage the students by
emphasising elements of fun and play. In addion, the
plaorm provides the teacher with a greater
understanding of the students’ current knowledge.
The experiment lasted 15 weeks, with 88 parcipang
informaon and management majors from a college
in Taiwan. In the experimental group, 44 students (14
females and 30 males) used a learner as leader
strategy, meaning that the students played the role of
leaders by taking turns hosng the IRS acvies. In
the control group, 44 students (6 females and 38
males) learned with a teacher leader strategy, where
the teacher designed quesons and items and
administered the IRS acvity every two weeks. To
explore how the two strategies facilitated learning
and whether it contributed to the students’ self-
regulated learning, quesonnaire surveys were
administered at the beginning, in the middle and aer
the experiment. The students were also asked to
record their learning reecons in a weekly diary. IRS
formave tests were conducted every one or two
weeks, and test results were recorded for further
analysis.
The experiment shows that using IRS in course
teaching and learning not only facilitated interacon
between teachers, students and peers, but also
enhanced their movaon to learn the target subject
and promoted learners’ self-directed movaon. The
KNOWLEDGE CENTRE FOR EDUCATION
|
43
IRS keeps students alert and focused on what is being
taught during lectures. The compeve factor triggers
students to read the textbooks before class in order
perform beer and to correctly answer the quesons
raised by the teacher or peers. In addion, students
showed iniave in reviewing their learning
performance and scheduling learning progress aer
each IRS acvity. No signicant gender dierences
were found, with both male and female students
expressing posive feedback on using the IRS for
learning.
The data analysis found that using both teaching
strategies with the IRS acvity had posive eects on
the students’ learning. However, using the learner as
leader strategy promoted students’ learning interest
more quickly than with the instructor as leader
strategy. The learner as leader strategy promoted
interacon between the teacher and peers and
enhanced discussion in groups, especially for the
leading groups. The use of the IRS with the learner as
leader strategy beneted those who acted as leaders
in taking the iniave to learn the content, while also
engaging the students because the leaders of the
course were their classmates – not the teacher.
has
shown that games must be goal directed, compeve,
and designed within a framework of choices and
feedback to enable teachers and students to monitor
learning progress. For games to support students’
learning, teachers must provide meaningful feedback
at all stages and assist students. How feedback is
designed and performed is key for students’ learning
experience and outcome. Teachers must be aware of
and responsive to potenal frustraon of students
who struggle with complex or ill-dened game
acvies. Playing and designing games can contribute
to acve, engaging, and authenc educaonal
experiences. The IRS Kahoot! is found to keep
students alert and focused on what is being taught
during lectures and triggers students to read
textbooks before class. The evidence whether serious
games enhance student learning is inconclusive.
3.3.3
Increasingly researchers argue that the successful use
of technology in educaon is a queson of pedagogy,
rather than technology. When new, digital tools are
introduced in higher educaon, they tend to be
adapted to tradional pracces, instead of
contribung to innovaons. Four studies have
invesgated this paradox, and nd that while
academics need technological know-how and
support, professional training courses should
emphasise pedagogy over technology.
argued that the
contemporary (binary) discourse posions digital as
new, modern, superior, represenng the future; while
non-digital is the past. The dichotomy digital-future
versus non-digital past makes non-digital teaching
and learning pracces appear outdated, instead of
co-exisng. The authors’ concern is that this binary
thinking priorises digital over non-digital and that in
the rush to digise higher educaon, best-pracce
teaching and learning based on sound pedagogy may
suer. The dichotomy digital/non-digital tends to
overshadow the fact that pedagogical quality is the
most important issue in both modes of educaonal
provision.
Using Deleuze and Guaari’s image of the non-
hierarchical rhizome97 (a space of interconnected
possibilies, likened to a tree with roots and
branches), the arcle proposes to see the course as
an ecosystem, with several coexisng learning
habitats98, to promote opmal engagement for
students with diering needs. While a university
course has a (pre-dened) clear purpose, a xed set
of content, acvies, assessment standards etc., and
a series of expected learning outcomes, the rhizome
should be perceived as a complex map of non-linear
and non-hierarchical intersecons, with the capacity
to foster student engagement. The use of ecological
(instead of technological) metaphors in course design
is intended to re-empower university teachers to trust
their experience, acvate their creavity and make
pedagogically driven decisions.
has studied how teachers integrate
web-based technologies and their percepons of
cloud pedagogy (an instruconal framework to
promote social construcvist learning)99.
97 Deleuze, G., & Guaari, F. (1987). A thousand plateaus: Capitalism &
schizophrenia. Minnesota, MN: University of Minnesota.
98 Wenger, E., White, N., & Smith, J. D. (2009). Digital habitats: Stewarding
technology for communies. CPsquare.
99 Barak (2017) describes social construcvism as a learning theory
contending that cognive development is a social, meaningful process
derived from communicaon with people or from the use of mediators.
44
|
KNOWLEDGE CENTRE FOR EDUCATION |
STUDY OVERVIEW
DIGITAL COMPETENCE
48 university teachers teaching
subject-maer courses or teaching
methods courses.
A range of disciplines, varied
digital experse.
73 pre-service science teachers
aending a course focusing on
methods of teaching science and
technology.
The course implemented the cloud
pedagogy framework that ulizes
digital technologies to promote
social construcvist learning.
The cloud pedagogy framework facilitates individual
and collaborave, synchronous and asynchronous
acve learning, in class and outdoors. Cloud pedagogy
includes a social construcvism layer and a techno-
instruconal layer. The social construcvism layer
includes: (1) exploring new venues – learning by
invesgang and discovering scienc principles; (2)
co-construcng contents – learning in teams; (3)
providing and receiving feedback; and (4) increasing
engagement – learning by interacng with peers. The
techno-instruconal layer included studio-based
instrucon100, embedded assessment linking
formave and summave evaluaons to learning
acvies and cloud applicaons.
Barak (2017) found that university teachers sll
adhere to tradional, lecture-based teaching, typically
through learning management systems and mainly to
distribute learning materials or informaon. The
study revealed a paradox. While university lecturers
expect their student teachers to use advanced
pedagogy and technologies in their future school
teaching acvies, they do not themselves provide
sucient examples for such pracces when educang
teachers. Many university teachers were not up to
date with web 2.0 environments, such as Wikis, blogs,
social networks, or other cloud technologies, and
rarely used them when teaching. The potenal in
online technologies to facilitate social construcvist
pracces (small group discourse, collaborave
authoring, online peer assessment, and social
network), was not ulized. The survey data indicated
a need for professional development acvies that
can support the implementaon of web-based
technologies, student-centred instrucon and social
construcvist learning.
100 A teaching method consisng of sessions combined with longer periods
of acve learning
has invesgated technology use and
eorts to improve teaching and learning in higher
educaon. A survey was sent to academics and
support sta in 22 public Higer Educaon Instuons
in South Africa. The survey had 30 quesons exploring
technology use, innovave pracces, the reasons for
use, the eects on teaching and learning, constraints
and support from the instuon. Members of the
research team idened respondents, specically
targeted for their reputaon as early adopters of new
technologies, including lecturers, support sta,
directors of teaching and learning and senior
academics. 262 educators responded to the survey,
and 18 were selected for an in-depth analysis.
Respondents were asked to list technologies they had
not heard about. Most had never used remote
instrumentaon (85%), tablet computers (76%), web
conferencing (66%), argumentaon visualisaon
(27%), reusable learning objects (23%) and RSS feed
(13%). The most frequently used was Learning
Management System (24%) followed by blogging
(8%), pod-/vodcasng (8%) and microblogging (3%).
Educators primarily used emerging technologies (ETs)
to support prescripve pracces and only a few
reported on how technology use was changing their
pracce.
Ng’ambi (2013) argues that deep and meaningful
learning can only be achieved with the eecve
pedagogical uses of ETs, and proposes Cultural-
Historical Acvity Theory (CHAT) as an analycal
framework. In CHAT, Educaonal goals are dened
(step 1), the relaonal agency (step 2) makes explicit
assumpons about learning, distributed intelligence/
experse (step 3) describes the object of the acvity,
a learning acvity is shaped by an awareness of
capabilies of available technologies (step 4),
appropriate tools are chosen (step 5), students create
an artefact (step 6) and reect on their learning
experience (step 7). The paper concludes that the
KNOWLEDGE CENTRE FOR EDUCATION
|
45
proposed framework would serve as a guide to
eecve pedagogical uses of emerging technologies.
has conducted an empirical study
to determine the eecveness of the virtual
microscopy adapve tutorials (VMATs) and whole
slide images (WSI) to learn diagnosc cytopathology;
a form of clinical decision-making, where the
diagnosis is based on the study of cells. Even though it
is an important part of professional everyday medical
pracce, this subject is rarely taught in medical
educaon.
The aim of WSI is to imitate tradional microscopy in
a digital environment. VMATs are interacve online
tutorials developed using the Adapve e-Learning
Plaorm, an intelligent tutoring system providing
individual students with adapve feedback. Previous
research conducted with pathology specialist trainees
indicates that VMATs are perceived more posively
than tradional learning methods. 35 senior medical
students with no previous experience with
cytopathology parcipated in the randomized
crossover trial. They were divided into two groups of
17 and 18. The trial included three weeks of classes,
each concluded with an online assessment with only
one aempt and one-hour exam me. Each
assessment queson was linked to a WSI and
evaluated either on the diagnosis or on idencaon
of cellular features. Other data sources were students’
self-reported study me, prior academic
performance, and the results of online surveys
evaluang user experience with WSI and VMATs and
their value as educaonal tools.
There was no signicant dierence in the mean
self-reported study me and in the prior academic
performance (measured by the mean weighted
average mark) between the groups. Student’s t-test
was used to analyse the online assessment results.
The group using WSI and WMAT had higher scores
percentage-wise in both FNA Cytology Assessment
(aer Week 2) and Fluid Cytology Assessment (aer
Week 3), but only the result for Diagnosis in FNA
Cytology Assessment was stascally signicant. This
indicates that VMATs and WSI could be more eecve
than tradional approaches. Online surveys revealed
that students preferred VMATs and WSI over
tradional methods, suggesng more adapve
features are favoured. At the same me, they had a
signicant preference for VMATs than WSI alone.
VMATs were evaluated as “more useful in developing
skills in cytopathology (...) more me ecient (...) and
providing more equitable opportunies” (p.5). In
comparison to WSI alone, VMATs enriched the
learning environment with immediate feedback and
interacvity. This adapve approach embeds the tutor
in the tool and is promising.
indicated that introducing new technology does not,
in itself, guarantee innovave pracces in higher
educaon instuons. Studies nd that prescripve
pracces persist. Instead of taking the opportunity to
introduce student centred teaching methods, sta
tend to adapt new technologies to tradional
pracce. If the introducon of technology in higher
educaon teaching aims at more student acve
learning, instuons must develop policies for how
they want to educate young technology users, lead
and closely follow the implementaon of the policies.
The dichotomy digital/non-digital should not
overshadow the fact that pedagogical quality is the
most important issue in both modes of educaonal
provision.
The following chapters 3.4 and 3.5 present two
themes that are also crossing through all the included
studies: collaborave learning and barriers to
technology use and innovave teaching.
3.4 COLLABORATIVE LEARNING
Digital age learners are used to networking and
expect modern higher educaon instuons to be on
top of the digital development. Research, however,
nds a gap between students’ expectaons and
academic digital use and experse. It is argued that
instuons must develop visions and policies,
46
|
KNOWLEDGE CENTRE FOR EDUCATION |
priorise and iniate instuon-wide competence
development to provide academics with the adequate
competence and skills needed to ulise possibilies in
new technologies.
This chapter presents ve studies on collaborave
learning approaches in online learning and teaching.
First, a study in the CSCL-tradion invesgates how
conversaonal agents may promote academically
producve talk. The second study provides an
overview of modalies and pracces in
telecollaboraon, and the third idenfy factors that
promote and hinder technology use in higher
educaon. Lastly, two studies have invesgated social
learning pracces in apps and wikis.
AUTHOR COUNTRY HAVE INVESTIGATED METHODS USED
Greece/Denmark/
Germany
Academically producve
talk
Pre-test, post-test
experimental design
USA Telecollaboraon Scoping review
UK Web 2.0 (Wikis) Interview
South-Africa What`s App Quesonnaire and
qualitave data
USA Wikis and collaborave
learning
Surveys, documents and
qualitave data
Research in computer-supported collaborave
learning (CSCL) has emphasised the importance of
dialogical interacons among learners101. Depth and
quality of peer interacons, in conict resoluon,
mutual regulaon or explicit argumentaon, is found
to play a catalyc role in how students comprehend
the topic in queson and learn from collaborave
acvies102. Despite this insight, research also nds
that student dialogues are oen unproducve103.
Simply placing students together to discuss a topic
does not ensure their engagement in eecve
collaborave behaviour104. This directs the aenon
to how CSCL environments can be designed to
provide scaolding during group discussions105.
101 Stahl, G., Cress, U., Ludvigsen, S., & Law, N. (2014). Dialogic foundaons
of CSCL. Internaonal Journal of Computer-Supported Collaborave
Learning, 9(2), 117-125.
102 Asterhan, C. S., & Schwarz, B. B. (2016). Argumentaon for learning:
Well-trodden paths and unexplored territories. Educaonal Psycholo-
gist, 51(2), 164-187.
103 Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for
computer-supported collaborave learning. Journal of computer
assisted learning, 23(1), 1-13.
104 Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2017). Socio-cognive
scaolding with computer-supported collaboraon scripts: A
meta-analysis. Educaonal Psychology Review, 29(3), 477-511.
105 Ludvigsen, S., & Mørch, A. (2010). Computer-supported collaborave
learning: Basic concepts, mulple perspecves, and emerging trends.
The internaonal encyclopedia of educaon, 5, 290-296.
drew on research indicang the
eecveness of exible conversaonal agents in
producve online peer dialogue. A conversaonal
agent is a third-party intervener in an online dialogue,
serving as an aenon-grabbing strategy to keep
students focused on task. A congurable APT-agent
was used to prompt peers in online discussions to
build on prior knowledge and logically connect their
contribuons to domain concepts that would support
their claims and arguments. APT priorises students’
reasoning and does not expect the teacher to
maintain complete control over student discussions.
APT aims to orchestrate a more student-centred
discussion, where students are movated and
challenged to think profoundly and use their scienc
reasoning skills to solve problems. APT assumes that
knowledge is constructed during peer interacon
through a series of steps where learners’ mental
models are explicitly shared, mutually examined and
possibly integrated106. The arcle reports from a
pre-test post-test experimental design study, involving
96 computer science students, comparing three
condions:
106 Stahl, G., & Rosé, C. P. (2011). Group cognion in online teams. Theories
of team cognion: Cross-disciplinary perspecves. New York, NY:
Routledge/Taylor & Francis. Web: hp://GerryStahl. net/pub/gcot. pdf.
KNOWLEDGE CENTRE FOR EDUCATION
|
47
COLLABORATIVE ACTIVITY
Students discuss in dyads
No agent intervenon
(Control)
U agent intervenon
(U-treatment)*
D agent intervenon
(D-treatment)**
Students individually answer a
post-test; answer a student
opinion quesonnaire and
parcipate in a focus group
session
*U agent intervenon: Undirected BPK (Building-on-Prior-Knowledge) intervenons while collaboraon in dyads (U-treatment condion)
**D agent intervenon: Directed BPK intervenons (D-treatment condions)
Students who interacted with the conversaonal
agent in the two treatment condions (U and D) came
out of the collaborave acvity with a domain
knowledge advantage over the students in the control
group. Students in the control group (no agent
intervenon) perceived the collaborave acvity as
less helpful for enhancing their domain knowledge
than the treatment students.
The analysis revealed that agent intervenons had a
signicant eect on the levels of explicit reasoning
exhibited during the collaborave acvity. The
frequencies of explicit arguments were substanally
higher in the treatment group where students were
pressed for clear statements backed by concrete
evidence. This conrms other studies showing that an
agent prompng students to follow academically
producve pracces can amplify students’ scienc
reasoning107.
The agent also had a posive impact on dyad
performance in the task. The dyads in treatment
groups (U and D) provided more comprehensive and
accurate answers to the learning quesons. Overall,
the conversaonal agent, commied to geng the
facts right, seemed to play a crical role by asking
students to consider themselves responsible for the
accuracy and validity of their claims. Encouraging
students to make their knowledge sources explicit is
considered vital in academic sengs for increasing
collecve reasoning levels and improving
collaborave learning outcomes.
107 Dyke, G., Adamson, D., Howley, I., & Rosé, C. P. (2013). Enhancing
scienc reasoning and discussion with conversaonal agents. IEEE
Transacons on Learning Technologies, 6(3), 240-247.
Students in the D condion appeared to be feeling
personally responsible for giving a comprehensive
response to the agent. The agent impact on individual
learning appears to be amplied when the agent
employs a directed intervenon method targeng a
parcular peer, rather than an undirected
intervenon method, addressing both peers in a dyad
simultaneously. Researchers reported that direcng
prompts to individual learners by an agent seems to
be a feasible way to reduce diusion of responsibility
and facilitate equal parcipaon in reasoning
processes.
have conducted a
scoping review of 55 telecollaboraon (TC) projects,
with the aim to idenfy pedagogical pracces
commonly used in telecollaboraon, dened as
instuonalised, electronically mediated intercultural
communicaon under the guidance of a teacher. TC
has ulised asynchronous tools (email, bullen board/
online forums, blogs) and synchronous tools,
(videoconferencing, Skype, MSN Messenger). The
review aimed to idenfy the most commonly used
tools in telecollaboraon projects in university foreign
language classes and how tools have changed over
the last 20 years.
Most projects were either mono- or bilingual. Email
was used as the main tool of interacon; to nd me
for meengs, and to reinforce feedback by combining
synchronous and asynchronous feedback.
48
|
KNOWLEDGE CENTRE FOR EDUCATION |
DEMOGRAPHICS PEDAGOGY
The average duraon of a project
was about 10,54 weeks.
Five types of interacon formaon when
parcipants engage synchronously was
idened: 1) 1-1, 2) 1-2, 3) small groups,
4) mid-size group 5) class vs class.
About 60% of projects included
asynchronous interacon in
addion to synchronous while the
rest only used synchronous.
Many projects were text-based (k=23) or
combined text chat with video interacon
(k=12). Projects also video chat only,
audio chat, audio graphic and both audio
and video chat.
62% of messages was via e-mail,
16% via blogs, 14% via Wikis or
websites, and 11% via discussion
forums. Only one study used
Facebook.
Most studies used informaon exchange
tasks, and language-focused tasks were
the least common. Twelve projects used
co-construcon tasks.
Six typical arrangements of synchronous
telecollaboraon projects were idened (Akiyama &
Cunningham, 2018, p. 63-64):
1. Tandem: a synchronous session is divided into two
parts (English 30 min, German 30 min).
2. Socialisaon: Languages are kept disnct, sessions
are synchronous (one session in English, the next
in German).
3. Co-construcon: engage parcipants in creang
artefact (blogs, presentaons). Co-construcon
usually has no strict rules for language separaon
or for how oen parcipants need to interact as
long as they create a cultural product.
4. Apprenceship: the exchange takes place between
FL learners and teacher trainees. One group is
learning how to teach their partners TL. The
interacon is usually monolingual.
5. Cultural Exploraon: interacon takes place
monolingually in FL learners` TL. The partner
group`s main objecve for parcipang is to
increase familiarity with the target culture rather
than language learning or teaching.
6. Lingua Franca: monolingual arrangement, but the
language of interacon is none of the parcipants’
rst language. Emphasizes content learning over
language learning and involves dialogue about
polical issues and acquision of sociological
knowledge.
KNOWLEDGE CENTRE FOR EDUCATION
|
49
Tandem has been and is the most popular
arrangement, more recent studies tend to belong to
Apprenceship, Cultural Exploraon or Lingua Franca.
This indicates that there is now a wider range of
partners who parcipate in TC for various purposes
other than language learning.
have invesgated how
eLearning with eResources (eRes) encouraged
academics to use web 2.0 technologies. Data were
collected through interviews with teachers in
physiotherapy, midwifery, archaeology, markeng and
design engineering. The table below shows
e-resources used and learning acvity in each course.
Academics were interviewed individually about their
experience with the project both during and at the
end of the project.
PEDAGOGY LEARNING ACTIVITY
e-Journals, blogs Finding and criquing arcles
using a blog
Physiotherapy
Blackboard, scholar Sharing using social
bookmarking
Midwifery
Wikis Creang group based resources Archaeology
e-Journals, e-news Criquing and nding with
med use of blogs
Markeng
e-journals, blogs,
wikis
Developing soluons in groups
using blogs and wikis
Design engineering
The study shows that academic teaching can be
changed with Web 2.0 technologies. Two issues were
idened: (a) scalability: Most academics required a
high level of support from pedagogical and technical
specialists, and b) professional development: academics
acknowledged their professional development
requirements in relaon to technology, but not the
need to change their pedagogical approach. As Web
2.0 tools (i.e., blogs, wikis) are integrated in the
instuons’ LMS, students and sta have easy access
to Web 2.0 tools. When academics hesitate to
introduce Web 2.0 in their teaching, it is because it
requires a dierent pedagogical approach. The authors
ask if academics do not take ownership of their
professional development and responsibility for their
learning, but expect external iniaves and support.
report from a study using
WhatsApp in an informaon technology course at a
South African university. They argue that mobile
messaging (MIM) is qualitavely and visually disnct
from email systems and has the potenal to create
dialogic spaces for students and trigger academic
parcipaon. MIM is one of the least exploited
funconalies of mobile devices in higher educaon
and there is lile research on how MIM inuences
pedagogy, for instance lecturers’ instrucon and
students’ academic parcipaon.
A case study was conducted with 95 third-year
technology students (59 female and 36 male) with a
diverse language background108. The lecturer
introduced WhatsApp to boost parcipaon, and
interacon lasted for a semester. WhatsApp did not
replace teaching acvies, but served to extend
academic consultaon during and aer hours. The
students were grouped in 12 clusters; each cluster
had students with varied academic capabilies, and
students were anonymous.
The lecturer was available between 8 am and 10 pm.
To promote peer-based interacon, reduce lecturer
dominance and ensure students ownership of their
learning, he was only acvely involved when students
were stuck. A researcher from another university
followed the acvity in WhatsApp, providing general
guidance to students upon request. He observed
interacons and interviewed 15 students about their
experiences of using WhatsApp (how it aected their
emoons, parcipaon etc.). A quesonnaire was
used to invesgate WhatsApp’s physical, technical and
funconal aordances in relaon to their pedagogical
value.
108 6 were English language speakers, 14 Africaans, 15 Xhosa, 2 Chinese and
55 Sesotho.
50
|
KNOWLEDGE CENTRE FOR EDUCATION |
Findings indicate increased student parcipaon, the
fostering of learning communies for knowledge
creaon and shis in the lecturers’ instrucon.
Problems encountered were resentment of the
merging of academic and family life occasioned by
WhatsApp consultaon aer hours and ambivalence
among students on the wide-scale-roll-out in dierent
academic programs.
argue that wikis as a CSCL
environment cannot facilitate classroom collaboraon
without an eecve learning design. The authors
therefore developed strategies for designing wiki-
supported curriculum. The paper outlines theories
and prior research upon which the design was based,
the implementaon of the iterave design-based
project, and the teaching and learning strategies
developed in the study. Researchers reported that
collaborave wring on wikis promotes the co-
creaon of knowledge and can, in theory, support the
development of learning communies.
The research was conducted over four semesters in
2007 and 2009, in a Web 2.0 Tools and Social Learning
course at a university in northern China. It followed
an iterave cycle: a wiki-based learning acvity was
designed; the design was implemented and data
collected using a variety of methods; the design was
then evaluated and analysed for problems. Following
this, an aempt to address these problems was
implemented in the redesign, which then followed
the same cycle through four iteraons. Parcipants
were postsecondary students.
Data included parcipant observaon, surveys,
interviews and parcipant-produced documents. At
the end of the fourth acvity, a survey was
administrated that queried students’ parcipaon in
and percepons of the wiki-based collaborave
acvity. Survey responses were analysed using
descripve stascs. Interviews with 4-5 parcipants
were conducted in all acvies, to ask about students’
general opinions of acvies and challenges
encountered during the acvies. Documents –
including student and teacher wiki work and
parcipaon on social media – were also collected.
During the process of detecng problems and rening
the design, three instruconal strategies emerged:
Developing a learning community; Forming groups;
Role assignment.
At the beginning, most students were excited and
movated about this project. Only 21% felt obligated
to parcipate because it was a class assignment and
only 7% did not want to parcipate. Students’
movaon mainly came from their own interest,
followed by incenves from instructor or peers. 100%
of the survey respondents agreed that wiki is a
favourable tool to support collaborave learning.
Strategies developed in this study may enable
teachers conducng similar collaborave acvies to
avoid problems related to instruconal designs.
Future research should not only address the
development of teaching strategies, which may be
context- and plaorm specic, but also iterave
design approaches for rening these strategies.
has shown that when
students work in groups, responsibility is frequently
dispersed. This highlights the need for learning designs
that support collaboraon and acvate each student.
Students in higher educaon are expected to learn to
argue. In academically producve talk (APT), students
learn scienc reasoning through building on prior
knowledge and logically connect their contribuons to
domain concepts to support their claims and
arguments. Encouraging students to make their
knowledge sources explicit is considered vital in
academic sengs. Studies also nd that student
collaboraon happens more spontaneously in apps
designed for social media use than in more formal
learning technologies. Research on telecollaboraon
reveals tradional teaching pracces with email
dominang the communicaon. Depending on the
design, Wikis are perceived as a favourable tool to
support collaborave learning. Researchers also ask
why academics don’t recognize their own responsibility
for professional development in technology use in
teaching, but expect external iniaves.
3.5 BARRIERS TO TECHNOLOGY USE AND
INNOVATIVE TEACHING
Five studies nd barriers to technology use in higher
educaon instuons that may explain why teaching
in higher educaon instuons remain teacher-
centred, while the intenon is a student-acve
learning approach. Despite much talk about the
potenal of technology to transform teaching and
learning in higher educaon, much university
teaching remains fundamentally unchanged.
A tension between external and internal is detected,
explaining concerns raised by sta. The introducon
KNOWLEDGE CENTRE FOR EDUCATION
|
51
of technology is oen described as top-down,
externally driven. If technology use is enforced by the
faculty, teachers can lose their sense of personal
agency; some even fear that if students can learn
online, they will stop aending the lectures. At the
same me, research nds inera in the instuons.
Implementaon progress is slow and sharing of
innovave pracces appears not to be happening.
Educaonal technology is more about technology, less
about pedagogy and learning design, and there is a
gap between the instuonal rhetoric of TEL and the
reality.
AUTHOR COUNTRY HAVE INVESTIGATED METHODS USED
Canada Integrang technologies in higher
educaon
Semi-structured interviews
UK Technology enhanced learning Essay
UK Why university lecturers stop using
technology in teaching
Qualitave (Interviews)
UK Super innovators – understanding
the use of LSM
Qualitave (Interviews)
UK Technology enhanced learning Quantave (longitudinal
survey data)
interviewed 24 technology
specialists and teaching centre experts in academic
instuons, to idenfy how online technologies can
enable eecve collaboraon in university learning
environments. All informants were experienced
technology users, specialists in university teaching
and/or directors of teaching and learning centres. The
study nds that technology innovaon was mostly
triggered by external reasons, such as “fad, cure-all
illusion, pressure from students or compeon from
the online educaon market etc.” (p. 21). Only 25 %
of the informants menoned internal reasons, such as
collaborave work or distance educaon. Tools
menoned in the interviews were LMS, wireless, Web
2.0 technologies and video, mobile devices. The
author noces that these technologies are not the
newest, and infer that this might indicate a digital gap
between higher educaon instuons and the rest of
the society.
argued that university
teachers perceive teaching dierently. Some have
teaching-focused concepons, others have learning-
focused concepons. Variaons in concepons of
teaching can account for how technologies or tools
are used. Teaching-focused individuals are more likely
to use technology to support exisng transmissive
teaching strategies, while learning-focused individuals
are more likely to use technologies that facilitate and
support students’ learning and development.
In a previous literature review109, the authors have
tried to idenfy a scholarly approach in the research
on technology use in educaon, asking:
• What evidence was being used to drive the
innovaon/intervenon?
• What evidence was gathered?
• What evidence illustrates changes in the
professional pracce of teachers in higher
educaon?
Few of the reviewed arcles exhibited a scholarly
approach to teaching, both in how technologies are
implemented and how researchers report from
intervenons. Much TEL research concentrates on
technology as the object of aenon and as the agent
of change – rather than teaching and/or learning.
According to the authors, transmissive teaching
beliefs permeate the sector. Even the most innovave
teacher can be constrained by instuonal contexts
or discouraged by professional development
programmes that focus primarily on ‘how to’
approaches instead of acvies that help them
reconsider deeply held beliefs about teaching. Too
oen, teachers seem to be asking ‘What can I use this
technology or tool for?’ rather than ‘How can I enable
109 Price, L. & Kirkwood, A. (2011). Enhancing professional learning and
teaching through technology: a synthesis of evidence-based pracce
among teachers in higher educaon. Higher Educaon Academy, York,
UK.
52
|
KNOWLEDGE CENTRE FOR EDUCATION |
my students to achieve the desired learning
outcomes?’ or ‘What forms of parcipaon or
pracce support learning?’. Professional development
of academics in technology use should primarily be
about their approaches to teaching.
explored how university lecturers used
technology, and why they stopped using it. Interviews
were conducted with eleven experienced university
educators from various facules. Findings indicate
that teachers don’t always see the replacement of old
technologies with new as an improvement because
they must learn new skills and unlearn old. If a new
technology is not aligned with the teachers’
pedagogical pracce, it is less likely to be used by the
teachers. Another reason to stop using technology is
bad experiences. Having experienced many technical
failures, or too lile student engagement, teachers
might revert to tradional teaching methods. The
study found the following reasons why lecturers
stopped using a technology: the emergence of a new
technology; when students consider certain
technologies as outdated; lack of professional
development; and lack of technical support. Social
media may be discarded by some teachers, because it
blurs the line between their professional and personal
life. When technology is integral to the course design,
it is harder for teachers to stop using it.
conducted a qualitave study in
one Finnish university (5000 students, 200 teaching
sta) and one Brish university (25.000 students and
2500 teaching sta), on the use of Learning
Management Systems (LMS). An LMS is an integrated
plaorm used to present resources, facilitate
administraon and communicaon, and support
learning acvies110. The study aimed to beer
understand the relaonship between LMS use and
teachers’ expressed beliefs and atudes, and how
instuons can support more innovave adopon
and development of pedagogy.
110 Costello, E. (2013). Opening up to open source: Looking at how Moodle
was adopted in higher educaon. Open Learning: The Journal of Open,
Distance and e-Learning, 28(3), 187-200.
Studies found that most sta use LMS’ only for very
basic funcons. An oen-menoned benet among
instructors (39 %) was beer communicaon to
students, while only 7 % thought it improved teaching
and learning111. There was lile indicaon that
pedagogy developed signicantly even aer years of
instuonal adopon.
Two LMS expert (Moodle) administrators, one from
each university, were interviewed about their
perspecves on the atudes of teaching sta that
they support. While there is a strong belief in the
movaonal eect of enthusiasc colleagues, the
informants in this study nd that sharing is not
happening. Most teachers were observed to start
with the basic funcons and never progress: ‘Most of
them who use it think that they can ulise it well. But
they can’t’ (p. 167). Both informants noted that sta
oen stated pedagogic, student-focused reasons for
using the LMS, while in pracce an esmated 50 % of
teaching sta (aer three years) and 15-20 % (aer 10
years) were not using the system at all, even for
purely informaonal purposes. One says: ‘They only
put their material in Moodle and then they think ok,
that’s it. I can stop here’ (p. 166).
Interviews were analysed in two themes, rst, what
teachers do and second, why they do it? Teacher
behaviour may be grouped in four categories: 1)
Iniators, 2) Followers, 3) The reluctant or unwilling
and 4) Non-users: ‘We sll have people who aren’t
using Moodle even though we have had it for 10
years’ (p. 165). Pedagogic iniators and innovators
were described as willing to explore, open to
experiment and risk-accepng; characteriscs seen as
lacking in most users. Barriers noted were described
as intrinsic, deep-rooted, individual, subjecve and
dicult to address: ‘It’s an ideological thing’ (p. 169).
The experts viewed pedagogical and conceptual
issues as fundamental inhibitors of progress, and did
not ascribe them to age, but to personality: ‘they are
scared of technology, and that’s their threshold’ (p.
167). Some teachers connect their reluctance to
technology use to personal weakness and failure,
even something shameful. Being a teacher implies to
know and be on top of things. Not fully mastering
111 Lonn, S., & Teasly, S. D. (2009). Saving me or innovang pracce.
Invesgang percepons and uses of learning management systems.
Computers & Educaon, 53(3), 686-694.
KNOWLEDGE CENTRE FOR EDUCATION
|
53
technology threatens their authority. For some, fear
of technology becomes fear of perceived failure and a
threat to professional standing. Other teachers feared
that students would stop aending the lectures
– ‘they don’t have to turn up to see the performance
of a lecture because they have access to all the
informaon in other ways’ (p. 168). A third group of
teachers argue that they ‘need to have the students in
a room to show something in a way that technology is
nowhere near close to doing’ (p. 169). These teachers
see their teaching pracce as sacred: ‘lectures are
seen somehow as the sacred thing that must connue
and everything else must to some extent bend around
it’ (p. 170).
Current instuonal support for sta is oen based
on training courses, online resources and individual
support: ‘We have just noced that the training
sessions that we have organised, it’s not a good idea.
Nobody comes and nobody learns anything’ (p. 170).
Researchers reported that beer understanding of
the reasons behind the lack of progression and an
approach to sta development which helps teachers
to understand and confront their conceptual barriers
is needed. The state of inera in technology use
idened in the study is partly related to concepons
of teaching and what it means to be a teacher. These
issues must be understood to meet the concerns of
teaching sta.
report on the
developments of Technology Enhanced Learning (TEL)
in higher educaon instuons in the UK. The study
draws on longitudinal survey data and case studies
from UCISA112 on TEL implementaon, from 2012,
2014 and 2016, in addion to qualitave interviews
with instuons about their approaches to TEL
developments. Although instuonal investment in
TEL has been signicant, there is no substanal
change in how the technologies are used. A gap is
revealed between the instuonal rhetoric of TEL and
the reality of its impact on academic pracce. A
112 The Universies and Colleges Informaon Systems Associaon (UCISA)
barrier idened in the 2016 UCISA survey was
departmental culture, related to many factors; lack of
me and support, healthy scepcism concerning the
value of digital provision in supporng student
learning, and resistance to top-down strategies from
instuonal management, which may lead to a lack of
commitment to change academic pracces.
Instruconal support for online learning requires
strategies to facilitate eecve group learning and
parcipant-led acvies. To develop these skills,
academics need professional development,
addressing both technology and pedagogic pracce.
Researchers reported that the introducon of TEL
tools in UK HE instuons has focused on instuonal
responsiveness to student expectaons and needs by
investments in centrally managed systems. However,
far less aenon has been paid to addressing
academic sta needs in the process.
has shown signicant barriers to technology
use in higher educaon instuons. One interesng
nding is that academics appear to not be using a
scholarly approach when implemenng technology in
educaon. A nding cung through all ve studies is
instuonal inera and a reluctance among lecturers
to change pracce. Researchers argue that this
reluctance must be addressed and understood, and
stress that the focus of sta development programs in
higher educaon must be on instructors’ percepon
of teaching and learning, as technology appears not
the main barrier. While sta obviously must know
how technology works, and be familiarised with the
potenal in technology, research indicates that
pedagogy is a more fundamental barrier to innovave
teaching in higher educaon than technology. To be
perceived as relevant for younger generaons,
instuons need heightened awareness of a
potenally emerging technological gap between the
instuons and the rest of society.
54
|
KNOWLEDGE CENTRE FOR EDUCATION |
4
LEARNING IN HIGHER EDUCATION
Chapter four brings together and synthesises ndings
from the 35 included arcles.
The systemac review was conducted to answer how
teaching with technology can support student acve
learning in higher educaon. The rst chapter,
Introducon, showed policy expectaons. White Paper
no. 16 (2016-2017) Culture for Quality in Higher
Educaon113, the long-term plan for research and
higher educaon114, the report from the EU
commission115, and the strategy for digitalisaon of
higher educaon116 all stress that technology should
be used innovavely, to support student acve
learning and develop new teaching strategies. Chapter
3 rst presented studies on learning analycs and
learning design. While big data bring new possibilies,
it also requires that instuons develop data literacy
as sta face the challenge to use abstract informaon
(numbers and percentages) pedagogically, when
developing learning designs. MOOCs were introduced
with ambious visions, but studies see few traces of
their prospected transforming potenal. Research on
capture technology suggests that sta should focus on
the why of technology use, not on the how. Studies on
mobile learning nd a persistent behaviourist learning
paradigm in the instuons, and conclude that mobile
learning need new and extended learning designs.
Augmented Reality and emerging technologies show
promise, but are sll at the early stages. Based on the
113 Meld. St. 16 (2016–2017). Kultur for kvalitet i høyere utdanning
hps://www.regjeringen.no/no/dokumenter/meld.-st.-16-20162017/
id2536007/
114 Meld. St. 7 (2014-2015). Long -term plan for research and higher
educaon 2015-2024
115 European Commission (2014). Report to the EU Commission on New
modes of learning and teaching in higher educaon hp://ec.europa.
eu/dgs/educaon_culture/repository/educaon/library/reports/
modernisaon-universies_en.pdf
116 hps://www.regjeringen.no/no/dokumenter/digitaliseringsstrategi-for-
universitets--og-hoyskolesektoren---/id2571085/
reviewed studies, it is reason to suspect that also new
technologies can risk being adapted to tradional
teaching. Throughout chapter 3, the included studies
show a consistent paern: while researchers assume
the transforming potenal of technology, studies nd
few examples of sustainable innovave teaching
pracces, few examples of successful student acve
learning designs and ndings on student movaon
and learning outcomes are inconsistent and
inconclusive.
Studies on barriers to technology use nd inera in
instuons. They conclude that sharing of exemplary
pracces is not happening; sta shows reluctance to
change; a behaviourisc mindset persists and
prescripve pracces dominate. The overall picture is
that tradional ideas about how students learn
dominate. As technology is mainly used
administravely and for one-way communicaon, the
interacve potenal in technology is underulised
and technological devices are adapted to familiar
work processes.
To narrow in on the queson how teaching with
technology can support student acve learning, the
included studies were uploaded in NVivo 11,
analysed, and coded according to the main paerns
idened across the studies. As chapter 3 shows,
most studies emphasised the need to change
teaching from content delivery to student acve
learning and most studies stressed the need for sta
professional development.
First paern: From content delivery to student
acve learning
Analysis shows that 25 of the 35 included studies
menon student acve or student-centred learning.
The arguments revolve around instruconal
approaches or learning designs that require students
to acvely collaborate in groups or on discussion
KNOWLEDGE CENTRE FOR EDUCATION
|
55
forums. Collaborave learning is oen used to
exemplify acve learning approaches, and technology
is referred to as a tool that can support student acve
learning and the co-construcon of knowledge. A
majority of the 25 studies menon technology, but
primarily as a tool with the purpose to administer
content, as a means for content delivery in MOOCs
(Toven-Lindsey et al., 2015), or to facilitate online
collaboraon by providing discussion forums, wikis,
possibilies to share documents and so forth (Blau &
Shamir-Inbal, 2017).
Six studies discuss collaboraon between students
(Blau & Shamir-Inbal, 2017; Cochrane 2014; Lee et al.,
2018; Rambe & Bere, 2013; Tegos et al., 2016;
Toven-Lindsey et al., 2015). Collaborave learning
means that students are engaged in discussions, share
what they have learned and provide feedback (Lee et
al., 2018), work inquiry-based when solving problems
and construcng knowledge (Blau & Shamir-Inbal,
2017; Toven-Lindsey et al., 2017). When they
collaborate on solving tasks, students need a variety
of social skills such as mutual respect, listening to
others, understanding, cooperang, and avoiding
conict situaons (Blau & Shamir-Inbal, 2017). Three
studies menon collaboraon between students and
teachers, but without elaborang (Amemado, 2014;
Barak, 2017; Blau & Shamir-Inbal, 2017).
Collaboraon amongst teachers is only briey
menoned in two studies (Amemado, 2014;
Cochrane, 2014); without examples or further
elaboraon.
Student acve learning is used about instruconal
approaches that acvely engage students in the
learning process through collaboraon and
discussions rather than having them passively receive
informaon from their instructors (Lee, Morrone &
Sierring, 2018). It is argued that for acve learning to
succeed, educators must create new and extended
learning designs that link dierent pedagogical
strategies. When teaching with technology, learning
designs also span dierent contexts. Studies queson
if current sta training courses develop these skills
and competences, and call for new approaches to
professional development in higher educaon
instuons.
One soluon, frequently menoned in the studies, is
that teachers abandon a behaviourisc perspecve on
learning and instead adopt a socio-cultural,
construcvist approach. If this happens, technology
will, supposedly, facilitate the move from teaching as
content delivery to student-acve learning. This might
be easier said than done. A review of learning
research117 found that behaviourism, cognive and
socio-cultural learning theories have developed
historically, but not as major paradigmac changes.
Studies that built on behaviourism dened learning as
changed behaviour; studies that built on cognivism
dened learning as internalisaon of external
knowledge and studies taking a sociocultural
perspecve dened learning as situated, social and
acve processes where people learn through
parcipang in cultural and social pracces.
Researchers who draw on behaviourist and cognive
perspecves are primarily interested in individual
learning, while researchers with an interest in
collaborave learning acvies nd support in social
and cultural learning theories. A characterisc of the
educaonal ecosystem is, however, that these three
perspecves on learning live side by side and serve
dierent purposes. When studies suggest to abandon
the behaviourisc perspecve, this is therefore only,
at best, part of the soluon. Teachers take several
consideraon when planning their teaching. They
prefer methods they perceive as useful for the
purpose, easy to use, that can be adapted to the
students’ needs and ts the physical surroundings.
Second paern: Sta professional development
Studies nd that pedagogical use of technology in
teaching is challenging. Technical training in how to
use technology is necessary, but not sucient, when
the goal is innovave teaching and more student
acve learning. Researchers argue that pedagogical
consideraons must be integrated in all eorts to
movate teachers to use technology. 19 studies
menon dierent training needs, spanning learning
about the potenal of technology and technical
details; pedagogical training i.e., learn new teaching
methods and data literacy i.e., learn how to use data
producvely to achieve meaningful results (Avella et
al., 2016), but also more general professional
development acvies.
As technology oen is iniated from the top, not
based on teachers’ needs, technology enhanced
learning is frequently also technology driven. Several
117 Murphy, P. K., & Knight, S. L. (2016). Exploring a Century of Advance-
ments in the Science of Learning. Review of Research in Educaon,
40(1), 402-456.
56
|
KNOWLEDGE CENTRE FOR EDUCATION | / LEARNING AND TEACHING WITH TECHNOLOGY IN HIGHER EDUCATION – A SYSTEMATIC REVIEW
studies highlight the need for a pedagogical
framework that take teachers’ concepon of learning
and why teachers chose to teach as they do into
consideraon. Researchers argue that quesons
related to pedagogy must guide the use of technology
in teaching, and not vice versa (Barak, 2017;
Cochrane, 2014; Kirkwood & Price, 2013; Newland &
Byles, 2014; Walker et al., 2017), and several studies
conclude that professional development programmes
with the aim to promote technology use in teaching
should movate teachers to reect on their beliefs
about teaching.
Key to answering the queson how teaching with
technology may support student acve learning
appears to be how sta professional development
courses are designed and conducted. The tradional
model of taking lecturers out of their everyday work
situaon to inform them about the potenals of new
technology and alternave teaching approaches
appears unproducve. Based on the analysed arcles,
two main topics must be central in higher educaon
professional development for teaching with
technology to support student acve learning:
learning design and collaborave learning.
This nding has implicaons for how instuons fund,
plan and structure professional development.
Learning design goes beyond tradional planning for
a lesson, and requires joint eort by a group of
teachers. Collaborave learning is central to learning
design, and teachers are currently expected to teach
students how to collaborate, while most teachers
work individually.
A scholarly approach to teaching
Challenges related to teaching are more oen shared
across than within academic disciplines. For example,
will themes such as teaching with technology or
student acve learning transcend disciplinary
boundaries. However, opportunies for sta to
collaborate and learn from one another are limited
because there are few mechanisms in place to
support academics’ teaching and few incitements to
support teacher collaboraon. In a systemac review
on campus development118, it was noted that while
118 Lillejord, S., Børte, K., Nesje, K., & Ruud, E. (2017). Campusuorming for
undervisning, samarbeid, forskning og læring – en systemask
kunnskapsoversikt. Oslo: Kunnskapssenter for utdanning. www.
kunnskapssenter.no
higher educaon instuons have a well-developed
infrastructure to support research, a similar
infrastructure appears to be lacking for teaching.
Paradoxically, work methods dier when academics
conduct research and when they teach. When
researching, academics use inquiry-based,
invesgave approaches, work collaboravely
co-author and disseminate ndings. Increasingly,
research is perceived as a collecve responsibility, but
teaching remains, predominantly, an individual
responsibility. While research is perceived as a
knowledge intensive, cumulave knowledge-building
endeavour with a joint knowledge base, guidelines,
methods and ethical boards, teaching in higher
educaon has not yet gained similar status.
As shown in 3.5, Barriers to technology use and
innovave teaching, Kirkwood and Price (2013) argue
for a scholarly approach to teaching:
“The scholarship of teaching and learning is, at its
core, an approach to teaching that is informed by
inquiry and evidence (both one’s own, and that of
others) about student learning. It is not so much a
funcon of what pedagogies [teachers] use. Rather, it
concerns the thoughulness with which they
construct the learning environments they oer
students, the aenon they pay to students and their
learning, and the engagement they seek with
colleagues on all things pertaining to educaon in
their disciplines, programs, and instuons”
(Kirkwood & Price 2013, p. 329119)
Without using the term scholarly, a majority (22) of
the included studies argue for similar perspecves on
teaching, when they refer to inquiry-based and
iterave learning designs, student acve learning and
collaboraon. Other studies highlight scienc
reasoning as an approach to student acve learning in
higher educaon (Tegos et al. 2016) or ontological
shis, emerging from sustained interacons
(Cochrane, 2014).
119 Cing Hutchins, P., M. T. Huber, & Ciccone, A. (2011). Geng There: An
Integrave Vision of the Scholarship of Teaching and Learning,
Internaonal Journal for the Scholarship of Teaching & Learning 5 (1).
KNOWLEDGE CENTRE FOR EDUCATION
|
57
Many academics lack fundamental professional
development and are not encouraged to keep up to
date with teaching research. Professional
development provision tends to be under-resourced
and disconnected from discipline acvies120. Newly
appointed academics may nd their rst teaching
experience stressful, and report feeling thrown in at
the deep end with lile support121. On this
background, researchers suggest that academic work
should be regarded as a professional pracce122.
Instuons expect sta to teach to a certain standard,
and should provide training, with the ambion to
develop scholarly teachers123, who are research-
informed, inquire into their teaching pracce, and
disseminate what they nd. Scholarly teachers take
advantage of instuonal programs and iniate their
own professional learning. One study (Cochrane,
2014), suggests Professional Learning Communies
(PLCs) as sites where sta can collaborate to develop
their teaching pracce124. PLCs have supporve
leadership and an acon- and results-oriented focus
on collaboraon and experimentaon to support
teaching and learning, and aims to de-privase
teaching125.
This is not a new idea. Structured, muldisciplinary
Faculty Learning Communies (FLCs)126, were
developed at Miami University in 1979, with the goal
to develop a scholarly product, usually Scholarship of
120 Boud, D., & Brew, A. (2013). Reconceptualising academic work as
professional pracce: Implicaons for academic development.
Internaonal Journal for Academic Development, 18(3), 208-221.
Roxå, T., & Mårtensson, K. (2009). Signicant conversaons and
signicant networks–exploring the backstage of the teaching arena.
Studies in Higher Educaon, 34(5), 547-559.
121 Fraser, K., Greeneld, R., & Pancini, G. (2017). Conceptualising
instuonal support for early, mid, and later career teachers.
Internaonal Journal for Academic Development, 22(2), 157-169.
122 Boud, D., & Brew, A. (2013). Reconceptualising academic work as
professional pracce: Implicaons for academic development.
Internaonal Journal for Academic Development, 18(3), 208-221.
123 Mya, P., Gannaway, D., Chia, I., Fraser, K., & McDonald, J. (2018).
Reecng on instuonal support for SoTL engagement: developing a
conceptual framework. Internaonal Journal for Academic Develop-
ment, 23(2), 147-160.
124 Cherrington, S., Macaskill, A., Salmon, R., Boniface, S., Shep, S., & Flutey,
J. (2017). Developing a pan-university professional learning community.
Internaonal Journal for Academic Development, 1-14.
125 DuFour, R., & Eaker, R. (1998). Professional Learning Communies at
Work: Best Pracces for Enhancing Students Achievement. Bloomington
IN: Naonal Educaonal Service.
Hipp, K. K., & Human, J. B. (Eds.) (2010). Demysfying professional
learning communies: School leadership at its best. Lanham, MD:
Rowman & Lileeld Educaon.
126 Cox, M. D. (2013). The impact of communies of pracce in support of
early-career academics. Internaonal Journal for Academic Develop-
ment, 18(1), 18-30
Teaching and Learning (SoTL)127, professional
development that promotes research-informed
teaching. Others, and similar, iniaves build on the
observaon that academics do not generally engage
with systemac peer-review of teaching128 with
construcve feedback 129. To be sustainable,
procedures for instuonalised, connuous
professional development, require procedures for
knowledge accumulaon and sharing, leadership and
processes for renewal. As the authority of
professional experse is more respected in academic
instuons than tradional forms of posional
power130, the status of teaching must be heightened
and an infrastructure developed to support
connuous inquiry into quesons of pedagogy and
didaccs.
Findings from studies in this systemac review has
implicaons for how instuons plan and conduct
programs for academic development131. Provision-
driven sta development builds on a decit-model,
which assumes that someone is lacking something,
for instance knowledge or skills. Programs that aim at
movang academics to teach with technology in a
way that promotes student acve learning must build
on the assumpon that academics have the necessary
competences, but that they need leader support and
supporng structures.
127 Fanghanel, J., Pritchard, J., Poer, J., & Wisker, G. (2016). Dening and
supporng the scholarship of teaching and learning (SoTL): A
sector-wide study. York: HE Academy.
128 Barnard, A., Nash, R., McEvoy, K., Shannon, S., Waters, C., Rochester, S.,
& Bolt, S. (2015). LeaD-In: a cultural change model for peer review of
teaching in higher educaon. Higher Educaon Research & Develop-
ment, 34(1), 30-44.
129 Kolb, D. (1984). Experienal learning as the science of learning and
development. Englewood Clis, NJ: Prence Hall.
130 Bento, F. (2011). A discussion about power relaons and the concept of
distributed leadership in higher educaon instuons. The Open
Educaon Journal 4, 17-23.
131 Boud, D., & Brew, A. (2013). Reconceptualising academic work as
professional pracce: Implicaons for academic development.
Internaonal Journal for Academic Development, 18(3), 208-221.
58
|
KNOWLEDGE CENTRE FOR EDUCATION |
5
Law for universies and colleges § 1-3, state that
teaching in Norwegian higher educaon instuons
must be based on R&D, research- and experience
based development132. In a previous review133, the
Norwegian Knowledge Centre concluded that modern
universies, expected to develop new teaching
methods, need an infrastructure for teaching in
addion to the already established infrastructure for
research. One consequence of perceiving academic
work as a professional pracce, is that universies
and colleges must establish higher educaon teaching
as a knowledge eld with a knowledge base,
equipment, tools and collecve work processes. This
work needs a supporng infrastructure and
leadership.
In chapter four, it was argued that a scholarly
approach to teaching is a prerequisite to develop
student acve learning. There is no reason why
teaching should not be an inquiry-based acvity. It
should, however, be acknowledged that individual
teachers who work in lecturing halls designed for
raonal one-way transmission of content from one
teacher to many students nd it dicult to change
pracces deeply ingrained in structure, history and
culture. Teachers prefer methods they nd easy to
use, that t the physical surroundings, are useful for
the purpose, and can be adapted to their students’
needs.
132 hps://lovdata.no/dokument/NL/lov/2005-04-01-15
133 Lillejord, S., Børte, K. Nesje, K. & Ruud, E. (2017). Campusuorming for
undervisning, forskning, samarbeid og læring – en systemask
kunnskapsoversikt. www.kunnskapssenter.no
An instuon-wide scholarly approach to teaching is
suggested as a mean to obtain student acve
learning. The analysed studies show that plans and
strategies communicate high expectaons, while
responsibility for the follow-up appears to be
somewhat dispersed in the sector. The systemac
review therefore concludes that teaching with
technology can promote student acve learning only
through a joint, coherent, mul-level eort.
In the introducon, it was referred to the
digitalisaon strategy (2017-2021)134, developed by
the Norwegian Ministry of Educaon and Research.
The strategy argues that the conscious use and
development of technology must be an integral part
of naonal and instuonal strategies. This supports
the argument for a mullevel eort, and Figure three
shows responsibilies at naonal and instuonal
levels.
A core message in this systemac review is that
technology implementaon in higher educaon
intuions must follow a scholarly approach; be
aligned with goals for teaching and research stated in
naonal and instuonal plans and strategies, what
students expect to learn in higher educaon, tested in
a variety of formats, evaluated and renewed in
accordance with acknowledged and familiar academic
work procedures, big data, student feedback, teacher
feedback and new research. This work needs
leadership and can only be achieved through a
collaborave eort.
134 hps://www.regjeringen.no/no/dokumenter/digitaliseringsstrategi-for-
universitets--og-hoyskolesektoren---/id2571085/
KNOWLEDGE CENTRE FOR EDUCATION
|
59
Who are responsible for what?
NATIONAL LEVEL
Naonal Policy, Priories & Strategies
INSTITUTIONAL LEVEL
Leadership responsibilies
Infrastructure for teaching:
• Instuonal ICT policy iniave
• Funding
• Training to develop scholarly teachers
- learning design
- collaborave learning
• Establish a knowledge base for teaching
• Develop ethical guidelines and methods
• Aenon to ethical issues
Develop a «scholarly» approach to teaching:
• Research and experience informed teaching
• Inquire into own teaching pracce
• Disseminate findings
• Take advantage of instuonal programs
• Iniate own professional learning
• Maintain and renew the knowledge base
Staff responsibilies
Figure 3. Responsibilies at naonal and instuonal levels.
KNOWLEDGE GAPS
The systemac review has idened these
knowledge gaps:
• There is a need for longitudinal studies to
invesgate how technology is adopted over longer
periods of me, not only early adopon.
• For the progressive knowledge development
within this research eld, there needs to be a
change from the current focus on simply exploring
the latest technology in quasi- experimental
evaluaons.
• Characteriscs of benecial student acve
learning should be empirically invesgated.
• There is a need for more consistent and rigorous
study designs (common methods, consistent
concept use, measures and reporng standards)
and objects of study
• Studies should establish characteriscs of eecve
knowledge scaolding, social factors, feedback,
ming, assessment modalies etc.) to help
understand what aributes of a learning
environment leads to improved learning
outcomes.
• Empirical research on teaching strategies and
learning outcomes associated with MOOCs is
limited.
• There is limited evidence in the literature that the
ipped classroom and personalised learning leads
to beer grades and improved learning outcomes.
• Currently lacking in the literature is research about
what level of control is benecial for students, and
at which level of exibility higher educaon
courses are eecve in improving student
engagement, experience and learning outcomes.
• Future research should not only address the
development of teaching strategies, which may be
context- and plaorm specic, but also iterave
design approaches for rening these strategies.
• More systemac reviews are needed to establish
the knowledge status of various topics and
research strands.
60
|
KNOWLEDGE CENTRE FOR EDUCATION |
Akiyama, Y., & Cunningham, J. D. (2018). Synthesizing the
Pracce of SCMC-based Telecollaboraon: A Scoping Review.
calico journal, 35(1), 49-76.
Al-Nashash, H., & Gunn, C. (2013). Lecture capture in engineering
classes: Bridging gaps and enhancing learning. Educaonal
Technology & Society, 16(1), 69-78.
Ali, M., Bilal, H. S. M., Razzaq, M. A., Khan, J., Lee, S., Idris, M., ...
& Kang, B. H. (2017). IoTFLiP: IoT-based ipped learning plaorm
for medical educaon. Digital Communicaons and Networks,
3(3), 188-194.
Amemado, D. (2014). Integrang technologies in higher
educaon: The issue of recommended educaonal features sll
making headline news. Open Learning: The Journal of Open,
Distance and e-Learning, 29(1), 15-30.
Avella TJ, Kebritchi M, Nunn SG, Kanai T (2016) Learning analycs
methods, benets, and challenges in higher educaon: a
systemac literature review. Online Learning
Barak, M. (2017). Cloud Pedagogy: Ulizing Web-Based
Technologies for the Promoon of Social Construcvist Learning
in Science Teacher Preparaon Courses. Journal of Science Educa-
on and Technology, 26(5), 459-469.
Blanco-Fernández, Y., López-Nores, M., Pazos-Arias, J. J., Gil-Solla,
A., Ramos-Cabrer, M., & García-Duque, J. (2014). REENACT: A
step forward in immersive learning about Human History by
augmented reality, role playing and social networking. Expert
Systems with Applicaons, 41(10), 4811-4828.
Blau, I., & Shamir-Inbal, T. (2017). Re-designed ipped learning
model in an academic course: The role of co-creaon and
co-regulaon. Computers & Educaon, 115, 69-81.
Cochrane, T. D. (2014). Crical success factors for transforming
pedagogy with mobile Web 2.0. Brish Journal of Educaonal
Technology, 45(1), 65-82.
Dennen, V. P., & Hao, S. (2014). Intenonally mobile pedagogy:
The M-COPE framework for mobile learning in higher educaon.
Technology, Pedagogy and Educaon, 23(3), 397-419.
Edmonds, R., & Smith, S. (2017). From playing to designing:
Enhancing educaonal experiences with locaon-based mobile
learning games. Australasian Journal of Educaonal Technology,
33(6).
Hung, I. C., & Chen, N. S. (2018). Embodied interacve video
lectures for improving learning comprehension and retenon.
Computers & Educaon, 117, 116-131.
Jones, A., & Benne, R. (2017). Reaching beyond an online/
oine divide: invoking the rhizome in higher educaon course
design. Technology, Pedagogy and Educaon, 26(2), 193-210.
Kirkwood, A., & Price, L. (2013). Missing: Evidence of a scholarly
approach to teaching and learning with technology in higher
educaon. Teaching in Higher Educaon, 18(3), 327-337.
Lameras, P., Arnab, S., Dunwell, I., Stewart, C., Clarke, S., &
Petridis, P. (2017). Essenal features of serious games design in
higher educaon: Linking learning aributes to game mechanics.
Brish Journal of Educaonal Technology, 48(4), 972-994.
Lee, D., Morrone, A. S., & Siering, G. (2018). From swimming pool
to collaborave learning studio: Pedagogy, space, and technology
in a large acve learning classroom. Educaonal Technology
Research and Development, 66(1), 95-127.
Maringe, F., & Sing, N. (2014). Teaching large classes in an
increasingly internaonalising higher educaon environment:
pedagogical, quality and equity issues. Higher Educaon, 67(6),
761-782.
Mesh, L. (2016). A curriculum-based approach to blended
learning. Journal of e-Learning and Knowledge Society, 12(3).
Newland, B., & Byles, L. (2014). Changing academic teaching with
Web 2.0 technologies. Innovaons in Educaon and Teaching
Internaonal, 51(3), 315-325.
Ng’ambi, D. (2013). Eecve and ineecve uses of emerging
technologies: Towards a transformave pedagogical model.
Brish Journal of Educaonal Technology, 44(4), 652-661.
Pimmer, C., Mateescu, M., & Gröhbiel, U. (2016). Mobile and
ubiquitous learning in higher educaon sengs. A systemac
review of empirical studies. Computers in Human Behavior, 63,
490-501.
Rambe, P., & Bere, A. (2013). Using mobile instant messaging to
leverage learner parcipaon and transform pedagogy at a South
African University of Technology. Brish Journal of Educaonal
Technology, 44(4), 544-561.
Rienes, B., & Toetenel, L. (2016). The impact of learning design
on student behaviour, sasfacon and performance: A cross-
instuonal comparison across 151 modules. Computers in
Human Behavior, 60, 333-341.
KNOWLEDGE CENTRE FOR EDUCATION
|
61
Shelton, C. (2017). Giving up technology and social media: why
university lecturers stop using technology in teaching. Techno-
logy, Pedagogy and Educaon, 26(3), 303-321.
Sinclair, J., & Aho, A. M. (2018). Experts on super innovators:
understanding sta adopon of learning management systems.
Higher Educaon Research & Development, 37(1), 158-172.
Tegos, S., Demetriadis, S., Papadopoulos, P. M., & Weinberger, A.
(2016). Conversaonal agents for academically producve talk: A
comparison of directed and undirected agent intervenons.
Internaonal Journal of Computer-Supported Collaborave
Learning, 11(4), 417-440.
Toven-Lindsey, B., Rhoads, R. A., & Lozano, J. B. (2015). Virtually
unlimited classrooms: Pedagogical pracces in massive open
online courses. The internet and higher educaon, 24, 1-12.
Van Es, S. L., Kumar, R. K., Pryor, W. M., Salisbury, E. L., & Velan,
G. M. (2016). Cytopathology whole slide images and adapve
tutorials for senior medical students: a randomized crossover
trial. Diagnosc pathology, 11(1), 1.
Vlachopoulos, D., & Makri, A. (2017). The eect of games and
simulaons on higher educaon: a systemac literature review.
Internaonal Journal of Educaonal Technology in Higher
Educaon, 14(1), 22.
Walker, R., Jenkins, M., & Voce, J. (2017). The rhetoric and reality
of technology-enhanced learning developments in UK higher
educaon: reecons on recent UCISA research ndings
(2012–2016). Interacve Learning Environments, 1-11.
Wang, Y. H. (2017a) Using augmented reality to support a
soware eding course for college students. Journal of Computer
Assisted Learning.
Wang, Y. H. (2017b). The eecveness of integrang teaching
strategies into IRS acvies to facilitate learning. Journal Of
Computer Assisted Learning, 33(1), 35-50.
Wanner, T., & Palmer, E. (2015). Personalising learning: Exploring
student and teacher percepons about exible learning and
assessment in a ipped university course. Computers &
Educaon, 88, 354-369.
Wion, G. (2017). The value of capture: Taking an alternave
approach to using lecture capture technologies for increased
impact on student learning and engagement. Brish Journal of
Educaonal Technology, 48(4), 1010-1019.
Zheng, B., Niiya, M., & Warschauer, M. (2015). Wikis and
collaborave learning in higher educaon. Technology, Pedagogy
and Educaon, 24(3), 357-374.
62
|
KNOWLEDGE CENTRE FOR EDUCATION |
Search string (Scopus syntax)
TITLE-ABS-KEY(“1 to 1 computer” OR “blended
learning” OR “CAI” OR “CAL” OR “CBL” OR “cloud
compung” OR “collaborave learning” OR
“computer aided” OR “computer assisted instrucon”
OR “computer assisted learning” OR “computer based
instrucon” OR “computer based learning” OR
“computer based teaching” OR “computer
simulaon*” OR “computer supported” OR “computer
technology” OR “computer use” OR “computer-aided”
OR “computer-assisted instrucon” OR “computer-
assisted learning” OR “computer-based instrucon”
OR “computer-based learning” OR “computer-based
teaching” OR “computerized instrucon” OR
“computers and learning” OR “computers in
educaon” OR “computer-supported” OR “compung
educaon” OR “digital learning” OR “digital
technology” OR “educaonal technology” OR
“e-learning” OR “electronic learning” OR “game*” OR
“ICT*” OR “informaon communicaon technolog*”
OR “innovave technology” OR “Instruconal
technologies” OR “intelligent tutoring system*” OR
“interacve learning environment*” OR “interacve
learning object*” OR “interacve simulaon*” OR
“Interacve white board*” OR “learning eect*” OR
“local area network*” OR “massive open online
courses” OR “media in educaon” OR “mobile
learning” OR “MOOC” OR “mulmedia learning” OR
“OER” OR “one to one computer” OR “one2one
computer” OR “online learning” OR “online learning
communies” OR “online open educaonal
resources” OR “online self study” OR “online study”
OR “rich media” OR “serious game*” OR “simulaon
based educaon” OR “simulaon based teaching” OR
“simulaon-based educaon” OR “simulaon-based
teaching” OR “simulaons” OR “social network” OR
“supplemental CAI” OR “tablet*” OR “technology
enhanced instrucon” OR “technology enhanced
learning” OR “technology use” OR “technology-
enhanced instrucon” OR “technology-enhanced
learning” OR “TEL” OR “tutoring system*” OR “virtual
learning” OR “virtual reality” OR “VLE” OR “web-
based instrucon*” OR “web-based learning” OR
“web-based training” OR “wireless network*”) AND
TITLE-ABS-KEY(“acve learning” OR (“collaborat*”
W/5 “lecturer”) OR (“collaborat*” W/5 “student”) OR
(“collaborat*” W/5 “teacher”) OR “eecve learning”
OR “enhanc* learning” OR (“innovave” W/5
“learning”) OR (“innovave” W/5 “teaching”) OR
(“interact*” W/5 “lecturer”) OR (“interact*” W/5
“student”) OR (“interact*” W/5 “teacher”) OR
“learning delivery” OR “learning design” OR
(“learning” W/5 “exible”) OR (“learning” W/5
“personali?ed”) OR “pedagog*” OR “teaching
delivery” OR “teaching method*” OR “teaching
model*”) AND TITLE-ABS-KEY(“college” OR “faculty”
OR “HE” OR “higher educaon” OR “university”)
KNOWLEDGE CENTRE FOR EDUCATION
|
63
QUALITY
REFERENCE METHOD QUALITY
Akiyama & Cunningham (2018) Literature review Medium
Al Nashash & Gunn (2013) Quantave and qualitave Medium
Ali et al. (2017) Theorecal Medium
Amemado (2014) Qualitave Medium
Avella et al. (2016) Systemac review High
Barak (2017) Mixed methods High
Blanco-Fernández et al. (2014) Qualitave Medium
Blau & Shamir-Inbal (2017) Qualitave High
Cochrane (2014) Qualitave Medium
Dennen & Hao (2014) Qualitave Medium
Edmonds & Smith (2017) Quantave and qualitave High
Hung et al. (2018) Quantave Medium
Jones & Benne (2017) Theorecal High
Kirkwood & Price (2013) Literature review Medium
Lameras et al. (2017) Literature review High
Lee et al. (2017) Mixed methods Medium
Maringe & Sing (2014) Literature review High
Mesh (2016) Quantave Medium
Newland & Byles (2014) Qualitave High
Ng'ambi (2013) Quantave and qualitave High
Pimmer et al. (2016) Systemac review High
Rambe & Bere (2013) Quantave and qualitave Medium
Rienes & Toetenel (2016) Quantave High
Shelton (2017) Qualitave High
Sinclair & Aho (2018) Qualitave Medium
Tegos et al. (2016) Quantave High
64
|
KNOWLEDGE CENTRE FOR EDUCATION |
REFERENCE METHOD QUALITY
Toven-Lindsey et al. (2015) Qualitave High
Van Es et al. (2016) Quantave and qualitave High
Vlachopoulos & Maki (2017) Systemac review High
Walker et al. (2017) Qualitave High
Wang (2017a) Quantave and qualitave High
Wang (2017b) Quantave and qualitave High
Wanner & Palmer (2015) Quantave and qualitave High
Wion (2017) Quantave and qualitave Medium
Zheng et al. (2015) Quantave and qualitave Medium
PREVIOUS PUBLICATIONS FROM
THE KNOWLEDGE CENTRE FOR EDUCATION
Lillejord S. & Børte K. (2018). Mellomledere i skolen:
Arbeidsoppgaver og opplæringsbehov
– en systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord, S., Elstad, E., & Kavli, H. (2018). Teacher
evaluaon as a wicked policy problem. Assessment
in Educaon: Principles, Policy & Pracce, 1-19.
DOI: 10.1080/0969594X.2018.1429388
Lillejord S., Børte K., Nesje K., & Ruud E. (2017).
Campusuorming for undervisning, forskning,
samarbeid og læring
- en systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord S., Børte K., Ruud E. & Morgan K. (2017).
Stress i skolen – en systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord, S., Johansson, L., Canrinus, E., Ruud, E.
& Børte, K. (2017). Kunnskapsbasert språkarbeid i
barnehager med erspråklige barn – en systemask
forskningskartlegging.
Oslo: Kunnskapssenter for Utdanning,
www.kunnskapssenter.no
Lillejord, S. & Børte, K. (2017). Lærerutdanning som
profesjonsutdanning - forutsetninger og prinsipper
fra forskning. Et kunnskapsgrunnlag.
Oslo: Kunnskapssenter for Utdanning,
www.kunnskapssenter.no
Lillejord, S., Børte, K., Halvorsrud, K., Ruud, E.,
& Freyr, T. (2017). Transion from Kindergarten
to school: A systemac review.
Oslo: Knowledge Centre for Educaon,
www.kunnskapssenter.no
Morgan, K., Morgan, M., Johansson, L. & Ruud, E.
(2016). A systemac mapping of the eects of ICT
on learning outcomes.
Oslo: Knowledge Center for Educaon.
www.kunnskapssenter.no
Lillejord, S., Vågan, A., Johansson, L., Børte, K.
& Ruud, E. (2016). Hvordan fysisk akvitet i skolen
kan fremme elevers helse, læringsmiljø og
læringsutbye. En systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for Utdanning.
www.kunnskapssenter.no
Børte, K., Lillejord, S. & Johansson, L. (2016).
Evnerike elever og elever med stort læringspotensial:
En forskningsoppsummering.
Oslo: Kunnskapssenter for Utdanning.
www.kunnskapssenter.no.
Lillejord, S., & Børte, K. (2016) Partnership in teacher
educaon – a research mapping. European Journal
of Teacher Educaon, 39(5), 550-563
Lillejord, S., Børte, K., Halvorsrud, K., Ruud, E., &
Freyr, T. (2015). Tiltak med posiv innvirkning på
barns overgang fra barnehage l skole:
En systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord, S., Halvorsrud, K., Ruud, E., Morgan, K.,
Freyr, T., Fischer-Griths, P., Eikeland, O. J.,
Hauge, T. E., Homme, A. D., & Manger, T. (2015).
Frafall i videregående opplæring:
En systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord, S., Ruud, E., Fischer-Griths, P., Børte, K.,
& Haukaas, A. (2014). Forhold ved skolen med
betydning for mobbing. Forskningsoppsummering.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Lillejord, S. & Børte, K. (2014). Partnerskap i
lærerutdanningen – en forskningskartlegging.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
Wasson, B & Morgan, K. (2014). Informaon and
Communicaons Technology and Learning:
State of the Field Review.
Oslo: Knowledge Centre for Educaon,
www.kunnskapssenter.no
Baird, J-A., Hopfenbeck, T. N., Newton, P., Stobart,G.
& Steen-Utheim A. T. (2014). Assessment and
Learning: State of the Field Review.
Oslo: Knowledge Centre for Educaon,
www.kunnskapssenter.no
Lillejord, S., Børte, K., Ruud, E., Hauge, T. E.,
Hopfenbeck, T. N., Tolo, A., Fischer-Griths, P.
& Smeby, J.-C. (2014). Former for lærervurdering som
kan ha posiv innvirkning på skolens kvalitet: En
systemask kunnskapsoversikt.
Oslo: Kunnskapssenter for utdanning,
www.kunnskapssenter.no
KNOWLEDGE CENTRE FOR EDUCATION
TELEPHONE: +47 22 03 70 00
EMAIL: kunnskapssenter@forskningsradet.no
INTERNET: www.kunnskapssenter.no
FACEBOOK: kunnskapssenter
TWITTER: kunnskapsrad
