Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Jul 14, 2019
Content may be subject to copyright.
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
A Pedagogical Framework for Mixed Reality in Classrooms based on a
Literature Review
Christopher Kommetter
Educational Technology, Graz University of Technology, Austria
kommetter@tugraz.at
Martin Ebner
Educational Technology, Graz University of Technology, Austria
martin.ebner@tugraz.at
Abstract: Virtual reality devices are not an invention of the previous years. In fact, those are
applicable since the 60s and therefore many scholars have elaborated in the potential of virtual
reality as a learning platform. There are many diverse ways to integrate these techniques into the
classroom. Affordable devices in combination with concepts like making legitimate the usage for
learning. Pedagogical motivation factors like collaborations and gamification amply justify the
usage of modern VR technologies as a learning platform. A variety of application have been used
already in classrooms but in higher education, there is a need to catch up. Those examples have
been divided by the way they make use of virtual systems and how they provide supervision. This
literature review describes the functional classification of virtual reality, how it typically applies to
learning and how it represents the key features for a didactically valuable usage. As a result of the
literature research, a framework for the use of mixed reality in education has been developed
allowing to spot new capabilities for the implementation of future VR or AR applications. This
paper will also clarify the proper direction of development and the potential impact on future
educational domains.
Introduction
Augmented reality (AR) Applications, as well as virtual reality (VR), was developed around the same time
periods in the late 1960s where AR was considered as a gradual extension of VR (Cai, Chiang, & Wang, 2013). To
achieve a Virtual World experience, Head Mounted Displays using two CRT's1 (one for each eye) were developed
(Sutherland, 1968). In recent years, head-mounted display technology has been major improved. So-called VR
headsets as Oculus Rift, HTC Vive, Samsung Gear VR, etc. make virtual reality available to the public (Sternig,
Spitzer, & Ebner, 2017). Especially when Google releases the Google Cardboard back in 2014 everyone can afford
the VR experience and the challenge presently becomes to discover possible ways of implementing this in an
educational sector (Boyles, 2017).
This research work is structured in three sections. To begin with, the research method will be described. In
the second part, the authors will give an overview of the continuum of mixed reality. The last part then presents the
results of the empirical research and also examples of the different implementations are shown in this review. The
authors will also explain how the examples are clustered. Studies will be presented as to indicate how the various
examples could scientifically prove to be used in educational purpose. Given examples on VR and AR technologies
used in educations are classified due to the output of the pedagogical concepts found in the literature. As a result of
all the shown examples, a couple of didactical concepts will present to underpin the usage of these in an educational
matter and furthermore the authors will present their developed framework as an outcome of the literature research.
In the conclusions, the authors are going to quote some thoughts on future work. The aim of this work is to gain a
clear idea of the potential of using virtual reality components in education. The authors will offer an overview of the
1 CRT: cathode-ray tube
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
significant technologies for supporting education. This research will furthermore declare the direction of
development in mixed reality items.
Research Design
Before actual reviewing didactical concepts, the authors must first clarify the consistent definition of virtual
reality (VR) and have to define the classification on VR systems. The various types of VR systems have been
clustered due to the definitions found in scholarly literature regarding the appropriate technology they were using.
The authors started searching on academic databases like ERIC, WebOfScience, ACM Digital Library and also
SpringerLink A search using Google Scholar was also performed. Search strings like "virtual reality", "VR",
"augmented reality", "mixed reality" and "AR" were used logical connection with the terms "classroom", "school",
"education" and "training" to gain results. A number of 213 papers were selected and been clustered in a second
analysis according to the kind of technology used. A number of these papers primarily examine technical
specifications and implementations, so they have been dropped for further analysis. The remaining ones have then
been reduced to those who are either describing use cases or pedagogical concepts.
Classification of mixed reality
Taxonomy of MR
Augmented and virtual reality is sometimes getting messed up, although AR systems are primarily
considered as a part of virtual reality. Augmented reality assumes the place between a completely computer
generated world (VR) and the real world because it is developed to overlay the real world by placing computer-
generated objects into real environments (Azuma, 1997; Zagoranski & Divjak, 2003). In contrast to virtual reality,
AR interfaces use handheld or head-mounted displays to see through or overlay graphics on video of the
surrounding world against other computer interfaces which draw users away from the surrounding world into a
virtual one (Billinghurst, 2002).
Figure 1: The Virtuality Continuum (VC) of Milgram defines the area between Real to Virtual environment as mixed reality,
where AR is supposed to augment the real world with virtual objects while AV is augmenting a virtual world with real-world
objects adopted from Milgram & Kishino (1994)
Milgram defined the area between the real word and a fully immersive world as a continuum where every
technology is succinctly summarized as mixed reality (figure 1). Augmented reality is one part of it. Augmented
virtuality (AV) as well as Virtual Environments (virtual reality), the environment surrounding a user is virtual in
contrast to AR where the environment is real (Azuma et al., 2001). In conclusion, mixed reality (MR) can be defined
as an interactive setting, where the user is either placed in the real world with virtual assets augmenting (AR) or in a
virtual world augmenting real-world objects (VR) (Hughes & Stapleton, 2005; Milgram & Kishino, 1994).
Augmented reality (AR)
AR, a 3D technology that fuses the physical and digital world in real time (Pasaréti et al., 2011), represent a
variety of virtual reality (Azuma, 1997). Users are able to interact in a seamless way with the virtual and computer-
generated images using real objects (Zhou, Duh, & Billinghurst, 2008). Natural operations within the real world
enable the users a seamless and real-time interaction with the virtual objects (Cai et al., 2013). AR would also be
able properly to apply to all senses, for now, its focus on sight to include virtual objects to a real environment
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
(Azuma, 1997). AR became popular to the public in 2016 because of the AR Game Pokèmon Go which was the
hype in the Summer of 2016 (Bonner & Reinders, 2018; Koll-Schretzenmayr & Casaulta-Meyer, 2016). AR
applications can be classified in marker-based, marker-less and location-based (Chen & Tsai, 2012). Pokèmon Go
would be a representative of a location-based AR system. Marker-based systems require labels containing a pattern
like a QR-Code whereas the third classification is typically based on the recognition of object shapes (Fotaris,
Pellas, Kazanidis, & Smith, 2017).
AR Application has been used for classroom-based learning within different subjects for appropriate K-12
education or higher but is not strongly represented in academic settings (Lee, 2012). But there are interests in
applying AR to universities even though the usage of these technologies does not lead to increased interaction
between students and the professor (Redó et al., 2008). Modern usage of AR integrated into real teaching
environments, like a lecture theatre, would at least allow students to experiment with virtual teaching material in an
engaging manner (Liarokapis & Anderson, 2010).
Virtual reality (VR)
The authors employ the term virtual reality as a synonym for augmented virtuality (AV) as shown above.
Unlike augmented reality, VR produces a world of its own, rather than expanding the real one. The dominant
features, according to Dede, Salzman, & Loftin (1996) of virtual worlds are:
• Immersion: Users typically interact within a completely by computers generated world (Nooriafshar,
Williams, & Maraseni, 2004).
• Telepresence: An Instructor, for example, can help remotely to find a solution for a given problem
(Spitzer, Nanic, & Ebner, 2018). Virtual reality technologies are capable to overlay an output from a 3D
Camera over the real world and are therefore capable to bridge over the practical limitations of traditional
video conferencing technologies like lack of spatial cues, limited screen space, and separation between real
and digital objects (Haller, 2004).
• Multisensory simulation: This can support users to investigate things trough VR by relying on their senses
to promptly recognize phenomena (Salzman, Dede, Loftin, & Chen, 1999). User's hand motion can be
tracked by using special gloves (Theng, Lim Mei-Ling, Liu, & Cheok, 2007) to interact with the system
and then get instantaneous feedback on their interactions.
• Motivation: Students genuinely enjoy the ability to move and handle with the object within an (immersive)
virtual world (Bricken & Byrne, 1992).
• Multiple representations and 3D frames of reference: A user can decide which information will be
visible by viewing in a certain direction in order to alter the angle of view which helps to concentrate on the
desired object (Erickson, 1993).
Virtual reality systems can be categorized in five sections: (i) simulation-based, (ii) avatar image-based,
(iii) projector-based, (iv) desktop-based and (v) true immersive based VR (Nachimuthu & Vijayakumari, 2009).
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
Figure 2: Classification of virtual reality (VR) based on type of technology used to achieve virtual worlds and interact with them
adopted from Muhanna (2015)
Muhanna (2015) classified VR in two types, depending on the type of technology used and further on the level of
mental immersion. Figure 2 shows the classification of VR in two groups, basic and Enhanced, depending on the
technology being used. The separation of enhanced VR systems was classified according to the degree of
immersion. Immersion is a term for technology designed to make a user feel a sense of being a presence in a virtual
world (Southgate, 2018) and the scale how much the user feels to be in a virtual world is called the degree of
immersion (Bronsch, 2018). According to Muhanna (2015) immersion is one of the Key elements of virtual reality
experience. Immersion allows students to discover the surrounding in a natural way an, therefore, aids them to learn
by allowing them to construct concepts by their direct experience of the environment (Jackson & Fagan, 2000;
Winn, Windschitl, Fruland, & Lee, 2002). Fully immersive virtual reality can be realistically achieved using a
CAVE Environment (Cave Automatic Virtual Environment). Projectors, or multiple LCD Screens (CAVE2), are
constructing a virtual room by surrounding the walls around users. Users may also wear 3D-Glasses to enhance
immersion. Using a CAVE multiple users can interact within the same virtual room at the same time and share same
experiences (Boyles, 2017; Cruz-Neira, Sandin, & DeFanti, 1993; Manjrekar et al., 2014).
Results and discussion
Pedagogical context
One significant advantage of the use of virtual reality systems is the ability to animate object, respond by
the user's action and to break the borders on physical limitations of real objects (Woods et al., 2004). Virtual reality
can also enhance distance learning, due to interfaces supporting remote actions with other learners or teachers
connecting to existing objects (Mellet-d'Huart, 2012). AR can enhance the way students learn in a creative and
convenient way: for example, an ordinary building can be augmented with educational content, so that the students
would be able to read things while they remain the way of interaction with the environment as they were used to it
(Kaufmann, 2003). In this context, the operation site will be examined in the examples found in the literature, if
either the students used the setting at school or alone at home. Using it at school does not typically mean the
building, further is it used as a term for an educational setting with a present teacher, this could also be the case at an
excursion.
Using fully immersive virtual worlds allows users interacting with objects in a natural way as they were
used to it (Winn, 1993), but users won't be able to share gestures. One essential issue with collaboration is that users
can see each other's facial expressions, gestures and body language which will increase communication
opportunities (Billinghurst & Kato, 1999). To increase a intend learning outcome, cooperative interactions with
adults and peers will help achieve that by exposing their peers thinking process, knowledge, and skills
(Thorsteinsson & Page, 2007). Augmented reality systems can establish that in contrast to a fully immersive VR
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
system. Students can see each other while they can see and virtual objects and interact with them. Among
Billinghurst and Kato (1999) AR systems offer besides the enhance of face-to-face collaboration also the ability for
a remote collaboration because they allow users to use traditional tools and workplace practices. However, there are
not many learning systems using AR designed for multi-user collaborations, instead they are currently designed that
one person is controlling the AR system while others are only observing via a shared display. A social interaction
between students within the same physical space should be a very important goal for an educational environment
(Roussos et al., 1999). Sternig et al. (2017) gave an example for combining making, a movement emerged in the last
years where children are supposed to create things by their own (Schön, Ebner, & Kumar, 2014), by building a
simple VR headset like the google cardboard. Using action-based learning can positively enhance the relationship
between reflection and action by working together (Wagner & Ip, 2009). Therefore, we are interested to determine
the scope of the used learning environments, if the pupils either been using the system on their own or if they use the
same virtual space for collaboration.
To increase student motivation, engagement and enjoyment on learning something, game-like elements can
be applied (Kavanagh et al., 2017). Such elements could be, for example, a ranking list where students get certain
points for an exercise and can then compete for each other. Through the use of Gamification, there is a
transformation from a teacher-centered learning environment to an activity-based approach with social interactions
(Jensen, 2017). This approach could be further applied on video games to make use of gamification for educational
purpose. Since the first video game was produced, the idea of learning while playing was present (Sternig et al.,
2017). Game-based learning environments attempt to motivate and challenge learners to solve problems and offer
playful access to gain knowledge and different knowledge concepts as well as competences (Zumbach & Moser,
2012). Virtual Worlds, like Second Life, offer a variety of Action Based Learning possibilities. Users can practice
behaviors, repeat them to gain practice and observe the outcome to adjust behaviors based on the outcomes (Wagner
& Ip, 2009). In such environments, students may not feel they are learning (Cheal, 2009). He also pointed out, that
some students concern about the validity of leaning in a game-like environment.
As augmented reality systems offer a new way to interact with computers, it allows users to interact with
object and events in a natural way, which gains the potential for the use as an educational tool (Kesim & Ozarslan,
2012). Although AR technology can be used in a wide range of domains, using it for educational experience is
different for a number of reasons, as Billinghurst (2002) mentioned:
• Support of seamless interaction between real and virtual environments
• The use of a tangible interface metaphor for object manipulation
• The ability to transition smoothly between reality and virtuality
Physical activity is linked to conceptual understanding, and students can increase their learning outcome when they
use their hands to interact with scenes from an augmented story (Radu, 2014). Augmented reality systems can
establish such an interactive learning environment. AR provides students with a 3D view and continuously changing
content by interacting, increasing their interest in learning (Cai et al., 2013).
Although VR environments are largely self-explanatory, a guideline should be provided to the users in
order to ensure learning benefits (Mellet-d'Huart, 2012). This should provide the user with information on how to
use for example the controller. Combining the usage of an MR system with game-based learning concepts, easy
navigation is essential for being successful (Sternig et al., 2017). In order to portray the actual scenario in teaching
activities, the presence of a supervisor must be observed. A supervisor could either be next to the students or even
could be a part of the virtual environment either as virtual human or artificial intelligence.
Examples in teaching
In our research examples were categorized according to the used MR technology and it was noted in which
setting they were used. This are just selected examples. The aforementioned pedagogical approaches were recorded.
Name
Type
Operation
site
Teaching
method
Level of
pervasion
Supervision
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
Aurasma/HP
Reveal
AR
School &
excursion
single
enhancing
1:n instructor in real world guides pupils
Anatomy 4D
and similar
APPS
AR
School &
At home
single
enhancing
1:n teacher is present in real world;
instructions within the application are
served without any AI2
MARIE
AR
School
single
enhancing
1:1 teacher control parts of the system
Studierstube &
Construct3D
AR
School
single or
group
replacing
1:1, 1:n teacher is present in real
environment next to student(s)
MagicBook
AR/VR
School
single or
partner
enhancing
1:n typically school setting where a
teacher in real environment is guiding
students
Second Life
VR
At home
group
replacing
1:n in virtual word, avatar controlled by
a human
Vicher I and II
VR
School
single
replacing
1:1, Instructions in a virtual welcome
center, fully computer-generated without
AI
ScienceSpace
VR
School
single
replacing
1:n teacher in real environment guides
students
Table 1: Literature examples categorized
AR based systems
There is a variable number of Applications using AR or VR made for the educational market. The free iOS
and Android Application called HP Reveal (formerly called Aurasma) uses trigger images and overlays them with
augmented elements like images, videos, websites and 3D object as seen in figure 3 (Panciroli, Macauda, & Russo,
2018). Students, for example, could take pictures of anything, e.g. Buildings, and overlay them with educational
material such as videos (Bonner & Reinders, 2018; Buchner, 2017a). Besides the Application also an Online portal
is offered, where, for example, a teacher can prepare any so-called auras and the students can then subscribe to the
teacher's channel and will then be able to use them. A similar APP is called blippar (Bonner & Reinders, 2018;
Stankovi'c, 2015). Both mobile applications are ascribable to AR and both of them were developed back in 2011.
The field of application is restricted to schools or field trips, e.g. school trip to a museum, and needs a teacher to
prepare AR materials and to guide pupils on the usage of this technique. Each student operates the given tasks using
an own device to enhance his surroundings.
Using the App Anatomy 4D students are able to augment a human body in life size to the room as figure 4
shows. With the application, they are able to have a look into a human body as a kind of an AR T-shirt which allows
selecting layers like muscles or nerves (Buchner, 2017b). The user can freely move around the augmented human
body and can, therefore, control the desired angle of view. Most parts of this application are self-explaining.
However, a teacher is supposed to give students instructions tasks to fulfill using the application on their own.
Due to the fact, that most virtual reality systems are specially made to fit the need of a single, specific
domain, researchers at the Technical University of Vienna started working on Studierstube trying to describe how to
use an AR-based media in general environment (Schmalstieg et al., 2002). Users wear a see-through head-mounted
display (HMD) to collaboratively view 3D models augmented onto the real world and can freely move around. To
interact they can make use of Personal Interaction Panels (PIP) which is according to user reports a very intuitive
interface and conducive to real-world collaboration (Szalavári, Schmalstieg, Fuhrmann, & Gervautz, 1998). Based
on Studierstube Construct3D was created and this tool has been already used on teaching geometry and mathematics
in secondary education (Kaufmann, 2011; Kaufmann & Schmalstieg, 2002; Kaufmann, Schmalstieg, & Wagner,
2000). Construct3D allows teachers and students to collaborate within interactive scenarios (Liarokapis & Anderson,
2 AI: artificial intelligence
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
2010) and fits the category of the game based learning to teach geometric construction education (Leitão, Rodrigues,
& Marcos, 2014).
Figure 3 left: Example of using the APP HP Reveal (former Aurasma) on augmenting real object like schoolbooks or printed
papers as a trigger image with multimedia content like Videos, Pictures, 3D Objects or Websites
Figure 4 right: Anatomy 4D is a free APP using augmented reality which allows students to interact with a virtual human body
augmented into the room using an iPad as a handheld display (HHD)
Another AR Project is MARIE (Multimedia Augmented Reality Interface for E-Learning) with the Aim of
to enhance the teaching and learning process using an augmented reality e-learning environment (Liarokapis,
Petridis, Lister, & White, 2002). These authors note that augmented reality is more effective than virtual
environments when comparing price, real-time augmentation, and interactivity.
VR based systems
MagicBook is a mixed reality environment which cannot be clearly assigned to either VR or AR
(Billinghurst, Kato, & Poupyrev, 2001b). It is able to create a seamless transition along the Virtuality Continuum
(figure 1) Billinghurst, Kato and Poupyrev (2001a) developed the MagicBook. The MagicBook looks like a normal
textbook or storybook and can for sure be used as such. When users look at the pages of the book through a Head-
mounted Display (HMD) or a Handheld Display (HHD), 3D object and virtual scenes look like to sit on the page
(Cho, Lee, Lee, & Yang, 2007). HHD or HMD along a computer and at least one physical book remain the three
core components of the Magic Book (Billinghurst et al., 2001b). Users can turn the book around or move around the
book to alter the angle of view (Zumbach & Moser, 2012) as known from HP Reveal. When users want to
concentrate on a certain scene, they simply fly into the page by decreasing the distance between the Head and the
book and can then experience it as an immersive virtual environment (Cai et al., 2013). This allows students to read
together and see the virtual object from their own viewpoint and can fly into the virtual space individual and see
others than as an avatar in the virtual story space (Billinghurst et al., 2001a).
Second Life is a virtual social space allowing users to connect with each other and play together. Academic
institutes started to use Second Life (SL), due to its popularity and 3D environmental structure, for training. SL
offers a high degree of customizability which makes the usage as a learning environment a logical step (Taylor &
Chyung, 2008). A lecturer can operate a virtual campus to teach using PowerPoint for example and participants can
join in real-time and interact with each other (Laws, Forsyth, & Baskett, 2009) in a virtual environment represented
by avatars. In Contrast to other MUVE (massive multiplayer virtual environment), Second Life was a subject for
instruction and instructional support in higher education in many studies and articles over the year 2009 (Brown &
Green, 2009). Due to the fact that the students among each other, as well as the teacher, are only interacting with
each other in a virtual world, it's not possible to share expressions and gestures with each other.
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
Vicher (Virtual Chemical Reaction Module), a virtual reality based simulator, was developed at the
University of Michigan Chemical Engineering Department for undergraduate chemical kinetics and reactor design
education (Bell & Fogler, 1995, 1996). Vicher provides a fully immersive virtual. A User evaluation has shown that
users more accurate and more proper understanding of engineering concepts and 80 % felt they had learned
something of the experience (Youngblut, 1998). To eliminate any ambiguity, a Welcome Center, implemented as a
virtual room, familiarizes the user with the scope of all possibilities to interact with the system (Bell & Fogler, 1995,
1996, 1998; Youngblut, 1998) as a form of interactive tutorials within the immersive virtual world.
The 1994 developed virtual world ScienceSpace consists of three applications: NewtonWorld,
MaxwellWorld, and PaulingWorld to simulate physical phenomenon's (C. Dede, Salzman, & Bowen Loftin, 1996).
In NewtonWorld a student can learn about the Laws of Motion by interacting with the virtual world using his hand
(Boas, 2013). MaxwellWorld would help students obtaining a view if the electric field produced by a charge
configuration (Salzman et al., 1999). PaulingWorld is designed to be a research tool as well as a teaching tool to
examine the structure of molecules from a certain angle of view (C. Dede et al., 1996).
Framework for mixed reality use in education
As a result of the found examples in literature, a framework was developed to cluster given mixed reality
applications for the usage in an educational setting. Thus it is possible to spot new capabilities for the
implementation of future VR or AR applications in classrooms. Figure 5 gives an overview on the framework,
where the first part defines a categorization of the type of mixed reality used on basis of the definition above. A
further subdivision of a MR application is, according to the framework, given by the site of operation. Typically,
this kind of applications are assumed to be used either in a classroom or any other setting where a teacher is present.
However, an educational MR implementation could also be designed to be used at home without any educator.
Besides the operation site, the teaching method needs to be specified. Mixed reality programs could be established to
be used in groups, alone or in pairs. Students can either have to fulfill a simple task using a MR domain by assisting
or enhancing their environment or they can even do a full learning experience where an application is replacing their
common learning setting with mixed reality. The last segment of the framework defines the type of supervision. An
instructor can either be present within the virtual world as an avatar controlled by a teacher, for example, or even
been controlled by any kind of artificial intelligence. But the teacher could also be present in the real world to guide
students. Therefore, the relation between tutor and learner needs to be observed, if either a teacher guides a group of
students (1:n) at the same time, or only one student (1:1).
Figure 5: Framework for mixed reality use in education, examples in table 1 are examined according this framework
The selected examples recorded in table 1 are examined according to the framework build by the authors
(see figure 5). Remembering the example of the AR application HP Reveal (Aurasma) it can be shown that this has
been used within classrooms and excursions and therefore the operation site was classified as “school”. Each student
used their own mobile device to complete several tasks. Therefore, the teaching method is in this case “single” and
the level of pervasion was defined as “enhancing”. A teacher issues instruction to the whole class in real world next
to the students, so supervision is specified as “1:n by teacher in real world”. Second Life on the other hand side
school
home
AR
VR
1:1
1:n
assisting
enhancing
replacing
group
single
partner
Type
Operation site
Teaching
method
Level of
pervasion
Supervision
Application
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
makes use of VR technology. Learners use this application at home and participate in courses in a group, so
operation site is “home” and teaching method is “group”. By the fact, that students use Second Life on learning
whole new things instead of just fulfill some tasks to retain new knowledge, level of pervasion could be defined as
“enhancing”. A course leader is present within the virtual world as a human controlled avatar instruction the whole
group at the same time, therefore supervision is “1:n by virtual teacher controlled by human”.
Most of the examples shown require any kind of instruction, as table 1 shows. Most time a teacher was
present in the real world next to the students for supervision either for one student (1:1) or for a group of students
(1:n). A few examples used any kind of virtual guidance, but none of them used an automatic tool to instruct
students. Even full immersive virtual worlds which are used to help students solve complex tasks need a teacher to
be present. A kind of AI in form of an avatar, like in Second Life human-driven, could improve a virtual
environment through the ability to individually assist each student. Therefore, a usage beyond a school-setting could
be established. Also, the availableness of cheap VR Headsets presents a step forward in this direction to make use of
VR as learning platform instead of an only entertaining one.
Conclusion and outlook on future work
During the review, the authors discovered that newer papers addressed AR systems. This could be due to a
change in research from augmented virtuality System to augmented reality ones. AR applications, as the MagicBook
shown formerly, could be easily be realized and can enhance a traditional schoolbook for example on showing 3D
visualizations of the teaching content (Zumbach & Moser, 2012). MagicBook is capable of providing transmission
from a real world to a fully immersive virtual world over the entire continuum of mixed reality. An easy to use AR
enhanced schoolbook can, for example, be achieved by using the APP HP Revival. An AR learning environment can
be implemented more cost-effectively than a VR system and also the effort of designing learning material can be
moderate compared to VR (Chen & Tsai, 2012). Among the usability of software and interfaces, Bricken (1991)
addresses cost and fears about technology as a challenge for the use of VR in educational settings. Virtual reality
systems have a chicken and egg problem, cause it takes a huge amount of time to create content to make a system
for teaching and this content is not generated until there are enough systems developed to make use of it (Allison &
Hodges, 2000).
The authors also mentioned that interaction degree and further the learning motivation is more pronounced
on using AR instead of VR. AR also increases creativity and grasping the power of the mind (Gurjar & Somani,
2016). Future work can build on this to generate a framework for creating effective educational AR Applications.
Furthermore, changes in the interface to fit the needs of students for interacting with the system as Shelton & Hedley
(2004) pointed out, should be part of further research. AR would also be capable of being used in multi-user settings
in classrooms and auditoriums, little has been researched on this topic (Shelton & Hedley, 2004). There is still a
need for research to investigate purposes of the use of AR in lecture halls. Modern equipment, as well as fast
network connections, make augmented reality systems capable to be used as multi-user settings in lectures.
Although AR and VR systems are not as common in higher education, they have been used for training in the
military sector and also the NASA is adopting this technology for training and preparation for technicians for many
years (Pantelidis, 1997). Didactical concepts of this form of andragogy could be served as a basis for further
research on applying virtual reality systems to higher education.
Acknowledgements
This activity has received funding from the European Institute of Innovation and Technology (EIT), a body
of the European Union, under the Horizon 2020, the EU Framework Programme for Research and Innovation.
Further information about MiReBook: https://eitrawmaterials.eu/course/mirebooks-mixed-reality-handbooks-for-
mining-education/
References
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
Allison, D., & Hodges, L. F. (2000). virtual reality for Education? In Proceedings of the ACM Symposium on virtual reality
Software and Technology (pp. 160–165). New York, NY, USA: ACM. https://doi.org/10.1145/502390.502420
Azuma, R. T. (1997). A Survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.
Azuma, R. T., Behringer, R., Feiner, S., Julier, S., MacIntyre, B., & Baillot, Y. (2001). Recent advances in augmented reality.
IEEE Computer Graphics and Applications, 2011(November), 1–27. https://doi.org/10.1109/38.963459
Bell, J. T., & Fogler, H. S. (1995). The investigation and application of virtual reality as an educational tool. In Proceedings of
the American Society for Engineering Education Annual Conference (pp. 1718–1728).
Bell, J. T., & Fogler, H. S. (1996). Vicher: a virtual reality based educational module for chemical reaction engineering.
Computer Applications in Engineering Education, 4(4), 285–296.
Bell, J. T., & Fogler, H. S. (1998). Virtual reality in chemical engineering education. In Proceedings of the 1995 Illinois/Indiana
ASEE Sectional Conference (pp. 16–18).
Billinghurst, M. (2002). augmented reality in Education. New Horizons for Learning, 12(5).
Billinghurst, M., & Kato, H. (1999). Collaborative mixed reality Games. Proceedings of the First International Symposium on
mixed reality (ISMR ’99), 261–284.
Billinghurst, M., Kato, H., & Poupyrev, I. (2001a). MagicBook: transitioning between reality and virtuality. In CHI’01 extended
abstracts on Human factors in computing systems (pp. 25–26).
Billinghurst, M., Kato, H., & Poupyrev, I. (2001b). The MagicBook: a transitional AR interface. Computers & Graphics, 25(5),
745–753.
Boas, Y. A. G. V. (2013). Overview of virtual reality technologies. In Interactive Multimedia Conference (Vol. 2013).
Bonner, E., & Reinders, H. (2018). Augmented and virtual reality in the Language Classroom: Practical Ideas: EBSCOhost.
Teaching English with Technology, 18(3), 33–53. Retrieved from
https://web.a.ebscohost.com/ehost/detail/detail?vid=0&sid=3c7a4151-977d-4c42-ab6d-
bc2d23f4d3d4%40sessionmgr4009&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3D%3D#AN=EJ1186392&db=eric
Boyles, B. (2017). virtual reality and augmented reality in Education.
Bricken, M. (1991). Virtual reality learning environments: potentials and challenges. ACM SIGGRAPH Computer Graphics,
25(3), 178–184.
Bricken, M., & Byrne, C. M. (1992). Summer Students in virtual reality: A Pilot Study on Educational Applications of virtual
reality Technology.
Bronsch, J. (2018). Einsatz von Virtual-Reality Techniken in Lernumgebungen. Hamburg University of Applied Sciences.
Brown, A., & Green, T. (2009). Issues and trends in instructional technology: Web 2.0, second life, and STEM share the
spotlight. In Educational media and technology yearbook (pp. 7–23). Springer.
Buchner, J. (2017a). Offener Geschichtsunterricht mit augmented reality. Medienimpulse,(1), 1–8.
Buchner, J. (2017b). Selbstbestimmtes Lernen mit augmented reality. Zentrum Für Lernende Schulen, 70.
Cai, S., Chiang, F. K., & Wang, X. (2013). Using the augmented reality 3D Technique for a Convex Imaging Experiment in a
Physics Course. International Journal of Engineering Education, 29(4), 856–865.
https://doi.org/10.1109/icassp.2010.5496212
Cheal, C. (2009). Student perceptions of a course taught in Second Life. Innovate: Journal of Online Education, 5(5).
Chen, C.-M., & Tsai, Y.-N. (2012). Interactive augmented reality game for enhancing library instruction in elementary schools.
Computer & Education, 59(2), 638–652. https://doi.org/10.1109/COMPSACW.2013.128
Cho, K., Lee, J., Lee, J. S., & Yang, H.-S. (2007). A realistic e-learning system based on mixed reality. In 13th International
Conference on Virtual Systems and Multimedia (pp. 57–64).
Cruz-Neira, C., Sandin, D. J., & DeFanti, T. A. (1993). Surround-Screen Projection-Based virtual reality: The Design and
Implementation of the CAVE. Proceedings of ACM SIGGRAPH ’93, 135–142.
Dede, C., Salzman, M. C., & Bowen Loftin, R. (1996). ScienceSpace: virtual realities for learning complex and abstract scientific
concepts. Proceedings of the IEEE 1996 virtual reality Annual International Symposium, 246–252.
https://doi.org/10.1109/VRAIS.1996.490534
Dede, C., Salzman, M. C., & Loftin, R. B. (1996). MaxwellWorld: learning complex scientific concepts via immersion in virtual
reality. Proceedings of the 1996 International Conference on Learning Sciences. https://doi.org/10.1016/S0151-
9638(18)30048-6
Erickson, T. (1993). Artificial realities as data visualization environments: Problems and prospects. In A. Wexelblat (Ed.), virtual
reality: Applications and Explorations (pp. 3–22).
Fotaris, P., Pellas, N., Kazanidis, I., & Smith, P. (2017). A systematic review of augmented reality game-based applications in
primary education. In Proceedings of the 11th European conference on games based learning (ECGBL17). Graz, Austria
(pp. 181–190).
Gurjar, S., & Somani, H. (2016). A Survey on Use of augmented reality in Education.
Haller, M. (2004). mixed reality@ Education. In Multimedia Applications in Education Conference, MApEC (Vol. 2004, p. 13).
Hughes, C. E., & Stapleton, C. (2005). mixed reality in Education, Entertainment, and Training, (November), 24–30.
https://doi.org/10.1109/MCG.2005.139
Jackson, R. L., & Fagan, E. (2000). Collaboration and learning within immersive virtual reality. In Proceedings of the third
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
international conference on Collaborative virtual environments (pp. 83–92).
Jensen, C. G. (2017). Collaboration and Dialogue in virtual reality. Journal of Problem Based Learning in Higher Education,
5(1), 85–110. https://doi.org/10.5278/OJS.JPBLHE.V0I0.1542
Kaufmann, H. (2003). Collaborative augmented reality in Education. Institute of Software Technology and Interactive Systems,
Vienna University of Technology.
Kaufmann, H. (2011). Virtual Environments for Mathematics and Geometry Education. Themes in Science and Technology
Education, 2(1–2), 131--152.
Kaufmann, H., & Schmalstieg, D. (2002). Mathematics and geometry education with collaborative augmented reality. In ACM
SIGGRAPH 2002 conference abstracts and applications (pp. 37–41).
Kaufmann, H., Schmalstieg, D., & Wagner, M. (2000). Construct3D : A virtual reality Application for Mathematics and
Geometry Education. Education and Information Technologies, 5(4), 263–276.
Kavanagh, S., Luxton-Reilly, A., Wuensche, B., & Plimmer, B. (2017). A Systematic Review of virtual reality in Education.
Themes in Science and Technology Education, 10(2), 85–119. https://doi.org/10.1007/s00170-017-1116-1
Kesim, M., & Ozarslan, Y. (2012). augmented reality in Education: Current Technologies and the Potential for Education.
Procedia - Social and Behavioral Sciences, 47(222), 297–302. https://doi.org/10.1016/j.sbspro.2012.06.654
Koll-Schretzenmayr, M., & Casaulta-Meyer, S. (2016). augmented reality. DisP - The Planning Review, 52(3), 2–5.
https://doi.org/10.1080/02513625.2016.1235863
Laws, A. G., Forsyth, H. L., & Baskett, M. (2009). MUVE, the future of e-learning: Building a virtual learning world.
Proceedings - International Conference on Developments in ESystems Engineering, DeSE 2009, 307–313.
https://doi.org/10.1109/DeSE.2009.34
Lee, K. (2012). augmented reality in Education and Training. TechTrends, 56(2), 13–21. https://doi.org/10.1007/s11528-012-
0559-3
Leitão, R., Rodrigues, J. M. F., & Marcos, A. F. (2014). Game-based learning: augmented reality in the teaching of geometric
solids. International Journal of Art, Culture and Design Technologies (IJACDT), 4(1), 63–75.
Liarokapis, F., & Anderson, E. F. (2010). Using augmented reality as a Medium to Assist Teaching in Higher Education. In L.
Kjelldahl & G. Baronoski (Eds.), Eurographics 2010 (pp. 9–16). Eurographics Association. Retrieved from
http://eprints.bournemouth.ac.uk/20907/
Liarokapis, F., Petridis, P., Lister, P. F., & White, M. (2002). Multimedia augmented reality Interface for E-Learning (MARIE).
World Transactions on Engineering and Technology Education, 1(2), 173–176. https://doi.org/DOI 10.1007/s10055-006-
0036-
Manjrekar, S., Sandilya, S., Bhosale, D., Kanchi, S., Pitkar, A., & Gondhalekar, M. (2014). CAVE: An Emerging Immersive
Technology--A Review. In Computer Modelling and Simulation (UKSim), 2014 UKSim-AMSS 16th International
Conference on (pp. 131–136).
Mellet-d’Huart, D. (2012). Virtual reality for training and lifelong learning. Themes in Science and Technology Education, 2(1–
2), 185–224.
Mikropoulos, T. A., & Natsis, A. (2011). Educational virtual environments: A ten-year review of empirical research (1999-2009).
Computers and Education, 56(3), 769–780. https://doi.org/10.1016/j.compedu.2010.10.020
Milgram, P., & Kishino, F. (1994). A Taxonomy of mixed reality Visual Displays. IEICE TRANSACTIONS on Information and
Systems, 77(12), 1321–1329.
Muhanna, M. A. (2015). Virtual reality and the CAVE: Taxonomy, interaction challenges and research directions. Journal of
King Saud University-Computer and Information Sciences, 27(3), 344–361.
Nachimuthu, K., & Vijayakumari, G. (2009). virtual reality Enhanced Instructional Learning. Journal of Educational
Technology, 6(1), 1–5. Retrieved from
http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ1098062&site=ehost-live
Nooriafshar, M., Williams, R., & Maraseni, T. N. (2004). The use of virtual reality in education. In Proceedings of the 7th
American Society of Business and Behavioral Sciences International Conference (ASBBS 2004).
Panciroli, C., Macauda, A., & Russo, V. (2018). Educating about Art by augmented reality: New Didactic Mediation Perspectives
at School and in Museums. Proceedings, 1(10), 1107. https://doi.org/10.3390/proceedings1091107
Pantelidis, V. S. (1997). Virtual reality and engineering education. Computer Applications in Engineering Education, 5(1), 3–12.
Pasaréti, O., Hajdin, H., Matusaka, T., Jambori, A., Molnar, I., & Tucsányi-Szabó, M. (2011). augmented reality in education.
INFODIDACT 2011 Informatika Szakmódszertani Konferencia.
Passig, D. (2010). The future of virtual reality in education : A future oriented meta analysis of the literature. Themes in Science
and Technology Education, 2(1–2), 269–293.
Radu, I. (2014). Augmented reality in education: a meta-review and cross-media analysis. Personal and Ubiquitous Computing,
18(6), 1533–1543.
Redó, M. N., Torres, A. Q., Quirós, R., Redó, I. N., Castelló, J. B. C., & Camahort, E. (2008). Collaborative augmented reality
for Inorganic Chemistry Education. 5th WSEAS / IASME International Conference on ENGINEERING EDUCATION
(EE’08).
Roussos, M., Johnson, A., Moher, T., Leigh, J., Vasilakis, C., & Barnes, C. (1999). Learning and building together in an
Draft – originally published in: Kommetter, C. & Ebner, M. (2019). A Pedagogical Framework for Mixed Reality in
Classrooms based on a Literature Review. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning
(pp. 901-911). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE).
Retrieved July 14, 2019 from https://www.learntechlib.org/primary/p/210261/.
immersive virtual world. Presence: Teleoperators and Virtual Environments, 8(3), 247–263.
https://doi.org/10.1162/105474699566215
Salzman, M. C., Dede, C., Loftin, R. B., & Chen, J. (1999). A Model for Understanding How virtual reality Aids Complex
Conceptual Learning. Presence: Teleoperators and Virtual Environments, Special Issue on Education.
Schmalstieg, D., Fuhrmann, A., Hesina, G., Szalavári, Z., Encarnaçao, L. M., Gervautz, M., & Purgathofer, W. (2002). The
Studierstube augmented reality Project. Teleoperators & Virtual Environments 11, 1, 33–54. Retrieved from
https://github.com/snowpuppy/augreality/wiki
Schön, S., Ebner, M., & Kumar, S. (2014). The Maker Movement. Implications of new digital gadgets, fabrication tools and
spaces for creative learning and teaching. ELearning Papers, 39, 14–25.
Shelton, B., & Hedley, N. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality.
Technology, Instruction, Cognition and Learning, 1(4), 323–357. https://doi.org/10.1021/om990262d
Southgate, A. E. (2018). Immersive virtual reality , children and school education : A literature review for teachers., (6).
Spitzer, M., Nanic, I., & Ebner, M. (2018). Distance Learning and Assistance Using Smart Glasses. Education Sciences, 8(1), 21.
https://doi.org/10.3390/educsci8010021
Stankovi’c, S. (2015). virtual reality and Virtual Environments in 10 Lectures. Synthesis Lectures on Image, Video, and
Multimedia Processing, 8(3), 1–197. https://doi.org/10.2200/s00671ed1v01y201509ivm019
Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and Evaluation of a VR Math-
Game. Mobile Technologies and augmented reality in Open Education.
Sutherland, I. E. (1968). A head-mounted three dimensional display. Fall Joint Computer Conference, 757–764.
Szalavári, Z., Schmalstieg, D., Fuhrmann, A., & Gervautz, M. (1998). “Studierstube”: An environment for collaboration in
augmented reality. virtual reality, 3(1), 37–48. https://doi.org/10.1007/BF01409796
Taylor, K. C., & Chyung, S. Y. (2008). Would you adopt Second Life as a training and development tool? Performance
Improvement, 47(8), 17–25.
Theng, Y.-L., Lim Mei-Ling, C., Liu, W., & Cheok, A. D. (2007). mixed reality Systems for Learning: A Pilot Study
Understanding User Perceptions and Acceptance. In R. Shumaker (Ed.), virtual reality (pp. 728–737). Berlin, Heidelberg:
Springer Berlin Heidelberg.
Thorsteinsson, G., & Page, T. (2007). Computer Supported Collaborative Learning in Technology Education Through virtual
reality Learning Environments. Bulletin of the Institute of Vocational and Technical Education, Graduate School of
Education and Human Development, Nagoya University, Japan,(4), 6–19.
Wagner, C., & Ip, R. K. F. (2009). Action Learning with Second Life - A Pilot Study. Journal of Information Systems Education,
20(2), 249–258. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=lxh&AN=42009003&site=ehost-
live
Winn, W. (1993). A conceptual basis for educational applications of virtual reality. Human Interface Technology Laboratory.
Technical Report TR-93-9, 1–14.
Winn, W., Windschitl, M., Fruland, R., & Lee, Y. (2002). When does immersion in a virtual environment help students construct
understanding. In Proceedings of the International Conference of the Learning Sciences, ICLS (pp. 497–503).
Woods, E., Billinghurst, M., Looser, J., Aldridge, G., Brown, D., Garrie, B., & Nelles, C. (2004). Augmenting the science centre
and museum experience. In Proceedings of the 2nd international conference on Computer graphics and interactive
techniques in Australasia and South East Asia (pp. 230–236).
Youngblut, C. (1998). Educational Uses of virtual reality Technology. IDA Document D-2128, (January), 131.
https://doi.org/10.1287/isre.2013.0480
Zagoranski, S., & Divjak, S. (2003). Use of augmented reality in education. In The IEEE Region 8 EUROCON 2003. Computer
as a Tool. (Vol. 2, pp. 339–342 vol.2). https://doi.org/10.1109/EURCON.2003.1248213
Zhou, F., Duh, H. B., & Billinghurst, M. (2008). Trends in augmented reality tracking, interaction and display: A review of ten
years of ISMAR. Proceedings of the 7th IEEE/ACM International Symposium on Mixed and augmented reality, 193–202.
Zumbach, J., & Moser, S. (2012). augmented reality--Multimediale Lernumgebung der Wahl im 21. Jahrhundert. Zukunft Des
Lernens, 154–164.
