Learning in a Virtual Environment: Implementation and Evaluation of a VR Math-Game

Abstract

With the introduction of Google Cardboard, a combination of mobile devices, Virtual Reality (VR) and making was created. This “marriage” opened a wide range of possible, cheap Virtual Reality applications, which can be created and used by everyone. In this chapter, the potential of combining making, gaming and education is demonstrated by evaluating an implemented math-game prototype in a school by pupils aged 12-13. The aim of the virtual reality game is to solve math exercises with increasing difficulty. The pupils were motivated and excited by immerging into the virtual world of the game to solve exercises and advance in the game. The results of the evaluation were very positive and showed the high motivational potential of combining making and game-based learning and its usage in schools as educational instrument.

Full text

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Learning in a Virtual
Environment:
Implementation and Evaluation of a VR
Math-Game
Christof Sternig
Graz University of Technology, Austria
Michael Spitzer
Graz University of Technology, Austria
Martin Ebner
Graz University of Technology, Austria
ABSTRACT
With the introduction of Google Cardboard, a combination of mobile devices, Virtual Reality (VR)
and making was created. This “marriage” opened a wide range of possible, cheap Virtual Reality
applications, which can be created and used by everyone. In this chapter, the potential of combining
making, gaming and education is demonstrated by evaluating an implemented math-game prototype
in a school by pupils aged 12-13. The aim of the virtual reality game is to solve math exercises with
increasing difficulty. The pupils were motivated and excited by immerging into the virtual world of the
game to solve exercises and advance in the game. The results of the evaluation were very positive and
showed the high motivational potential of combining making and game-based learning and its usage
in schools as educational instrument.
Keywords: Virtual Reality, Game-Based Learning, Making, Educational Game, Google Cardboard,
Gamification, Technology Enhanced Learning, Math
INTRODUCTION
Since the dawn of virtual reality (VR) headsets there have been major improvements in the head-
mounted display technology in recent years. With the evolution of VR headsets and the
announcements of products as Oculus Rift, HTC Vive, Samsung Gear VR etc., more and more
manufacturers will release their headsets to the market in the near future to make Virtual Reality
available to the public. Mobile devices are complex computers which are gradually replacing Personal
Computers (PCs) in all-day life (Gartner Press Release, 2016). With their high availability, the
variety of embedded sensors, cameras and the availability of high computational power, mobile
phones especially lie in the focus of researchers and developers. A new movement emerged in the last
few years, namely, the Maker Movement (Schön, Ebner, & Kumar, 2014). Small computers as the
Raspberry Pi or the Arduino offer new possibilities to invent and experiment with ideas and
technologies. Makers meet at annual maker fairs to showcase inventions and experiments. When

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Google introduced Google Cardboard, a virtual reality headset can be made of a cardboard combined
with a mobile phone. Therefore, making was combined with VR and mobile technology. The
marriage of making, VR and a mobile device offers a great possibility to use concepts of digital game-
based learning to create an immersive learning experience. Using VR techniques for learning,
education or advanced training is an interesting research field with high potential.
The following research questions were examined:
RQ1: How could a VR device be integrated into a mathematical learning scenario?
RQ2: Which lessons could be learned while evaluating the learning scenario in a secondary school
class?
RQ3: How could the learning experience be improved?
For this work, a simple virtual reality game has been implemented to evaluate the need for such
educational games, supporting and motivating children in exercising math and the educational and
motivational effects of combining making, gaming and learning. Finally, this chapter gives some
recommendations how to integrate and augment school lessons with making and game-based learning.
BACKGROUND
In the following three sections, some background information is given about virtual reality (VR),
game-based learning and the maker movement. The sections outline the key definitions used
throughout this chapter.
Virtual Reality
Virtual Reality is a computer generated illusion of the real world. The perfect Virtual Reality
manipulates the human senses to be indistinguishable from the real world. Although, it is a utopian
desire to create such a perfect virtual world (Stanković, 2015, p. 4). Common sense defines VR by
head-mounted displays as Oculus Rift and/or data gloves, but this is not an adequate definition
(Burdea, & Coiffet, 2003, p. 1). VR can also be accomplished by projectors in combination with
Personal Computers (PCs) called the CAVE (Cruz-Neira, Sandin, & DeFanti, 1993). All sorts of
computer games are also capable of creating a VR experience. The differences between VR and 3D
movies are the possibilities to interact with the created world, change the state of the world and get a
feedback (Stanković, 2015, p. 9). By having the possibility of interaction one can immerge into the
virtual world. According to Burdea and Coiffet (2003, p. 3), interaction and immersion are two of
three key features of VR. The third feature is imagination. Virtual Reality is often used to simulate
real world processes. To restrict the parameters of the simulation to fully map a real world process,
without breaking the simulation, is a difficult task which has to be solved by VR-developers and their
imagination. Another important definition with respect to Virtual Reality is the Virtual Environment
(VE). “VEs provide the illusion of presence in a place different from one's current physical
surrounding.” (Stanković, 2015, p. 10). Stanković (2015) differs between four different types of VEs.
● Single user, text only (2D): i.e. text adventures.
● Single user, realistic 3D: AAA games (video games of high quality and produced with high
development budgets are called “triple-A”-games) on modern console generations or the PC.
● Multi user, text only (2D): social networks like Facebook.
● Multi user, realistic 3D: Massively Multiplayer Online (MMO) games or general purpose
VEs.
The rise of VR started again after Palmer Luckey started a successful Kickstarter campaign on August
1, 2012, for his VR headset Oculus Rift. Luckey showed the world that it is possible to create a high-
end VR headset priced under $300. Facebook acquired the company in 2014 for 2 billion dollars and

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
showed the world VR is, again, the next big thing (Kuusisto, 2015). Virtual Reality already was the
“next big thing” back in the 1990s but Virtual Reality devices failed to enter the market (Stanković,
2015). The desire of buying a head-mounted display (HMD) to enter virtual worlds rose when
affordable Virtual Reality headsets and applications where available. Although the available devices
are using the newest technologies to create those virtual worlds, the used concepts behind all modern
HMDs go back to 1968, when Sutherland (1969) introduced the first head-mounted display. It
featured two cathode ray tube (CRT) displays in front of every eye and a tracking device to track
movements of the head.
In 2014 Google introduced Google Cardboard at its Google I/O conference. It combines a mobile
phone and a cardboard construction to create an HMD. With the high availability of mobile phones in
the public nowadays, immerging into a virtual world is possible for everyone simply by owning a
mobile phone. The prototype implemented and described in this work is developed in Java for
Android devices and uses Google Cardboard to create the needed headset. This artefact enables pupils
to immerge into the created virtual world to learn math. Considering the previous definitions, the
game implements a single user, 3D Virtual Environment.
Video Games in Education
The Video game industry has steadily grown over the years. With the evolution of games and game
mechanics, realistic graphics and sounds, more and more people are playing video games. The
Entertainment Software Association (ESA) releases a yearly report, showing essential facts about the
computer and video game industry. The current report points out that the average American gamer is
35 years old (Entertainment Software Association [ESA], 2015), in 2005 the average gamer's age was
30 (ESA, 2005). 51% of American households own a dedicated game console and 42% are playing
video games on a regularly basis (three hours or more a week) (ESA, 2015). With this statistics
increasing on a yearly basis, using video games for educational purposes gets an interesting industrial
factor. Kapp (2012) reviews researches and studies about the effectiveness of games and game
elements used in learning contexts and he concludes, that learners can benefit from such technologies
when implemented and presented in the right way.
The following sections explain some key terms as digital game-based learning and gamification and
their use in learning applications.
(Digital) Game-Based Learning
The term digital game-based learning (DGBL) describes the process of learning, while playing a
computer game (Prensky, 2007). It can be seen as a specialization of the umbrella term game-based
learning (GBL) which describes the process of learning by playing a game. In terms of GBL any
game, i.e. board games or card games, match the definition. This work focuses on DGBL, therefore
the following definitions concentrate on learning by playing video games.
Since the creation of the first video games in the 70s and 80s, the idea of learning by playing
computer games was present. With the evolution of the game market, from a small industry for young
men interested in technology, to a billion dollar industry, more and more studies about learning in
combination with video games emerged. From the early 2000s the two terms 'Serious Games' and
'Digital Game-Based Learning' replaced the term 'Edutainment', short for education and entertainment
(Breuer, 2011).
The term serious game, defined by Abt in 1971, describes all games with an educational purpose
without the intention to be played only for amusement (Abt, 1987). Digital game based learning
should feel like playing a computer game without recognizing the learning effect or discovering the
content to be taught by the game (Prensky, 2007). It is “any marriage of educational content and
computer games” (Prensky, 2007, p. 145). Breuer (2011) points out, that DGBL and serious games as
defined before, have common elements as shown in the following enumeration, but serious games go

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
beyond, when for example being used as distraction for painful therapies. Although studies and
researches differentiate between DGBL and serious games, the aim of both concepts is to use games
for an educating purpose going beyond mere entertainment. Breuer (2011) lists six common features
of DGBL and serious games:
● Interactivity: learning by doing and experimenting.
● Multimedia-based: visualizing/preparing content and feedback by using 3D models, audio
etc.
● Involvement: the game should be fully engaging to keep the player from distractions.
● Challenge: increasing difficulty but beginner friendly, the game should always challenge the
individual skills to keep players motivated.
● Reward: rewards and feedback of progression should push self-efficacy and motivation.
● Social Experience: providing communication channels to connect players.
Considered as sets, DGBL is a proper subset of serious games and game-based learning and further of
entertainment education. Serious games, e-learning as whole and entertainment education have
common properties but differentiate in their base definitions. Figure 1 shows the different sets and
their connections (Breuer, 2011).
Figure 1. The different learning approaches displayed as sets and their connections (Breuer, 2011)

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Gamification
According to Deterding, Dixon Khaled and Nacke (2011) gamification has been a contested term
since its first appearance in 2008. There have been several definitions of gamification. With the lack
of academic attempts to define gamification, Deterding mentions two ideas. Current usages of the
word are fluctuating between: “The first is the increasing adoption, institutionalization and ubiquity of
(video) games in everyday life” (Deterding et al., 2011, pp. 9-10). The second idea is the usage of
game design and game elements in non-game contexts or products, adding enjoyment which in turn is
leading to engagement, motivation and duration. Deterding et al. (2011) define gamification as “the
use of game design elements in non-game contexts” (p. 10) and thus favors the second idea stated
before.
Kapp (2012) defines gamification as the usage of “game-based mechanics, aesthetics and game
thinking to engage people, motivate action, promote learning, and solve problems” (p. 17). This
definition of gamification will be used when the term is used within this work. It fits the implemented
prototype and its methods to convey the learning content efficiently. The next section will discuss
different types of gamification and sample applications to get a better understanding how gamification
can be used and implemented.
Applications
Implementing learning applications using game elements, gamifying existing systems and contents or
designing a digital game-based learning application from scratch is a challenging task. Kapp, Blair
and Mesh (2014) define the term Interactive Learning Event (ILE) to combine games, gamification
and simulation in one term. To create a successful ILE, Kapp et al. (2014) list four important points to
keep in mind:
● Game-based (fun) elements and instructional, non-entertaining elements should co-exist and
grow together to keep the right balance.
● Interactivity ensures player engagement which results in learning more and keeping the
knowledge for a longer time.
● Create a good story. The better the story and the storytelling, the more the player is engaged.
Story elements and learning goals should be linked for a better learning experience.
● Proper testing needs to be done in order to evaluate the ILE and its elements.
For the simple implemented prototype described in this work, three of the four previous mentioned
principles (game-based elements, interactivity and the evaluation) have been considered. Creating a
good story and implementing story elements was not considered due to the first attempt but should be
implemented in later versions of the prototype to keep players motivated and to achieve a better
engagement.
Making and Education
With today's availability of cheap technology and tools as 3D printers, Personal Computers (PCs),
tiny computers as the Raspberry Pi or the Internet of Things (IoT), inventing, creating and making can
be done by everyone. Magazines as 'Make', FabLabs, conferences and fairs made 'Making' popular
and more and more people are joining this movement. The maker movement stands for creating and
developing new things using the aforementioned technologies. This can happen in fabrication labs
(FabLabs), Makerspaces or on the desktop at home (Schön, Ebner, & Kumar, 2014). This section
outlines the history of the maker movement and shows how making and education can be combined to
create a new learning experience, not only for children, but students and adults.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
According to Libow and Stager (2013), Seymour Papert can be seen as the father of today's maker
movement. He wanted to revolutionize education by using modern technologies as computers not
only for problem solving, as they are and have been used, but even more for creating some action. In
the paper “Twenty Things to Do With a Computer” these aforementioned actions as composing
music, programming, etc. are described and shown (Solomon & Papert, 1971). But Papert not only
showed how to use technology for better educational methods, he was a maker himself. He invented
the programming language Logo to teach children the concepts of programming. He also worked on
the first programmable robotic construction kits for LEGO (Libow & Stager, 2013). He coined the
term 'Constructionism' which means that learners gain knowledge by using tools to realize their ideas
(Papert, 1986).
With the invention of computers, the desire of using them to teach pupils was evident. But the first
computers were big machines and programming was done by experts. Adults, apart from experts, did
not know how to program these complex computers, so how could it be taught to children? With the
previously mentioned invention of Logo by Seymour Papert, things changed and schools used Logo to
teach programing and mathematics to pupils. The same applies to the maker movement. As stated
before, FabLabs teach how to use technology to create new tools, technology or products. Policy
recognized the importance of making. Curricula of schools were expanded to teach these new working
skills, but eventually, in 1999 this skill-based approach was abandoned due to the rapid evolution of
technology. The aim is to teach technology and how it works rather than teaching skills how to work
with specific items (Blikstein & Krannich, 2013).
Nevertheless, Halverson and Sheridan (2014) point out that “learning in making is, emphatically, not
interchangeable with schooling” (p. 498). With the invention of FabLabs and adaptations for schools,
the learning focus is placed on “principles of engineering, robotics, and design” (p. 499). Researches
show how to rebuild a classroom to turn it into a makerspace, to enable schoolkids experimenting
with tools and technology. There has not been found a consensus yet, how making should fit into
existing curricula or if it should replace those curricula. How tests and grading should look like as
making could not only combine the learning of different skills as mathematics, physics etc., but also
leadership (Thompson, 2014). Makers fear that institutionalizing making through school programs
“will quash the emergence, creativity, innovation and entrepreneurial spirit that are hallmarks of the
“maker revolution”” (Halverson & Sheridan, 2014, p. 500). According to Halverson and Sheridan
(2014) “the great promise of the maker movement in education is to democratize access to the
discourses of power that accompany becoming a producer of artifacts, especially when those artifacts
use twenty-first-century technologies.” (p. 500). Learning through making has the potential to reach
the institutional and policy goals for learning in science, technology, engineering and mathematics
(Halverson & Sheridan, 2014), so-called STEM education.
Closely related to making is the concept of Project-Based Learning (PBL). It was introduced by John
Dewey in the 1890s. He noticed the importance to integrate real-world problems and situations with
the contents taught in school. Project-Based Learning evolved and developed over the years and there
exist many definitions and different interpretations of the term (Habók & Nagy, 2016). Markham,
Lamer & Ravitz (2003) define PBL as “a systematic teaching method that engages students in
learning knowledge and skills through an extended inquiry process structured around complex,
authentic questions and carefully designed products and tasks” (p. 4). PBL has become more popular
but faces challenges as how to integrate it into school lessons and how teachers and students should
handle this form of schooling. Studies showed that Project-Based learning results in higher
engagement of students or pupils and also teaching how to plan and organize a project and
communicate with others about aspects of the project (Hwang and Kim, 2006). Although, Project-
Based Learning combines many positive aspects of teaching and learning, it is not suitable for

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
learning basic skills. These basic skills or knowledge as math, writing, reading, etc. are required
beforehand to accomplish the PBL project (Markham et al. 2003).
This work uses a kind of an inverse approach to PBL. The described math game was implemented and
evaluated in school. After the evaluation, a tinkering project was realized with the evaluated children
to create their own cardboard headsets to play the prototype on their own. After the evaluation the
pupils were highly motivated and everyone managed to realize the complex tinkering project. Figure 2
shows a final cardboard headset, work in progress and a pupil who is crafting his cardboard headset.
[ADD Fig. 2 HERE]
Figure 2. The image shows three pictures of a pupil crafting his cardboard headset
MAIN FOCUS OF THE CHAPTER
Nowadays mobile devices are gradually replacing PCs or Laptops. The increasing computational
power and their available embedded sensors open up many possibilities for new applications.
Especially smartphones lie in the focus of developers due to their high availability in public. Children
are growing up with these technologies and are therefore used to mobile devices. Children know how
to interact with them and how to use them for surfing the internet, messaging, gaming, etc. (Grimus &
Ebner, 2014). When Google introduced Google Cardboard at its Google I/O conference in 2014 they
managed to combine the Maker Movement with mobile phones, to create a cheap virtual reality
headset. Figure 3 shows the cardboard headset used during the evaluation of the prototype.
As stated in the introduction, the aim of this work was to evaluate the need for educational games
used to teach content to young pupils. Learning basic math operations as multiplication and addition
need a lot of repetition to internalize them. The idea was to create an educational math game to test a
game-based approach for learning and repeating these operations.
This app was developed based on several years of experience in designing mathematical learning
applications (Ebner, 2015; Ebner, Schön, Taraghi & Steyrer, 2014).
To excite the pupils and keep them motivated to play the game, Google Cardboard was used to create
a mobile, virtual reality game. By testing the game, the pupils should also be motivated to create their
own headsets to play the game, and, therefore, learn at home.
There is no other mathematical learning VR game available for google cardboard yet (Jan, 2016)
hence his concept was tested from scratch.
The next section describes the game mechanics, game items and how to play the game.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Figure 3. An image of the cardboard headset used to evaluate the prototype
Game Mechanics
The game mechanics section is divided in three main parts. The navigation mechanics of the game are
described in the first part of this section. The different game items and the level structure are outlined
in the second part. The third part describes the gameplay, how to solve exercises, finish the game and
the possible “Game Over”-states.
Navigation
An easy navigation and good controls are key elements of successful games. Sophisticated control
mechanics can decrease the overall gaming experience of the player. Due to the lack of input devices,
a very simple navigation scheme was implemented within this prototype. When a new level is loaded,
the player's avatar is placed in the center of the level platform. The camera used to render the scene
sits atop the avatar as illustrated in figure 4. The avatar automatically moves along the viewing
direction with constant speed. By moving around the own axis, the player changes the viewing
direction and thus, the moving direction of the avatar in the virtual world.
Due to the moving avatar, suggesting the player's brain a motion although the player himself is not
moving, causing a discrepancy. This discrepancy, called visually induced motion sickness, “typically
occurs in the absence of real physical motion, thus, motion sickness in simulators, cinemas, video-
games, or virtual reality” (Keshavarz & Hecht, 2014, p. 521) causes symptoms like nausea and
dizziness. There have been researches about motion sickness prevention i.e. using pleasant music as
countermeasure against visually induced motion sickness (Keshavarz & Hecht, 2014). The induced
symptoms could lead to difficulties in keeping the balance while standing and playing the game. To
keep the prototype's implementation simple, no visually induced motion thickness prevention methods
have been implemented. The simplest prevention of losing the balance is to sit down on a revolving
chair and do the navigation by rotating around the chair’s axis.
The evaluation of the game showed that neither motion sickness nor symptoms like nausea or
dizziness were a problem for the children testing the game. The previous simple prevention method of
using a revolving chair was not needed by any of the pupils.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
[ADD Fig. 4 HERE]
Figure 4. A simple cone model representing the player's avatar. The camera, visualized as the frustum
of a pyramid, is placed on top of this cone
Game Items and Level Structure
Every level of the game consists of three key elements; the avatar, the floor and collectible items. All
game objects are represented by simple geometric forms to keep the rendering simple and fast and
therefore, prevent frame rate issues.
The avatar of the player, a simple cone illustrated in figure 4, represents the player in the virtual
world. The player can navigate the avatar by changing the view direction as described before, to
collect the collectible items and solve exercises. The avatar is placed onto a platform, the floor, when
the game level is loaded. The floor is another key element and is represented by a scaled cube. It
bounds the area and therefore the virtual world, in which the player can navigate his avatar.
Collectible items are divided into two types, the digits and pickups, respectively.
Digits are represented by their corresponding 3D objects and are collected to solve the current
exercise. The pickups are green, yellow and red colored cubes with a symbol on each side. The color
visualizes the possible effect of the pickup, green stands for a positive, yellow for a neutral and red for
an undesired effect. The symbols are used to distinguish between pickups of the same color resulting
in different effects. Pickups and digits are placed onto the floor when the level is loaded.
The following enumeration lists all pickups available in the game with images and descriptions of
their effects; figure 5 shows the game representation of the pickups.
● Time Bonus: adds thirty seconds to the remaining game time.
● Speed Up: doubles the speed of the avatar for ten seconds. If a second Speed-Up-pickup is
collected, while the first speed-up is still active, the speed is doubled again.
● Jump: the avatar jumps up into the air. This can be used to get a better view onto the
platform or to jump over pickups and digits which the player does not want to collect.
Navigation is still possible while airborne.
● Invert Controls: the controls are inverted, changing the viewing direction to the right results
in the avatar moving left and vice versa. Additionally, the moving direction of the avatar
changes from forward to backward. The effect lasts five seconds. This is rather short but as
testing showed, inversion of the controls quickly leads to dizziness or motion sickness.
With the description of the navigation mechanics, the key game items and the structure of the levels
finished, the gameplay is described in detail in the next section.
[ADD FIG 5 HERE]
Figure 5. The four different pickups collectible in the game. In the top left corner the Time-Bonus-
pickup, top right the Invert-Controls-pickup, bottom left the Speed-Up-pickup and bottom right the
Jump-pickup
Gameplay
This section explains in detail how to solve exercises and collect items to accomplish a level. When
the level is loaded the avatar is placed in the center of the platform and the collectible items are
distributed randomly over the gaming platform. More precisely, ten digits, nine possibly wrong results
and the correct results, and ten pickups are created. The generation of the pickups is completely

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
random; there are no restrictions or rules the pickup creation follows. It is possible that seven time
bonuses and three speed-up pickups are created, but no control-inversion or jump-pickup exists.
The generation of the exercises depends on the chosen settings from the main menu. Possible settings
are only additions, only multiplications or a mix of these two operations. The generation of the
exercises follows the rules of the regular expression shown in listing 1. Before the level starts, five
exercises are generated. The player has to solve them sequentially to advance to the next level.
[1-9]? [0-9] ( (+[1-9] ? [0-9]) | (*[1-9]) )
Listing 1. The regular expression used to generate the exercises for each level
When the level is loaded successfully and all items have been generated, the avatar starts moving and
the clearing time for the level starts decreasing. The clearing times vary from level to level and are
chosen tighter for higher levels. The remaining time can be increased by solving exercises or
collecting time-pickups. Collectible items can be “picked” up by bumping the avatar into the desired
item. When the avatar collides with the object, a sound effect is played to indicate the player that the
item was successfully collected. Three different scenarios are possible when collecting items: a
pickup is collected, the player picked the wrong digit or the player solved the current exercise by
choosing the correct digit. The following enumeration explains every scenario in detail:
1. A pickup is collected: the simplest of all scenarios, the pickup is applied. The cube
representing the pickup is destroyed. The possible effects of pickups are explained in the
previous section.
2. The wrong result is collected: a sound effect and a message displayed on the screen
signaling an incorrect calculation. The player's remaining time is decreased by five seconds,
the overall score decreases by ten points. The collected digit is destroyed and a new one is
generated randomly, the current exercise does not change.
3. The correct result is collected: as with the wrong result, a sound effect and a message
indicate that the player solved the current calculation. The remaining time is increased by five
seconds and additionally, 50 points are added to the player's score. The next calculation is
displayed on the screen and a digit representing the correct result is generated and placed
randomly on the game platform.
While looping through the above scenarios, the player may solve five exercises within the time limit
of the level. If so, the level is completed, the remaining time is added to the player's high score and the
next level, with increased difficulty resulting in a tighter time limit and more challenging calculations,
is loaded. The game ends if one of the following scenarios occurs.
1. Time limit is exceeded: the remaining time reaches zero and the player was not able to solve
five calculations.
2. Falling off the platform: if the avatar is navigated beyond the borders of the game floor, the
avatar falls into the void and the game ends immediately, even though time is still remaining.
3. The player completes all challenges: if the player solves 25 exercises within the time limits
of the five levels, the player has completed the game and wins.
Regardless of which ending scenario occurred, the player’s current score can be saved and viewed in
the high score list. Even though a non-winning scenario is reached, the points, won by previous
mastered levels or correct calculations, can be saved. Table 1 shows the time limits and number
ranges for the generated calculations. Figure 6 shows a screenshot of the game with descriptions of

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
the User Interface (UI) and the visible game items. Figure 7 shows the virtual world from different
angles from the user’s point of view.
Figure 6. The game screen from the player's point of view. The green circles indicate the digits; the
yellow circles are showing pickup items. The bar in the center of the screen represents the remaining
time; the current calculation is displayed in the upper half. The green disc at the bottom displays a
small fraction of the avatar
[ADD Fig. 7 HERE]
Figure 7. The game screen from the player's point of view from different angles
Table 1. The level properties for the five levels of the prototype
Number
Name
Time Limit
Addition Range
Multiplication Tables
1
Plaza
60
0 – 9
3
2
City
55
0 – 19
5
3
Jungle
50
0 – 30
6
4
Desert
45
0 – 50
8
5
Mountain
40
0 – 99
7
The implemented math game covers five of the six principles of DGBL defined by Breuer (2011) in
the background section of this chapter. The player can interact with the virtual game world

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
(Interactivity and Multimedia). The virtual reality effect creates high engagement and the feeling of
presence in the world keeps the player focused on the current exercise (Involvement). The difficulty is
increasing with every level by generating harder exercises and a tighter time limit (Challenge).
Different themed levels and a simple scoring system should keep players motivated to play on
(Reward). According to Kapp et al. (2014) the implemented prototype can be classified as a testing
game, where previous learned mathematical skills are tested via solving exercises. A learning process
is created by repeatedly playing the game and getting better and faster in solving the exercises.
This part of the chapter described the key game elements and the gameplay in detail. The next section
examines the evaluation of the prototype by pupils in a secondary school.
Prototype Evaluation
This section describes the evaluation process and its results in detail. It is divided into two main parts.
The first part explains the evaluation process and setup, the second part shows the results of the
evaluation and discusses issues and observations made during the evaluation.
Evaluation Setup
The evaluation took place on the 18th of December 2015 in the “Neuen Mittelschule (NMS) Fröbel”
(secondary school) and lasted about two hours. The class, taught by Mrs. Silvana Aureli, counts 15
pupils, one child was missing on the evaluation day. The proportion of boys and girls was almost
equal; the children were aged 12 to 13. After a short introduction to the game, the evaluation and their
tasks, the pupils were split into groups of three and one group of two. The evaluation took place in a
separate room with only the current testing group present. Every group ran through a four-step-
process to test and evaluate the game.
1. Introduction: the evaluating group of pupils got a brief introduction about the game and its
mechanics. Neither of the children in the group had played the game before nor had
experience with VR-headsets or VR games. Therefore, the children got instructions on how to
use the cardboard VR headset used for the evaluation, how to navigate through the virtual
world and what the aim of the game is.
2. First play-through: after introducing the game and the VR headset, the first child of the
group started playing the game for the first time. Every child had the chance to play until it
failed or completed the game. Each child of the testing group played the game once and after
that first play-through, possible questions were answered, some advice was given to avoid
common mistakes made in the first iteration of the test.
3. Second play-through: the second play-through did not differ from the first; the children
played the game again, considering the tips given after the first play-through.
4. Evaluation: the evaluation of the game consisted of five statements about the game. The
current evaluation group had to rate the statements by marking them with one of four
differently tempered smileys shown in figure 8. One statement had to be rated with one
smiley by the whole group. This method was used to gain some extra information about the
feelings of the children during their discussion on how to rate the current statement. This
approach has already been used in previous studies (Spitzer & Ebner, 2015).
The evaluation process lasted, as stated before, approximately two hours. Each of the five evaluation
groups ran through the above described evaluation process and rated the five evaluation statements.
Two months after the evaluation took place, the pupils created their own VR-headsets in six school
lessons. The construction manual needed to tinker the headsets and the bi-convex lenses to create the
stereoscopic vision of the 3D world created by the prototype, were offered by the Technical

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
University of Graz. According to the teachers supervising the tinkering lessons, all pupils enjoyed
creating their own personalized headsets.
The results of the evaluation and the ratings of the statements are presented in the upcoming section.
[ADD FIG 8 HERE]
Figure 8. The image shows the five statements used for evaluating the prototype and the smileys used
to rate each statement
Evaluation Results
As described in point 4 of the evaluation process, each evaluation group rated five statements by
marking them with smileys. The statements were read out loud with additional explanations about
each statement’s intentions, to be sure every child understands the meaning of every statement, being
able to rate them properly. To get a measureable result, every type of smiley is assigned a score from
-2 to +2. The scores are added up group by group with the sum being the final score of a statement.
The meanings and scores of the different smileys can be found in figure 9. The following sections
describe each statement, the group-specific answers and a brief interpretation of each rating. In
parentheses below each statement, the original German statement, as used during the evaluation, is
shown. Observations made during the play-through-steps of the evaluation process and the
discussions, on how to rate the statements, respectively, are described and interpreted afterwards.
Figure 9. The meaning of the smileys used to rate the statements and the points of each smiley used to
interpret the results
Statement 1
“The game was fun.”
(Das Spiel hat mir Spaß gemacht.)
The second column in table 2 shows the final results for statement one. The groups all agreed to the
first statement and rated it with the highest possible score. The children liked to immerge into the
virtual world and navigate through it. It was an overwhelming experience for the children to be part of

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
a virtual world. The fact that no child of the evaluating groups had experience with Virtual Reality
certainly had a big influence on the rating.
Statement 2
“Calculating was fun using the game.”
(Mit dem Spiel hat mir das Rechnen Spaß gemacht.)
Statement two had the same rating as statement one, shown in column three of table 2. The intention
of this statement was to check, if the children like solving math exercises while playing a computer
game. On the one hand the high rating could mean that all evaluating children liked the game-based
approach of learning math, on the other hand, while discussing in the group, some pupils pointed out
that the calculations were too easy. The reason for this could be that the prototype’s difficulty was
designed for pupils attending primary school. Maybe the rating of statement two could have been
worse by generating calculations with a higher difficulty. A few children also stated that they did not
try to solve the exercises but they liked moving through the virtual world. Again the effect of diving
into a virtual world via the VR headset could have influenced the rating of this statement.
Statement 3
“It was easy to navigate through the virtual world.”
(Es fiel mir leicht, mich in der Spielwelt zu bewegen.)
A good navigation mechanic is an important requirement to create a good user experience. The
intention of statement three was to evaluate the implemented control mechanics of the game. With a
final score of seven points, all groups rated the game's navigation positive, although three groups did
not rate the statement with the full score. Playing a computer game always implies the use of an input
device as a game controller, a keyboard or fingers touching the screen of a mobile device to control
and navigate the game's avatar. Since the evaluating pupils were not used to Virtual Reality games
and VR-Headsets, respectively, navigating by moving the whole body took some time until the
children got used to it. All evaluating children managed to navigate through the world and play the
game properly on their second play-through therefore, the navigation mechanics can be considered a
success. Column “Points S3” in table 2 shows the ratings of statement three.
Statement 4
It would be fun, playing the game together with my friends.
(Es würde mir Spaß machen, wenn ich das Spiel mit meinen Freunden/Freundinnen spielen könnte.)
The aim of statement four was to evaluate the need for a multi-player mode. The final score of seven
points (shown in table 2, column four) indicates that multiple players playing the game together in the
same virtual world would be a good extension to the current prototype of the game. Surprisingly, one
group did not like the idea of playing against other players and rated the statement with -1 points, but
a multi-player mode being a good idea was the general opinion. By integrating a multi-player mode
the prototype would cover the sixth DGBL principle defined by Breuer (2016), because it would add a
social component to the game.
Statement 5
I would play the game at home.
(Ich würde das Spiel gerne zu Hause spielen.)

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
The intention of this statement was to find out, if the evaluating group members would download the
game and play it at home with their own VR-headset. All groups top-rated this statement which
indicates the success of the combination of game-based learning in a virtual world combined with
making. The virtual reality experience motivated the children to tinker their own VR-Headset, which
was done in six lessons. For the sake of completeness, the last column of table 2 shows the final
results of statement five.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Table 2. The ratings for every statement by group and the final results for every statement. The letter
S in the column names stands for statement
Group No.
Points S1
Points S2
Points S3
Points S4
Points S5
1
2
2
1
-1
2
2
2
2
2
2
2
3
2
2
1
2
2
4
2
2
2
2
2
5
2
2
1
2
2
Result
10
10
7
7
10
Following the results of the evaluation, the next part of the evaluation section describes some
observations made during the evaluation process and gives some explanations. The list follows no
specific order.
● The testing pupils did not see the current calculation to solve: this observation was mostly
made during the first play-through. When the evaluating person was told to look up a little to
see the calculation, the person lifted the head which in turn moves the game's camera upward,
causing the calculation to move upward as well. The eyes have to be moved upwards to see
the calculation without changing the camera angle in the game. This could take some time to
get used to it.
● Bending the whole body or tilting the head to the left or right was used to navigate the
avatar: due to the new way of navigation without input devices and only a brief introduction
to the new navigation mechanics, some pupils needed some extra time to get used to
navigation of the avatar and tried different motions to control the avatar. After short
instructions during playing the game, the pupils internalized the navigation process.
● The first play-through during the evaluation process was worse than the second play-
through: the first play-through sometimes lasted only a few seconds because the avatar fell
off the game floor due to bad navigation. If this was the case, the child was instructed again
and could start the play-through one more time, which resulted in a far better first play-
through. The cause of this problem is the start of the game. The avatar immediately moves
forward after the game started. Until the evaluating person has adjusted the VR-Headset the
avatar may have moved close to one edge of the platform.
● Motion sickness did not occur: this was a very positive observation because previous, adult
test persons could not play the game for a long time due to motion sickness.
● During the evaluation-step there was less discussion between the group members: the
intention of forming groups to evaluate the game was, to observe the discussion during the
rating of the statements. Unfortunately there was less discussion between the group members,
almost every group had some kind of leader, who took the rating-smileys and distributed
them, with the other group members agreeing in most of the time.
The results of the evaluation have been very positive and they suggest that there is a high potential in
game-based learning combined with making when used for educational purpose. The motivation to

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
practice and play with the game could be increased if pupils create their own practice devices
(cardboards VR-headsets), because they are proud what they made.
The next section describes issues which should be tackled, when extending the prototype and outlines
potential improvements to the current prototype.
SOLUTIONS AND RECOMMENDATIONS
During the testing and evaluation of the prototype some issues were identified. This section is split
into two subsections with the first discussing the issues of the prototype’s gameplay and the second
listing problems which occurred during the evaluation process.
Gameplay
The game design and the navigation mechanics were held simple to create a beginner friendly, Virtual
Reality experience. Navigating by changing the viewing direction is an easy way to control the avatar,
but was hard to explain, while introducing the game's mechanics to the pupils. After a first hands-on
and a few minutes with the game, all pupils got used to it. The game objects could clearly be
recognized and the children understood how to solve calculations and leverage the pickups. Changing
the level after three correct solved exercises also raised the fun factor of the game and motivated the
pupils in the group to solve more exercises than the previous group member to advance to the next
level.
Some pupils pointed out that calculations were too easy to solve. A better Random Number Generator
(RNG) has to be implemented to solve this problem to get a better distribution of the exercises.
Another problem is a drop of the frame rate, which occurs when too much objects have to be
rendered. This could also be prevented by a more advanced RNG, distributing the created objects
equally over the gaming platform. Nevertheless, a better distribution will not completely fix this
problem. If the avatar is positioned near a boundary of the game floor and the player changes the
viewing direction to get back to the middle of the platform, the player gains a big field of view,
containing many game objects, which, in turn, have to be rendered and a frame rate drop could occur.
The engine has to be extended by more advanced rendering patterns to get a steady frame rate in all
possible situations. Positively, the frame rate problem either did not occur during the evaluation nor it
did not disturb the game experience for the pupils although it occurred. If the prototype is extended or
revised, these problems have to be addressed.
The current calculation to solve, displayed in the upper area of the screen, was not seen by all pupils
at once. A better way to display text on the screen should be found to make the players see the
calculations immediately without help.
The combination of the simple game mechanics and the new experience of emerging into a virtual
world via a VR-Headset was an exciting experience for the pupils. There were no complaints about
the simplistic graphics or missing realism, which is rather standard in AAA-games on the new console
generations or the PC nowadays.
As suggested by Kapp et al. (2014), the prototype should be extended to offer a story and proper
story-telling to provide better engagement and long-term motivation. Adding new challenges as boss-
fights, a multi-player mode, enabling comparison with other human players, may increase engagement
as well as adding connection to social media. Showing the learning progress to friends via social
media could lead to engaging friends to replay the game and progress further in the game.
Evaluation
The four step evaluation process was an adequate method to gain user feedback of the prototype. The
evaluating group size of three pupils per group and one group of two was a good decision. The groups

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
were big enough to have the attention of every group member while introducing the game, the headset
and to deal with the limited time for the whole evaluation process.
A problem of multiple groups was the introduction of the game for every group. Future groups may
have gotten a more detailed introduction due to mistakes made by previous groups. Also two play-
throughs have not been planned, but after the first play-through of group one, a second play-through
after a second briefing, seemed to be necessary and it showed that every group needed this extra
briefing and a second play-through.
Evaluating the game via rating statements using smileys was very intuitive for the children. The
intention of three group members having only one vote for a statement was to observe the pupils,
while discussing how to rate the statement and gain some additional information about the thoughts
and feelings of the children. Unfortunately, as stated in the evaluation chapter, every group had some
kind of leader, distributing the smileys at first. The other group members agreed in most cases without
having a discussion about the rating. Rating the statements by pupil and a little interview on the one
hand could have resulted in some additional or more precise information but on the other hand would
have exceeded the time limit for this evaluation.
Despite these shortcomings, the evaluation process itself was good enough to gain some vital
information and results to build atop for improvements and extensions of the prototype. It clearly
showed that the pupils are willing to use and play educational games and enjoyed the game-based
learning approach. Table 3 summarizes the known issues and possible solutions as listed in the above
text.
Table 3. A list of the issues and possible solutions identified during the evaluation and testing of the
prototype
No.
Issue
Solution
1
Too easy exercises.
Add a more advanced random number generator.
2
Possible drops of the frame rate resulting in
jerking and thus could result in motion sickness.
Implement a more advanced rendering algorithm
including better Level Of Detail (LOD)
mechanics.
3
Current exercise is difficult to see.
Implement a better User Interface (UI) and find
better ways to position text elements on screen.
4
No story was added to the game.
Add a compelling story for a higher engagement
level.
5
Common game elements are missing.
Add well-known game elements as a multi-
player mode, enemies, boss fights, etc.
6
Introduction of the game to every evaluating
group separately, was biased.
Create a script to follow when introducing the
game.
7
No discussion during the rating of the
statements.
Rating the statements by pupil including a short
interview.
FUTURE RESEARCH DIRECTIONS
The evaluation of the prototype in school showed that there is a high potential in educational games.
Children are used to play games and the combination of playing and learning is a great way to break
up old structures of today’s teaching. A more advanced prototype could be used for testing pupils or

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
to submit results of homework. All pupils get the same exercises and play the game at home. Their
results are submitted as homework. Another step would be the integration into lessons to make the
game a tool for the teachers to teach new content to pupils. For that purpose, a good feedback system
to better evaluate and visualize the learning progress of every pupil has to be implemented. Also,
further testing of the prototype has to be done to fix unknown flaws in the gameplay.
Using the prototype during school lessons and integrating such an educational medium into learning
scenarios in school demands some initial training for teachers to use this new schooling method
properly. It should be evaluated how such training for teaching staff should look like and if it
demands explicit training lessons for teachers or if precise guidelines on how to integrate and use such
media in school will be sufficient.
The aforementioned integration of story elements should be evaluated as well. How could adding a
story be done and how will the story elements fit with the math content tested by the current
prototype? How can challenges of the game mask mathematical calculations and integrate them into
the gameplay to blend learning and playing without being recognized by the player?
CONCLUSION
The aim of this work was to evaluate the need for educational VR computer games in combination
with making. Other researchers already investigated educational VR learning modules. They
confirmed the potential of VR based applications to help students learn mathematical concepts.
Wang, Yang & Lian (2009) already used VR learning modules for engineering students. They
transferred real-world engineering challenges to help students to better realize and recognize
mathematical concepts. They visualized a projectile in 3D space to show the parabola flight curve
following the quadratic equation of a falling object. Students could freeze the flight of the projectile to
show the current parameters of the equation. The feedback of the students was very good. They
believe that the VR modules are helping them to better understand mathematical concepts.
Kaufmann, Schmalstieg & Wagner (2000) build a VR application for mathematics and geometry
education. They used an HMD which was used to display the 3D models. Students were able to
manipulate the 3D geometry of the shown objects. The students liked the experience even though the
performance and accuracy caused some difficulties.
These two examples showed the potential of VR learning applications which, among other things,
inspired us to build the VR math game.
The game mechanics have been described in the previous sections of this work, the making-part
consisted of tinkering a VR-Headset made of cardboard. The game was evaluated by 14 pupils with
one headset and a testing device. After the evaluation, the children created their own VR-Headset with
the help of their teachers to play virtual reality games. In six lessons the pupils created their own
personalized virtual reality headsets. This follows the maker approach. The teachers mentioned the
positive attitude of the pupils during the tinkering lessons and were amazed by the high motivated
children, although it was not an easy task to tinker the cardboard headsets. The lessons were a big
success and the pupils were happy with their headsets. Figure 10 shows examples of the headsets
created by the pupils (RQ1).
Although the game was kept very simple in its presentation due to performance bottlenecks, resulting
from mobile hardware and the simple rendering system, the pupils were excited to play the game.
Emerging into the virtual world via the cardboard headset was an overwhelming experience, because
none of the evaluating children have ever used a VR-Headset. The results of the evaluation have been
positive throughout the evaluation process and none of the pupils felt motion sickness nor criticized
the game or its mechanics. Being able to navigate within the virtual world needed some time,

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
nevertheless, the children got used to it. The learning curve was steep and every child was able to
solve exercises and therefore, was motivated to play on (RQ2).
The results of the evaluation clearly show that educational games in combination with making can be
a great success. The prototype provided an extended learning experience for the pupils. Using their
mathematical knowledge to advance in the game motivated the pupils to keep on playing and try
again after failing to finish a level or the game. By using and playing this game, the process of
repetition to tighten the mathematical skills of the pupils is enriched with a fun and motivational
component, leading to long-term engagement.
To improve the learning experience the mathematical content must be highly adapted to the level of
education of the users. Additionally, a compelling story should be added and the user interface should
be improved to keep the pupils motivated and to reduce teachers’ introductions (RQ3).
[ADD FIG 10 HERE]
Figure 10. Images of some headsets tinkered by the pupils, which evaluated the prototype
The impact of a better looking, realistic 3D game using educational elements and integrating one or
more new elements, mentioned in the recommendations in the last section should have an even greater
impact and show great potential to be used in schools and for education of pupils. Maybe the positive
and educational effects of learning by doing and game-based learning will revolutionize learning as
we know it today.
REFERENCES
Abt, C. C. (1987). Serious games. Lanham, MD: University Press of America.
Blikstein, P., & Krannich, D. (2013). The makers’ movement and FabLabs in education. In J. P.
Hourcade, N. Sawhney, & E. Reardon (Eds.), Proceedings of the 12th International Conference on
Interaction Design and Children (p. 613). doi:10.1145/2485760.2485884
Breuer, J. (2011). Spielend lernen? Eine Bestandsaufnahme zum (Digital) Game-Based Learning.
Landesanstalt für Medien NRW. Retrieved May 4, 2016, from http://lfmpublikationen.lfm-
nrw.de/index.php?view=product_detail&product_id=190
Burdea, G., & Coiffet, P. (2003). Virtual reality technology. Academic Search Complete. Wiley.
Retrieved April 16, 2016, from https://books.google.ca/books?id=0xWgPZbcz4AC
Cruz-Neira, C., Sandin, D. J., & DeFanti, T. A. (1993). Surround-screen projection-based virtual
reality: the design and implementation of the CAVE. In M. C. Whitton (Ed.), SIGGRAPH ’93
Proceedings of the 20th annual conference on Computer graphics and interactive techniques (pp.
135–142). New York: ACM. doi:10.1145/166117.166134
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to
gamefulness. In A. Lungmayr (Ed.), Proceedings of the 15th International Academic MindTrek
Conference Envisioning Future Media Environments (pp. 9–15). New York, NY: ACM.
doi:10.1145/2181037.2181040
Ebner, M. (2015) Mobile Learning and Mathematics. Foundations, Design, and Case Studies.
Crompton, H., Traxler, J. (ed.). Routledge. New York and London. pp. 20-32

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Ebner, M., Schön, M., Taraghi, B., & Steyrer, M. (2014) TEACHERS LITTLE HELPER: MULTI-
MATH-COACH, IADIS International Journal on WWW/Internet, 11, 3, pp.1-12
Entertainment Software Association. (2005). Essential Facts about the Computer and Video Game
Industry. Retrieved May 4, 2016, from http://www.tntg.org/documents/gamefacts.pdf
Entertainment Software Association. (2015). Essential Facts about the Computer and Video Game
Industry. Retrieved May 4, 2016, from http://www.theesa.com/about-esa/esa-annual-report/
Gartner Press Release (2016). Worldwide Device Shipments to Grow 19 Percent in 2016, While End-
User Spending to Decline for the First Time. (2016, January 20). Retrieved 30 May, 2016, from
http://www.gartner.com/newsroom/id/3187134
Grimus, M. & Ebner, M. (2014). Learning with Mobile Devices Perceptions of Students and Teachers
at Lower Secondary Schools in Austria. In Proceedings of World Conference on Educational
Multimedia, Hypermedia and Telecommunications 2014 (pp. 1600-1609). Chesapeake, VA: AACE
Habok, A., & Nagy, J. (2016). In-service teachers' perceptions of project-based learning.
SpringerPlus, 5, 83. doi:10.1186/s40064-016-1725-4
Halverson, E. R., & Sheridan, K. (2014). The Maker Movement in Education. Harvard Educational
Review, 84(4), 495–504. doi:10.17763/haer.84.4.34j1g68140382063
Hwang, S. Y., & Kim, M. J. (2006). A comparison of problem-based learning and lecture-
based learning in an adult health nursing course. Nurse education today, 26(4), 315-321.
Kapp, K. M. (2012). The gamification of learning and instruction: Game-based methods and
strategies for training and education. Pfeiffer essential resources for training and HR professionals.
San Francisco, CA: Pfeiffer.
Kapp, K. M., Blair, L., & Mesch, R. (2014). The gamification of learning and instruction fieldbook:
Ideas into practice. San Francisco, CA: Wiley.
Kaufmann, H., Schmalstieg, D., & Wagner, M. (2000). Construct3D: a virtual reality application for
mathematics and geometry education. Education and information technologies, 5(4), 263-276.
Keshavarz, B., & Hecht, H. (2014). Pleasant music as a countermeasure against visually induced
motion sickness. Applied ergonomics, 45(3), 521–527. doi:10.1016/j.apergo.2013.07.009
Kuusisto, F. (2015). Vr head-mounted displays. XRDS: Crossroads, The ACM Magazine for Students,
22(1), 65. doi:10.1145/2837754
Libow Martinez, S., & Stager, G. (2013). Invent to learn: Making, tinkering, and engineering in the
classroom. Torrance, Calif.: Constructing Modern Knowledge Press.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Markham, T., Larmer, J., & Ravitz, J. L. (2003). Project based learning handbook: a guide to
standards-focused project based learning for middle and high school teachers (2nd ed., p. 1). Novato,
Calif.: Buck Institute for Education.
Papert, S. (1986). Constructionism: A New Opportunity for Elementary Science Education.
Prensky, M. (2007). Digital game-based learning (Paaragon House ed.). St. Paul,
Minn.: Paragon House.
Schön, S., Ebner, M., & Kumar, S. (2014). The Maker Movement. Implications of new digital
gadgets, fabrication tools and spaces for creative learning and teaching. Retrieved May 6, 2016, from
http://www.openeducationeuropa.eu/de/article/The-Maker-Movement.-Implications-of-new-digital-
gadgets,-fabrication-tools-and-spaces-for-creative-learning-and-teaching
Solomon, C., & Papert, S. (1971). Twenty Things to Do With a Computer. Retrieved May 6, 2016,
from http://www.stager.org/articles/twentythings.pdf
Spitzer, M., & Ebner, M. (2015). Collaborative Learning Through Drawing on iPads. In Proceedings
of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2015. pp.
633-642 Chesapeake, VA: AACE.
Stanković, S. (2015). Virtual Reality and Virtual Environments in 10 Lectures. In A. C. Bovik (Ed.),
Synthesis Lectures on Image, Video, and Multimedia Processing (Vol. 8, pp. 1–197). Morgan &
Claypool Publishers. doi:10.2200/S00671ED1V01Y201509IVM019
Sutherland, I. E. (1968). A head-mounted three dimensional display. In AFIPS (Ed.), Proceedings of
the December 9-11, 1968, fall joint computer conference, part I
(pp. 757–764). doi:10.1145/1476589.1476686
Thompson, G. (2014). The Maker Movement Conquers the Classroom. THE Journal (Technological
Horizons In Education). Retrieved May 6, 2016, from
https://thejournal.com/Articles/2014/04/30/The-MakerMovement-Conquers-the-
Classroom.aspx?Page=2
Wang, Y., Cui, S., Yang, Y., & Lian, J. A. (2009). Virtual reality mathematic learning module for
engineering students. Technology Interface Journal, 10(1), 1-10.
KEY TERMS AND DEFINITIONS
Avatar: A 3D model of a human, animal, thing, etc. representing the person in a virtual world
enabling the person to interact with elements of the virtual world.
Digital Game-Based Learning: The approach of combining learning and teaching with playing
computer games.
Gamification: Learning material is augmented with game-like elements to keep learners motivated
and engaged.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
Head-Mounted Display: A computer display worn like goggles to immerge into virtual worlds.
Immersion: Completely dive into a virtual world. A high immersion leads to a user’s perception of
being part of this virtual world.
Maker Movement: A trending movement consisting of people implementing do-it-yourself (DIY)
projects using available technology to create, share and use their invention and ideas.
Project-Based Learning: An approach to use learned contents in a project to find a proper solution to
an existing problem.
Virtual Reality: A computer generated virtual world the user can interact with.

Draft – originally published in: Sternig, C., Spitzer, M., & Ebner, M. (2017). Learning in a Virtual Environment: Implementation and
Evaluation of a VR Math-Game. In G. Kurubacak, & H. Altinpulluk (Eds.), Mobile Technologies and Augmented Reality in Open
Education (pp. 175-199). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-2110-5.ch009
NOT FOR PUBLISHING
REVIEW RESPONSE
We received three reviews. This is our response to these reviews and the information what we
changed and added:
Review 1
• We described a learning scenario how to implement a VR game to train basic arithmetical
operations hence the game design and the mathematical concepts were quite straight forward.
Based on our experience (additional references added in the chapter) we developed the
concept and implementation of the app.
• We reformatted the headers/sequences to fit into the guidelines of IGI.
• We added more screenshots of the app.
Review 2
• We added links to the maker part, why the maker part is positive for the success of this
project
• We decided not to include the links to the neurological background of gamification, because
we build a prototype to test the VR technology if it is technically possible to implement a
simple VR learning app with google cardboard. The elements of the learning game are quite
simple and straight forward without any deeper neurological background.
• Additionally, the general trend is difficult to analyze because on this topic (google cardboard
math learning apps) there were no other math learning apps available at the time when we
wrote this chapter. Our main goal was to do first prototype tests with the implemented
learning game and to investigate the user response of this application.
• We added backlinks from the results section to the background section.
• We simplified sentences and corrected sentence fragments.
• We shortened sentences and used a simpler syntax
Review 3
• We did an intense proof reading of the chapter
• We removed or corrected the sentence fragments.
• We added references to research of other VR learning apps.