Conference PaperPDF Available

Pathomon: A Social Augmented Reality Serious Game

A Social Augmented Reality Serious Game
Daniel Rapp, Jonas M¨
uller, Kristina Bucher and Sebastian von Mammen
University of W¨
urzburg, Germany
Abstract—The release of Pok´
emon GO attracted a huge player
base ranging from children to adults, thereby establishing aug-
mented reality (AR) on the mass market. In this paper, we
present Pathomon, a social AR serious game which combines
the location-based game mechanics of Pok´
emon GO with the
benefits of using AR in a serious context. We describe the
concept of the game which is based on cooperatively scanning
QR codes, enabling the players to work together towards their
common goal of eradicating viruses in their environment, while
at the same time acquiring knowledge about these viruses.
Furthermore, we present a first user study suggesting favorable
results with respect to game experience, yet indicating room
for improvement concerning the social game aspects. Finally,
we outline the opportunity of the game to serve as a starting
point for the development of a more versatile social AR platform
capable of including arbitrary contents beyond the context of
virus infections.
Index Terms—Educational technology, Mobile learning, Mobile
applications, Augmented reality, Games, Biological information
theory, Information sharing.
The release of the 2016 blockbuster title Pok´
emon GO
granted its developing company Niantic an unprecedented
success. Due to the popularity of its brand, the game was
downloaded more than 30 million times within the first two
weeks after its release [1]. Along with that, it targets a
huge and diverse audience ranging from children to adults
alike [2]. Furthermore, by making use of location-based AR,
emon GO represents the first game to have popularised this
technology among global mainstream [3].
In this paper, we present Pathomon, a serious game which
integrates Pok´
emon GO’s main game mechanics into the
serious context of infectious diseases. Following the basic
concept of Pok´
emon GO, three main reasons lead to selecting
the domain of virus transmission as context of the game: First,
education in the field of contagious diseases is a fundamen-
tal prerequisite to reduce their spreading [4]. Second, since
Pathomon targets the same diverse audience as Pok´
emon GO,
we wanted to pick a domain which children and adults can
be educated on alike. Third, the location-based structure and
mechanics of Pok´
emon GO qualify excellently for adaptation
in the serious context of diseases, mimicking the ubiquitous
and path-dependent characteristics of virus transmission.
Furthermore, like Pok´
emon GO, our game Pathomon also
emphasises the social aspect between players by encouraging
and rewarding collaboration in different ways. Accordingly,
the contribution of our work is two-fold: On the one hand,
we have transposed the Pok´
emon GO game mechanics to a
serious, educative context. On the other hand, this work paves
the way for developing a general set of game mechanics and
linking them to serious contents beyond our target domain.
In the following section, we will give a brief overview
on related work concerning the usage of AR in learning
environments. In section III, we present Pathomon’s concept
and functionality in further detail and discuss the results of a
user study. Eventually, we conclude with a summary and give
an outlook on future work in section IV.
The use of AR provides several affordances for educational
purposes in comparison to non-AR environments. According
to [5], AR provides the possibility of (1) presenting learning
content as three-dimensional representation, (2) ubiquitous,
collaborative and situated learning, (3) the invocation of the
learners’ senses of presence, immediacy and immersion, (4)
visualising the invisible and (5) bridging formal and informal
learning. As summarised in [6], these affordances can lead po-
tential advantages like (1) better learning of spatial structures
among various domains (e.g. astronomy [7], chemistry [8] or
anatomy [9]), (2) higher long-term memory retention as well
as (3) improved collaboration and motivation. These findings
portray the usage of AR technology as a promising opportunity
to convey knowledge in an effective and collaborative way.
Aside from this, studies suggest that it could be beneficial
to integrate education within the learner’s lifeworld, e.g. by
incorporating learning into everyday life, and thus making
the learned content more authentic and applicable [10]. To
achieve this, the use of games for learning tasks appears to
be a suitable way [11]. Therefore, combining the benefits of
AR technology with gamified environments which refer to
the player’s lifeworld seems like a promising approach for
education. An example for the integration of such a game-
based approach into everyday life is the 2007 AR game
Outbreak @ The Institute [12]. The game put students in
different roles in which they had to prevent the spread of
an infectious disease. However, as the hard- and software of
handheld devices was still limited at that time, the use of AR
was limited to location-based information and did not include
augmenting the real world with real-time virtual 3D models.
Fig. 1. Game Procedures of Pathomon, showing the repeatable game loop
Apart from these positive aspects, there are also problems
associated with AR learning: Learners might experience issues
with the technology in use [13]. Moreover, the use of AR
can lead to attentional tunneling or cognitive overload [14].
According to [15], users might also “lose sight of where the
game ends and reality begins”. These shortcomings have to be
carefully considered when creating an AR learning experience.
As outlined in the previous section, new forms of tech-
nology, especially AR, might improve learning environments.
However, such systems are yet to be fully established. Path-
omon aims at transforming information into a system using
the discussed benefits of AR and gamification.
A. Concept
1) Story: Pathomon takes the real world and the real issues
of spreading diseases and transforms it into a journey: players
dive into the role of young scientists and have to fight viruses.
In order to do that, the players soon realize that they have to
find the right ingredients to craft antidotes that are then used
to eradicate the viruses. However, killing them at one (real-
world) location does not completely erease them. Players need
to team up with others to share knowledge and fight stronger
viruses in the hope of finally erasing them completely.
2) Game Procedures: The game procedures are depicted
in Figure 1. Players first have to find QR codes which can
be located within the play area. The player will be presented
an AR real-time virtual animation of either a virus or an
ingredient after scanning a code. By collecting ingredients and
subsequently crafting (Figure 3) antidotes, players can attack
the viruses and earn experience points (XP). Some highly
resistant viruses make it necessary for players to team up with
others in order to fight them. Other ways of social engagement
are sharing knowledge about QR code, and hence, antidote and
virus locations. Finally, progress is tracked and summed over
all players, leading to the uniting aim to eradicate all viruses.
3) Mechanics: First, players create their personal account
(Figure 2). Then, they obtain a profile which designates them
as expert for one particular virus (Figure 3). Due to the prox-
imity to a viral specimen, the player has the possibility to leave
the virus with QR codes he interacts with, forcing subsequent
Fig. 2. Login (left), Pathodex (center) and AR view (right)
Fig. 3. Profile (left), Inventory (center) and Crafting (right)
players to attack them. Progress is achieved through various
game mechanics: First, players earn XP that transform into
levels (Figure 3). Higher levels allow the crafting of new an-
tidotes and therefore to fight higher-level viruses. Players can
also compare themselves on a score ladder, which is accessible
through their in-game profile. Additionally, achievements can
be earned. Concerning the viruses, each player is decorated
with a personal success rate in their Pathodex, a collection of
viruses this player has fought so far (Figure 2). Players earn
progress and unlock information about defeated viruses and
thereby increase their rates. Additionally, all players share a
global rate. This leads to the fact that the whole player base
is working together to eradicate viruses by fulfilling personal
rates in order to take the global rate to its maximum.
The main interactions in the game are collecting, crafting
and fighting. Players can scan QR codes to find ingredients,
which are then stored in their inventory (Figure 3). They can
cooperate by sharing their knowledge about the locations of
the codes. In the next step, players combine ingredients into
antidotes using the crafting option (Figure 3). They need to
discover the right combinations first or ask other players about
them. Finally, players fight viruses by using antidotes in the
AR view (Figure 2). Resistant viruses can only be killed by
coordinating attacks with other players.
4) Conveying Knowledge: Three different strategies are
used to convey knowledge about viruses within Pathomon:
First, each virus contains facts that can be unlocked and are
explicitly attached to those viruses. This textual information
includes size, time of discovery, lethality, type of vaccination,
symptoms, incubation time, method of transmission and out-
breaks. Some “fun facts” like “swine flue is not transmitted
by swines” are also stated. Second, players learn about the
appearance of the viruses, since models with realistic shapes
are used. Third, the game mechanics implicitly relate to the
characteristics of viruses, like players who can spread (drop)
them (onto QR codes).
All in all, the game closely bridges between its knowl-
edge information and game mechanics, since “specific game
mechanics require gaming skills which are then mapped to
general physiological as well as cognitive and social human
skills” [16], allowing players to learn this knowledge in an
entertaining way.
B. Implementation
Unity3D was used for development in order to achieve
multi-platform support. For the AR aspects of the game, the
Kudan framework was used. The server-side implementation
and API was hosted on an Amazon AWS EC2 instance at-
tached to an Amazon AWS RDS database instance. Pathomon
was published both on iOS and Android.
C. User Study
Pathomon was first played and evaluated at the 2017 Inter-
national Genetically Engineered Machine (iGEM) Conference
in Boston, MA. By the end of the conference, 162 players
had downloaded and tried out the game. We conducted a user
study with a subset of these players to evaluate on the general
game experience. Due to the social features of the game, the
extent of social involvement with respect to other players was
examined as well. However, this study should not be seen as
a summative evaluation since other important aspects, like the
effectiveness of learning, have not been tested yet.
1) Participants: By the end of the conference, players were
selected at random and asked to take part in the study. How-
ever, only data of players that had gained XP were analysed
in order to make sure they had at least some experience with
the game. A total of 23 people participated, of which 10 were
female, 9 were male and 4 did not state their gender. All of
the subjects were attendees of the 2017 iGEM Competition and
thus came from a variety of different countries. On average, the
participants were 22.00 years old (SD = 1.98). The subjects
stated that they had played the game for 44.47 minutes on
average (SD = 27.10 minutes).
2) Instruments: For quantitative data collection, two mod-
ules of the Game Experience Questionnaire (GEQ) [17] were
used. (1) The Ingame Questionnaire allowed us to raise data
about experience components such as immersion or challenge.
(2) By means of the Social Presence Module, we examined
the psychological and behavioural involvement during the
game. Answers were collected in an online questionnaire
with a 5-point Likert scale (ranging from 0 (not at all) to
4 (extremely)). The participants were also asked to express
open coded qualitative feedback. Finally, additional qualitative
feedback was provided by the iGEM competition’s judges via
the iGEM website [18].
3) Procedure: At the beginning of the conference, the
required QR codes were distributed at the conference venue.
Simultaneously, the game was promoted to the participating
teams by approaching them at their exhibition booths and
asking them to try it out with their private smartphone during
the conference days. Playing took mostly place during poster
sessions and was not monitored. At the end of the confer-
ence, players were approached again at random and asked to
take part in the study. The participants filled out the online
questionnaires using their smartphones.
4) Results: The results of the Ingame Questionnaire can
be found in table I. The components ‘Sensory and Imaginative
Immersion’, ‘Positive Affect’ as well as ‘Competence’ showed
relatively high values, whereas the game does not seem to
evoke negative affects or tension. The components ‘Flow’ and
‘Challenge’ are on a moderate level.
RES ULTS GEQ Ingame Questionnaire
Component n M SD
Sensory and Imaginative Immersion 22 3.07 1.00
Flow 19 1.95 1.19
Competence 20 2.51 1.11
Positive Affect 18 2.90 0.89
Negative Affect 21 0.70 1.09
Tension 19 0.86 0.89
Challenge 22 2.29 1.09
The results for the Social Presence Module are depicted
in table II. Players experienced empathy on a moderate level
and did not have many negative feelings towards others. The
relatively low value of ‘Behavioural Involvement’ indicates
that they experienced only little to moderate influence of their
actions on the actions of the others.
RES ULTS GEQ Social Presence Module
Component n M SD
Psychological Involvement – Empathy 17 2.28 1.10
Psychological Involvement – Negative Feelings 18 1.60 1.46
Behavioural Involvement 20 1.55 1.28
The qualitative feedback contained several positive state-
ments, but also a couple of critical mentions. Among these
were technical issues, especially with the Android version.
The latter seemed to have a less stable performance compared
to the iOS version. In addition, the participants of the game
provided suggestions to improve the game concept as well.
Some players for example perceived the start and progress
of the game as too hard. The judges’ qualitative feedback
was very positive, stating i.a. the “AR game Pathomon is a
wonderful way to engage others”.
5) Discussion: The results of the Ingame Questionnaire
are quite favorable as the game seems to evoke feelings
of immersion and positive affects while not being tension-
ing or bringing out negative emotions. The high values for
‘Competence’ however contradict the qualitative expressions
stating that the start of the game was too hard. A possible
explanation for this is that the players felt competent with
respect to understanding the game controls and rules, but
were overwhelmed by the difficulty in the initial phase of
the game. Additionally, the players did not have the feeling
that their actions would influence the actions of the others
in a considerable way according to the the low values for
Behavioural Involvement’. An explanation for this could be
that the players might have focused on their own progress,
thereby overlooking their influence on the others. It should
therefore be investigated in what way a better perception of
this impact can be evoked in order to strengthen the game’s
social aspect.
Aside from this, it must be noted, that the study itself is
subject to a couple of limitations as well. First and foremost,
it only considered the experience of the player during the
game. An evaluation of the effectiveness of the game for
learning about viruses has not been conducted yet. Second,
neither the game-playing nor the collection of the data could
be done in a uniform way under laboratory conditions due to
the circumstances at the conference. This might have led to
undesired deflections. Third, the number of participants was
relatively low. For more reliable results, a follow-up study with
a larger number of participants should be carried out.
This paper presented the mobile social AR serious game
Pathomon. An analysis of related work focusing on the usage
of AR within the context of learning deems this technology as
a promising approach. Concordantly, Pok´
emon GO used this
technology effectively to attract a huge player base.
Pathomon resembles a combination of the location-based
game mechanics of Pok´
emon GO with scientific knowledge
about viruses. By grouping up, players can help each other
with sharing QR code locations, fighting together against
very resistant viruses and work towards the common goal of
eradicating all viruses.
A first user study conducted at the 2017 iGEM conference,
revealed several positive aspects of the game as well as
suggestions for improvement. These include, for example, the
wish for a smoother and easier progress in the game.
While knowledge about viruses is a good start, the game
mechanics developed in this paper could be used in various
other contexts, as long as well-defined, localised interactions
between the AR contents and the player can be established.
With respect to Pok´
emon GO and Pathomon, these interactions
are, for instance, finding, collecting, fighting, crafting, and
spreading. More systematically speaking, it makes sense to
provide game mechanics that contribute to a socially and spa-
tially distributed game setting that involves local interactions,
combination and transport. Therefore, a possible step in the
future might be the further development of the game towards a
versatile platform whose content for learning can be exchanged
The authors would like to thank the 2017 iGEM Team
Franconia and especially the W¨
urzburg fraction for their great
support and the great project experience. They also express
special thanks to Andreas Knote who always helped with all
sorts of technical difficulties.
[1] A. Biseria and A. Rao, “Human computer interface-augmented reality,
International Journal of Engineering Science, vol. 2594, 2016.
[2] M. Sonders, “Pok´
emon go demographics: The evolving player mix of a
smash-hit game,” Survey Monkey Intelligence blog (December 7, 2016)
Retrieved from, 2016.
[3] J. Paavilainen, H. Korhonen, K. Alha, J. Stenros, E. Koskinen, and
F. Mayra, “The pok´
emon go experience: A location-based augmented
reality mobile game goes mainstream,” in Proceedings of the 2017 CHI
Conference on Human Factors in Computing Systems. ACM, 2017,
pp. 2493–2498.
[4] D. Kirby, B. Laris, and L. Rolleri, Impact of sex and HIV education
programs on sexual behaviors of youth in developing and developed
countries. Family Health International, YouthNet Program North
Carolina, 2005.
[5] E. Klopfer, K. Squire, and H. Jenkins, “Environmental detectives: Pdas
as a window into a virtual simulated world,” in Wireless and Mo-
bile Technologies in Education, 2002. Proceedings. IEEE International
Workshop on. IEEE, 2002, pp. 95–98.
[6] I. Radu, “Augmented reality in education: a meta-review and cross-
media analysis,” Personal and Ubiquitous Computing, vol. 18, no. 6,
pp. 1533–1543, 2014.
[7] R. Lindgren and J. M. Moshell, “Supporting children’s learning with
body-based metaphors in a mixed reality environment,” in Proceedings
of the 10th International Conference on Interaction Design and Chil-
dren. ACM, 2011, pp. 177–180.
[8] Y.-C. Chen, “A study of comparing the use of augmented reality and
physical models in chemistry education,” in Proceedings of the 2006
ACM international conference on Virtual reality continuum and its
applications. ACM, 2006, pp. 369–372.
[9] A. Nischelwitzer, F.-J. Lenz, G. Searle, and A. Holzinger, “Some aspects
of the development of low-cost augmented reality learning environments
as examples for future interfaces in technology enhanced learning,”
in International Conference on Universal Access in Human-Computer
Interaction. Springer, 2007, pp. 728–737.
[10] W. Schreiber, “Historisches lernen und lebenswelt,” 2005.
[11] M. Virvou, G. Katsionis, and K. Manos, “Combining software games
with education: Evaluation of its educational effectiveness,Journal of
Educational Technology & Society, vol. 8, no. 2, 2005.
[12] E. Rosenbaum, E. Klopfer, and J. Perry, “On location learning: Authentic
applied science with networked augmented realities,” Journal of Science
Education and Technology, vol. 16, no. 1, pp. 31–45, 2007.
[13] H.-K. Wu, S. W.-Y. Lee, H.-Y. Chang, and J.-C. Liang, “Current
status, opportunities and challenges of augmented reality in education,”
Computers & education, vol. 62, pp. 41–49, 2013.
[14] M. Dunleavy, C. Dede, and R. Mitchell, “Affordances and limitations of
immersive participatory augmented reality simulations for teaching and
learning,” Journal of science Education and Technology, vol. 18, no. 1,
pp. 7–22, 2009.
[15] E. Klopfer and K. Squire, “Environmental detectives—the development
of an augmented reality platform for environmental simulations,Ed-
ucational Technology Research and Development, vol. 56, no. 2, pp.
203–228, 2008.
[16] S. Oberd¨
orfer and M. E. Latoschik, “Develop your strengths by gaming:
Towards an inventory of gamificationable skills.” in GI-Jahrestagung,
2013, pp. 2346–2357.
[17] W. IJsselsteijn, Y. De Kort, and K. Poels, “The game experience
questionnaire,” Manuscript in preparation, 2008.
[18] 2017. [Online]. Available: Page
... AR applications presented in four subgroups, including the emergency management of a) any infectious disease epidemics [45][46][47] , b) influenza [20,48] , and c) dengue [18] ; (3) AR and VR applications for the COVID-19 disease presented in four sections, i.e., a) potential applications [49][50][51] , b) clinical context [52][53][54][55][56] , c) Teleservices [57][58][59][60] , and d) education [54,61,62] . Finally, a taxonomy of the potential applications of VR and AR in the COVID-19 outbreak was generated in Section 4. ...
... The and sharing knowledge about viruses through the game. The use of this game can be a good method for improving knowledge about viruses during outbreaks [47]. Through projection mapping, a mixed reality space was used to teach hand hygiene as an important way to improve hand hygiene and prevent the transmission of infectious diseases. ...
Full-text available
The pandemics of major infectious diseases often cause public health, economic, and social problems. Virtual reality (VR) and augmented reality (AR), as two novel technologies, have been used in many fields for emergency management of disasters. The objective of this paper was to review VR and AR applications in the emergency management of infectious outbreaks with an emphasis on the COVID-19 outbreak. A search was conducted in MEDLINE (PubMed), Embase, IEEE, Cochrane Library, Google engine, Google scholar, and related websites for papers published up to May 2, 2020. The VR technology has been used for preventing or responding to infections by simulating human behaviors, infection transmission, and pathogen structure as a means for improving skills management and safety protection. Telehealth, telecommunication, and drug discovery have been among the other applications of VR during this pandemic. Moreover, AR has also been used in various industries, including healthcare, marketing, universities, and schools. Providing high-resolution audio and video communication, facilitating remote collaboration, and allowing the visualization of invisible concepts are some of the advantages of using this technology. However, VR has been used more frequently than AR in the emergency management of previous infectious diseases with a greater focus on education and training. The potential applications of these technologies for COVID-19 can be categorized into four groups, i.e., 1) entertainment, 2) clinical context, 3) business and industry, and 4) education and training. The results of this study indicate that VR and AR have the potential to be used for emergency management of infectious diseases. Further research into employing these technologies will have a substantial impact on mitigating the destructive effects of infectious diseases. Making use of all the potential applications of these technologies should be considered for the emergency management of the current pandemic and mitigating its negative impacts.
... However, in existing serious games, collaboration is mostly used as a method to reinforce specific learning outcomes, rather than being an outcome itself. For example, in the serious game Pathomon [37], players use AR with the primary aim of learning about viruses, but also collaborate by sharing with each other the correct ingredient combination for an antidote. However, players did not feel that their actions influenced the actions of others, which demonstrates an opportunity for greater emphasis on collaboration. ...
Full-text available
Early adolescence is a time of major social change; a strong sense of belonging and peer connectedness is an essential protective factor in mental health during that period. In this paper we introduce LINA, an augmented reality (AR) smartphone-based serious game played in school by an entire class (age 10+) together with their teacher, which aims to facilitate and improve peer interaction, sense of belonging and class climate, while creating a safe space to reflect on mental health and external stressors related to family circumstance. LINA was developed through an interdisciplinary collaboration involving a playwright, software developers, psychologists, and artists, via an iterative co-development process with young people. A prototype has been evaluated quantitatively for usability and qualitatively for efficacy in a study with 91 early adolescents (agemean=11.41). Results from the Game User Experience Satisfaction Scale (GUESS-18) and data from qualitative focus groups showed high acceptability and preliminary efficacy of the game. Using AR, a shared immersive narrative and collaborative gameplay in a shared physical space offers an opportunity to harness adolescent affinity for digital technology towards improving real-world social connection and sense of belonging.
... A lot of governmental and health institutions for example recommend behavioral rules to avoid infection with the currently prevailing SARS-CoV-2-Virus, e.g., cover nose and mouth, wash hands, avoid handshaking or touching the face [1,2]. Methods for raising awareness for the spread of pathogens are wide-ranging, e.g., videos [5,2], apps, visualizations [22], websites, and serious games [18]. There are a lot of videos, which simulate the spread of pathogens and visualize fomites [2,13,22]. ...
Full-text available
Several times a day, we touch our faces and all kinds of surfaces, and often we are not even aware that pathogens are transported via surfaces and our hands. There are already a large number of educational or governmental videos that aim to raise people’s awareness towards health and hygienic behavior, but whether videos simulating the spread of pathogens can really influence people’s behavior and awareness has not been investigated yet. Using a virtual copy of a students’ office room and spatial visualizations of touched surfaces, our approach visually simulates the spread of pathogens of smear infections. In a user study, we show videos with or without visualizations of potentially contaminated surfaces in a students’ office room to participants, and afterwards ask them to describe their own potential subsequent behavior when entering the same room. In addition to open questions, we also use the PPQ to investigate the persuasive potential of our approach. With this approach we investigate the impact of the visualization of potentially contaminated surfaces on people’s behavior and hygiene-oriented attitude. While our results did not reveal a significant effect on the consideration of behavior change assessed by a questionnaire, the participants’ descriptions in the qualitative part of the study show more hygiene-oriented behavior descriptions in the visualization conditions.
... The number of Mobile Augmented Reality (MAR) applications has been increasing steadily in the last few years, with many applications being launched for the wider consumer market (Hilken et al., 2017). Different to a few years ago, consumers today are aware of the technology and understand how it can be used in a number of contexts, from retail (Dacko, 2017) to social engagement (Rapp et al., 2018) and for leisure purposes (Han et al., 2018). However, while the expanding awareness of AR technology has sparked more interest among businesses to design and enhance consumer experiences, MAR still struggles to make the breakthrough with regards to mass usage in the consumer market. ...
Conference Paper
Full-text available
As the expanding awareness of AR technology has sparked more interest among businesses to design and enhance experiences, there is an emerging discussion on different ways MAR applications can contribute to the user experience. A prevalent way to make information more attractive to consumers is argued to be the employment of storytelling. This study aims to provide insights and recommendations on designing emotionally engaging and memorable AR user experiences using storytelling as the mediating tool. It is based on a MAR application that uses historical locations related to personal stories from WWII called Brabant Remembers. The findings of the paper provide a more holistic perspective on the literature of MAR experience design that blends the context (physical location), user-relevance and emotional user engagement. Through the use of a temporal case study, the paper discusses practical implications with regards to stakeholder management for MAR design and implementation. Finally, the study offers insights and recommendations on how the design and implementation of MAR can be approached using storytelling as the mediating tool to emotionally engage users.
... For example, in 2016, the game Pokemon Go allowed smartphone users to see an augmented environment in different parts of the world and play the game worldwide. Later, Rapp et al. [25] developed another location-based AR game. In this game, users could scan Quick Response (QR) codes with their smartphones and receive the visuals and descriptions of viruses spread around the world. ...
Full-text available
Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.
... Even though these preliminary indications are encouraging, VoIP solutions do not take advantage of immersive technologies, which allow the users to experience a shared virtual environment. Recent advancements in head-mounted display-(HMD) and virtual reality (VR) technologies have brought a wave of social immersive experiences, which are referred to as "social VR" [15] or "social augmented reality" [37,39]. ...
Conference Paper
This study used augmented virtuality with an overall aim to prevent undernourishment among older adults. The purpose was to facilitate remote social eating for solitary older adults, as people tend to eat more when socializing. In this study, an augmented virtuality application was developed in which a physical cake buffet and a virtual living room coexisted and interacted in real time. This was possible using an Oculus Rift CV1 HMD with an Intel Realsense SR300 depth sensor mounted on top of the HMD. The study included initial workshops with 30 experts and 16 older adults, prototyping with 7 mobility-restricted older adults, and a final user study with 27 older adults. In the user study, we evaluated the user experience of a system designed to establish a meal conversation between three older friends while eating a solitary meal in augmented virtuality. Within three overall factors (user, context, and system), we outlined sub- elements that constituted both opportunities and limitations, which included interactions, perceptions, and behavior in the augmented virtuality. The virtual living room was described very positively by all participants. However, there were also some technological limitations in terms of fidelity, HMD comfort, and quality. When developing virtual environments, we found it very important to include very specific elements within the user, contextual and system categories, as well including qualitative methods throughout the entire design process.
LegionARius is a real-time strategy game using Augmented Reality (AR) to let players experience Roman and Germanic history in their own home. Being rooted in the tradition of Serious Games (SG), LegionARius aims at providing historically accurate knowledge about ancient everyday life in an entertaining manner. The players have to decide the destiny of a Roman cohort or a Germanic tribe living alongside the fortifications of Limes during the second century AD. As Roman commander or chief of a Germanic tribe they build, produce, trade and use their environment to choose a path between peaceful coexistence and war. With the help of AR the player can view and inspect the historically accurate 3D models of Roman or Germanic buildings. In addition, the player gets to know the historical context and background in a playful way. This knowledge is not only expanded in the course of the game, but is also a key element for success.
Serious games continue to attract the interest of developers and users. This project aims to design a serious game called Win the City to explore famous attractions e.g. buildings in the city of Valletta in Malta. The Adobe PhoneGap, Google Maps API, and Firebase platform for Android were used to design the prototype for the game. The game uses the real Valletta map to locate the targeted buildings as presented on Google maps. All the provided details and options on Google Maps are shown on the game’s prototype. The game can be installed on mobile devices that support the Android platform. The designed prototype was tested by a number of users (residents and visitors) to collect feedback about the game idea, its design, the included functionalities and features, and its overall usability. It’s expected that this game will enhance the knowledge of local people as well as the visitors about the targeted famous buildings and sites. Also, the game will provide valuable feedback to the city council about how such buildings are located, frequency of visits, and any physical issues with the buildings. On the other hand, playing the game will promote more exercise activities and enhance the wellbeing of the players.
In 5G and beyond, the adequate interaction between densely deployed internet of things (IoT) devices and cellular users will generate massive cyber-physical information stream in real-time manner. How to capture insights underneath these data in a smart city context is gaining great attention nowadays. In this paper, we introduce a highly function-differentiated metropolitan scenario, which is covered by multiple unmanned aerial vehicles (UAVs) serving as cache-enabled edge computing nodes. With the help of wireless backhaul technology, coverage capability of UAV can be dynamically configured through smart 3-D placement, where trajectories of two types of UAVs are optimized. A social augmented reality (AR) based use case is proposed and discussed in proposed scenario, from which we derive the fundamental mechanisms and strategies to maintain a green and sustainable edge/fog computing framework. We sequentially establish two non-convex programming problems and optimize delay and energy performance during AR data acquisition and AR content downloading, respectively. Two convex approximation skills are applied to transform the original problems into tractable form. Experimental results show that our proposed edge computing framework can help provide energy-efficient AR service to cellular users, catering to pretty tight delay constraints.
Full-text available
The learning of science can be made more like the practice of science through authentic simulated experiences. We have created a networked handheld Augmented Reality environment that combines the authentic role-playing of Augmented Realities and the underlying models of Participatory Simulations. This game, known as Outbreak @ The Institute, is played across a university campus where players take on the roles of doctors, medical technicians, and public health experts to contain a disease outbreak. Players can interact with virtual characters and employ virtual diagnostic tests and medicines. They are challenged to identify the source and prevent the spread of an infectious disease that can spread among real and/or virtual characters according to an underlying model. In this paper, we report on data from three high school classes who played the game. We investigate students’ perception of the authenticity of the game in terms of their personal embodiment in the game, their experience playing different roles, and their understanding of the dynamic model underlying the game.
Full-text available
The purpose of this study was to document how teachers and students describe and comprehend the ways in which participating in an augmented reality (AR) simulation aids or hinders teaching and learning. Like the multi-user virtual environment (MUVE) interface that underlies Internet games, AR is a good medium for immersive collaborative simulation, but has different strengths and limitations than MUVEs. Within a design-based research project, the researchers conducted multiple qualitative case studies across two middle schools (6th and 7th grade) and one high school (10th grade) in the northeastern United States to document the affordances and limitations of AR simulations from the student and teacher perspective. The researchers collected data through formal and informal interviews, direct observations, web site posts, and site documents. Teachers and students reported that the technology-mediated narrative and the interactive, situated, collaborative problem solving affordances of the AR simulation were highly engaging, especially among students who had previously presented behavioral and academic challenges for the teachers. However, while the AR simulation provided potentially transformative added value, it simultaneously presented unique technological, managerial, and cognitive challenges to teaching and learning.
Full-text available
The form factors of handheld computers make them increasingly popular among K-12 educators. Although some compelling examples of educational software for handhelds exist, we believe that the potential of this platform are just being discovered. This paper reviews innovative applications for mobile computing for both education and entertainment purposes, and then proposes a framework for approaching handheld applications we call “augmented reality educational gaming.” We then describe our development process in creating a development platform for augmented reality games that draws from rapid prototyping, learner-centered software, and contemporary game design methodologies. We provide a narrative case study of our development activities spread across five case studies with classrooms, and provide a design narrative explaining this development process and articulate an approach to designing educational software on emerging technology platforms. Pedagogical, design, and technical conclusions and implications are discussed.
Conference Paper
Pokémon GO is a location-based augmented reality mobile game based on the Pokémon franchise. After the game was launched globally in July 2016, it quickly became the most successful mobile game in both popularity and revenue generation at the time, and the first location-based augmented reality game to reach a mainstream status. We explore the game experiences through a qualitative survey (n=1000) in Finland focusing on the positive and the negative aspects of Pokémon GO as told by the players. The positive experiences are related to movement, sociability, game mechanics, and brand while the negative experiences emerge from technical problems, unequal gaming opportunities, bad behavior of other players and non-players, and unpolished game design. Interestingly, the augmented reality features, safety issues or the free-to-play revenue model did not receive considerable feedback. The findings are useful for academics and industry practitioners for studying and designing location-based augmented reality game experiences.
Augmented reality (AR) is an educational medium increasingly accessible to young users such as elementary school and high school students. Although previous research has shown that AR systems have the potential to improve student learning, the educational community remains unclear regarding the educational usefulness of AR and regarding contexts in which this technology is more effective than other educational mediums. This paper addresses these topics by analyzing 26 publications that have previously compared student learning in AR versus non-AR applications. It identifies a list of positive and negative impacts of AR experiences on student learning and highlights factors that are potentially underlying these effects. This set of factors is argued to cause differences in educational effectiveness between AR and other media. Furthermore, based on the analysis, the paper presents a heuristic questionnaire generated for judging the educational potential of AR experiences.
Although augmented reality (AR) has gained much research attention in recent years, the term AR was given different meanings by varying researchers. In this article, we first provide an overview of definitions, taxonomies, and technologies of AR. We argue that viewing AR as a concept rather than a type of technology would be more productive for educators, researchers, and designers. Then we identify certain features and affordances of AR systems and applications. Yet, these compelling features may not be unique to AR applications and can be found in other technological systems or learning environments(e.g., ubiquitous and mobile learning environments). The instructional approach adopted by an AR system and the alignment among technology design, instructional approach, and learning experiences may be more important. Thus, we classify three categories of instructional approaches that emphasize the “roles,” “tasks,” and “locations,” and discuss what and how different categories of AR approaches may help students learn. While AR offers new learning opportunities, it also creates new challenges for educators. We outline technological, pedagogical, learning issues related to the implementation of AR in education. For example, students in AR environments may be cognitively overloaded by the large amount of information they encounter, the multiple technological devices they are required to use, and the complex tasks they have to complete. This article provides possible solutions for some of the challenges and suggests topics and issues for future research.
Conference Paper
We describe an approach to designing immersive learning experiences for children using body-based metaphors. Previous research shows benefits for learning through physical interactions in virtual spaces (e.g., [1, 16])---here we look specifically at using mixed reality to embed children as elements within the systems they are attempting to learn. Using gross body-movements the children are able to test predictions and have their intuitions challenged, laying the foundation for deeper conceptual understanding. We present data from a study we conducted comparing the mixed reality experience with a desktop version of the same simulation. Results suggest that children's interactions with designs supporting body-based metaphors can lead them to better grasp the "deep structure" of the learning domain.
Conference Paper
Augmented Reality (AR) applications are becoming increasingly available for everyday applications. In this paper, we concentrate on how to design and develop applications for educational purposes with the use of the ARToolkit and present an example from the area of human medicine. Since good Usability is essential, especially for non-expert end users, including both children and the elderly, we put particular emphasis on how to ensure good Usability. For both groups of end users, this technology has high potential and can be of great benefit. Within a small scale usability study amongst children, we could gain first experiences in their interaction with such applications. An essential factor is the notion of play, which could be beneficial to transport difficult learning material. On the basis of the gained insight, we provide an outlook for further work and we conclude that elderly could also benefit from this technology.