Content uploaded by Mauricio Sáenz
Author content
All content in this area was uploaded by Mauricio Sáenz on Jul 31, 2015
Content may be subject to copyright.
Enhancing Navigation Skills
through Audio Gaming
Abstract
We present the design, development and initial
cognitive evaluation of an Audio-based Environment
Simulator (AbES). This software allows a blind user to
navigate through a virtual representation of a real
space for the purposes of training orientation and
mobility skills. Our findings indicate that users feel
satisfied and self-confident when interacting with the
audio-based interface, and the embedded sounds allow
them to correctly orient themselves and navigate within
the virtual world. Furthermore, users are able to
transfer spatial information acquired through virtual
interactions into real world navigation and problem
solving tasks.
Keywords
Orientation and Mobility, Virtual Environment, Visual
Impairment, audio games, videogames
ACM Classification Keywords
K.4.2 [Computing Milieux]: Computers and Society –
Social Issues: Assistive technologies for persons with
disabilities
General Terms
Design, Experimentation
Introduction
Several different approaches have been developed to
assist the blind with orientation and mobility (O&M).
Copyright is held by the author/owner(s).
CHI 2010, April 10–15, 2010, Atlanta, Georgia, USA.
ACM 978-1-60558-930-5/10/04.
Jaime Sánchez
Department of Computer Science
Center for Advanced Research in Education (CARE)
University of Chile
Blanco Encalada 2120.
Santiago, Chile
j
sanchez@dcc.uchile.cl
Mauricio Sáenz
Department of Computer Science
Center for Advanced Research in Education (CARE)
University of Chile
Blanco Encalada 2120.
Santiago, Chile
msaenz@dcc.uchile.cl
Alvaro Pascual-Leone
Berenson-Allen Center for Noninvasive Brain Stimulation,
Department of Neurology, Beth Israel Deaconess Medical
Center, Harvard Medical School
Lotfi Merabet
Berenson-Allen Center for Noninvasive Brain Stimulation,
Department of Neurology, Beth Israel Deaconess Medical
Center
,
Harvard Medical School
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3991
One possibility to help them become more autonomous
is to provide them with virtual-based training which
could ultimately be transferred to real world settings.
Along these lines, a number of studies using virtual
environment simulators allow a blind user to interact
through both audio [1], [4] and tactile cues [6].
Another possibility would be through the use of audio-
based games. Some studies have pointed out the
importance of gaming for improving problem solving
skills [10]. Moreover, the possibility of using games for
learning in pedagogical contexts opens up enormous
opportunities to bring education closer to students’
everyday life experiences, increasing motivation,
commitment to learning, and better shaping the
students’ current learning styles [2].
There have been numerous insights gained from the
design and use of videogames for visually impaired
people. When legally blind people interact with
videogames that include visual cues, they take
advantage of whatever residual vision they have in
order to achieve better results from the interaction
(e.g. through the use of high magnification). Certainly,
for a totally blind user this is not possible and it is thus
necessary to provide them with relevant information
regarding the environment through other sensory
channels such as touch and hearing [7]. There are also
studies implementing videogames for learning
mathematics in blind students [7]. Another study has
used audio-based gaming to reinforce science concepts
in a ludic environment for visually impaired children
[8]. As the child interacts with the game to fulfill the
underlying mission, he/she develops problem-solving
skills while learning science curriculum. Other
videogames assisted the development of spatial
knowledge in blind children [5]. Therefore, if
videogames can improve the development of different
types of skills, can they also improve the development
of navigation skills in blind children? The development
of orientation and mobility skills (O&M) is essential for
the autonomous navigation of a blind user.
The purpose of this research was to evaluate an audio-
based virtual environment simulator developed by our
group called Audio-based Environment Simulator
(AbES) designed to improve orientation and mobility
skills in blind users.
AbES
The simulator was developed to represent a real,
familiar or unfamiliar environment to be navigated by a
blind person. In the virtual environment, there are
different elements and objects (walls, stairwells, doors,
toilets or elevators) through which the user can
discover and come to know his/her location.
The simulator is capable of representing any real
environment by using a system of cells through which
the user moves. The user has audio feedback in the
left, center and right side channels, and all his/her
actions are carried out through the use of a traditional
keyboard, where a set of keys have an associated
action. All of the actions in the virtual environment
have a particular sound associated to them. In addition
to this audio feedback, there are also spoken audio
cues that provide information regarding the various
objects and the user’s orientation in the environment.
Orientation is provided by identifying the room in which
the user is located and the direction in which he/she is
facing, according to the cardinal compass points (east,
west, north and south). AbES includes three modes of
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3992
interaction: Free Navigation, Path Navigation and Game
Mode.
The free navigation mode provides the blind user with
the possibility of exploring the building freely in order
to become familiar with it (Figure 1). For a beginning
user, we found it useful to include the option that all
the doors in the building are open, making the
navigation simpler. In the same way, for beginners it is
necessary to hear all of the instructions that the
simulator provides. For this reason the “Allow Text-To-
Speech to end before any action” option is necessary.
Path navigation provides the blind user with the task of
finding a particular room. The facilitator must choose
the departure and arrival room and select how many
routes he/she deems it is necessary to take. When all
the routes have been selected, the user begins his/her
interaction with the simulator and has to navigate all
the chosen paths, thus training in, surveying and
mapping the building.
The game mode provides blind users with the task of
searching for “jewels” placed in the building. The
purpose of the game is to explore the rooms and find
all the jewels, bringing them outside one at a time and
then going back into the building to continue exploring.
Enemies are randomly placed in the building, and try to
steal the user’s jewels and hide them elsewhere. There
is a verbal, audio warning when the user is facing two
cells away from a jewel or an enemy. The enemies
always remain inside the building. In this game mode,
the facilitator can choose the number of jewels to find
(two, four or six) and the number of monsters (two,
four or six).
Preliminary Cognitive evaluation
Sample
The first part of the study included the participation of
seven children aged ten to twelve years old who attend
the Santa Lucia School for Blind Children in Santiago,
Chile. None of the participants had any other
neurological deficits and their visual status was
confirmed by their medical records or an
ophthalmological evaluation.
Instruments
Checklists were designed for each of the activities.
These checklists were based on standard orientation
and mobility instruments [3] and contain both common
and specific indicators that measure different aspects of
the students’ levels of progress (spatial orientation,
spatial knowledge and spatial representation) when
working on the various activities. A Likert-type scale
with scores ranging from 1 (never) to 4 points (always)
was used to quantify the results and calculate
percentages of achievement for each indicator.
Procedure
All the activities with the children were carried out in 6
sessions lasting three hours and fifteen minutes each.
During this time, five activities were performed in which
the children participated by interacting with AbES. Each
of these activities was evaluated by considering the
attainment of three navigation skills: Spatial
Orientation, is the ability to locate oneself within the
simulated map and direct efficiently from one point to
another; Spatial Knowledge, is the ability to
recognize, identify and remember the location of the
elements that can be found in the environment; and
Spatial Representation, is the ability to create a
mental image of the space that has been navigated.
Figure 1. Real and virtual environments in
AbES. (A) The first floor plan of the St. Paul
building. (B) Virtual representation of the
same floor in AbES showing various objects
the user interacts with. (C) Floor plan of
the Santa Lucia building. (D) Virtual
representation of the first floor of the Santa
Lucia building
Figure 2. (A) Tactile model representing
the space that the students will travel
through virtually using the AbES simulator.
(B) Blind participant exploring the model of
the real space through touch.
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3993
1. Initial interaction with a Concrete Model.
Previous studies have shown that children are likely
to understand certain processes more fully when
modeling and solving tasks using concrete materials
that supplement interactions with a virtual
environment [7]. In this study, this cognitive task
consisted of exploring with a concrete tactile model
of the environment that would be navigated using
AbES for the remainder of the time. This model
contains the building’s main structural divisions, as
well as the names of the different spaces written in
Braille (Figure 2). Once the exploration of the model
had been completed, the students had to construct
a spatial representation of the space through the
use of concrete materials such as plasticine, LEGO
or by making a drawing. In order to not
contaminate the sample, the maps used in this
stage were different from those used in later stages.
2. Free Navigation. To explore the environment, the
students’ interacted AbES using free navigation
mode, traveling at their own pace through all of the
spaces represented. Once they had finished
exploring the virtual model, the students had to
recreate the spaces they could remember using
concrete material of their choice.
3. Path Navigation Mission. In this activity the
students had to start from a predetermined start
point (the computer room) and travel to four
different destinations distributed throughout the first
floor of the building (the massage therapy room, the
fourth grade classroom, the front hall and the early
childhood intervention room). They were instructed
to take the shortest possible route (Figure 3). Once
they had finished exploring the virtual model, the
students had to represent the spaces they could
remember through the use of concrete material.
4. The Game. In this activity the students had to
interact with AbES in the game mode, seeking out
the hidden jewels and bringing them to the school’s
inner yard. This activity allows the students to take
different routes from those used in the “Path
Navigation Mission”, as they now knew of other
rooms and were constructing a mental image of the
spaces traveled. In this activity, the students
interacted for half of the time with the simulator and
during the other half of the time they performed the
same task in the real environment (Figure 3). This
way of interacting increases the participation and
motivation of the children, thus facilitating their
learning [9].
5. Concrete Representation. In this activity the
students interacted freely with the environment
(just as they had done in activity 2). Once this free
navigation was completed, they were asked to
represent the spaces traveled with concrete material
(Lego® bricks and drawings) (Figure 4).
Results
In general, the students obtained high scores on all the
activities held. For Spatial Orientation, the five activities
with the AbES simulator demonstrate very high
achievement percentage scores (Concrete Model: 74%,
Free Navigation: 84%, Path Navigation Mission: 81%,
The Game: 77%, Concrete Representation: 80%), with
free navigation representing the activity that obtained
the best results (Figure 5) (Chi Square= 1.895; dof=4;
p > 0.05) showing no evidence of significant difference.
By navigating freely, the students are focused only on
the task of moving through the environment. At the
same time, the students ask themselves more
questions about the virtual surroundings, which allow
them to become even more oriented. Spatial knowledge
also has high scores for all the activities held (Concrete
Model: 81%, Free Navigation: 87%, Path Navigation
Mission: 80%, The Game: 88%, Concrete
Representation: 79%), with The Game activity having
the highest achievement percentage score (Chi
Square= 9,714; dof=4; p < 0.05) (Figure 5) showing
evidence of significant difference. In playing, the
students had to travel through the space while
Figure 3. Students playing to get the
jewel in the real environment
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3994
concentrating and paying close attention to details, as
the activity implied locating the jewel and bringing it to
the schoolyard. To do this, they not only had to know
where they were but also remember well the paths to
be able to get out and leave the jewel in the right
place, without being caught by the monster. All this
information was successfully transferred when they
played in the real environment.
Finally, the activity with the highest spatial
representation scores was The Game (Concrete Model:
42%, Free Navigation: 52%, Path Navigation Mission:
53%, The Game: 86%, Concrete Representation:
59%), which resulted in much higher scores that those
obtained for the other activities (Chi Square= 5,837;
dof=4; p > 0.05) (Figure 5), showing no evidence of
significant difference. In playing at finding the jewel,
the students travel throughout the entire environment,
picking up information on the spaces and the objects
within the environment, and thus being able to
successfully play the game in the real spaces. In this
way, they are able to obtain more information from the
space and to improve their mental representation,
which is then successfully transferred to the real world
spaces.
In summary the global activity that generated the best
results is The Game (84%) (Figure 6), (Chi Square=
4.000; dof=4; p > 0.05), although there is no evidence
of significant difference. When the students played,
they obtained better scores than when they performed
other activities with AbES. When playing, they
remained more concentrated and focused on fulfilling
the goals of the game, being able to pick up on more
information provided by the simulator, and in a more
efficient manner.
Conclusion
The purpose of this research was to evaluate an audio-
based virtual environment simulator developed by our
group called Audio-based Environment Simulator
(AbES) designed to improve orientation and mobility
skills in blind users. O&M training remains a mainstay
in blind rehabilitation and with systematic and rigorous
training, individuals with visual impairment can gain
functional independence. Here, we show that the
creative use of interactive virtual navigation
environments such as AbES combined with other
strategies may provide for flexibility adjusting for a
person’s own needs, strengths and weaknesses to
supplement their O&M training curricula.
Of particular note was the robust nature of spatial
cognitive information that could be obtained by
interacting with AbES the gaming mode. We intended
for users to be able to play and enjoy the game and in
doing so, learn to navigate their surrounding
environment, understand the spatial organization and
layout of its spaces, its dimensions and the
corresponding objects Key to this approach is the fact
that this information is learned implicitly though
gaming interactions rather than explicit route learning.
As users became more skilled at playing AbES through
navigating freely at their own pace, they were in fact
laying the foundations for transferring virtual learning
to real world navigation. This game mode has been the
activity that generated the best results as far as the
students’ spatial representations (although there was
no evidence of significant difference), showing that this
kind of interaction requires them to focus on the tasks
that they are carrying out. It makes them more
attentive careful and resourceful through constant
inquisition about the places they are traveling through.
Figure 4. The students’ representations
made with concrete material. (A)
Representation with Legos® bricks. (B)
Drawn representation
Figure 5. Graphic shows the results
obtained by the students for the three
aspects evaluated (Spatial Orientation,
Spatial Knowledge and Spatial
Representation)
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3995
They thus obtain more robust information regarding
their surroundings translating to better results in the
transfer of this knowledge to the real physical world.
Future Work
We continue to investigate the feasibility, effectiveness,
and potential benefits of training and learning to
navigate unfamiliar environments using virtual
auditory-based gaming systems. In parallel, we are
also developing methods for quantifying behavioral
gains as well as uncovering brain mechanisms
associated with navigational skills. A key direction for
future research will be to understand what aspects of
acquired spatial information are actually transferred
from virtual to real environments, and the conditions
that promote this transfer. This implies the use of
experimental designs in order to clearly determine the
impact that the use of this technology has on the
development of navigation skills. We further propose
that understanding how the brain creates spatial
cognitive maps used for navigation and over time, as
well as a function of an individual’s own experience and
motivation will have potentially important repercussions
in terms of how rehabilitation is carried out and,
ultimately, an individual’s overall rehabilitative success.
Acknowledgements
This report was funded by the Chilean National Fund of
Science and Technology, Fondecyt #1090352 and
Project CIE-05 Program Center Education PBCT-Conicyt
References
[1] Amandine, A., Katz, B., Blum, A., Jacquemin, C.,
Denis, M. (2005) A Study of Spatial Cognition in an
Immersive Virtual Audio Environment: Comparing Blind
and Blindfolded Individuals. Proc. of 11
th
ICAD, pp.
228-235
[2] Cipolla-Ficarra, F. (2007). A Study of Acteme on
users Unexpert of VideoGames. J. Jacko (Ed.): Human-
Computer Interaction, Part IV, HCII 2007, LNCS 4553,
pp. 215-224
[3] González, F., Millán, L., Rodríguez, C. (2003).
Orientación y Movilidad. Apuntes del curso
“Psicomotricidad, y Orientación y Movilidad para la
persona con discapacidad visual”, VII semestre
Trastornos de la visión, Universidad Metropolitana de
Ciencias de la Educación. Sin numeración.
[4] Kehoe, A., Neff, F., Pitt, I. (2007) Extending
traditional user assistance systems to support an
auditory interface. Proc. of the 25th IASTED
International Multi-Conference: artificial intelligence
and applications, pp. 637 – 642
[5] Lumbreras, M. & Sánchez, J. (1999). Interactive 3D
sound hyperstories for blind children. Proceedings of
the ACM-CHI '99, Pittsburgh, USA, pp. 318-325
[6] Murai, Y., Tatsumi, H., Nagai, N., Miyakawa, M.
(2006). A Haptic Interface for an Indoor-Walk-Guide
Simulator. K. Miesenberger et al. (Eds.); ICCHP 2006,
LNCS 4061, pp. 1287 – 1293
[7] Sánchez, J. (2008). User-Centered Technologies for
Blind Children. Human Technology Journal, 45(2), pp.
96-122
[8] Sánchez, J., Elías, M. (2007). Science Learning by
Blind Children through Audio-Based Interactive
Software. 12th Annual CyberTherapy 2007 Conference:
Transforming Healthcare Through Technology, pp. 40
[9] Squire, K. (2003). Video games in education.
International Journal of Intelligent Simulations and
Gaming, 2(1).pp. 1-16.
[10] Steinkuehler, C. (2008). Cognition and literacy in
massively multiplayer online games. In Leu, D., Coiro,
J., Lankshear, C. & Knobel, K. (Eds.), Handbook of
Research on New Literacies. Mahwah NJ: Erlbaum, pp.
1-38
Figure 6. Total results obtained by the
students for each of the activities.
CHI 2010: Work-in-Progress (Spotlight on Posters Days 3 & 4)
April 14–15, 2010, Atlanta, GA, USA
3996
