Conference PaperPDF Available

Design, implementation and evaluation of audio for a location aware augmented reality game


Abstract and Figures

In this paper, the development and implementation of a rich sound design, reminiscent of console gaming for a location aware game, Viking Ghost Hunt (VGH) is presented. The role of audio was assessed with particular attention to the effect on immersion and emotional engagement. Because immersion also involves the interaction and the creation of presence (the feeling of being in a particular place) these aspects of the sound design were also investigated. Evaluation of the game was undertaken over a three-day period with the participation of 19 subjects. The results gained imply that audio plays an important role in immersing a player within the game space and in emotionally engaging with the virtual world. However, challenges in regards to GPS inaccuracy and unpredictability remain, as well as device processor constraints, in order to create an accurate audio sound field and for the real-time rendering of audio files.
Content may be subject to copyright.
Design, Implementation and Evaluation of Audio for a
Location Aware Augmented Reality Game
Natasa Paterson Katsiaryna Naliuka Soren Kristian Jensen Tara Carrigy Mads Haahr
Fionnuala Conway
Trinity College Dublin
College Green
Dublin 2, Ireland
In this paper, the development and implementation of a rich sound
design, reminiscent of console gaming for a location aware game,
Viking Ghost Hunt (VGH) is presented. The role of audio was
assessed with particular attention to the effect on immersion and
emotional engagement. Because immersion also involves the
interaction and the creation of presence (the feeling of being in a
particular place) these aspects of the sound design were also
investigated. Evaluation of the game was undertaken over a
three-day period with the participation of 19 subjects. The results
gained imply that audio plays an important role in immersing a
player within the game space and in emotionally engaging with
the virtual world. However, challenges in regards to GPS
inaccuracy and unpredictability remain, as well as device
processor constraints, in order to create an accurate audio sound
field and for the real-time rendering of audio files.
Categories and Subject Descriptors
C.5.3 [Computer System Implementation]: Microcomputers;
H.1.2 [Models and Principles]: User/Machine Systems; H.5.1
[Information Interfaces and Presentation]: Multimedia
Information Systems; H.5.2 User Interfaces; H.5.5 Sound and
Music Computing.
General Terms
Design, Experimentation, Human Factors.
Sound Design, Location Aware Gaming, Immersion, Engagement.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
Fun and Games 2010, September 15-17, 2010, Leuven, Belgium.
Copyright 2010 ACM 978-1-60558-907-7/10/09…$10.00.
In the last 35 years, game audio has evolved considerably by
moving away from the dependence on midi based wavetable
synthesis bleeps, beeps and simplistic melodies to three
dimensional (3D) sound effects, orchestral soundtracks and
believable dialogue. Console gaming has more processing power
available to synthesise sounds and real-time complex sound
effects, 3D localisation and reverberation – especially in a 5.1
surround sound configuration. Hence there is a trend to replace a
part of recorded audio by real time generated sounds and music.
Therefore, for a composer and sound designer there are many
more avenues available in creating a complex and immersive
soundscape. Such game console examples are Silent Hill [21] and
Thief [24] which present engaging sound designs and an
interactive use of audio.
Immersive gaming soundscapes have mainly been associated with
a static console set up but with the introduction of Personal
Digital Assistants (PDAs) capable of greater processing power
than their predecessors, gaming audio can now move away from
the dependability on midi wavetable synthesis sound to include
richer soundscapes. An example of this is Soul Trapper [22],
which uses spatialisation and a complex audio backdrop to convey
narrative elements of an audio adventure game with some visual
interaction. However, most mobile phone based games still pay
minimal attention to the overall audio design, which are mainly
interested on conveying information and appropriate user
interface sounds.
The increased use of GPS (Global Positioning System) enabled
smartphones, have allowed developers to create real world
location aware gaming where the virtual world can be overlaid
onto the physical world. As audio has developed for console
applications, the challenge is now to incorporate some of these
features in a location aware scenario on the mobile platform that
is not only for navigational purposes but that is immersive,
reactive to the gamers’ movements and that adds to the sense of
presence. Developing a realistic and ambitious sound design on
the limited mobile phone CPU (Central Processing Unit) creates
challenges and requires innovative approaches and ideas. The
emotional and immersive impact of complex sound designs in
location gaming has as yet, not been thoroughly explored.
The aim of this research is to develop a sound design with a
complex and varied soundscape that is immersive, emotionally
engaging and supportive of the game narrative. The goal is to
implement this on the mobile platform with all the technical
constraints that mobile gaming can bring. In location aware
gaming, visual displays are often small therefore other sensory
modalities such as sound can be investigated in regards to player
game experience. As well as the implementation of an augmented
reality game, the sound design will be assessed for its ability to
convey the desired soundscape environment relative to the game
context, the development of player immersion and emotional
engagement. First we shall take a brief look at the background of
location aware gaming on the mobile platform, then describe
Viking Ghost Hunt as our working prototype and the sound design
and subsequent implementation. The user trials undertaken
examined the usability of the game interface and the overall game
experience, which included audio elements. Specifically the
immersive quality of the audio and how this added to the game
experience was investigated. This testing process will be
described together with the results obtained and their significance.
Finally we conclude on the possible role of audio in location
aware gaming and directions for further research.
There has been increasing interest in location aware gaming since
the availability of PDAs that incorporate integrated GPS
capabilities. This integration allows for the overlay of a digital
mediascape onto the physical world that responds to contextual
cues such as GPS. Location aware gaming layers a virtual world
onto the physical world in such a way that physical elements of
the gamers real world can be integrated within the game structure
thereby merging the virtual and physical worlds [5] which in turn
can produce a sense of a place (a space is a physical environment
and a place has greater meaning) [25]. The sound design of a
location aware game can convey important game information such
as navigational sounds and instructional dialogue thereby enabling
the user to look away from the visual graphical interface and to
remain engaged within their physical environment. Listening to
audio requires less cognitive attention than visual information and
therefore reduces the amount of player distraction from the game,
hence encouraging the player to remain in the game space [5].
Research has already shown that the use of a realistic sound
design can help build excitement and tension in the game world
and an enhanced sense of immersion, without the use of visual
graphics [26]. To facilitate user engagement within the virtual
world, a sense of presence (the perception of being in a particular
space) must be created which is interrelated and dependant on
player emotional engagement and immersion. Hence a
multisensory approach of using both visual and audio interfaces is
increasingly being implemented for enhancing immersion and it
has been found that audio is a significant element for multimodal
applications such as location aware gaming [11].
Investigations into using non-speech audio messages (earcons)1
concluded that audio cues were significant for conveying
information and for the development of semantic meaning [12].
This is important as it suggests that audio can provide useful
information about virtual environments and support perceptual
integration. In regards to location aware applications, most audio
investigation has focused on non-gaming applications, artistic
installations or navigational tools for the visually impaired.
Research into spatialised sound in a location aware application
1 Sounds used to convey information about computer systems.
was undertaken by Dr. Kirsten Cater and Dr. Richard Hull et al
[3], with initial results indicating that the soundscape of a PDA
application can help users navigate in a physical environment by
conveying relevant information through audio. In their
application, a 2-axis compass, Bluetooth GPS and HP iPAQ was
used. Sound installations such as The Tactical Sound Garden’
[23] explore the concept of overlaying soundscapes onto a
specific urban space. In this example, wireless (WiFi) ‘hot zones’
were utilised to install a ‘sound garden’ for public use which used
an element of audio for navigation and spatialisation. Demor [7]
is a location aware 3D audio first person shooter game primarily
designed for the blind but which sighted players can also take
part. This game investigated psychoacoustic properties for the
presentation of a 3D soundscape that would be accurately
presented and used by players for real physical space navigation.
In this example, a laptop, GPS tracker and headphones with a
head-tracking device together with a joystick is used. These
applications mainly focus on sound as navigation and not on
potential immersive or experiential properties. Additionally, the
applications were not based on the mobile platform and often
involved complex technical set-ups.
An example of location aware audio in gaming is ‘The Songs of
North’ [8]. This application investigates the use of audio for
navigation and in the conveying of game information with the
research containing some reference to the possible role of
immersion in the game experience. This location aware mixed
reality game features audio as an important aspect, however it
must be noted that headphones were not used in this example.
The application was also hampered by challenging technical
constraints in the presentation of an audio soundscape due to
memory and processor limitations.
There are many more examples of the use of audio in location
aware gaming, however most research has focused on audio as a
function of navigation and in the conveying of game information
rather than its emotional and immersive effect. There is currently
research on console game audio [9], however game audio mobility
and its implications has not been thoroughly dealt with.
Viking Ghost Hunt (VGH) is implemented on the Android
platform using the HTC T-Mobile G1 and is a working prototype
for a location aware game in Dublin city centre, Ireland. The
game is narrative driven and played around the old Viking sites of
Dublin with the player acting as a paranormal investigator. The
gamer uses various visual and audio interfaces together with
stereo headphones to locate and interact with the ghosts. The VGH
aim is to immerse the player in the game world which is merged
with the physical world, with the PDA acting as a paranormal
investigative device. The aim of the audio design was to create a
soundscape that is both informative of the game environment and
that is immersive and emotionally engaging. Sound also supports
the contextual element of VGH to create a believable space and to
blur the real and virtual world borders. The aim of the game is to
locate paranormal activity and gather evidence that manifests
visually or aurally. This evidence may give information about the
ghostly activity and clues for moving through the game space.
The paranormal device (mobile phone) has different modes or
interfaces that can be used to locate paranormal manifestations in
the real world space and to capture audio-visual evidence. The
modes of paranormal investigation consist of camera/x-ray mode,
Figure 1. Viking Ghost Hunt prototype
map mode, radar mode and frequency scanner mode. The
frequency scanner mode is an audio interface reminiscent of
Electromagnetic Voice Phenomenon (EVP) often used by
paranormal investigators. Paranormal investigators believe that
‘ghostly voices’ can be heard when radio static is analysed.
Gamers must use the frequency scanner in order to find the
correct ghost frequency (Hertz), record the white noise audio and
subsequently hear the decoded ghostly message on playback.
This is an integral part in the game mechanics for player
interaction, game information, evidence retrieval and in the
delivery of the game narrative.
Figure 2. Radar mode
In addition to the frequency scanner, audio is also used for all of
the user interface sounds in order to support the concept that the
mobile phone is a paranormal investigative device. Automated
ambient audio and sound effects are triggered when in proximity
to GPS locations or regions to assist the player in navigating to the
correct locations and for immersion and engagement in the game
The aim of the sound design is to create an immersive experience
in which the player remains engaged with their physical
environment. Therefore a balance between the ghostly
atmosphere of the game and the location environment was
maintained. Hence, sounds used are representative of both the
location environment and game atmosphere. The sound samples
used for VGH were stereo field recordings, sourced samples or
created electronically using a midi synthesizer and audio
sequencer. The required sounds were placed into four categories:
paranormal, environmental, musical and user interface sounds.
The table below shows some of the sounds used.
Table 1. Sound Categories
User Interface
high pitch
battle sounds
animal sounds
church bells
children playing
school bell
paper flying
dog barking
wind in trees
people talking
minor scales
pitched and
ghostly pan-
old AM radio
white noise
audio static and
metal detector
dial clicks
Geiger metre
button sounds
sonar sound
compass sounds
From a design perspective, audio is used similarly as in the genre
of film and divided into background sound, sound effects and
dialogue with the addition of user interface elements. The
background consists of the creation of an atmospheric soundscape
that represents the game space and the physical location of the
given narrative. This background sound sets the mood of the
location and virtual game space by using randomised, overlapping
looped audio files in order to avoid habituation and hence player
disinterest. Sound effects are then added that are relevant to the
location and game narrative together with appropriate dialogue,
which can be accessed using the frequency scanner mode.
The sound is triggered by creating physical zones in the form of
concentric rings organised around a GPS location with varying
radial distances. This enables different sound files to be triggered
due to ongoing GPS update information provided by player
movements. For example, as the player comes within a
paranormal zone a looped drone sound (rumbling, low frequency)
and some environmental sounds (e.g. wind) begin. The mixing of
environmental and game sound supports the blending of the
virtual and real world to such an extent that the player may
believe some sounds to be part of the physical locations. As the
player continues to move towards the central point of activity,
sound files will start that are related to the paranormal entity.
These sounds are descriptive of the game narrative and historical
location of the game and consist of short sound effects (e.g. chains
etc) that may be randomly triggered or looped. Hence players can
obtain information related to the game space without looking at
the mobile device which enables them to remain engaged in the
physical space.
Table 2. Example of sound design for physical locations
Stairway to
going down
into a tunnel)
drone using
As the player reaches the GPS defined location, an escalation of
sounds due to increasing amplitude and sound file additions
signals to the player that they are entering a paranormal zone.
The visual interface indicates whether the paranormal
manifestation is visual or aural and the player must choose the
appropriate mode to gather the evidence. As the player leaves the
location after obtaining the evidence, these sounds gradually
dissipate in intensity and fade away. Therefore different sounds
enter and remain for different lengths of time depending on the
location of the player in relation to the GPS point, hence
providing an interactive, immersive experience. The ambient
audio soundscape is always triggered automatically by player
proximity to physical game locations.
Within the sound design the audio can also be used for
navigational support by providing verbal instructions in the
frequency scanner mode dialogue and by triggering sounds near
paranormal location that increase in volume and complexity as the
player moves closer to the location. This navigational strategy
also includes the addition of various sound effects related to the
paranormal activity that are triggered as the player moves toward
the physical location, again informing the player that they are
entering a paranormal zone. Constant background sound between
paranormal locations is avoided so as not to overload the user
experience and take away from the immersive experience.
In order to design and implement a complex and engaging
soundscape, properties of sound must be explored and integrated
in order to develop an engaging and realistic audio design. A
somewhat ‘natural’ audio presentation that reflects real world
sound has been found to be more immersive and engaging for
augmented reality applications [14]. The psychoacoustics of
sound in the real world in relation to spatialisation and
reverberation is an important factor in creating the sound design
for VGH as well as the creative composition. Sound is presented
in a three dimensional manner (3D) in the environment and
human survival depends on the ability to establish a sense of
spatial orientation. This is needed for localisation, distance
perception and recognition [1] of objects, and spatialisation in an
artificial soundscape can mimic how sound is heard in the real
world. In order to perceive from where a sound source is
originating, the human brain uses information derived from the
interaction of the sound wave with that of the torso, shoulders,
head and ears. Interaural time differences (ITD) and interaural
level differences (ILD) of sound as it interacts with the head help
to localise sound in space. For example, if the sound were located
to the right, the right ear would hear the sound first and then the
left with a slight time delay (ITD). The amplitude (volume) level
would also be greater on the right ear than the left (ILD) and
therefore the sound is perceived to originate from the right [10].
ITD is used to differentiate the location of lower frequencies
while ILD is used to differentiate the location of higher
frequencies. Perceptually humans use a combination of these
systems, however ITD and ILD do not work as well for sound
originating on the azimuth axis (vertical plane) and therefore can
be poor for front and back sound localisation. For a true 3D
representation of audio the use of Head Related Transfer
Functions (HRTFs) must be implemented to give a realistic
spatialisation of sound as it takes into account the accurate
individual interaction of the user’s head with the sound wave [11].
By using HRTFs individually a player sometimes cannot tell
virtually spatialised sound from the real thing.
Figure 3. ILD, ITD and HTRF [14]
However the use of HRTFs is not realistic for a mobile phone
(PDA) application as it requires real-time processing of audio
filters which is CPU intensive and unachievable at this level for
PDAs at this time. Therefore in order to spatialise sound for the
VGH prototype, a stereo representation of sound was found to be
adequate with the use of ILD and ITD parameters in either a pre-
rendered format or by real-time live processing of audio files,
CPU permitting. For the pre-rendered formats, sound files were
spatialised using a binaural simulator (software designed to
imitate realistic spatialisation) or panned by changing right and
left volume levels (ILD) in relation to the virtually placed source.
Therefore sounds such as whispers or dialogue can be perceived
as if to be moving around and in and away from the player.
Spatialised audio can be integral to navigate through a virtual
world and create realism and facilitate immersion. Immersion is
an important aspect of a location aware game as the user’s full
attention can be captured by the experience and research as shown
that spatialised sound can increase immersion [14].
Reverberation is another important psychoacoustic consideration
when developing the sound design. Reverberation is sound that is
reflected from physical surfaces and back into the environment.
Numerous ‘early’ reflections are then themselves reflected and
form an ‘ambient’ sound field [18] and these sound reflections
have been found to contribute to the sense of a space and the
perception of a sound source size and its distance [2]. The
addition of realistic artificial reverberation in the sound design of
a location aware game can add a sense of realism and
envelopment (sense of immersivity in a reverberant sound field)
for the user. Hence the use of reverberation was an important
stylistic element in creating a ghostly atmosphere as research as
found that reverberant ambient sound can add to immersion and
player engagement [14]. In the prototype, audio files were
processed using different types of reverberation settings which
were dependant on the context of the game locations.
In order to create a dense and engaging game atmosphere, the
sound design involved the playback of a complex configuration of
multiple and varied simultaneous audio files that were created
with keeping in mind psychoacoustic principles, the blending of
the physical and virtual worlds and the game narrative. Hence the
sound design was delivered with the use of closed supra-aural
headphones (to allow some external environmental sound to be
heard) without the use of a head-tracking device. Therefore it was
suggested to players that they face the direction of the device to
ensure proper delivery of spatialised audio events.
5.1 Method of evaluating the sound design
Objective findings such as physiological changes are difficult to
measure for emotional response and immersivity to sound and
music [17] as it does not take into account cognitive aspects.
Therefore for a phenomenological experience (experienced from
the first-person point of view), subjective reports were used for
the assessments. The location aware prototype, VGH was
evaluated over a period of three days. The 19 participants were
each provided with a G1 HTC mobile phone and headphones.
Participant demographics included male and female volunteers
with ages ranging from 20 years to 45 years with a mixture of
previous game experience to little or no game experience.
Volunteers were not briefed regarding the importance and
evaluation goals of the sound design. Game feedback focused on
the prototype user interface usability and game experience and
took the form of a questionnaire, which included certain questions
focusing on the effect and role of the sound design for emotional
response and immersion.
The questionnaire consisted of open-ended questions (e.g. “Did
you feel more or less engaged at different stages of the game?
Which part of the game was immersive and why?” and “Which
features of the interface most helped guide you to the places
where the ghosts were located? Please describe) which were
designed specifically not to lead the volunteers to comment on the
audio unless it had a significant impact on the game experience.
It also consisted of a set of statements in which 9 were specifically
related to audio with players responding by using a 5-item scale
(strongly disagree to strongly agree). Audio statements included:
“The sound made the game feel scary”, “I feel that the sound was
reactive to my movements” and “The audio was seamless and felt
a real part of the game”.
5.2 Results of the game audio experience
Generally, the overall game experience was well received with
70% of participants responding that they enjoyed the game.
An important question in regards to sound was in participant
expectation of the role of audio in augmented reality gaming. It
was found that the respondents expected the role of audio to be
roughly equal in the form of narrative and sound effects.
Figure 4. Audio expectations in augmented reality gaming
Overall, the participants responded positively (71%) to the sound
design, with comments such as:
“Addition of audio greatly increased the atmosphere and
“I felt immersed in the game”
“It was a very immersive experience. A couple of times I found
myself not realising that I am on a street and there are people
around me.”
“At the moment it seems like the atmosphere is the unifying
element (also one of the strongest aspects of the game).”
This positive feedback regarding the sound design was an
encouraging outcome.
Figure 5. Summary of audio feedback
The overall response to the role of audio for game immersion was
high with 70% of players agreeing or strongly agreeing to the
immersion statements. By creating an engaging and complex
soundscape and dialogue taking into account psychoacoustic
considerations, participants felt involved in the game and in their
role as a paranormal investigator. This engagement and
immersion adds to the possible development of an emotional
response felt or perceived within a game space and a sense of
game presence. When asked whether the game audio contributed
to the game feeling scary, 63% of participants responded in
agreement or strong agreement that the “sound effects created a
scary atmosphere.”
Additionally, as well as investigating audio for immersion and
emotional engagement, user interface usability was also
investigated. For audio, this included an element of interactivity
between the player, physical locations and the device user
interface. As discussed previously, audio reactivity to player
movement was achieved by volume changes and the addition and
deletion of sound files triggered by varying distances from given
GPS locations, together with the implementation of realistic
paranormal investigative device sounds. Congruency between
audio changes and player movement can support player
involvement and therefore immersion, as the device and player
actions are perceived to be coupled and realistic. In multi-modal
presentations, congruency is an important factor in reinforcing a
natural mode of interaction [27]. That is, the game world should
reflect the experiences of real world interactions which increases
the likelihood that there is a wiling suspension of belief and hence
immersion [4]. Participants were asked whether they found the
sound reactive to the physical environment and if they thought the
user interface sounds complemented the visual interfaces and
paranormal device sounds. In response, 84% of testers agreed or
strongly agreed that the user interface sounds supported the role-
play element of the game play and 68% felt that the game sound
was reactive which supports the blending of the virtual world
and physical one:
“The backing audio changing as I moved location was a good
“Loved the sound – that was new for mobile gaming location
Additionally, 79% of testers felt that the audio supported the game
environment in the physical locations.
In creating a complex sound design with many layers and types of
sounds playing simultaneously, care must be taken in the
presentation of the audio files. This was evident in some of the
feedback as sound effects were at times found to overpower the
narrative dialogue, which is essential to the game play. Hence a
balance must be struck between important audio game
information and ambient background or sound effects:
Felt a little scary, however the first time the ghost spoke I had
difficulty understanding what he said over the sound effects.
Though the sound was quite good.”
This brings to light another challenge for location-based audio
external noise and busy environments. Even though in this
example most testers did not feel distracted or interrupted by
external noise this might have been due to the fact that a quiet
location was sourced as a testing ground. Therefore augmented
reality game locations must be carefully sourced:
“Stopping in the middle of a path was distracting as I felt I was in
the way [of people]….isolated paths and lanes were more
atmospheric and I felt more immersed.”
Generally the results obtained from the user trials were extremely
positive in regards to the sound design aim of trying to emulate
console gaming audio and to be immersive and potentially
emotive. As one tester noted:
“The sound – that was new and reminded me of Silent Hill on PS2
(brilliant because of sound).”
Therefore mobile phone gaming in a location aware application
has the potential to be immersive and engaging even when faced
with the challenges of slower processing speeds and limited
memory space when compared to its static console counterpart.
In designing, implementing and testing the ambitious soundscape
design for Viking Ghost Hunt, it was noticed that there are two
main challenges when developing a location aware game that
impacted on all aspects of the prototype GPS inaccuracy,
unreliability and mobile device technical constraints. During the
user trials, GPS updates at certain times either did not occur or
were inaccurate in determining the player location. This produced
graphical user interface anomalies and resulted in audio files not
being triggered at the intended locations. It was obviously a
distraction from the game play and affected enjoyment and
The other main issue for audio in augmented reality applications
is the processing speed of the device and the memory available.
In the Viking Ghost Hunt prototype, the game application is stored
on the mobiles’ internal memory with the game assets residing on
the SD (Secure Digital Memory Card) card. Very little processing
power is available for any real-time audio manipulation. In order
for audio to be truly reactive and to accurately represent sound
spatially, live processing of sound in regards to GPS and compass
positions in a physical location, must be possible. An example of
this presentation is Demor [7], which uses real-time processing of
spatialised sound to create a truly 3D audio virtual space. Players
can accurately hear sounds emanating from the left and right in
relation to their head positions and physical location. Players can
also judge if a sound is located at a distance or placed close by. In
this application, audio is reactive to the players’ GPS locations
with audio files being adjusted accordingly by live processing on
a specially written audio-engine.)The audio engine designed for
Demor [7] would most likely have used a non-individualised
HRTF (Head Related Transfer Function) database for the 3D
representation, involving large amounts of data processing.
Simplified psychoacoustical considerations have been
implemented in the VGH prototype however this has centered on
including most of the spatialised sound in a pre-rendered format.
Some location reactive left and right audio panning has been
designed and implemented that is able to represent virtual sound
in a static physical location by using GPS co-ordinates and
internal phone compass sensors. However, the location and
direction sensitive audio panning is currently device dependant
and has not been fully tested. Player immersion and engagement
could be improved in location aware gaming with a more accurate
presentation of audio on the mobile platform that included a non-
invasive head-tracking compass.
Careful consideration must also be taken when choosing locations
for location aware applications. Such issues as noise levels,
environmental player distractions and safety must be taken into
account. Certain locations would be more immersive than others
and combing this with narrative relevance would require thorough
investigation before game development and implementation.
Safety for gamers when wearing headphones can be an issue,
especially if the audio is particularly immersive as high levels of
audio immersion may take away from the intended physical
location creating a sense of disengagement and pose problems in
regards to traffic or unwanted attention.
Creating a sound design that intends to bring the sonic richness of
console gaming into location aware applications requires the
consideration of many factors outside the realm of audio. The
combination of technical aspects, location, game narrative and
mechanics all add to the game experience. Sound design is not
simply a process of developing impressive quality audio samples
but instead requires a sensitive compositional approach, as
excellent sound quality does not always equate to immersion.
In this paper we have presented the sound design, implementation
and evaluation of our working prototype Viking Ghost Hunt. The
design demonstrated that complex audio similar to that of popular
console games can to some extent, be implemented on a mobile
device, even with the challenging technological constraints. This
research highlights the importance of audio in location aware
gaming for immersion and engagement, and hence for the overall
game experience. Audio in location aware gaming is an area still
not thoroughly investigated in regards to emotional impact and
immersiveness and the role that this could potentially play in this
genre. Also, research has not delved into the area of representing
audio in a psychoacoustically correct way that would increase
perception of sound naturalness and hence increase player
engagement. Further research into these areas together with a
possible subjective and objective investigation of player
emotional and physiological reactions could be undertaken.
Location aware games can provide the user with more
understanding and knowledge of their environment through both
the audio and visual mediums. It is the hope of this investigation
to present the importance of sound for immersion and engagement
and that it may inform future applications for this increasingly
popular genre.
We’d like to thank the rest of the VGH team; Alan Duggan,
Daniel Crowley, Tina Hedayet and Roisin Cotton for their input
and knowledge in the development and evaluation of the location
aware prototype. We’d also like to thank the National Digital
Research Centre in Dublin Ireland, for their continued generous
support of the project.
[1] Ashmead, D.H., Hill, E.W., & Talor, C.R. Obstacle
perception by congenitally blind children. Perception and
Psychophysics, 46(5) (1989), 425–433.
[2] Begault, D. 3D Sound for Virtual Reality and Multimedia.
Academic Press Inc, 1994.
[3] Cater, K., Hull, R., Melamed, E., Hutchings, R. An
Investigation into the use of Spatialised Sound in Locative
Games. CHI 2007, April 28 - May 3, San Jose, USA.
[4] Coleridge, S. T. The Collected Works of Samuel Taylor
Coleridge: Biographia Literaria, edited by James Engell and
W Jackson Bate, Princeton university Press, Princeton, New
Jersey, USA, 1983.
[5] Dede, C. Immersive interfaces for engagement and learning.
Science, 323 (5910) (2009), 66–69.
[6] De Souza e Silva, A. Location Based Games: Blurring the
borders between physical and virtual spaces, ISEA (2004).
[7] Demor –
[8] Ekman, I., Ermi, L., Jussi, L., Nummela, J., Lankoski, P.,
Mäyrä, F. Designing Sound for a Pervasive Mobile Game,
ACE 2005.
[9] Collins, K. Game Sound: An Introduction to the History,
Theory and Practice of Video Game Music and Sound
Design. MIT Press, USA, 2008.
[10] Gardner, W.G. Spatial Audio Reproduction: Toward
Individualized Binaural Sound. The Bridge, V 34 (2004).
[11] Gardner, W.G., & Martin. K. D. HRTF measurements of a
KEMAR. Journal of the Acoustical Society of America, 97
(6) (1995), 3907 – 3908.
[12] Gaver, W. W., Smith, R. B., O’Shea, T. Effective Sounds in
Complex Systems: The Arkola Simulation. Proceedings of
the SIGCHI Conference on Human Factors in computing
systems, p. 85 – 90, 1991.
[13] Hendrix, C. & Barfield, W. The sense of presence within
auditory virtual environments. Presence: Teleoperators and
Virtual Environments, 5 (1996), 274– 289.
[14] Hollier, M. P., Rimell, A. N., Burraston, D. Spatial audio
technology for telepresence. BT Technology Journal, Vol 15
(1997) No 4.
[15] Keller, P. & Stevens, C. Meaning from environmental
sounds: Types of Signal-Referent Relations and their effect
of recognizing Auditory Icons. Journal of Experimental
Psychology: Applied, 10, 1 (2004), 3-12.
[16] Maynes, C. Worldizing – take studio recordings to the field
and make them sound organic. The Editors Guild Magazine
Vol. 25 (2004), No.2.
[17] Meyer, L., Emotion and Meaning in Music. University of
Chicago Press, Chicago, USA, 1956.
[18] Rumsey, F. Spatial Audio. Focal Press, Oxford, UK, 2001.
[19] Sanchez, J., & Lumbreras, M. 3D aural interactive
hyperstories for blind children. International Journal of
Virtual Reality, 4(1) (1998), 20–28.
[20] Sheridan T.B. Musings on Telepresence and Virtual
Presence. Presence: Teleoperators and Virtual Environments,
1 (1) (1992), 120-125.
[21] Silent Hill
[22] Soul Trapper
[23] Tactical Sound Garden
[24] Thief
[25] Nisi, V., Oakley, I., Haahr, M. Location-Aware Multimedia
Stories: Turning Spaces into Places. ARTECH 2008.
[26] Valente, L., Sieckenius de Souza, C., Feijo, B. An
exploratory study on non-visual mobile phone interfaces for
games. IHC 2008.
[27] Witmer, B.G., Singer, M.J., Measuring Presence in Virtual
Environments: A Presence Questionnaire. Presence, Vol. 7
(1998), No. 3, 225-240.
... Another study suggests that the visual display bears some weaknesses that the audio modality can overcome, such as limited screen space, overload of information, vulnerability to sunlight, and the necessity for constant attention [35]. Furthermore, many experiments argue that augmentation through data sonification and sonic interaction design techniques can significantly enhance players' immersion level in the augmented environment [29,36] and increase their emotional engagement with the virtual world [37,38]. It has also been observed that in some cases, users could not distinguish the real from the virtual component [39]. ...
... The auditory culture of space itself should be considered more thoroughly [27,41]. Last, the technical requirements for an efficient ARA experience should be further investigated, including spatialization techniques, headset usage, GPS and other sensors' accuracy, and computation power [29,38,[42][43][44]. ...
... Other projects have added more complex modes of audio interaction to just navigating to the sound. In "Viking Ghost Hunt", once players enter the closest proximity zone of a virtual sound, they use the game's visual interface to record the audio sample [38]. In "Eidola", players have to locate the virtual sounds and then press a button on the display once standing at their position [1], whereas in the multi-player version of the same game, the movement of the virtual components is controlled by the movement of other players in a different room [53]. ...
Full-text available
Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express their content and mechanics. Existing ARAG implementations have focused on exploring the surroundings and navigating to virtual sound sources as the main mode of interaction. This paper suggests that gestural activity with a handheld device can realize complex modes of sonic interaction in the augmented environment, resulting in an enhanced immersive game experience. The ARAG “Audio Legends” was designed and tested to evaluate the usability and immersion of a system featuring an exploration phase based on auditory navigation, as well as an action phase, in which players aim at virtual sonic targets and wave the device to hit them or hold the device to block them. The results of the experiment provide evidence that players are easily accustomed to auditory navigation and that gestural sonic interaction is perceived as difficult, yet this does not affect negatively the system’s usability and players’ immersion. Findings also include indications that elements, such as sound design, the synchronization of sound and gesture, the fidelity of audio augmentation, and environmental conditions, also affect significantly the game experience, whereas background factors, such as age, sex, and game or music experience, do not have any critical impact.
... Accordingly, in recent years, the industry has seen an increase in technical possibilities concerning sound and graphical aspects (Gallagher & Park, 2002;Schilling, 2003). Auditory features went from bare bleeps to simple melodies resulting in 3D soundtracks recorded by actual orchestras (Paterson et al., 2010). With regard to innovation of graphical features of games, there have been around five different stages of technical innovations within the video game industry (Gallagher & Park, 2002). ...
... Every few years new generations of gaming platforms, computers and consoles alike, appear on the market with enhanced technical capabilities (Cadin et al., 2006;Gallagher & Park, 2002;Marchand & Hennig-Thurau, 2013). As the graphical and auditory elements of presentation are steadily increasing in their performance (Paterson et al., 2010;Situmeang et al., 2016), good presentation has only a positive impact on short-term success because in the long-term perspective it is already outdated and replaced by new technical possibilities after several months. For example, the development of deeper resolution lead from the introduction of HD-standard to UltraHD in only a few years (ITU, 2012), which is currently the best high-resolution standard in the entertainment industry. ...
Full-text available
In recent years, the video game industry has invested significant amounts in the development of innovative elements for its products. However, it is still subject to debate whether these R&D investments always pay off. While prior research has confirmed that enhancing product innovativeness increases business performance in certain industries, corresponding findings for the video game market are lacking. This might be a result of missing theoretical conceptualizations and adequate empirical operationalizations of game innovativeness. Addressing this research gap, this study provides the first conceptualization and operationalization of game innovativeness, shedding first light on its performance effects. Based on longitudinal data on 351 computer games, our findings confirm that innovations in the game's presentation and principle enhance short-term success, whereas innovations in a game's storyline can be more of a hindrance than a godsend for companies. However, our results also show that performance effects of game innovativeness diminish over time. Supplementary information: The online version contains supplementary material available at 10.1007/s12525-022-00521-7.
... A small number of research groups have focused studies on the design of augmented audio systems that combine real scenery with artificial audio environments (Cohen et al., 2004;Lyons et al., 2000;Paterson et al., 2010). At the Consumer Electronic Show (CES) in 2015, researchers from Intel presented a wearable device that used RealSense technology. ...
... In the exergame Sonic-Badminton (Kim et al., 2016), designers modified a typical badminton racket and shuttlecock with wireless sensors so the device provides players with information about position and distance.Research teams have also experimented with a player's physical movement and position by creating prototypes of 'real walking' or location-based audio games. These games use the Global Positioning System (GPS)(Chatzidimitris et al., 2016;Paterson et al., 2010;Velleman et al., 2004) or indoor positioning systems (virtual reality)(Allain et al., 2015;Andrade et al., 2019;Moustakas et al., 2009;Podkosova et al., 2016). ...
Full-text available
According to the World Health Organization (2019), an estimated 217 million people worldwide are visually impaired and 36 million are blind. Although there are approximately 114,000 video games currently in active circulation (MobyGames, 2020), just over 700 of these are accessible to the visually impaired (, n.d.-a). This practice-oriented research project investigates the potential of audio games through the design and development of the Audio Game Hub and Blind Cricket. The games were created through iterative cycles of prototyping and public releases and stimulated and refined through the agency of voluntary user feedback. They were released on iOS and Android platforms and over a period of two years were downloaded over 130,000 times. They gathered insightful user reviews and won multiple nominations and awards. The project was presented at several conferences and featured on television and the Internet. The research was activated by a form of agency I define as an Indie Designer/Developer. Here, one is an integrated agent who develops work through critical reflection from online reviews, relying heavily on the implementation of tacit knowing (Polanyi, 1967; Schön, 1984). As a ‘generalist’, the Indie Designer/Developer combines the role of researcher, designer, reflective practitioner, developer, publisher and entrepreneur.
... Τα εικονικά ηχητικά ερεθίσματα έχει αποδειχτεί πως συμβάλλουν θετικά στην ικανότητα προσανατολισμού (Stahl, 2007;McGookin, Brewster & Priego, 2009) και στον εντοπισμό χωροθετημένων πληροφοριών (Sodnik et al., 2006) απευθυνόμενα εξίσου σε χρήστες διαφορετικών βαθμών εξοικείωσης με ακουστικές και μουσικές έννοιες (Holland, Morse & Gedenryd, 2002). Παράλληλα, η αίσθηση της ανακάλυψης προάγει την ευχαρίστηση, τη συναισθηματική σύνδεση και την εμβύθιση των χρηστών ως προς τη δραστηριότητα στην οποία συμμετέχουν στο επαυξημένο περιβάλλον (Reid, et al., 2005;Paterson et al., 2010;Vazquez-Alvarez, Oakley & Brewster, 2012;Chatzidimitris, Gavalas & Michael, 2016), επιδράσεις από τις οποίες μπορεί να ωφεληθεί ιδιαίτερα η εκπαιδευτική διαδικασία. ...
Conference Paper
Full-text available
Το πεδίο της Επαυξημένης Πραγματικότητας αφορά τεχνολογίες, οι οποίες εμπλουτίζουν τον φυσικό κόσμο με εικονικές πληροφορίες και παρέχουν στους χρήστες δυνατότητες αλληλεπίδρασης με το περιβάλλον τους που δεν θα ήταν εφικτές μέσω της άμεσης αισθητηριακής αντίληψης. Εξαιτίας των εγγενών αυτών χαρακτηριστικών, η Επαυξημένη Πραγματικότητα χρησιμοποιείται όλο και περισσότερο σε τομείς, όπως η εκπαίδευση, η μουσειακή και γενικότερα πολιτιστική περιήγηση, η προσβασιμότητα, η θεραπεία, και η ψυχαγωγία, γεγονός που τονίζει την ανάγκη εξοικείωσης μαθητών και σπουδαστών με το νέο αυτό μέσο, ώστε να αντεπεξέλθουν στις σύγχρονες και μελλοντικές προκλήσεις σε κοινωνικό και εργασιακό επίπεδο. Η αξιοποίηση ηχητικών ερεθισμάτων ως στοιχείων επαύξησης και η ως εκ τούτου πραγμάτωση της επαυξημένης αλληλεπίδρασης μέσω του ακουστικού καναλιού συνιστούν το πεδίο της Ακουστικώς Επαυξημένης Πραγματικότητας. Το Εργαστήριο Επεξεργασίας Οπτικοακουστικών Σημάτων (ΕΠ.ΟΑ.ΣΗ) του Τμήματος Τεχνών ‘Ήχου και Εικόνας του Ιονίου Πανεπιστημίου δραστηριοποιείται τα τελευταία χρόνια στον σχεδιασμό και την ανάπτυξη ακουστικώς επαυξημένων ηχητικών παιχνιδιών με σκοπό αφενός τη διερεύνηση των τρόπων αλληλεπίδρασης και εμβύθισης σε ένα ακουστικώς επαυξημένο παιγνιακό περιβάλλον και αφετέρου την εφαρμογή ακουστικώς επαυξημένων τεχνικών για την επιμόρφωση και ευαισθητοποίηση των χρηστών σχετικά με γνωστικά αντικείμενα που άπτονται της εκπαίδευσης και του πολιτισμού. Η παρούσα εισήγηση επιχειρεί να αναδείξει τις δυνατότητες αξιοποίησης ακουστικώς επαυξημένων ηχητικών παιχνιδιών στην εκπαίδευση μέσα από το πρίσμα των πεδίων της Ακουστικής Επαύξησης και της Πολιτιστικής Περιήγησης. Έπειτα, παρουσιάζει το ακουστικώς επαυξημένο ηχητικό παιχνίδι «Audio Legends», περιγράφοντας τον επιμορφωτικό σχετικά με την πολιτιστική κληρονομιά της Κέρκυρας χαρακτήρα του παιχνιδιού, τη σύνθεση του σεναρίου και των εκπορευόμενων παιγνιακών μηχανικών, τις τεχνικές σχεδιαστικές προδιαγραφές, την πειραματική δοκιμή του και τα συμπεράσματα που αντλήθηκαν σχετικά με την αποδοχή και εμβυθιστικότητα του νέου αυτού μέσου.
... An infrared transmitter detects the visitor's location, which signals a computer to play or stop playing a pre-recorded description audio message. Paterson et al. [135] stressed that immersion, besides interaction, involves the "creation of presence", which is the feeling of being in a particular place. They have created a location-aware augmented reality game where the user navigates a field looking for ghosts. ...
Full-text available
“Interaction” represents a critical term in the augmented and mixed reality ecosystem. Today, in mixed reality environments and applications, interaction occupies the joint space between any combination of humans, physical environment, and computers. Although interaction methods and techniques have been extensively examined in recent decades in the field of human-computer interaction, they still should be reidentified in the context of immersive realities. The latest technological advancements in sensors, processing power and technologies, including the internet of things and the 5G GSM network, led to innovative and advanced input methods and enforced computer environmental perception. For example, ubiquitous sensors under a high-speed GSM network may enhance mobile users’ interactions with physical or virtual objects. As technological advancements emerge, researchers create umbrella terms to define their work, such as multimodal, tangible, and collaborative interactions. However, although they serve their purpose, various naming trends overlap in terminology, diverge in definitions, and lack modality and conceptual framework classifications. This paper presents a modality-based interaction-oriented diagram for researchers to position their work and defines taxonomy ground rules to expand and adjust this diagram when novel interaction approaches emerge.
... As well as the voice, there has been an increased interest in the impact of in-game audio. Paterson et al. developed an audio design with a complex and immersive soundscape, which is emotionally engaging and supports the game narrative [45]. Similarly, Roden et al. proposed a framework for interactive narrativebased audio only adventure games [47], and Sliwinski et al. explored the development of an audio-visual game to induce wellbeing and mindfulness [56] Similarly, Rogers et al. discussed games which are considered relaxing and encourages research directions for exploring the role of game audio specifically, to improve player wellbeing, via stress reduction [48]. ...
Our daily interaction with the soundscape is in flux, and complex natural sound combinations have shown to have adverse implications on user experience. A computational approach to stabilise the sonic environment, tailored to a user’s current affective state may prove beneficial in a variety of scenarios, including workplace efficiency, and exercise. Herein, we present initial perception test results, from a rudimentary approach for soundscape augmentation utilising chromatic feature sonification. Results show that arousal and valance dimensions of emotion can be altered through augmentation of three classes of natural soundscape, namely ‘mechanical’, ‘nature’, and ‘human’. Proceeding this we outline a possible approach for an affective audio-based recognition and generation system, in which users (either individually or as a group within a specific environment) are provided with an augmentation of their current soundscape, as a means of improving wellbeing.
... Beyond DMIs, the role of the music and audio in video game interactive experiences has been evaluated in Paterson et al. [286] and Prechtl [53] by open or direct, Likertscale questions regarding the effect that the music or audio had on the overall experience of participants. For the evaluation of FATRA open questions or Likert-type questions intended to capture information relevant to the impact of the generated music on the overall EPE. ...
Conference Paper
Music is used to enhance the experience of participants and visitors in a range of settings including theatre, film, video games, installations and theme parks. These experiences may be interactive, contrastingly episodic and with variable duration. Hence, the musical accompaniment needs to be dynamic and to transition between contrasting music passages. In these contexts, computer generation of music may be necessary for practical reasons including distribution and cost. Automated and dynamic composition algorithms exist but are not well-suited to a highly interactive episodic context owing to transition-related problems including discontinuity, abruptness, extended repetitiveness and lack of musical granularity and musical form. Addressing these problems requires algorithms capable of reacting to participant behaviour and episodic change in order to generate formic music that is continuous and coherent during transitions. This thesis presents the Form-Aware Transitioning and Recovering Algorithm (FATRA) for realtime, adaptive, form-aware music generation to provide continuous musical accompaniment in episodic context. FATRA combines stochastic grammar adaptation and grammar merging in real time. The Form-Aware Transition Engine (FATE) implementation of FATRA estimates the time-occurrence of upcoming narrative transitions and generates a harmonic sequence as narrative accompaniment with a focus on coherent, form-aware music transitioning between music passages of contrasting character. Using FATE, FATRA has been evaluated in three perceptual user studies: An audioaugmented real museum experience, a computer-simulated museum experience and a music-focused online study detached from narrative. Music transitions of FATRA were benchmarked against common approaches of the video game industry, i.e. crossfading and direct transitions. The participants were overall content with the music of FATE during their experience. Transitions of FATE were significantly favoured against the crossfading benchmark and competitive against the direct transitions benchmark, without statistical significance for the latter comparison. In addition, technical evaluation demonstrated capabilities of FATRA including form generation, repetitiveness avoidance and style/form recovery in case of falsely predicted narrative transitions. Technical results along with perceptual preference and competitiveness against the benchmark approaches are deemed as positive and the structural advantages of FATRA, including form-aware transitioning, carry considerable potential for future research.
... A third type of locative media for cultural heritage are actual locative games, such as REXplorer (Ballagas et al. 2008) and Viking Ghost Hunt Paterson et al. 2010), and also the Haunted Planet games discussed later in this article. Like modern games, which use techniques such as environmental storytelling that allow a player to explore a storyworld and piece the snippets together themselves, locative games for cultural heritage can present a narrative as a puzzle to be explored, or solved to different degrees, depending on the players' preferences. ...
Locative gaming dates back to the early 2000s, and with the success of Ingress (2012) and Pokémon GO (2016), locative games have now entered the mainstream in a very serious way. However, while the genre holds considerable promise for cultural heritage, it has yet to make a real impact for this purpose. A particular challenge is to reconcile the two apparently conflicting concerns of ensuring immersion into the experience without compromising the audience’s sense of presence in the physical space. For example, Ingress and Pokémon GO offer excellent immersion in the gameplay but at the cost of near-total loss of the player’s sense of presence in the physical environment, even to the extent that accidents occur. For cultural heritage, presence is not only about safety but also about the audience experiencing the site and not only the digital content. In this article, we argue that for locative games to be successful for cultural heritage, they must bridge the design tension between offering immersion and presence. We use two of our own titles to shed light on the design concerns and show how careful use of locative game mechanics and narrative techniques can help reconcile these two design pressures and create a new type of engagement with cultural heritage.
A consistent finding of research into mobile learning guides and outdoor learning games has been the value of audio as a medium of communication. This paper discusses the value of location-based and movement-sensitive audio for learning. Three types of audio learning experience are distinguished, based primarily upon differing levels of narrative cohesion: audio vignettes, movement-based guides and mobile narratives. An analysis of projects in these three areas has resulted in the formulation of guidelines for the design of audio experiences. A case study of a novel audio experience, called ‘A Chaotic Encounter,’ delivers an adaptive story based on the pattern of movements of the user.
Full-text available
This paper explores the idea of interactive narrative systems which are site specific and rely on large-scale movements around that space to mediate access to their content. Essentially, by walking around an area with a handheld device, the audience gains access to different short video clips related to (and usually depicting) their immediate environment. The motivations behind this work are that the overlap of space and narrative will provide a deeply compelling experience and a high level of immersion. Furthermore, such a system may also speed up the process by which the audience forms a semantically rich conception of place from the socially empty physical space. In line with an experience design methodology, a case study is presented and key features of its creation, development, use and evaluation are highlighted. The paper concludes with recommendations for future designers of location-aware narratives.
Full-text available
Full-text available
Interactive stories are commonly used for learning and entertaining purposes enhancing the development of several perceptual and cognitive skills. These experiences are not very common among blind children because most computer games and electronics toys do not have appropriate interfaces to be accessible by them. This study introduces the idea of interactive Hyperstories performed in a 3D acoustic virtual world. The hyperstory model enables us to build an application to help blind children to enrich their early world experiences through exploration of interactive virtual worlds by using 3D aural representations of the space. We have produced AudioDoom, interactive model-based software for blind children. The prototype was qualitative and quantitatively field-tested with several blind children in a Chilean school setting. Our preliminary results indicate that when acoustic-based entertainment applications are careful applied with an appropriate methodology can stimulate diminished cognitive skills. We also found that spatial sound experiences can create spatial navigable structures in the mind of blind children. Methodology and usability evaluation procedures and results appeared to be critical to the efectiveness of interactive Hyperstories performed in a 3D acoustic virtual world.
Explores the principles and practical considerations of spatial sound recording and reproduction. Particular emphasis is given to the increasing importance of multichannel surround sound and 3D audio, including binaural approaches, without ignoring conventional stereo. The enhancement of spatial quality is arguably the only remaining hurdle to be overcome in pursuit of high quality sound reproduction. The rise of increasingly sophisticated spatial sound systems presents an enormous challenge to audio engineers, many of whom are confused by the possibilities and unfamiliar with standards, formats, track allocations, monitoring configurations and recording techniques. The author provides a comprehensive study of the current state of the art in spatial audio, concentrating on the most widely used approaches and configurations. Anyone wishing to expand their understanding of these cutting-edge technologies will want to own this book. Keep up to date with the latest technology - 3D audio, surround sound and multichannel recording techniques Enables you to cut through the common confusions and problems associated with the subject Further your knowledge with this comprehensive study of the very latest spatial audio technologies and techniques
An abstract is not available.
An extensive set of head-related transfer function (HRTF) measurements of a Knowles Electronics Mannequin for Acoustic Research (KEMAR) has recently been completed. The measurements consist of the left and right ear impulse responses from a Realistic Optimus Pro 7 loudspeaker mounted 1.4 m from the KEMAR. Maximum length (ML) pseudorandom binary sequences were used to obtain the impulse responses at a sampling rate of 44.1 kHz. In total, 710 different positions were sampled at elevations from -40 deg to +90 deg. These data are being made available to the research community on the Internet via anonymous FTP and the World Wide Web.
The ability to perceive objects from a distance and navigate without vision depends principally on auditory information. Two experiments were conducted in order to assess this ability in congenitally blind children aged 4 to 12 years who had negligible amounts of visual experience or formal mobility training. In Experiment 1, children walked along a sidewalk toward a target location to get some candy. A box was placed along the path on some trials, and the children were instructed to avoid the box if it was present. The children spent more time in the region just in front of the box than in the region just behind it, indicating that they perceived the box and acted so as to navigate around it. In Experiment 2, children attempted to discriminate whether a nearby disk was on their left or on their right. The children performed at above-chance levels, again indicating distal perception of objects. The results of both experiments suggest that blind children with little or no visual experience or formal training utilize nonvisual information, presumably auditory, to perceive objects. The specific nature of this auditory information requires further investigation, but these findings imply that the underlying perceptual ability does not require experience in spatial vision or deliberate training and intervention.
As people start to exploit new telepresence technologies to meet and work, they will be able to exploit all of their senses as they transmit, receive, and monitor information. An essential part of such three-dimensional spaces is the audio landscape. People are able to detect a wide variety of sounds and separate them in space. Spatial separation improves the detection and intelligibility of speech from multiple talkers, and enables simultaneous monitoring of multiple information streams through the use of multiple alert sounds. On-going research at BT Laboratories into spatial audio has resulted in a number of leading edge demonstrations and patent applications. This paper introduces the technologies employed to create spatial audio for real-time synthetic worlds including single and multiple users, and non-ideal acoustic environments.