Content uploaded by Mads Haahr
Author content
All content in this area was uploaded by Mads Haahr on Sep 29, 2017
Content may be subject to copyright.
Design, Implementation and Evaluation of Audio for a
Location Aware Augmented Reality Game
Natasa Paterson Katsiaryna Naliuka Soren Kristian Jensen Tara Carrigy Mads Haahr
Fionnuala Conway
Trinity College Dublin
College Green
Dublin 2, Ireland
natasa.paterson,katsiaryna.naliuka,soren.jensen,tara.carrigy@ndrc.ie
mads.haahr@cs.tcd.ie, conwayfi@tcd.ie
ABSTRACT
In this paper, the development and implementation of a rich sound
design, reminiscent of console gaming for a location aware game,
Viking Ghost Hunt (VGH) is presented. The role of audio was
assessed with particular attention to the effect on immersion and
emotional engagement. Because immersion also involves the
interaction and the creation of presence (the feeling of being in a
particular place) these aspects of the sound design were also
investigated. Evaluation of the game was undertaken over a
three-day period with the participation of 19 subjects. The results
gained imply that audio plays an important role in immersing a
player within the game space and in emotionally engaging with
the virtual world. However, challenges in regards to GPS
inaccuracy and unpredictability remain, as well as device
processor constraints, in order to create an accurate audio sound
field and for the real-time rendering of audio files.
Categories and Subject Descriptors
C.5.3 [Computer System Implementation]: Microcomputers;
H.1.2 [Models and Principles]: User/Machine Systems; H.5.1
[Information Interfaces and Presentation]: Multimedia
Information Systems; H.5.2 User Interfaces; H.5.5 Sound and
Music Computing.
General Terms
Design, Experimentation, Human Factors.
Keywords
Sound Design, Location Aware Gaming, Immersion, Engagement.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
Fun and Games 2010, September 15-17, 2010, Leuven, Belgium.
Copyright 2010 ACM 978-1-60558-907-7/10/09…$10.00.
1. INTRODUCTION
In the last 35 years, game audio has evolved considerably by
moving away from the dependence on midi based wavetable
synthesis bleeps, beeps and simplistic melodies to three
dimensional (3D) sound effects, orchestral soundtracks and
believable dialogue. Console gaming has more processing power
available to synthesise sounds and real-time complex sound
effects, 3D localisation and reverberation – especially in a 5.1
surround sound configuration. Hence there is a trend to replace a
part of recorded audio by real time generated sounds and music.
Therefore, for a composer and sound designer there are many
more avenues available in creating a complex and immersive
soundscape. Such game console examples are Silent Hill [21] and
Thief [24] which present engaging sound designs and an
interactive use of audio.
Immersive gaming soundscapes have mainly been associated with
a static console set up but with the introduction of Personal
Digital Assistants (PDAs) capable of greater processing power
than their predecessors, gaming audio can now move away from
the dependability on midi wavetable synthesis sound to include
richer soundscapes. An example of this is Soul Trapper [22],
which uses spatialisation and a complex audio backdrop to convey
narrative elements of an audio adventure game with some visual
interaction. However, most mobile phone based games still pay
minimal attention to the overall audio design, which are mainly
interested on conveying information and appropriate user
interface sounds.
The increased use of GPS (Global Positioning System) enabled
smartphones, have allowed developers to create real world
location aware gaming where the virtual world can be overlaid
onto the physical world. As audio has developed for console
applications, the challenge is now to incorporate some of these
features in a location aware scenario on the mobile platform that
is not only for navigational purposes but that is immersive,
reactive to the gamers’ movements and that adds to the sense of
presence. Developing a realistic and ambitious sound design on
the limited mobile phone CPU (Central Processing Unit) creates
challenges and requires innovative approaches and ideas. The
emotional and immersive impact of complex sound designs in
location gaming has as yet, not been thoroughly explored.
The aim of this research is to develop a sound design with a
complex and varied soundscape that is immersive, emotionally
engaging and supportive of the game narrative. The goal is to
implement this on the mobile platform with all the technical
constraints that mobile gaming can bring. In location aware
gaming, visual displays are often small therefore other sensory
modalities such as sound can be investigated in regards to player
game experience. As well as the implementation of an augmented
reality game, the sound design will be assessed for its ability to
convey the desired soundscape environment relative to the game
context, the development of player immersion and emotional
engagement. First we shall take a brief look at the background of
location aware gaming on the mobile platform, then describe
Viking Ghost Hunt as our working prototype and the sound design
and subsequent implementation. The user trials undertaken
examined the usability of the game interface and the overall game
experience, which included audio elements. Specifically the
immersive quality of the audio and how this added to the game
experience was investigated. This testing process will be
described together with the results obtained and their significance.
Finally we conclude on the possible role of audio in location
aware gaming and directions for further research.
2. RELATED BACKGROUND
There has been increasing interest in location aware gaming since
the availability of PDAs that incorporate integrated GPS
capabilities. This integration allows for the overlay of a digital
mediascape onto the physical world that responds to contextual
cues such as GPS. Location aware gaming layers a virtual world
onto the physical world in such a way that physical elements of
the gamers real world can be integrated within the game structure
thereby merging the virtual and physical worlds [5] which in turn
can produce a sense of a place (a space is a physical environment
and a place has greater meaning) [25]. The sound design of a
location aware game can convey important game information such
as navigational sounds and instructional dialogue thereby enabling
the user to look away from the visual graphical interface and to
remain engaged within their physical environment. Listening to
audio requires less cognitive attention than visual information and
therefore reduces the amount of player distraction from the game,
hence encouraging the player to remain in the game space [5].
Research has already shown that the use of a realistic sound
design can help build excitement and tension in the game world
and an enhanced sense of immersion, without the use of visual
graphics [26]. To facilitate user engagement within the virtual
world, a sense of presence (the perception of being in a particular
space) must be created which is interrelated and dependant on
player emotional engagement and immersion. Hence a
multisensory approach of using both visual and audio interfaces is
increasingly being implemented for enhancing immersion and it
has been found that audio is a significant element for multimodal
applications such as location aware gaming [11].
Investigations into using non-speech audio messages (earcons)1
concluded that audio cues were significant for conveying
information and for the development of semantic meaning [12].
This is important as it suggests that audio can provide useful
information about virtual environments and support perceptual
integration. In regards to location aware applications, most audio
investigation has focused on non-gaming applications, artistic
installations or navigational tools for the visually impaired.
Research into spatialised sound in a location aware application
1 Sounds used to convey information about computer systems.
was undertaken by Dr. Kirsten Cater and Dr. Richard Hull et al
[3], with initial results indicating that the soundscape of a PDA
application can help users navigate in a physical environment by
conveying relevant information through audio. In their
application, a 2-axis compass, Bluetooth GPS and HP iPAQ was
used. Sound installations such as ‘The Tactical Sound Garden’
[23] explore the concept of overlaying soundscapes onto a
specific urban space. In this example, wireless (WiFi) ‘hot zones’
were utilised to install a ‘sound garden’ for public use which used
an element of audio for navigation and spatialisation. Demor [7]
is a location aware 3D audio first person shooter game primarily
designed for the blind but which sighted players can also take
part. This game investigated psychoacoustic properties for the
presentation of a 3D soundscape that would be accurately
presented and used by players for real physical space navigation.
In this example, a laptop, GPS tracker and headphones with a
head-tracking device together with a joystick is used. These
applications mainly focus on sound as navigation and not on
potential immersive or experiential properties. Additionally, the
applications were not based on the mobile platform and often
involved complex technical set-ups.
An example of location aware audio in gaming is ‘The Songs of
North’ [8]. This application investigates the use of audio for
navigation and in the conveying of game information with the
research containing some reference to the possible role of
immersion in the game experience. This location aware mixed
reality game features audio as an important aspect, however it
must be noted that headphones were not used in this example.
The application was also hampered by challenging technical
constraints in the presentation of an audio soundscape due to
memory and processor limitations.
There are many more examples of the use of audio in location
aware gaming, however most research has focused on audio as a
function of navigation and in the conveying of game information
rather than its emotional and immersive effect. There is currently
research on console game audio [9], however game audio mobility
and its implications has not been thoroughly dealt with.
3. GAME PROTOTYPE
Viking Ghost Hunt (VGH) is implemented on the Android
platform using the HTC T-Mobile G1 and is a working prototype
for a location aware game in Dublin city centre, Ireland. The
game is narrative driven and played around the old Viking sites of
Dublin with the player acting as a paranormal investigator. The
gamer uses various visual and audio interfaces together with
stereo headphones to locate and interact with the ghosts. The VGH
aim is to immerse the player in the game world which is merged
with the physical world, with the PDA acting as a paranormal
investigative device. The aim of the audio design was to create a
soundscape that is both informative of the game environment and
that is immersive and emotionally engaging. Sound also supports
the contextual element of VGH to create a believable space and to
blur the real and virtual world borders. The aim of the game is to
locate paranormal activity and gather evidence that manifests
visually or aurally. This evidence may give information about the
ghostly activity and clues for moving through the game space.
The paranormal device (mobile phone) has different modes or
interfaces that can be used to locate paranormal manifestations in
the real world space and to capture audio-visual evidence. The
modes of paranormal investigation consist of camera/x-ray mode,
Figure 1. Viking Ghost Hunt prototype
map mode, radar mode and frequency scanner mode. The
frequency scanner mode is an audio interface reminiscent of
Electromagnetic Voice Phenomenon (EVP) often used by
paranormal investigators. Paranormal investigators believe that
‘ghostly voices’ can be heard when radio static is analysed.
Gamers must use the frequency scanner in order to find the
correct ghost frequency (Hertz), record the white noise audio and
subsequently hear the decoded ghostly message on playback.
This is an integral part in the game mechanics for player
interaction, game information, evidence retrieval and in the
delivery of the game narrative.
Figure 2. Radar mode
In addition to the frequency scanner, audio is also used for all of
the user interface sounds in order to support the concept that the
mobile phone is a paranormal investigative device. Automated
ambient audio and sound effects are triggered when in proximity
to GPS locations or regions to assist the player in navigating to the
correct locations and for immersion and engagement in the game
experience.
4. SOUND DESIGN
The aim of the sound design is to create an immersive experience
in which the player remains engaged with their physical
environment. Therefore a balance between the ghostly
atmosphere of the game and the location environment was
maintained. Hence, sounds used are representative of both the
location environment and game atmosphere. The sound samples
used for VGH were stereo field recordings, sourced samples or
created electronically using a midi synthesizer and audio
sequencer. The required sounds were placed into four categories:
paranormal, environmental, musical and user interface sounds.
The table below shows some of the sounds used.
Table 1. Sound Categories
Paranormal
Environmental
Musical
Elements
User Interface
footsteps
floorboards
creaking
bangs
high pitch
metallic
squeal
breathing
screams
whispers
faraway
voices
growls
scratching
moaning
modulated
voices
rattling
chains
battle sounds
ghost
dialogue
animal sounds
church bells
laughing
children playing
school bell
paper flying
around
thunder
wind
rain
traffic
dog barking
chimes
birds
leaves
wind in trees
people talking
fire
minor scales
chromatics
pitched and
unpitched
drones
drums
ghostly pan-
flute
augmented
intervals
tremolos
irregular
rhythms
old AM radio
sounds
white noise
audio static and
interference
metal detector
sounds
dial clicks
Geiger metre
clicks
button sounds
sonar sound
compass sounds
transitional
sound
From a design perspective, audio is used similarly as in the genre
of film and divided into background sound, sound effects and
dialogue with the addition of user interface elements. The
background consists of the creation of an atmospheric soundscape
that represents the game space and the physical location of the
given narrative. This background sound sets the mood of the
location and virtual game space by using randomised, overlapping
looped audio files in order to avoid habituation and hence player
disinterest. Sound effects are then added that are relevant to the
location and game narrative together with appropriate dialogue,
which can be accessed using the frequency scanner mode.
The sound is triggered by creating physical zones in the form of
concentric rings organised around a GPS location with varying
radial distances. This enables different sound files to be triggered
due to ongoing GPS update information provided by player
movements. For example, as the player comes within a
paranormal zone a looped drone sound (rumbling, low frequency)
and some environmental sounds (e.g. wind) begin. The mixing of
environmental and game sound supports the blending of the
virtual and real world to such an extent that the player may
believe some sounds to be part of the physical locations. As the
player continues to move towards the central point of activity,
sound files will start that are related to the paranormal entity.
These sounds are descriptive of the game narrative and historical
location of the game and consist of short sound effects (e.g. chains
etc) that may be randomly triggered or looped. Hence players can
obtain information related to the game space without looking at
the mobile device which enables them to remain engaged in the
physical space.
Table 2. Example of sound design for physical locations
Location
Ghosts
Intended
Mood
Background
Sound
Sound
Effects
Stairway to
Hell
(stairway
going down
into a tunnel)
Leper
Fearful,
dark,
uncertain
Low
frequency
rumbling
drone using
randomised
overlapping
minor
intervals
Chains,
moaning
Shallow
breathing
Fire
sounds
Footsteps
!"#$$%#$&'()
*+,()
-%+,(.+/)
,(01)1$)
2(34(5+%)
"#&6"#7)
8#$'1%/)
"#4%36(,)
9(64(:)
$;(,)
';+"()
<4,3).41#)
4,=6(>&(,1)
%$.)3+2;(3)
;(6"&''45()
?$$2'))
@#4%36(,)
;%+/4,A)
4,)1#()
34'1+,"()
!"#$$%)
?(%%')
.#4';(6')
As the player reaches the GPS defined location, an escalation of
sounds due to increasing amplitude and sound file additions
signals to the player that they are entering a paranormal zone.
The visual interface indicates whether the paranormal
manifestation is visual or aural and the player must choose the
appropriate mode to gather the evidence. As the player leaves the
location after obtaining the evidence, these sounds gradually
dissipate in intensity and fade away. Therefore different sounds
enter and remain for different lengths of time depending on the
location of the player in relation to the GPS point, hence
providing an interactive, immersive experience. The ambient
audio soundscape is always triggered automatically by player
proximity to physical game locations.
Within the sound design the audio can also be used for
navigational support by providing verbal instructions in the
frequency scanner mode dialogue and by triggering sounds near
paranormal location that increase in volume and complexity as the
player moves closer to the location. This navigational strategy
also includes the addition of various sound effects related to the
paranormal activity that are triggered as the player moves toward
the physical location, again informing the player that they are
entering a paranormal zone. Constant background sound between
paranormal locations is avoided so as not to overload the user
experience and take away from the immersive experience.
In order to design and implement a complex and engaging
soundscape, properties of sound must be explored and integrated
in order to develop an engaging and realistic audio design. A
somewhat ‘natural’ audio presentation that reflects real world
sound has been found to be more immersive and engaging for
augmented reality applications [14]. The psychoacoustics of
sound in the real world in relation to spatialisation and
reverberation is an important factor in creating the sound design
for VGH as well as the creative composition. Sound is presented
in a three dimensional manner (3D) in the environment and
human survival depends on the ability to establish a sense of
spatial orientation. This is needed for localisation, distance
perception and recognition [1] of objects, and spatialisation in an
artificial soundscape can mimic how sound is heard in the real
world. In order to perceive from where a sound source is
originating, the human brain uses information derived from the
interaction of the sound wave with that of the torso, shoulders,
head and ears. Interaural time differences (ITD) and interaural
level differences (ILD) of sound as it interacts with the head help
to localise sound in space. For example, if the sound were located
to the right, the right ear would hear the sound first and then the
left with a slight time delay (ITD). The amplitude (volume) level
would also be greater on the right ear than the left (ILD) and
therefore the sound is perceived to originate from the right [10].
ITD is used to differentiate the location of lower frequencies
while ILD is used to differentiate the location of higher
frequencies. Perceptually humans use a combination of these
systems, however ITD and ILD do not work as well for sound
originating on the azimuth axis (vertical plane) and therefore can
be poor for front and back sound localisation. For a true 3D
representation of audio the use of Head Related Transfer
Functions (HRTFs) must be implemented to give a realistic
spatialisation of sound as it takes into account the accurate
individual interaction of the user’s head with the sound wave [11].
By using HRTFs individually a player sometimes cannot tell
virtually spatialised sound from the real thing.
Figure 3. ILD, ITD and HTRF [14]
However the use of HRTFs is not realistic for a mobile phone
(PDA) application as it requires real-time processing of audio
filters which is CPU intensive and unachievable at this level for
PDAs at this time. Therefore in order to spatialise sound for the
VGH prototype, a stereo representation of sound was found to be
adequate with the use of ILD and ITD parameters in either a pre-
rendered format or by real-time live processing of audio files,
CPU permitting. For the pre-rendered formats, sound files were
spatialised using a binaural simulator (software designed to
imitate realistic spatialisation) or panned by changing right and
left volume levels (ILD) in relation to the virtually placed source.
Therefore sounds such as whispers or dialogue can be perceived
as if to be moving around and in and away from the player.
Spatialised audio can be integral to navigate through a virtual
world and create realism and facilitate immersion. Immersion is
an important aspect of a location aware game as the user’s full
attention can be captured by the experience and research as shown
that spatialised sound can increase immersion [14].
Reverberation is another important psychoacoustic consideration
when developing the sound design. Reverberation is sound that is
reflected from physical surfaces and back into the environment.
Numerous ‘early’ reflections are then themselves reflected and
form an ‘ambient’ sound field [18] and these sound reflections
have been found to contribute to the sense of a space and the
perception of a sound source size and its distance [2]. The
addition of realistic artificial reverberation in the sound design of
a location aware game can add a sense of realism and
envelopment (sense of immersivity in a reverberant sound field)
for the user. Hence the use of reverberation was an important
stylistic element in creating a ghostly atmosphere as research as
found that reverberant ambient sound can add to immersion and
player engagement [14]. In the prototype, audio files were
processed using different types of reverberation settings which
were dependant on the context of the game locations.
In order to create a dense and engaging game atmosphere, the
sound design involved the playback of a complex configuration of
multiple and varied simultaneous audio files that were created
with keeping in mind psychoacoustic principles, the blending of
the physical and virtual worlds and the game narrative. Hence the
sound design was delivered with the use of closed supra-aural
headphones (to allow some external environmental sound to be
heard) without the use of a head-tracking device. Therefore it was
suggested to players that they face the direction of the device to
ensure proper delivery of spatialised audio events.
5. EVALUATION
5.1 Method of evaluating the sound design
Objective findings such as physiological changes are difficult to
measure for emotional response and immersivity to sound and
music [17] as it does not take into account cognitive aspects.
Therefore for a phenomenological experience (experienced from
the first-person point of view), subjective reports were used for
the assessments. The location aware prototype, VGH was
evaluated over a period of three days. The 19 participants were
each provided with a G1 HTC mobile phone and headphones.
Participant demographics included male and female volunteers
with ages ranging from 20 years to 45 years with a mixture of
previous game experience to little or no game experience.
Volunteers were not briefed regarding the importance and
evaluation goals of the sound design. Game feedback focused on
the prototype user interface usability and game experience and
took the form of a questionnaire, which included certain questions
focusing on the effect and role of the sound design for emotional
response and immersion.
The questionnaire consisted of open-ended questions (e.g. “Did
you feel more or less engaged at different stages of the game?
Which part of the game was immersive and why?” and “Which
features of the interface most helped guide you to the places
where the ghosts were located? Please describe) which were
designed specifically not to lead the volunteers to comment on the
audio unless it had a significant impact on the game experience.
It also consisted of a set of statements in which 9 were specifically
related to audio with players responding by using a 5-item scale
(strongly disagree to strongly agree). Audio statements included:
“The sound made the game feel scary”, “I feel that the sound was
reactive to my movements” and “The audio was seamless and felt
a real part of the game”.
5.2 Results of the game audio experience
Generally, the overall game experience was well received with
70% of participants responding that they enjoyed the game.
An important question in regards to sound was in participant
expectation of the role of audio in augmented reality gaming. It
was found that the respondents expected the role of audio to be
roughly equal in the form of narrative and sound effects.
Figure 4. Audio expectations in augmented reality gaming
Overall, the participants responded positively (71%) to the sound
design, with comments such as:
“Addition of audio greatly increased the atmosphere and
engagement.”
“I felt immersed in the game”
“It was a very immersive experience. A couple of times I found
myself not realising that I am on a street and there are people
around me.”
“At the moment it seems like the atmosphere is the unifying
element (also one of the strongest aspects of the game).”
This positive feedback regarding the sound design was an
encouraging outcome.
Figure 5. Summary of audio feedback
The overall response to the role of audio for game immersion was
high with 70% of players agreeing or strongly agreeing to the
immersion statements. By creating an engaging and complex
soundscape and dialogue taking into account psychoacoustic
considerations, participants felt involved in the game and in their
role as a paranormal investigator. This engagement and
immersion adds to the possible development of an emotional
response felt or perceived within a game space and a sense of
game presence. When asked whether the game audio contributed
to the game feeling scary, 63% of participants responded in
agreement or strong agreement that the “sound effects created a
scary atmosphere.”
Additionally, as well as investigating audio for immersion and
emotional engagement, user interface usability was also
investigated. For audio, this included an element of interactivity
between the player, physical locations and the device user
interface. As discussed previously, audio reactivity to player
movement was achieved by volume changes and the addition and
deletion of sound files triggered by varying distances from given
GPS locations, together with the implementation of realistic
paranormal investigative device sounds. Congruency between
audio changes and player movement can support player
involvement and therefore immersion, as the device and player
actions are perceived to be coupled and realistic. In multi-modal
presentations, congruency is an important factor in reinforcing a
natural mode of interaction [27]. That is, the game world should
reflect the experiences of real world interactions which increases
the likelihood that there is a wiling suspension of belief and hence
immersion [4]. Participants were asked whether they found the
sound reactive to the physical environment and if they thought the
user interface sounds complemented the visual interfaces and
paranormal device sounds. In response, 84% of testers agreed or
strongly agreed that the user interface sounds supported the role-
play element of the game play and 68% felt that the game sound
was reactive – which supports the blending of the virtual world
and physical one:
“The backing audio changing as I moved location was a good
touch”.
“Loved the sound – that was new for mobile gaming – location
sensitive.”
Additionally, 79% of testers felt that the audio supported the game
environment in the physical locations.
In creating a complex sound design with many layers and types of
sounds playing simultaneously, care must be taken in the
presentation of the audio files. This was evident in some of the
feedback as sound effects were at times found to overpower the
narrative dialogue, which is essential to the game play. Hence a
balance must be struck between important audio game
information and ambient background or sound effects:
“ Felt a little scary, however the first time the ghost spoke I had
difficulty understanding what he said over the sound effects.
Though the sound was quite good.”
This brings to light another challenge for location-based audio –
external noise and busy environments. Even though in this
example most testers did not feel distracted or interrupted by
external noise this might have been due to the fact that a quiet
location was sourced as a testing ground. Therefore augmented
reality game locations must be carefully sourced:
“Stopping in the middle of a path was distracting as I felt I was in
the way [of people]….isolated paths and lanes were more
atmospheric and I felt more immersed.”
Generally the results obtained from the user trials were extremely
positive in regards to the sound design aim of trying to emulate
console gaming audio and to be immersive and potentially
emotive. As one tester noted:
“The sound – that was new and reminded me of Silent Hill on PS2
(brilliant because of sound).”
Therefore mobile phone gaming in a location aware application
has the potential to be immersive and engaging even when faced
with the challenges of slower processing speeds and limited
memory space when compared to its static console counterpart.
6. DISCUSSION
In designing, implementing and testing the ambitious soundscape
design for Viking Ghost Hunt, it was noticed that there are two
main challenges when developing a location aware game that
impacted on all aspects of the prototype – GPS inaccuracy,
unreliability and mobile device technical constraints. During the
user trials, GPS updates at certain times either did not occur or
were inaccurate in determining the player location. This produced
graphical user interface anomalies and resulted in audio files not
being triggered at the intended locations. It was obviously a
distraction from the game play and affected enjoyment and
immersion.
The other main issue for audio in augmented reality applications
is the processing speed of the device and the memory available.
In the Viking Ghost Hunt prototype, the game application is stored
on the mobiles’ internal memory with the game assets residing on
the SD (Secure Digital Memory Card) card. Very little processing
power is available for any real-time audio manipulation. In order
for audio to be truly reactive and to accurately represent sound
spatially, live processing of sound in regards to GPS and compass
positions in a physical location, must be possible. An example of
this presentation is Demor [7], which uses real-time processing of
spatialised sound to create a truly 3D audio virtual space. Players
can accurately hear sounds emanating from the left and right in
relation to their head positions and physical location. Players can
also judge if a sound is located at a distance or placed close by. In
this application, audio is reactive to the players’ GPS locations
with audio files being adjusted accordingly by live processing on
a specially written audio-engine.)The audio engine designed for
Demor [7] would most likely have used a non-individualised
HRTF (Head Related Transfer Function) database for the 3D
representation, involving large amounts of data processing.
Simplified psychoacoustical considerations have been
implemented in the VGH prototype however this has centered on
including most of the spatialised sound in a pre-rendered format.
Some location reactive left and right audio panning has been
designed and implemented that is able to represent virtual sound
in a static physical location by using GPS co-ordinates and
internal phone compass sensors. However, the location and
direction sensitive audio panning is currently device dependant
and has not been fully tested. Player immersion and engagement
could be improved in location aware gaming with a more accurate
presentation of audio on the mobile platform that included a non-
invasive head-tracking compass.
Careful consideration must also be taken when choosing locations
for location aware applications. Such issues as noise levels,
environmental player distractions and safety must be taken into
account. Certain locations would be more immersive than others
and combing this with narrative relevance would require thorough
investigation before game development and implementation.
Safety for gamers when wearing headphones can be an issue,
especially if the audio is particularly immersive as high levels of
audio immersion may take away from the intended physical
location creating a sense of disengagement and pose problems in
regards to traffic or unwanted attention.
Creating a sound design that intends to bring the sonic richness of
console gaming into location aware applications requires the
consideration of many factors outside the realm of audio. The
combination of technical aspects, location, game narrative and
mechanics all add to the game experience. Sound design is not
simply a process of developing impressive quality audio samples
but instead requires a sensitive compositional approach, as
excellent sound quality does not always equate to immersion.
7. CONCLUSIONS AND FUTURE WORK
In this paper we have presented the sound design, implementation
and evaluation of our working prototype Viking Ghost Hunt. The
design demonstrated that complex audio similar to that of popular
console games can to some extent, be implemented on a mobile
device, even with the challenging technological constraints. This
research highlights the importance of audio in location aware
gaming for immersion and engagement, and hence for the overall
game experience. Audio in location aware gaming is an area still
not thoroughly investigated in regards to emotional impact and
immersiveness and the role that this could potentially play in this
genre. Also, research has not delved into the area of representing
audio in a psychoacoustically correct way that would increase
perception of sound naturalness and hence increase player
engagement. Further research into these areas together with a
possible subjective and objective investigation of player
emotional and physiological reactions could be undertaken.
Location aware games can provide the user with more
understanding and knowledge of their environment through both
the audio and visual mediums. It is the hope of this investigation
to present the importance of sound for immersion and engagement
and that it may inform future applications for this increasingly
popular genre.
8. ACKNOWLEDGMENTS
We’d like to thank the rest of the VGH team; Alan Duggan,
Daniel Crowley, Tina Hedayet and Roisin Cotton for their input
and knowledge in the development and evaluation of the location
aware prototype. We’d also like to thank the National Digital
Research Centre in Dublin Ireland, for their continued generous
support of the project.
9. REFERENCES
[1] Ashmead, D.H., Hill, E.W., & Talor, C.R. Obstacle
perception by congenitally blind children. Perception and
Psychophysics, 46(5) (1989), 425–433.
[2] Begault, D. 3D Sound for Virtual Reality and Multimedia.
Academic Press Inc, 1994.
[3] Cater, K., Hull, R., Melamed, E., Hutchings, R. An
Investigation into the use of Spatialised Sound in Locative
Games. CHI 2007, April 28 - May 3, San Jose, USA.
[4] Coleridge, S. T. The Collected Works of Samuel Taylor
Coleridge: Biographia Literaria, edited by James Engell and
W Jackson Bate, Princeton university Press, Princeton, New
Jersey, USA, 1983.
[5] Dede, C. Immersive interfaces for engagement and learning.
Science, 323 (5910) (2009), 66–69.
[6] De Souza e Silva, A. Location Based Games: Blurring the
borders between physical and virtual spaces, ISEA (2004).
[7] Demor – www.student-kmt.hku.nl/~g7/redirect
[8] Ekman, I., Ermi, L., Jussi, L., Nummela, J., Lankoski, P.,
Mäyrä, F. Designing Sound for a Pervasive Mobile Game,
ACE 2005.
[9] Collins, K. Game Sound: An Introduction to the History,
Theory and Practice of Video Game Music and Sound
Design. MIT Press, USA, 2008.
[10] Gardner, W.G. Spatial Audio Reproduction: Toward
Individualized Binaural Sound. The Bridge, V 34 (2004).
[11] Gardner, W.G., & Martin. K. D. HRTF measurements of a
KEMAR. Journal of the Acoustical Society of America, 97
(6) (1995), 3907 – 3908.
[12] Gaver, W. W., Smith, R. B., O’Shea, T. Effective Sounds in
Complex Systems: The Arkola Simulation. Proceedings of
the SIGCHI Conference on Human Factors in computing
systems, p. 85 – 90, 1991.
[13] Hendrix, C. & Barfield, W. The sense of presence within
auditory virtual environments. Presence: Teleoperators and
Virtual Environments, 5 (1996), 274– 289.
[14] Hollier, M. P., Rimell, A. N., Burraston, D. Spatial audio
technology for telepresence. BT Technology Journal, Vol 15
(1997) No 4.
[15] Keller, P. & Stevens, C. Meaning from environmental
sounds: Types of Signal-Referent Relations and their effect
of recognizing Auditory Icons. Journal of Experimental
Psychology: Applied, 10, 1 (2004), 3-12.
[16] Maynes, C. Worldizing – take studio recordings to the field
and make them sound organic. The Editors Guild Magazine
Vol. 25 (2004), No.2.
[17] Meyer, L., Emotion and Meaning in Music. University of
Chicago Press, Chicago, USA, 1956.
[18] Rumsey, F. Spatial Audio. Focal Press, Oxford, UK, 2001.
[19] Sanchez, J., & Lumbreras, M. 3D aural interactive
hyperstories for blind children. International Journal of
Virtual Reality, 4(1) (1998), 20–28.
[20] Sheridan T.B. Musings on Telepresence and Virtual
Presence. Presence: Teleoperators and Virtual Environments,
1 (1) (1992), 120-125.
[21] Silent Hill www.konami.com/games/shsm
[22] Soul Trapper
http://www.rtaudioadventures.com/Soultrapper.html
[23] Tactical Sound Garden http://www.tacticalsoundgarden.net/
[24] Thief www.gamethief.com
[25] Nisi, V., Oakley, I., Haahr, M. Location-Aware Multimedia
Stories: Turning Spaces into Places. ARTECH 2008.
[26] Valente, L., Sieckenius de Souza, C., Feijo, B. An
exploratory study on non-visual mobile phone interfaces for
games. IHC 2008.
[27] Witmer, B.G., Singer, M.J., Measuring Presence in Virtual
Environments: A Presence Questionnaire. Presence, Vol. 7
(1998), No. 3, 225-240.
