ArticlePDF Available

Audio space invaders

Authors:

Abstract and Figures

Whilst advances are underway in various areas to ease and encourage disabled uptake of new technology, very little emphasis to date has been placed on making the games market accessible to all. The aims of the described work have been twofold. Firstly, to prove that the standard features of a traditional space invader game can be replicated using a 3-D audio (ambisonic) environment. Secondly, through combining audio and visual interfaces with force feedback joystick movement that it is possible to produce a multi-modal game that can be played by both sighted and non-sighted users, thereby enabling them to share the same gaming experience. This paper describes the development and features of the resultant Audio Space Invaders game.
Content may be subject to copyright.
Audio space invaders
R J McCrindle and D Symons
Department of Computer Science, The University of Reading,
Whiteknights, PO Box 225, Reading, Berkshire, UK
r.j.mccrindle@reading.ac.uk, david.symons@bigfoot.com
ABSTRACT
Whilst advances are underway in various areas to ease and encourage disabled uptake of new
technology, very little emphasis to date has been placed on making the games market
accessible to all. The aims of the described work have been twofold. Firstly, to prove that the
standard features of a traditional space invader game can be replicated using a 3-D audio
(ambisonic) environment. Secondly, through combining audio and visual interfaces with force
feedback joystick movement that it is possible to produce a multi-modal game that can be
played by both sighted and non-sighted users, thereby enabling them to share the same gaming
experience. This paper describes the development and features of the resultant Audio Space
Invaders game.
1. INTRODUCTION
Digital technologies such as hypermedia, virtual reality, digital video broadcasting, video conferencing, co-
operative working and the world wide web have been the subject of intense development over recent years
and together may be said to comprise the second information technology revolution. These developments
coupled with the continually decreasing costs of the enabling technology have resulted in significant
expansion in their use (Harlow and Gadher, 1999). One such area of expansion is that of the home
entertainment industry and in particular the video games market, to the extent that computer games are
played by almost all children, with many families having a home computer or games console. Most of these
games involve the use of computer graphics to navigate a fantasy world. Consequently, whilst the graphical
technology of these games has become very advanced and sophisticated, the audio component is often
mediocre and only used to add to the realism of a game rather than to assist play it. This can leave blind and
partially sighted children feeling excluded and set apart from their peers, as they are unable to participate in
these games.
That the inclusion of disabled users in all areas of the new technological revolution is of prime
importance is evidenced through the increasing awareness of the requirements and rights of the disabled
including those related to inclusive product design (UK Government, 1995; DRC, 2000; W3C, 2000;
Disability Now, 1999; FEFC, 1996; FEDA, 1998). With inclusive software design (McCrindle, 1999) in
mind we have developed a multimodal games interface for a space invader type of game. In contrast to
traditional games development, design of the Audio Space Invaders game has been based primarily around a
3-D surround sound environment, with the graphical interface being added later. Force feedback controls
further enhance the playing experience of the game.
2. EXISTING WORK
Whilst advances are underway in various areas to ease disabled uptake of new technology, for example use of
the World Wide Web (W3C, 2000), very little emphasis to date has been placed on making the games market
accessible to all. That such a concept is possible, has been demonstrated by Lumberas and Sánchez (1998)
who developed a 3D aural interactive hyperstory specifically aimed at blind children. The project proved that
blind children could interact with a computer using an audio interface. It also showed that playing the game
helped children build up a model of the fantasy world in which they were playing and resultantly improved
their spatial awareness of the real world. Another study into audio interfaces (Mereu and Kazman, 1996)
found that by using a 3D audio interface a blind person could locate a point in 3D space as accurately as a
sighted person could using a graphical interface, although the time taken by the blind person was
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6 59
significantly longer. It was also found that in a sound only environment visually impaired users were very
much more accurate than sighted users. Other ambisonic research (Cooper and Taylor, 1998; Lumberas et al,
1996) also substantiates the effectiveness of 3-D audio environments. Use of force feedback has also been
used to guide users through a representation of a GUI (Ramstein et al, 1996) and manipulate the
environment. By producing the Audio Space Invaders game we have taken the 3-D audio gaming concept a
stage further by combining audio and visual interfaces with force feedback joystick movement to produce a
game that can be played by both sighted and non-sighted users, thereby enabling them to share the same
gaming experience.
3. THE AUDIO SPACE INVADERS GAME
Audio Space Invaders has been implemented in Visual C++ combined with the APIs from Aureal’s A3D
Software Development Kit (SDK) 2.0 (1999) and Microsoft’s DirectX SDK 6.1 (1998, 1999). This has
proved to be a very powerful combination allowing the programming of advanced PC gaming features,
relatively good portability and excellent driver support.
As the primary aim of this project has been to prove that the standard features of a traditional space
invader game can be replicated using an audio environment, incorporation of the following game
requirements were deemed to be important:
The game should be based on a player who his defending himself against may enemy foes.
The enemy may shoot at a player. This will decrease the player’s life, when the player’s life reaches
zero the player dies and the game ends.
The player may shoot at the enemy, who also have a life that decreases when they are shot. When
the enemy dies, the player’s score is increased.
The game should have a number of different levels each of increasing difficulty.
The level reached and total score should enable players to rate themselves against each other.
4. GAME SPECIFICATION
The features currently incorporated in a fully working prototype may be briefly summarised as:
4.1 Type of Game
A shoot ‘em up style of game based on a futuristic space adventure has been developed, as such games are
currently very popular with the teenage gaming community. The game incorporates a number of scenario
levels, each one based around a player fighting against different types of enemy ships. Each ship type has a
certain ‘life’ that is automatically set to a particular value when the ship is created and decreased every time
it is shot. When the ‘life’ of a ship reaches zero the ship is destroyed. The ships also have different velocities,
directions of attack, flight patterns and firing rates. The scenarios are of increasing difficulty and are
presented to the player in sequence. For enhanced variability, enjoyment and challenge, each scenario
possesses its own unique difficulties and features and to proceed from one level to the next a number of
objectives must be met. Each scenario is guided by audio/textual information and played via interaction with
a 3-D audio/graphical/tactile interface. Points are obtained by shooting enemy vehicles and by completing the
mission objectives. The game can be considered complete when the player has completed the final mission.
A total score enables players to rate themselves against each other and against previous plays.
4.2 Adjusting Levels of Difficulty
The scenario levels are initially set with default values, however to adjust the complexity of the game to suit
different age groups, disabilities and experience levels, each scenario may be customised using the Level
Editor supplied with the game (see Figure 1). For example once a player has mastered the inbuilt scenarios,
they may be extended by adding extra missiles to the players arsenal.
4.3 Audio Interface
The game is based mainly around a 3-D audio environment. In order to achieve this most effectively, the
audio interface was the first part of the program to be designed and implemented. This ensured that other
parts of the program were an accurate representation of the audio data. This is in contrast to traditional games
development, which tends to concentrate development on the graphical interface, and then tries to associate
an audio interface with it. Ideally to give the highest sense of reality to the player, the audio environment
consists of a four-speaker surround sound set-up. However the program will also work adequately on sound
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6
60
cards that only support two speakers – in this mode it is especially effective with headphones. For further
ease of use instructions are given audibly, audio clues as to what is happening within the game are
implemented, and the control system has been made as simple as possible.
4.4 Visual Interface
The Audio Space Invaders game does not require a graphical interface for playing purposes. However, a
graphical interface has been included for a number of reasons: firstly to provide a more natural mode of
interaction for sighted users; secondly to provide a training mechanism to the game for both sighted and non-
sighted users; and thirdly to provide a comparative test-bed for research into human computer interaction
issues across sighted and non-sighted user communities. During testing it was found that newcomers to the
game often experienced initial difficulty in getting their bearings in the audio world. The graphical interface
allowed sighted people to work out their relative position to other objects in the game and it is envisaged that
via this method a sighted person would help a visually impaired person also learn the game. It may also be of
use during game play by partially sighted users. The visual interface may be turned off completely if desired.
Figure 1. Level-editor to set degree of difficulty of game and establish new scenarios
4.5 Interaction
There are two possible control methods for the game – a keyboard or joystick. The keyboard can be used to
replicate a number of basic controls: - up, down, left, right and fire. The joystick enables more sophisticated
interaction with the game since it allows more than one signal to be processed at once, so for example the
user can shoot and move at the same time. Simple keyboard interaction rather than a mouse driven interface
has been incorporated as it also assists users with co-ordination impairment since it eliminates the needs for
physical dexterity and accuracy that are associated with the use of a mouse. Additionally, blind users who
may not feel completely confident with a mouse or joystick may also prefer this method of interaction.
4.6 Force Feedback
The use of an optional force feedback joystick provides extra information and clues to the game situation if
the user has one of these devices. Through different movements of the stick the player is able to feel
themselves being shot, the shots they are firing etc. and in the future may be used to guide the user through
certain events or as a training mechanism. This extra information can be very useful, especially at higher
levels of the game when there are several activities going on and hence a variety of different sounds being
produced.
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6 61
4.7 Universality and Affordability
The game has been designed to run on a Windows 95/98 PC of average specification. The recommended
minimum specification is a Pentium II with 64mb RAM, Aureal 2.0 compatible sound card (4-speaker) and a
force feedback joystick. However acceptable performance is also achieved on a Pentium MMX or equivalent
with 32mb RAM and a standard sound card (2-speaker). A player can also turn off features such as the
graphics to improve the speed of the game. Although a force feedback joystick is very useful in supplying
extra sensory information to a player, it is an expensive device and hence is an optional feature of the game.
Additionally, research has also shown that some blind children find a keyboard a more natural method of
interaction and hence this is always an available control option.
5. GAME SCENARIO IMPLEMENTATION
There are currently six scenario levels, including an initial training level, each representing the defence of a
different planet from increasingly complex formations of enemy ships. Essentially, the player is positioned in
a turret on each planet’s surface, which is then attacked by the enemy ships. The player has the choice of
being reactive and firing at the enemy ships when they are detected audibly or visually within range, or they
may be proactive and seek out enemy ships to destroy.
Purposefully, the first scenario level is a training level to teach the users the basic controls and how to
play the game. At this level the user is in a turret with four stationary ships placed around them. A ship is
placed in front, behind and to either side of the turret. There is also a more powerful enemy ship incorporated
into this level to enable the player to recognise the difference between these ships and the normal ships. This
level is essential to help players not using the graphical interface learn how to locate ships by listening to
where the sounds come from and to differentiate between the different types of ships. Unlike the graphical
interface, which is a common way of playing games, the audio interface is quite different to anything else the
players will have used before. For example the more powerful ships are given a louder and more high-
pitched sound than the normal ships and must be distinguished as they are harder to destroy. Additionally, the
turret is given a long life-span in this level. This gives the user ample time to get used to the controls and the
ideas of the game. The training level also introduces Molly, a female robotic companion, who instructs the
user what to do at each level and what dangers are present at each level of the game. She also offers words of
encouragement and gives the user a sense of continuity throughout the game.
Figure 2. Snapshots of level-2 to show mission instructions, game in play and debriefing
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6
62
Figure 2 shows four screenshots taken at various points during Level-2 of the game. The first quadrant shows
the start of the scenario with Molly alerting the player to an imminent attack. Information in this mission
brief is given about the players position as well as the direction of the incoming ships. Quadrants 2 and 3
show snapshots of the game in progress. Quadrant 2 shows an approaching wave of ships, whilst quadrant 3
shows a ship that has flown over from behind the player. To make the ships sound as though they fly towards
and away from the player the Doppler effect is applied to the sound. This effect, which is apparent in normal
life, increases the pitch of the sound of the object as it comes towards them and decreases the pitch as the
sound moves away. The remaining player life-span and their score are given in the left and right corners of
the screen respectively. Such information is also accessible audibly by pressing a keyboard control key and is
given automatically should the life-span level become dangerously low. Once the mission has been
successfully completed, Molly gives an encouraging debrief to the players as shown in quadrant 4.
Scenario levels-3 to 6 take place on different planets and have increasingly complex flight patterns for the
enemy ships, shorter player life-spans, more of the powerful ships etc. However they all purposefully
incorporate a similar look-and-feel as shown in Figure 3, which incorporates screen shots taken from level-4.
Another ‘twist-in-the-tail’ for pitting blind players against sighted players is that as the levels become
progressively more difficult the degree of graphical help for sighted players is lessened by making the planets
very dark such that the approach of enemy ships cannot be so readily detected by visual means as shown in
Figure 4.
Figure 3. Consistent look & feel but increasing complexity. Figure 4. Difficult visual interface.
The game may be played using audio and visual information in combination or in isolation of each other. The
different models can be switched in real-time by pressing a single control key. The game play resulting solely
from using the audio interface is shown in Figure5. A series of menus, instruction files and other feedback
information screens such as those for game statistics are also incorporated into the game, some of which are
shown in Figure 6.
Figure 5. Game play solely through audio interface. Figure 6. Introductory and summary screens.
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6 63
5. TESTING
Testing has been ongoing throughout development and a number of important observations have been made.
In particular users initially found it difficult to distinguish between different sounds and from where they
were coming. However, with a little practice they soon became able to locate and shoot the enemy ships. The
graphical interface proved a useful training mechanism even for non-sighted users since with the help of a
sighted friend they were able to gain an understanding of where the sounds were coming from and what they
represented.
Other results were also gleaned about the type of sounds and general game play that were most effective,
for example:
Sounds with a smooth varying pitch were very difficult to locate. It gave the impression that these
sounds were moving when they were in fact not.
Sounds that changed could be located as long as the change was quite harsh. A helicopter’s propellers
or some other sort of machine were easy to locate whereas a sound like siren was not.
With more than two sounds the quieter or less distinguishable sounds became masked and were not
easy to hear until objects making the louder sounds were destroyed.
Two or more sounds which were the same or very similar were also difficult to locate, although not
impossible.
Too many sources of sound were found to be detrimental to the game. Originally every time a players
score increased or their life-span decreased the information was communicated audibly to the player.
This proved to mask out the ship sounds and disadvantage the player. This feature was therefore
subsequently changed to the score being read out on demand via a control key and only automatically
when a players life was becoming dangerously low.
There was a very steep initial learning curve particular for sighted people trying the play the game on
audio alone, as the task is quite peculiar to a lot of sighted people who rely mainly on images to locate
items.
The game became rapidly more challenging as ships flew with circular motions in addition to straight
lines. Further challenges were provided, by having ships flying in both directions over the user and
approaching from both sides simultaneously.
The game is playable using only two speakers although it is obviously better with four speakers. If
only two speakers are available it is best to use headphones instead.
Changing the height of the object has proved very difficult to use with a 4-speaker set-up. The
difference in sound is very small and so is currently not used. Further testing is required, possibly
within a CAVE environment, to make full use of this feature.
The positioning of the speakers was found to be essential for maximum efficiency and enjoyment of
the game. The speakers should be positioned at approximately head height with the player an equal
distance from each speaker.
The initial version of the game has been completed, is comparatively portable across a wide range of PC
specifications and is very robust. Preliminary feedback has been exceedingly encouraging. Unfortunately, the
more extensive and formal evaluations with blind/visually impaired teenagers originally scheduled for
March/April 2000 have had to delayed slightly due to the logistics of term-times and exam timetables.
However, these will shortly be underway and all feedback obtained will impact on the continued
development of the game.
6. FUTURE WORK
A number of advanced features are also planned for implementation such as incorporation of Artificial
Intelligence (AI) techniques to provide added realism to the game and to support help facilities; improved
text to speech and speech recognition functions; an increased number of joystick functions; three dimensional
graphics; and a networked version of the game to enable several players to interact simultaneously within the
same gaming environment. Additionally, exploitation of the new facilities in the recent releases of Aureal’s
A3D SDK (1999) and Microsoft’s DirectX 7.0 (1999) will improve the audio interface and force feedback
interfaces respectively.
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6
64
Proc. 3rd Intl Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000
2000 ICDVRAT/University of Reading, UK; ISBN 0 7049 11 42 6 65
7. SUMMARY
This paper has introduced the work associated with the creation of an Audio Space Invaders game.
Substantial progress has been made in a very short space of time and a fully working version of the game
produced which successfully incorporates the features described in Section 4 and proves that interactive
gaming features can be incorporated within an audio environment. Indeed, such is its success that discussions
are underway with a digital company with a view to professional production of the game and its movement
into the market place. Further user trails are continuing; a number of other audio games are being developed;
and further research into human computer interaction techniques for blind and visually impaired users being
conducted.
8. REFERENCES
Aureal (1999), A3D 2.0 Software Developers Kit, Aureal Semiconductor Inc., 1999
Aureal (1999) Aureal Announces A3D 3.0 Software Development Kit, Aureal Semiconductor Inc, Nov. 11,
http://www.a3d.com/newaureal/index.asp?title=news/title.html&home=news/releases/showpr.asp?releas
e=19991110&nav=news/nav.html, 1999
M Cooper and M E Taylor (1998), Ambisonic Sound in Virtual Environments and Applications for Blind
People, Proceedings of the 2nd European Conference on Disability, Virtual Reality and Associated
Technologies, 1998 pp.113-118
Disability Now (1999), Design for All, http://www.disabilitynow.org.uk/features.htm
DRC - Disability Rights Commission, Promoting Disabled People’s Right’s – a Summary,
http://www.disability. gov. uk/ dispr/summary
FEDA (1998). Equality Assurance: Self-Assessment for Equal Opportunities, FEDA
FEFC (1996), Inclusive Learning: Principles and Recommendations, FEFC.
J Harlow and D Gadher (1999), The Second Internet Revolution, Focus, The Sunday Times, Oct 17th.
M Lumberas and J Sánchez (1998), 3D Aural Interactive Hyper Stories for Blind Children, Proceedings of
the 2nd European Conference on Disability, Virtual Reality and Associated Technologies, 1998 pp.119-
128
M Lumberas, J Sánchez and M Barcia (1996), A 3D hypermedial System for the Blind, Proceedings of the 1st
European Conference on Disability, Virtual Reality and Associated Technologies, 1996 pp.187-191
R J McCrindle (1999), The Impact of New Technologies on Disabled Users, Proceedings European
Conference on Distributed Imaging, Oct, 1999, London, IEE.
S Mereu and R Kazman (1996), Audio Enhanced 3D Interfaces for Visually Impaired Users, Proceedings of
CHI ‘96, ACM Press
Microsoft (1998/1999), DirectX Foundation, MSDN Library, DirextX 6.1 Software Developers Kit, What’s
New in DirectX Foundation?,
http://msdn.microsoft.com/isapi/msdnlib.idc?theURL=/library/psdk/directx/dxintro_6d0v.htm, 1999
Ramstein, Arcand, Deveault (1996), Adaptive User Interfaces with Force Feedback, Proceedings of CHI ‘96,
ACM Press
UK Government (1995), The Disability Discrimination Act, http://www.disability.gov.uk/dda/dle/index.html
W3C (2000), Web Accessibility Initiative, http://www.w3.org/WAI/
... Several tools that support audio game designers, developers and researchers in producing audio games have been created during the last 20 years. These tools can be divided into node-based tools [5,41,45], map makers [30,31], game/level editors [7], code-based tools (software frameworks) [25], full game engines [2,51], or software that is usually used in audio-only disciplines (e.g. Max/MSP in audio engineering [48]). ...
... For evaluation purposes, it is imperative to describe the study, its user goal, and additional contextual information. A typical format for presenting audio games or audio game projects entails a summary of project goals and examples of games that had been developed during the corresponding project [2,5,6,17,20,40,52], as part of third-party projects [31], or that have been created for the commercial market and given to participants [55]. The descriptions of the audio games themselves varied in length, focus and detail (e.g., detailed [58] or rather brief [59]). ...
... However, some papers reported concerns that sighted participants (may) struggle in playing audio-only [31,42]. Therefore, participants were temporarily blinded (by blindfolding them or simulating visual impairment with blurry glasses) [60]. ...
Conference Paper
We look back on two decades of academic research on audio games. During this time, a substantial amount of research has explored many facets of this special genre of computer games. However, despite many publications, there is a lack of review papers, which help delineate this growing research field. For this reason, we take one step back and investigate 20 years of audio game research by synthesizing a literature review adopting grounded theory methods. The resulting research map provides an overview of efforts into audio games with a special focus on how to design for audio games. We observed three important trends or tensions in audio game research. Firstly, audio games research depended heavily on technological advancements during the last two decades. Secondly, most studies about audio games were conducted with novices to audio games in lab situations, that is, based on artificial situations and not on real gamers and their genuine experience. Thirdly, the audio game design process per se has been greatly neglected in the literature so far. We conclude the paper by discussing design or research implications.
... However, a large number of people face difficulties in playing video games due to one or more disabilities. Video games stimulate eyesight, hearing and touch, and require players to analyze this multimodal information and perform motor actions in return (i.e.: ability to hold a joystick or perform large movements) [2,3,4]. ...
... They require players to have sensory, mental and motor skills. These skills are required to interpret stimuli produced by the game and to control the game through input device control (handle a controller, keyboard, mouse, touchscreen interface, etc.) [2,3,4,10]. Yuan et al. [4] identified two categories of stimuli. ...
Chapter
Full-text available
Video games are an integral part of popular culture. The video game industry faces challenges with the increase in players’ numbers and application areas including serious games. The increase in the number of players includes disabled players. Therefore, serious games have to consider these audiences who may be affected by one or more temporary or ongoing disabilities. Universally accessible games (UA-Games) aim to create interfaces that can be accessed and manipulated by the largest number of players. Currently few serious games include accessibility features, while accessibility should be considered at the beginning of the serious game design. Then how can we help designers and developers to include accessible features in an existing serious game? To this end, game engines must be efficient but also scalable and modular. This paper studies the interest of the Entity-Component-System (ECS) software architecture to integrate accessibility features in an existing serious game. As a case study, we will take a serious game developed with ECS yet not accessible: E-LearningScape. We will present the accessible features that we have integrated into the serious game and discuss the pros and cons of this approach. This feedback shows us that ECS provides very useful design flexibility to integrate unanticipated interaction features into serious games.
... The third category has several works with auditory graphs (FLOWERS; HAUER, 1993;BROWN;BREWSTER, 2003), and sonifications techniques based on models (HER-MANN;HUNT, 2005). The fourth category, which is the one closest to the work presented in this paper, has applications in entertainment (for instance, audio-only versions of games) (WINBERG; HELLSTROM, 2001;MCCRINDLE;SYMONS, 2000), and in art (for instance, musical compositions made through data sonification) (QUINN, 2001;QUINN;QUINN;HATCHER, 2003). ...
... The third category has several works with auditory graphs (FLOWERS; HAUER, 1993;BROWN;BREWSTER, 2003), and sonifications techniques based on models (HER-MANN;HUNT, 2005). The fourth category, which is the one closest to the work presented in this paper, has applications in entertainment (for instance, audio-only versions of games) (WINBERG; HELLSTROM, 2001;MCCRINDLE;SYMONS, 2000), and in art (for instance, musical compositions made through data sonification) (QUINN, 2001;QUINN;QUINN;HATCHER, 2003). ...
Article
Full-text available
In this paper, we present an investigation on sonification of web pages involving three media: HTML struc- ture, page text and images. In our approach, HTML pages are read as musical scores; text elements are also read as musical scores following the idea present in notation systems like ABC; and images are read as wave files. We deve- loped a set of tools for website sonification which can be used to explore musical creativity and to create new soun- ds based on the web pages content. In addition, we show how to create sounds and visual feedback using this tool.
... The relevant papers are from 1999 to 2014, being 80% of the papers from 2008 on. The selected papers were: [13,51,2,35,3,4,8,15,12,17,18,41,42,24,25,30,28,29,32,33,31,37,39,38,40,36,34,47,48,49]. Among these, 25 papers described 21 distinct applications: 17 multimodal games and four multimodal navigation virtual environment. ...
... The improvement of this skill involves the user adopting and restructuring a mental model of spatial dimensions, based on audio and haptic cues after interacting with a video game [42]. The adequate orchestration between audio and haptics can also help the user who is blind to build up a model of the fantasy world and of how a game works [18,51]. Specific mental models are stimulated by 27% of the studied applications. ...
Conference Paper
Full-text available
Multimodal serious video games are relevant tools to enhance the cognitive skills of people who are blind. For this purpose, it is necessary that designers and developers be able to create interfaces and interactions using the multimodal elements properly. Thus, there is a need to know the relevant elements for such applications, as well as to understand their roles and relationships. In this paper, we propose and discuss a 4-dimension classification: Interface, Interaction, Cognition, and Evaluation. Such classification was assembled from the features related to the design and evaluation of 21 multimodal video games and environments, identified via a bibliographic review based on the systematic review approach. Besides, we classify and discuss the 21 multimodal applications into the proposed classification.
... Special decoders are used to play back Ambisonic sound, converting Ambisonic recordings into formats supported by conventional stereo systems or headphones, ensuring realistic sound reproduction in three dimensions. Also, research on three-dimensional audio spaces includes studies by R.J. McCrindle & D. Symons (2022). Scientists point to the importance of moving from stereo audio recording and playback to three-dimensional audio technologies. ...
Article
Full-text available
Immersive audio technology is increasingly used in music, with devices like cell phones and televisions replicating formats. The increasing adoption of immersive audio technologies by both consumers and professionals in the music industry has made this topic more relevant. The purpose of the research is to analyse immersive technologies that contribute to the creation of the musical audio environment. The research employed methods: literature and historical review, technical analysis, practical cases, statistical method, and the author’s personal empirical experience. The article examines the function of audio space in relation to the perception of musical art and the characteristics of musical audio space. The proposed classification categories musical audio spaces into two distinct types: natural and artificial, as well as offline and online. The article’s statistical data illustrates the changing patterns of growth in online audio consumers and the rivalry between radio stations and streaming platforms to enhance their impact on shaping listeners’ audio spaces. The research demonstrates that streaming platforms actively encourage the incorporation of musical content in immersive formats as a means to expand their market share and evoke heightened emotional and perceptual responses in customers. The article outlines techniques used in immersive music spaces to provide a distinct aural experience for the listener. The article’s practical analysis of immersive audio technologies encompasses the examination of requisite microphones and digital audio workstations that facilitate the transition from two-dimensional audio space formation to threedimensional. In contemporary circumstances, the distribution of musical items is progressively employing threedimensional immersive technologies. The article enumerates audio technologies that guarantee the creation of an immersive auditory environment for both listeners and musicians: Ambisonics, KLANG: technologies, and DPA 5100. The practical significance of this work rests in examining the perceptual attributes of users of immersive technologies and exploring the potential for developing immersive technologies, such as immersive musical instruments or immersive radio stations
... Development of user centered interfaces and technologies have become crucial in the process designing for the differently abled people. Adding an extra element is just not enough to assist the use of technology for the visually disabled [5][6][7]. Many device-based hardware and software technologies exist to assist the visually disabled. ...
Article
Full-text available
Focusing and addressing the problems faced by the differently abled people such as visually, audibly and vocally challenged, through a single device is a tough job. A lot of research has been done on each problem and solutions have been proposed separately. But not all of them are addressed together. The aim of the project is to create a single device solution in such a way that is simple, fast, accurate and cost-effective. The main purpose of the device is to make the differently abled people, feel independent confident by seeing, hearing and talking for them. The paper provides a Google API and Raspberry Pi based aid for the blind deaf and dumb people. The proposed device enables visually challenged people to read by taking an image. Further, Image to text conversion and speech synthesis is done, converting it into an audio format that reads out the extracted text translating documents, books and other available materials in daily life. For the audibly challenged, the input is in form of speech taken in by the microphone and recorded audio is then converted into text which is displayed in the form of a pop-up window for the user in the screen of the device. The vocally impaired are aided by taking the input by the user as text through the built-in customized on-screen keyboard where the text is identified, text into speech conversion is done and the speaker gives the speech output. This way the device speaks for the user.
Chapter
Full-text available
There has been an increasing popularity in audio-only games, particularly for mobile devices. These are usually based on spoken-word, sound-effect scene setting, or DJ/musical-instrument simulation. This chapter surveys the field and discusses the concept of games which are entirely abstract musical experiences, presenting a first example based on Minecraft and music memory games: Musicraft. The comparisons with Minecraft can be extended into the idea of a creative mode: a quite natural approach in music-only games which has not generally been considered in audiogame research.
Chapter
Full-text available
This Chapter introduces the concept of Universally Accessible Games (UA-Games) as a novel approach to creating games which are proactively designed to be concurrently accessible by people with a wide range of diverse requirements and / or disabilities. First, background information is provided on the various types of disabilities affecting game accessibility, the way that people with disabilities play games, and guidance and approaches for achieving game accessibility. Then, related work is presented, followed by an introduction to the concept and goals of Universally Accessible Games. The main part of the Chapter is dedicated to a step by step presentation of a structured method for designing Universally Accessible Games. Finally, some conclusions are offered along with directions for future work.
Article
Full-text available
Interactive stories are commonly used for learning and entertaining purposes enhancing the development of several perceptual and cognitive skills. These experiences are not very common among blind children because most computer games and electronics toys do not have appropriate interfaces to be accessible by them. This study introduces the idea of interactive Hyperstories performed in a 3D acoustic virtual world. The hyperstory model enables us to build an application to help blind children to enrich their early world experiences through exploration of interactive virtual worlds by using 3D aural representations of the space. We have produced AudioDoom, interactive model-based software for blind children. The prototype was qualitative and quantitatively field-tested with several blind children in a Chilean school setting. Our preliminary results indicate that when acoustic-based entertainment applications are careful applied with an appropriate methodology can stimulate diminished cognitive skills. We also found that spatial sound experiences can create spatial navigable structures in the mind of blind children. Methodology and usability evaluation procedures and results appeared to be critical to the efectiveness of interactive Hyperstories performed in a 3D acoustic virtual world.
Article
Full-text available
To date there has been much more effort directed to generating credible presentations of virtual worlds in a visual medium than in reproducing corresponding or even self contained worlds with synthesised or recorded sound. There is thus today a disparity in the relative fidelity of these 2 modes in most VR systems. While much work has been done in hi-fidelity and 3 dimensional sound reproduction in psycho-acoustic research and applications for audiophiles this has rarely been taken onboard by the VR community. This paper describes work ongoing to apply Ambisonic techniques to the generation of audio virtual worlds and environments. Firstly Ambisonics is briefly outlined then principles behind its implementation in this context described. The design of the implementations to date is described and the results of trials discussed. The strengths and limitations of the approach are discussed in the light of this practical experience. There is a range of applications envisaged for this technique that would be particularly of benefit to disabled people. The 2 principal areas under consideration by the authors is the extended use of the audio mode in HCI for people with disabilities and VR applications for blind and partially sighted people. Both of these areas are described and specific examples of applications given in each. The limitations given available low cost technology are highlighted and the technology evolution required to make such applications widespread commented on. The results of the development work and subsequent user trials undertaken to date are given and discussed. Lines of further exploration of this technique and its application are outlined.
Conference Paper
Three dimensional computer applications such as CAD packages are often difficult to use because of inadequate depth feedback to the user. It has, however, been shown that audio feedback can help improve a user's sense of depth perception. This paper describes an experiment which evaluates the use of three different audio environments in a 3D task undertaken by visually impaired users. The three audio environments map tonal, musical, and orchestral sounds to an (x, y, z) position in a 3D environment. In each environment the user's task is to locate a target in three dimensions as accurately and quickly as possible. This experiment has three important results: that audio feedback improves performance in 3D applications for all users; that visually impaired users can use 3D applications with the accuracy of sighted users; and that visually impaired users can attain greater target accuracy than sighted users in a sound-only environment
3D Aural Interactive Hyper Stories for Blind Children Virtual Reality and Associated Technologies A 3D hypermedial System for the Blind
  • M Lumberas
  • J Sánchez
  • J Lumberas
  • M Sánchez
  • Barcia
M Lumberas and J Sánchez (1998), 3D Aural Interactive Hyper Stories for Blind Children, Proceedings of the 2nd European Conference on Disability, Virtual Reality and Associated Technologies, 1998 pp.119- 128 M Lumberas, J Sánchez and M Barcia (1996), A 3D hypermedial System for the Blind, Proceedings of the 1st European Conference on Disability, Virtual Reality and Associated Technologies, 1996 pp.187-191 R J McCrindle (1999), The Impact of New Technologies on Disabled Users, Proceedings European Conference on Distributed Imaging, Oct, 1999, London, IEE
Audio Enhanced 3D Interfaces for Visually Impaired Users DirectX Foundation, MSDN Library, DirextX 6.1 Software Developers Kit, What's New in DirectX Foundation?
  • S Mereu
  • R Kazman
S Mereu and R Kazman (1996), Audio Enhanced 3D Interfaces for Visually Impaired Users, Proceedings of CHI '96, ACM Press Microsoft (1998/1999), DirectX Foundation, MSDN Library, DirextX 6.1 Software Developers Kit, What's New in DirectX Foundation?, http://msdn.microsoft.com/isapi/msdnlib.idc?theURL=/library/psdk/directx/dxintro_6d0v.htm, 1999
Adaptive User Interfaces with Force Feedback The Disability Discrimination Act
  • Arcand Ramstein
Ramstein, Arcand, Deveault (1996), Adaptive User Interfaces with Force Feedback, Proceedings of CHI '96, ACM Press UK Government (1995), The Disability Discrimination Act, http://www.disability.gov.uk/dda/dle/index.html W3C (2000), Web Accessibility Initiative, http://www.w3.org/WAI/
Design for Allfeatures.htm DRC -Disability Rights Commission, Promoting Disabled People's Right's – a Summary Equality Assurance: Self-Assessment for Equal Opportunities
  • Disability Now
Disability Now (1999), Design for All, http://www.disabilitynow.org.uk/features.htm DRC -Disability Rights Commission, Promoting Disabled People's Right's – a Summary, http://www.disability. gov. uk/ dispr/summary FEDA (1998). Equality Assurance: Self-Assessment for Equal Opportunities, FEDA FEFC (1996), Inclusive Learning: Principles and Recommendations, FEFC.
Aureal Announces A3D 3.0 Software Development Kit, Aureal Semiconductor Inc
Aureal (1999) Aureal Announces A3D 3.0 Software Development Kit, Aureal Semiconductor Inc, Nov. 11, http://www.a3d.com/newaureal/index.asp?title=news/title.html&home=news/releases/showpr.asp?releas e=19991110&nav=news/nav.html, 1999