Conference PaperPDF Available

AudioBattleship: blind learners collaboration through sound.

Authors:
Jaime Sánchez at University of Chile
Jaime Sánchez
Nelson Baloian at University of Chile
Nelson Baloian
Tiago Hassler
Tiago Hassler
Tiago Hassler
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Heinz Ulrich Hoppe
Heinz Ulrich Hoppe
Heinz Ulrich Hoppe
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Download full-text PDF
Read full-text
Download citation

Abstract

A growing number of audio-based applications for blind learners have being produced in the last few years. Many of them focus on the development of 3D audio interfaces to map the entire surrounding space. Other studies center on the impact of sound interaction on cognition by evaluating the usability of these applications. No previous work has centered on using spatialized sound to develop collaborative skills in blind learners. This ongoing research study introduces AudioBattleShip, an interactive audio-based environment to enhance collaboration and cognition in blind learners. AudioBattleship mimics the traditional game battleship for sighted people but without visual cues. A preliminary pilot study has been implemented showing that blind children collaboration can be enhanced through the interaction with spatialized sound.
Content uploaded by Nelson Baloian
Author content
All content in this area was uploaded by Nelson Baloian
Content may be subject to copyright.
AudioBattleship: Blind Learners Collaboration
through Sound
Jaime Sánchez, Nelson Baloian,
Tiago Hassler
Department of Computer Science
University of Chile
Blanco Encalada 2120, Santiago, CHILE
+ 56 2 673 1280
{jsanchez, nbaloian, thassler}@dcc.uchile.cl
Ulrich Hoppe
Institute for Computer Science
University of Duisburg
Lotharstr. 65, 47048, Duisburg, GERMANY
+ 49 203 379 3553
hoppe@informatik.uni-duisburg.de
ABSTRACT
A growing number of audio-based applications for
blind learners have being produced in the last few
years. Many of them focus on the development of 3D
audio interfaces to map the entire surrounding space.
Other studies center on the impact of sound interaction
on cognition by evaluating the usability of these
applications. No previous work has centered on using
spatialized sound to develop collaborative skills in
blind learners. This ongoing research study introduces
AudioBattleShip, an interactive audio-based
environment to enhance collaboration and cognition in
blind learners. AudioBattleship mimics the traditional
game battleship for sighted people but without visual
cues. A preliminary pilot study has been implemented
showing that blind children collaboration can be
enhanced through the interaction with spatialized
sound.
Keywords
Virtual acoustic environment, blind children,
collaborative learning, audio-based navigation,
usability, spatialized sound
INTRODUCTION
Diverse audio-based applications for blind children
have being produced in the last few years [1, 2, 3, 4, 5,
6, 7]. Many of them focus on the development of 3D
audio interfaces to map the entire surrounding space as
proof of concept concerning audio-based interfaces.
Other studies center on the effects and impact of sound
on the development of cognitive skills of blind people
by evaluating the cognitive usability of these
applications. No previous work has centered on using
spatialized sound to develop collaborative skills in
blind learners.
This ongoing research study introduces
AudioBattleship, an interactive audio-based
environment to enhance collaboration, abstract
memory, spatial abstraction, and haptic perception in
blind learners. AudioBattleship is a similar version of
the traditional game battleship for sighted people but
without visual cues. A preliminary pilot usability
study has been implemented showing that blind
learners collaboration skills can be enhanced through
the interaction with spatialized sound.
AUDIOBATTLESHIP
AudioBattleShip was built in Java SDK 1.4.1, using a
collaborative platform, MatchMaker, which allows
synchronizing a tree of distributed objects. Java Media
Framework was used to synchronize multimedia
events. The environment was divided in three classes:
communication, sound, and the main program.
AudioBattleShip has objects that store the information
of players, the state of the game (turns), and the
coordinates of a given point with the corresponding
state (ship, water, destroyed ship, shooting to water).
By using these objects a synchronization tree
(Matchmaker) is created with nodes that store
information about the players which also contains
objects corresponding with the size of the board.
AudioBattleShip contains a directory of sounds that
can be easily modified to change language and sounds.
Copyright is held by the author/owner(s).
CHI 2003, April 5–10, 2003, Ft. Lauderdale, Florida, USA.
ACM 1-58113-630-7/03/0004.
Short Talks: Specialized Section CHI 2003: NEW HORIZONS
798
Short Talk: Brains, Eyes and Ears CHI 2003: NEW HORIZONS
798
Copyright is held by the author/owner(s).
CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA.
ACM 1-58113-637-4/03/0004
Playing modes
AudioBattleShip can be played between blind to blind,
blind to sighted, and blind to computer. Blind to blind
mode presents the same interface to both players.
Blind to sighted mode provides a variety of tools to
the blind learner to minimize the disadvantages in
comparison with sighted learners who can have in any
moment snapshots of the state of actions. Blind to
computer mode gives intrinsic advantages to the
computer over the blind learner because of computer
memory and probability calculations, but this can be
diminished by limiting the algorithm and creating
different levels of complexity.
Development
AudioBattleShip was developed in three steps: 1.
Design and implementation of the basic objects, and
creation of the MatchMaker tree and internal models,
2. Input, interface, and feedback operations, and 3.
Turn-taking and control. We created two interfaces for
both blind and sighted learners. The interface for blind
learners consists of a board and help buttons. Sound
feedback is provided to inform about a specific spatial
location on the board and the occurrence of certain
actions. We use as input interface a tablet device and
built upon it a grille to represent an exact copy of the
visual interface. The tablet can map the entire screen
and by using a pen-based pointing device diverse
events can be triggered. The interface for sighted
learners consists of two boards, one per each player, as
well as a text area that provides feedback for actions.
Software modes
AudioBattleShip has three modes: Ship positioning
mode, creating a new session or joining to an ongoing
session mode, and shooting mode. Both players can
manipulate matrix with the board’s state (see Figure
1).
PRELIMINARY USABILITY TESTING RESULTS
AudioBattleShip was usability pilot tested with four
blind learners, two legally blind and two with residual
vision. They played the game and answered questions
during six sessions. Most questions were related to the
audio interface, graphic interface (for the residual
vision learners), and the use of input devices such as
the tablet. They were also observed in all sessions by
two people from the research team. Comments were
registered. The evaluation for cognitive achievement is
under way right now. As a result of the preliminary
testing, learners helped to redesign the software by
making insight and meaningful contributions about the
sounds used, intensity and quality, volume, feedback,
sound synchronicity, sound overlapping, sound help,
color contrast, size of the cursor, position
identification number, and tablet mapping.
The four learners enjoyed interacting with
AudioBattleShip (see Figure 2). There were diverse
forms of interaction between them. This is especially
important for blind learners because they are
accustomed to do individual work with little social
interaction when using digital devices. Collaboration
skills with obvious constrains can be enhanced with
audio-based applications such as AudioBattleShip. We
need to know what type of skills can be enhanced most
and ways of improving them through the interaction
with sound. A current full field study will give us
detailed insights and knowledge about how cognition
and collaboration can be enhanced by the interaction
with sound.
REFERENCES
1. Baldis, J. Effects of spatial audio on memory,
comprehension, and preference during desktop
conferences. Proceeding of the ACM CHI ´01,
Vol 3, 1, (2001), 166-173.
2. McCrindle, R. & Symons, D. Audio space
invaders. Proceedings of the Third International
Conference on Disability, Virtual Reality and
Associated Technologies, (2000), 59-65.
3. Mereu, S. & R. Kazman. Audio enhanced 3D
interfaces for visually impaired users.
Proceedings of CHI ‘ 96, ACM Press., (1996).
4. Sánchez, J. Interactive 3D sound hyperstories for
blind children. Proceedings of ACM-CHI 99,
(1999), 318-325.
5. Sánchez, J. Interactive virtual acoustic
environments for blind children. Proceedings of
ACM CHI ´2001, pp. 23-25. Seattle,
Washington, April, (2001), 2-5.
6. Sjostrom, C. Using haptics in computer
interfaces for blind people. Proceeding of the
ACM CHI ´01, Vol 3, 1, (2001), 245-246.
7. Tan, H. Haptic interfaces. Communications of
the ACM, 43(3), (2000), 40-41.
Short Talks: Specialized Section CHI 2003: NEW HORIZONS
799
Short Talk: Brains, Eyes and Ears CHI 2003: NEW HORIZONS
799
... In The Last Crusade [18], self-voicing is used to find out happening events in the game. In AudioBattleShip [42], audio cues are used to help the player to find the locations on the board or happening actions such as a dropping bomb over certain cells in the grid of the game. Tim's Journey is an adventure game with a non-linear narration that is open-ended. ...
Sonification and interaction design in computer games for visually impaired individuals
Article
Full-text available
Video games are changing how we interact and communicate with each other. They can provide an authentic and collaborative platform for building new communities and connecting people. Since video games generally rely on visual elements that must be recognized by the players, most of them are not accessible by visually impaired people. In this research, we study the sonification and interaction issues in the design of computer games for visually impaired individuals. We have proposed an audio game called GrandEscape with the focus on the special needs of visually impaired people while playing. A comprehensive set of user studies has been performed to evaluate different interaction and sonification techniques in terms of providing a sense of presence and gaming experience.
View
... Although they were inclusive, these two prototypes were evaluated by people with VI or blindfolded participants playing against the computer and not a sighted human. AudioBattleship [49] is a multiplayer adapted version of a battleship game that was developed to foster collaboration between blind learners. This game integrates balanced difficulty levels for playing with sighted people to ensure fairness (the same probability of winning independently of visual status). ...
Inclusive Adaptation of Existing Board Games for Gamers with and without Visual Impairments using a Spatial Augmented Reality Framework for Touch Detection and Audio Feedback
Article
  • Nov 2021
Board games allow us to share collective entertainment experiences. They entertain because of the interactions between players, physical manipulation of tokens and decision making. Unfortunately, most board games exclude people with visual impairments as they were not initially designed for players with special needs. Through a user-centered design process with an accessible game library and visually impaired players, we observed challenges and solutions in making existing board games accessible through handcrafted solutions (tactile stickers, braille labels, etc.). In a second step, we used Spatial Augmented Reality (SAR), to make existing board games inclusive by adding interactivity (GameARt). In a case study with an existing board game considered as not accessible (Jamaica), we designed an interactive SAR version with touch detection (JamaicAR). We evaluated this prototype in a user study with 5 groups of 3 players each, including sighted, low vision and blind players. All players, independent of visual status, were able to play the Augmented Reality game. Moreover, the game was rated positively by all players regarding attractiveness, play engrossment, enjoyment and social connectivity. Our work shows that Spatial Augmented Reality has the potential to make board games accessible to people with visual impairments when handcrafted adaptations fall short.
View
... An interactive, audio-based virtual environment called AudioBattleShip is presented in another research article by Sánchez, Baloian, Hassler et al. [56]. It aims to enhance collaboration and cognition in blind learners, and focuses on using spatialized sound to develop collaborative skills in blind learners. ...
Computer Entertainment Technologies for the Visually Impaired: An Overview
Article
Full-text available
  • Jan 2021
Over the last years, works related to accessible technologies have increased both in number and in quality. This work presents a series of articles which explore different trends in the field of accessible video games for the blind or visually impaired. Reviewed articles are distributed in four categories covering the following subjects: (1) video game design and architecture, (2) video game adaptations, (3) accessible games as learning tools or treatments and (4) navigation and interaction in virtual environments. Current trends in accessible game design are also analysed, and data is presented regarding keyword use and thematic evolution over time. As a conclusion, a relative stagnation in the field of human-computer interaction for the blind is detected. However, as the video game industry is becoming increasingly interested in accessibility, new research opportunities are starting to appear.
View
... For example, "Terraformers" [16] uses sound propagation to convey the proximity of obstacles. "Terraformers" [16] and "AudioBattleship" [17] give feedback for every action using a voice over. "Sonic-Badminton" [18] enables play regardless of visual impairment by presenting the position of the wings of the badminton with stereophonic sound. ...
Auditory Uta-Karuta: Development and Evaluation of an Accessible Card Game System Using Audible Cards for the Visually Impaired
Article
Full-text available
  • Mar 2021
Playing board games is important for people with a visually impairment, as it promotes interactive socialization and communication skills. However, some board games are not accessible to them at present. In this study, we proposed an auditory card game system that presents a card’s contents with auditory stimuli to all players, towards playing equally with others, regardless of whether they have a visual impairment or not as one of the solutions to make board games accessible. This proposal contributes significantly to expand the range of inclusive board games for the visually impaired. The purpose of this paper is to determine whether the game allows for fair competition for people with visual impairments and to clarify the effects of the valuable parameters of the system on the players. The effectiveness of the proposed system was verified by having experimental participants play “Auditory Uta-Karuta”. The results suggested that the proposed system has the potential for an accessible board game design regardless of visual impairment. In the following experiment, we investigated the impact of each valuable parameter of the system on the player’s perception of the board games to clarify the appropriate audio cue design method. The results of this experiment will greatly assist in designing an appropriate board game using the proposed system.
View
... Sánchez et al. [25] investigated the potential of audio games for supporting learning and mental map generation in blind school children. This work and related follow-up research [24,18,7] focused on learning processes mediated by audio games. Hence, the games per se and the underlying design processes were delineated as black boxes, and there were no audio gamers involved in the studies. ...
Unpacking the Audio Game Experience: Lessons Learned from Game Veterans
Conference Paper
Full-text available
  • Oct 2019
People with or without visual impairments play and enjoy audio games. While this genre of computer games has attracted a strong fan base and some attention in HCI, little research has been dedicated to the people who actually play audio games in their daily life. There is a pressing need to capture the viewpoints of authentic or expert players, designers and developers to advance audio game design. Thus, we give voice to seven game veterans of sound-based gaming, i.e., people who each have more than a decade of profound experience in playing or designing audio games. We conducted a total of 14 interviews and employed grounded theory methods to unpack their experiences. We found that audio games enriched their life through creativity, play, and social exchange. Those core concepts were influenced by peripheral concepts like, inter alia, aesthetics & enjoyability, accessibility, or the availability of audio games. We show how they relate to each other and discuss design implications.
View
... There is also an optional voice-based environment analyzer. AudioBattleShip (Sánchez et al. 2003) is also designed for people who are blind. Once again 3D sound is used to implement the game, in this case the traditional battleships board game. ...
Evaluating BCI for Musical Expression: Historical Approaches, Challenges and Benefits
Chapter
  • May 2019
A recurring challenge in the use of BCI (and more generally HCI) for musical expression is in the design and conduct of appropriate evaluation strategies when considering BCI systems for music composition or performance. Assessing the value of computationally assisted creativity is challenging in most artistic domains, and the assessment of computer assisted (or entirely computer generated) music is no different. BCI provides two unique possibilities over traditional evaluation strategies: firstly, the possibility of devising evaluations which do not require conscious input from the listener (and therefore do not detract from the immersive experience of performing, creating, or listening to music), and secondly in devising neurofeedback loops to actively maneuver the creator or listener through an expressive musical experience. Music offers some unusual challenges in comparison to other artistic interfaces: for example, often it is made in ensemble, and there is evidence to suggest neurophysiological differences are evident in ensemble measurement when compared to solo performance activities, for example see (Babiloni et al. in cortex 47:1082–1090, 2011). Moreover, a central purpose of music is often to incite movement (swaying, nodding head, dancing)—both in performer and audience—and as such this also offers up challenges for BCI/HCI design. This chapter considers historical approaches as well as making proposals for borrowing solutions from the world of auditory display (also referred to as sonification) and psychoacoustic evaluation techniques, to propose a hybrid paradigm for the evaluation of expression in BCI music applications.
View
... There is also an optional voice-based environment analyzer. AudioBattleShip (Sánchez et al. 2003) is also designed for people who are blind. Once again 3D sound is used to implement the game, in this case the traditional battleships board game. ...
Emotion in Video Game Soundtracking
Book
  • Feb 2018
This book presents an overview of the emerging field of emotion in videogame soundtracking. The emotional impact of music has been well-documented, particularly when used to enhance the impact of a multimodal experience, such as combining images with audio as found in the videogames industry. Soundtracking videogames presents a unique challenge compared to traditional composition (for example film music) in that the narrative of gameplay is non-linear – Player dependent actions can change the narrative and thus the emotional characteristics required in the soundtrack. Historical approaches to emotion measurement, and the musical feature mapping and music selection that might be used in video game soundtracking are outlined, before a series of cutting edge examples are given. These examples include algorithmic composition techniques, automated emotion matching from biosensors, motion capture techniques, emotionally-targeted speech synthesis and signal processing, and automated repurposing of existing music (for example from a players own library). The book concludes with some possibilities for the future.
View
View
Understanding Tabletop Games Accessibility: Exploring Board and Card Gaming Experiences of People who are Blind and Low Vision
Conference Paper
  • Feb 2022
Gaming accessibility research for blind or low vision (BLV) commu- nities largely focuses on digital games. There is a need for designers to understand BLV’s experience with tabletop games that involve the player’s physical interaction. In this study, we investigate BLV individuals’ experience with the accessibility of tabletop games. We conducted semi-structured interviews with 15 BLV participants and found four themes that uncovered participants’ tabletop gam- ing experiences: (1) properties of inaccessible games, (2) outcomes of inaccessible games, (3) properties of accessible games, and (4) outcomes of accessible games. Our findings demonstrate a richness and variety in BLV individuals’ tabletop gaming experiences. By providing discussions on the state of tabletop game interactivity and design recommendations, our work assists the creation of ac- cessible tangible games that make use of digital information and physical forms by affording designers the opportunity to under- stand how inaccessible interactions in tabletop games affect BLV populations.
View
View
Interactive virtual acoustic environments for blind children
Conference Paper
Full-text available
  • Jan 2001
Blind children tend to represent spatial environments with cognitive difficulty. The absence of visual stimuli processing impedes the construction of rich representations of the surrounding space. This can be decreased if they are exposed to interactive experiences with acoustic stimuli delivered through spatialized sound software complemented with cognitive tasks tailored to accomplish richer representations. A few studies have approached this issue by using interactive applications that integrate virtual reality and cognitive tasks to enhance spatial orientation skills. The aim of this research has been to design interactive software based on spatialized sound to help blind learners to construct cognitive spatial structures.
View
View
Interactive virtual acoustic environments for blind children: Computing, usability, and cognition
Article
Full-text available
  • Mar 2001
Blind children tend to represent spatial environments with cognitive difficulty. The absence of visual stimuli processing impedes the construction of rich representations of the surrounding space. This can be decreased if they are exposed to interactive experiences with acoustic stimuli delivered through spatialized sound software complemented with cognitive tasks tailored to accomplish richer representations. A few studies have approached this issue by using interactive applications that integrate virtual reality and cognitive tasks to enhance spatial orientation skills. The aim of this research has been to design interactive software based on spatialized sound to help blind learners to construct cognitive spatial structures.
View
Perceptual user interfaces: haptic interfaces
Article
Full-text available
  • Mar 2000
An abstract is not available.
View
Audio space invaders
Article
Full-text available
  • Jan 2000
Whilst advances are underway in various areas to ease and encourage disabled uptake of new technology, very little emphasis to date has been placed on making the games market accessible to all. The aims of the described work have been twofold. Firstly, to prove that the standard features of a traditional space invader game can be replicated using a 3-D audio (ambisonic) environment. Secondly, through combining audio and visual interfaces with force feedback joystick movement that it is possible to produce a multi-modal game that can be played by both sighted and non-sighted users, thereby enabling them to share the same gaming experience. This paper describes the development and features of the resultant Audio Space Invaders game.
View
Interactive 3D Sound Hyperstories for Blind Children.
Conference Paper
Full-text available
  • Jan 1999
Interactive software is currently used for learning andentertainment purposes. This type of software is not very commonamong blind children because most computer games and electronictoys do not have appropriate interfaces to be accessible withoutvisual cues.This study introduces the idea of interactive hyperstoriescarried out in a 3D acoustic virtual world for blind children. Wehave conceptualized a model to design hyperstories. ThroughAudioDoom we have an application that enables testing cognitivetasks with blind children. The main research question underlyingthis work explores how audio- based entertainment and spatial soundnavigable experiences can create cognitive spatial structures inthe minds of blind children.AudioDoom presents first person experiences through explorationof interactive virtual worlds by using only 3D auralrepresentations of the space.
View
View
Audio enhanced 3D interfaces for visually impaired users
Conference Paper
  • Jan 1996
Three dimensional computer applications such as CAD packages are often difficult to use because of inadequate depth feedback to the user. It has, however, been shown that audio feedback can help improve a user's sense of depth perception. This paper describes an experiment which evaluates the use of three different audio environments in a 3D task undertaken by visually impaired users. The three audio environments map tonal, musical, and orchestral sounds to an (x, y, z) position in a 3D environment. In each environment the user's task is to locate a target in three dimensions as accurately and quickly as possible. This experiment has three important results: that audio feedback improves performance in 3D applications for all users; that visually impaired users can use 3D applications with the accuracy of sighted users; and that visually impaired users can attain greater target accuracy than sighted users in a sound-only environment
View
Designing haptic computer interfaces for blind people
Conference Paper
  • Feb 2001
Certec has been working on touch interfaces - haptic interfaces since 1995, exploring the possibilities they can offer people with different kinds of disabilities. With a haptic computer interface a blind person can learn mathematics by tracing touchable mathematical curves, playing haptic computer games, and gaining better access to graphical user interfaces like Windows. This paper presents a brief overview of a set of tests that have been made and some of the results from these tests. This is followed by a set of design recommendations that we have been able to extract as an extended result of this research and development work. These guidelines are grouped under the headings navigation, finding objects, understanding objects, haptic widgets and physical interaction
View
Haptic interfaces impaired users. virtual acoustic pp. 23-25. Seattle, Short Talks: Specialized Section Short Talk: Brains
  • 40-41
  • H Tan
Tan, H. Haptic interfaces. Communications of the ACM, 43(3), (2000), 40-41. impaired users. virtual acoustic pp. 23-25. Seattle, Short Talks: Specialized Section Short Talk: Brains, Eyes and Ears CHI 2003: NEW HORIZONS CHI 2003: NEW HORIZONS 799 799