Content uploaded by Alexis Kirke
Author content
All content in this area was uploaded by Alexis Kirke on Apr 01, 2019
Content may be subject to copyright.
When the Soundtrack Is the Game: From Audio-Games to Gaming the Music
Abstract
There has been an increasing popularity in audio-only games, particularly for mobile devices. These are usually based on spoken-word, sound-effect scene setting, or DJ/musical-instrument simulation. This chapter surveys the field and discusses the concept of games which are entirely abstract musical experiences, presenting a first example based on Minecraft and music memory games: Musicraft. The comparisons with Minecraft can be extended into the idea of a creative mode: a quite natural approach in music-only games which has not generally been considered in audiogame research.
... People with visual impairment (PVI) have difculty accessing digital games due to the inherently visual nature of such games [15,98]. Although some solutions for PVI to engage with digital games have emerged, particularly in the form of audiogames and text-based adventures, providing PVI access to mainstream gaming experiences remains a challenge [4,38,85]. One particular challenge PVI face when engaging with digital games is difculty in acquiring information about the spatial characteristics of the virtual world. ...
... There is a growing interest in digital games targeted to PVI (commonly referred to as audiogames) and in improving accessibility in mainstream games [4,38,85]. However, for PVI, the transition from 2D to 3D graphics was an efective barrier that has prevented them from engaging with modern mainstream games [4,85]. ...
... It has been argued that playing games is a fundamental human desire [34] and specifc tools have been designed to bring access to PVI to specifc games [41,84,97]. However, one recurrent challenge these tools face is the sequential and non-persistent nature of sound [2,38]. Integrating diferent means to use active echolocation in virtual worlds-through sound pulses, footsteps, clapping sounds or others-may help address these challenges by giving agency to the player, who will decide when and how to receive information from their surroundings. ...
In virtual environments, spatial information is communicated visually. This prevents people with visual impairment (PVI) from accessing such spaces. In this article, we investigate whether echolocation could be used as a tool to convey spatial information by answering the following research questions: What features of virtual space can be perceived by PVI through the use of echolocation? How does active echolocation support PVI in acquiring spatial knowledge of a virtual space? And what are PVI’s opinions regarding the use of echolocation to acquire landmark and survey knowledge of virtual space? To answer these questions, we conducted a two-part within-subjects experiment with 12 people who were blind or had a visual impairment and found that size and materials of rooms and 90-degree turns were detectable through echolocation, participants preferred using echoes derived from footsteps rather than from artificial sound pulses, and echolocation supported the acquisition of mental maps of a virtual space. Ultimately, we propose that appropriately designed echolocation in virtual environments improves understanding of spatial information and access to digital games for PVI.
... color and contrast options) and depend on audio feedback. Past research has explored diferent strategies to adapt games that require vision to an accessible audio-based gameplay [4,33,39,40,48,58,61]. In most cases a substantial redesign is needed to ensure that the player is able to perceive the information, determine the correct action, and provide input. ...
... In some audio-based games there's an analogy with real-life navigation systems, such as compass and sonar like features [2,58]. Musicraft [33] is an audio game inspired by Minecraft, in which the world is depicted by abstract musical representations. ...
... Compilations such as Audio Game Hub 31 and Sammy Senter 32 explore different mechanics and similar mechanics applied to different narrative contexts. Curious games originate from real-life challenges where seeing is unimportant such as lock-picking 33 and musical performance [19,49,50]. ...
... https://store.steampowered.com/app/247020/Cook_ Serve_Delicious/ (Last visited on October 23rd, 2020) 50 VR The Diner Duo. https://store.steampowered.com/app/530120/VR_The_ ...
Noticeably, the majority of mainstream games — digital games and tabletop games — are still designed for players with a standard set of abilities. As such, people with some form of disability, often face insurmountable challenges to play mainstream games or are limited to play games specifically designed for them. By conducting an initial study, we share multiplayer gaming experiences of people with visual impairments collected from interviews with 10 adults and 10 minors, and 140 responses to an online survey. We include the perspectives of 17 sighted people who play with someone who has a visual impairment, collected in a second online survey. We found that people with visual impairments are playing diverse games, but face limitations in playing with others who have different visual abilities. What stood out is the lack of intersection in gaming opportunities, and consequently, in habits and interests of people with different visual abilities. In this study, we highlight barriers associated with these experiences beyond inaccessibility issues and discuss implications and opportunities for the design of mixed-ability gaming. As expected, we found a worrying absence of games that cater to different abilities. In this context, we explored ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. We designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.
... color and contrast options) and depend on audio feedback. Past research has explored different strategies to adapt games that require vision to an accessible audio-based gameplay [4,33,39,40,48,58,61]. In most cases a substantial redesign is needed to ensure that the player is able to perceive the information, determine the correct action, and provide input. ...
... In some audio-based games there's an analogy with real-life navigation systems, such as compass and sonar like features [2,58]. Musicraft [33] is an audio game inspired by Minecraft, in which the world is depicted by abstract musical representations. ...
The landscape of digital games is segregated by player ability. For example, sighted players have a multitude of highly visual games at their disposal, while blind players may choose from a variety of audio games. Attempts at improving cross-ability access to any of those are often limited in the experience they provide, or disregard multiplayer experiences. We explore ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. Our team designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.
... O Musicrafté um jogo ou ferramenta de edição ouáudios formatados [Kirke 2018]. O Musicrafté baseado no Minecraft com dois modos de ações: sobrevivência e criatividade. ...
Os jogos eletrônicos são produtos tecnológicos de produção cultural midiática com contribuição social de promover uma realidade alternativa que virtualiza diferentes papéis para os participantes. Essa experiência possibilita a imaginação interpessoal inerente, dentre tantos mais específicos, pelo enredo, qualidade visual, identidade cultural, utensílios, efeitos estratégicos e qualidade sonora. A trilha sonora está deixando de ser função de fundo ou reflexos de ações de participantes para atuações pertinentes e importantes durante a percepção do jogo. Este trabalho apresenta uma versão contextualizada do jogo Chrome T-Rex com personagens regionais que ao final gera uma música particular baseada na jogabilidade de cada participante. Há uma música de fundo, pré-selecionada, e para cada movimentação do personagem principal, demais personagens ou ainda eventos do cenário, uma sequência de sons é ativada representando instrumentos musicais ou efeitos sintetizados do gênero musical selecionado. No entanto, para criar uma música interessante é necessário sincronizar as ações de jogabilidade com o ritmo musical. Dessa forma, este jogo apresenta contribuições de entretenimento e iniciação a musicalização a partir de ações durante o jogo. Para testes, utilizou-se o gênero popular de tecnobrega.
... Interactive fction (also known as text adventures), text-based MMORPGs and MUDs are also popular among people with visual impairments [2]. Past research shows rhythm/music games to be efective in engaging both visually impaired and sighted players [18,29,30,53]. Other studies aimed to design multiplayer games for groups with mixed visual abilities. ...
Games bring people together in immersive and challenging interactions. In this paper, we share multiplayer gaming experiences of people with visual impairments collected from interviews with 10 adults and 10 minors, and 140 responses to an online survey. We include the perspectives of 17 sighted people who play with someone who has a visual impairment, collected in a second online survey. Our focus is on group play, particularly on the problems and opportunities that arise from mixed-visual-ability scenarios. These show that people with visual impairments are playing diverse games, but face limitations in playing with others who have diferent visual abilities. What stands out is the lack of intersection in gaming opportunities, and consequently, in habits and interests of people with diferent visual abilities. We highlight barriers associated with these experiences beyond inaccessibility issues and discuss implications and opportunities for the design of mixed-ability gaming.
The author of this study suggests an idea that the auditive element of digital games is soon going to be one of the most important factors influencing their overall success, popularity, and originality. He analyses the phenomenon of binaural and ambisonic sound, its evolution and uses in the context of modern audiovisual work, primarily focusing on games. The auditive component and its increasingly important role areanalyzed in connection with the graphic design of games, virtual reality, as well as the popularity of specific games. This paper also focuses on audio-games, the use of binaural sound (which was first used on a large scale in Hellblade: Senua's Sacrifice) and various hybrid digital games, which are balancing between classic games and audio-games. Argumentation is based on an assumption that binaural sound is the way to ambisonic sound, which (within the context of the immersive and interactive character of digital games) predestines the new standard and shows an entirely new way of creating and using digital games at the same time. All of this is reflected in the context of the graphic design of digital games and their future.
(Article issued in Acta Ludologica, Vol. 1. No. 1 https://actaludologica.com/volume-1/ )
As virtual environments—in the form of videogames and augmented and virtual reality experiences—become more popular, it is important to ensure that they are accessible to all. Previous research has identified echolocation as a useful interaction approach to enable people with visual impairment to access virtual environments. In this paper, we further investigate the usefulness of echolocation to explore virtual environments. We follow a participatory design approach that comprised a focus group session coupled with two fast prototyping and evaluation iterations. During the focus group session, expert echolocators produced a series of seven design recommendations, of which we implemented and trialed four. Our trials revealed that the use of ambient sounds, the ability to place landmarks, directional control, and the ability to use pre-recorded mouth-clicks produced by expert echolocators improved the overall experience of our participants by facilitating the detection of openings and obstacles. The recommendations presented and evaluated in this paper may help to develop virtual environments that support a broader range of users while recognising the value of the lived experience of people with disability as a source of knowledge.
Gamemusik unterhält unter anderem eine wichtige Verbindung zu Phänomenen am Rande des Musikalischen. Geräusche und andere amorphe Schallvorgänge können sich mit der Musik in Spielen verbinden und gemeinsam ganz neue Ästhetiken ausformen. In unserem Beitrag analysieren wir eine Passage aus Inside (Playdead, 2016), in der es zu einer solchen Verbindung kommt und beleuchten sie im Zusammenhang mit dem Begriff der musique concrète. Dabei zeigen wir auf, was die Musik von Inside einzigartig, innovativ und wegweisend für die Zukunft macht.
Training blind children to use audio-based navigation is a demanding and risky task, as children can walk into objects and hurt themselves. Furthermore, training outdoors is dangerous due to traffic, noise and weather conditions. Having a controlled indoor environment is safer but not always available. To tackle this problem, we developed an audio-based computer game, Legend of Iris (LOI), specifically designed to train navigation skills. The game is a 3D exploration game, which uses the headtracking capabilities of the Oculus Rift to create an immersive experience, and the new sound libraries AstoundSound and Phonon3D, to generate an accurate and realistic soundscape. These libraries use a head-related transfer function, allowing the player to localize the audio source in 3D space. The design of LOI involved selecting sounds that are easily recognizable to provide cues to blind people playing the game. A subset of these cues were incorporated into the game. To verify the effectiveness of the game in developing audio orientation and navigation skills, we performed a preliminary qualitative experiment with blind children in a dedicated school. LOI scored high in terms of accuracy and immersion, but a larger test is required to make statistical conclusions.
Ambisonics is a 3D recording and playback method that is based on the representation of the sound field excitation as a decomposition into spherical harmonics. This representation facilitates spatial sound production that is independent of the playback system. The adaptation to a given playback system (loudspeakers or motion-tracked headphones) is achieved by a suitable decoder. This contribution gives an overview of the current state-of-the-art in Ambisonics including content produc- Tion using Ambisonic main microphone arrays or panning of virtual sources, spatial effects, and reproduction by loudspeakers and headphones. The software for the whole production chain is already available as a VST-plugin suite for digital audio workstations.
This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user-friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in general.
Text-based games offer a compelling opportunity for sighted and non-sighted players to interact. Studies on navigation show that spatial mental models are encoded differently across both groups of players. While sighted people tend to view the environment from an overview representation (allocentric), non-sighted people more frequently do so from an egocentric perspective. We investigate whether a text-based game‟s navigation system should be presented using both perspectives to support immersion for sighted and non-sighted players alike. An evaluation of the fit of both systems using a between-subject user study supports the hypothesis that egocentric directions are easier to use and more immersive for non-sighted players, while sighted players perform better with allocentric directions. The discussion offers insights into how these results can be interpreted and operationalised for the design of text-based games or other inclusive software.
Over the past four decades video games have become a worldwide phenomenon and one of the preferred leisure options for many. However, game accessibility remains a pending issue, due to the visual and interactive nature of the medium. Mangiron and Zhang analyse the current state of game accessibility, particularly for the blind and visually impaired. They discuss the barriers and gaming options for this target group, such as audio games and ‘video-less’ games. Finally, the chapter explores the feasibility of applying audio description to video games, which would potentially enhance game accessibility for blind and visually impaired players.
Audio-Games (AGs) are electronic games that mainly (or exclusively) implement auditory means to express the game’s plot, mechanics and content. AG players need to concentrate on perceiving sound, in order to understand and play the game. Recent developments in the field of edutainment suggest that AGs can be implemented not only as entertaining systems, but also as valid tools for research and education on any curriculum related (but no limited) to acoustics and sound studies. This paper will first discuss existing approaches to organizing the sonic content of an AG. Then it will propose a methodology for designing an AG for educational purposes. Finally, a case–study consisting of the development of two educational AGs related to the issue of noise will be demonstrated, on which the proposed methodology is applied.
A lot of video games on the market are inaccessible to players with visual impairments because they rely heavily on use of graphical elements. This paper presents a project aimed at developing a point-and-click adventure game for smart phones and tablets that is equally functional and enjoyable by blind and sighted players. This will be achieved by utilizing audio to give blind players all necessary information and enjoyment without graphics. In addition to creating the game, the aim of the project is to identify design aspects that can be applied to more types of games to include more players. This paper also presents a pilot study that has been conducted on an early version of the game and the preliminary findings are discussed.
This paper examines the use of sound in designing and structuring the gameplay of Audio-Based games. The classification into different categories and the discussion of existing techniques conclude that there are issues that need to be dealt with: the conflict between a strict system of rules and free improvisation, the sonic description of visual elements and the limited educational role. A brief outline of the work-in-project Audio-Based Role-Playing-Game 'Kronos' suggests how a game can be merged with an instrument on an educational basis.