Research ProposalPDF Available

Generating real-time audio description for live streamed gaming

Authors:

Abstract

In films and TV shows a verbal commentary is included in order to help visually impaired people follow the plot, access information on set design and costumes as well as other visual aspects of the production. This verbal commentary is known as Audio Description (AD). Newer media experiences are still to catch up in terms of access services, and live streamed gaming is an example of that. Live streamed gaming is an emerging form of entertainment where video games being played are streamed in real time to thousands of spectators. Visually impaired people are already using text-to-speech software on the text chat of live streamed services; however, this technology does not exist on the video stream yet. This PhD project takes a user-centered approach to the study of the application of audio description to live streamed video games, by analysing its use from both theoretical and technological standpoints.






 !
       "     #$ % 
&               !!  
&!!!!!
!   
! &'' ( &     )  
!      &              *  +  "    '
          !
! !
%,-./-%
                           
 !"#$

%"#&'(    #)   
*+'
#, -./  ."#
*0#12
          #  1            
                      
3"#)04(5"
# 44(
0
6#7
4(#$     
2#
&'(89:
#
,010,/2*,-*-1
&;<
&2
=
7                              

>"#
&?"#@2
  A 9  
B-"#74(5"#&B4(
$!    A"# 7     

4(0#4
4(2#$
4( "#&
   ;    ! %"       2  
      #  1                     
 #7   
<#44(
 #C ,2
8/,2/2
   #1    9
94(#4
06#
D              
#E4(4(&
24(!.?"#7
4(3"4(
2#
4 
8#E35"#$4(
#,
/,/#F
       
4(#
4DG1 -?2>"1
  #  F                    
?"# 4       4(     
#1    # &  
#4
                ;          
#G     # &
#
&                              
#&0
#79
 9  
#720
#
42'(
#G4(
    ;    
8#(9
  , /   - ."# & 
! "#4
;4( " !"
H#1
4( %"#19
#
3--%4*0,1-%1030%
&'(8   
#1      0   
#&       
  &  ."# 1        
0#
& 0   1   1 
A?"#&
,/#A21&
              2              
0#
,050,0%/01
"(#4 &# I# A F# E2 J# C#&
#1 -->#
 "7#D#G$#7 )# K# & L   
# --?#
!")#K$#7#)<$#1
        4H->
3?M53EK2EKNF4 -->#4DA#
%"#J#@#7L2#A4#&!
#1!"
(1A$4H->!%M%EK2EKNF4 -->#4DA#
."J##<$H##$
., /<>M>3 -.#
3"*#7#4<4#% -3#
5"  O#'#  N    (#  1#  7#  N      <          
#>,!/<>?M -! --#
>"O#)D#''#4#DA#)#4,/<&#
1& ->#
?"4#@L2# 4<2  
#'5,%/<>!!M>%-E ->#
-"B##,##/#NBC@-#"4<
6<PP###PP0PP
4O# - -"#
"4D)#, -?/#APBJ4D)H -?44
  4  (A#  "  4  <  6<PP#P
Q$ Q>-Q?? -?4O# - -"#
 "4#FL22#, -/#&<4(#(
$(.% 3 #
!"*#7O#7#D0<2
# ,!/<% M% 3 -!#
%"A#(#E(#1#7A#@R#1;<$8
<4;#1 ->#
."*LA#I#SGTI#, ->/#4(NI<@2
H#)$*
!3,!/ 5%M ?#
3"  I#  G6  E#  E $#  F    O#  G#  $            
#1++,
-.>?M?>F# - #
5"  #  @ F # I  7#    A#  G    O# IL#        
<  4    #  1                 
(4D$H%%.<M%.<%EK2EKNF4 -%#4DA#
>"J#JO#GI#1#4,/H<42
#  1 /     +0 &   & 1  
DG1$4H?EK2EKNF4 -?#4DA#
?" (#*#4 < FH  
#AH -?#
 -"I#(E#U#24#7#4
#12
(%'$&J4H.EK2EKNF4 -.#4DA#
 " 7# J (# 7V '# 4# D  U# U# 4  < C8
0    # 1 ++    3    45    
67!?M! %F# - #
 " 4# @L2  *# 7#       <  H
# 3,/<3?M>! ->#
 !" &# )# F# A     # # ( 
*,/< -M 3?? #
 %"4#FL224#O22#&0#
# - #
 ." &## ,##/# ' # " 4 < 6<PP##PPP
4O# - -"#
ResearchGate has not been able to resolve any citations for this publication.
Thesis
Full-text available
In recent years, video and live streaming platforms have grown rapidly, which has accelerated the development of user-generated content. These new practices cannot be ignored by game designers as they may potentially have a high impact on how visible games are in the market. Some game developers admit considering videomakers and live streamers as target for their game. Some of them are also including special features that allow the audience to interact with the game in a live streaming context. We call these games: audience participation games (APG). This thesis will first explore live streaming integration in video games in order to frame the thesis subject to audience participation games. After doing a case study with a presentation and audience participation focused analysis of the games Choice Chamber and Dead Cells, the engagement of the audience in the different audience participation features of these games will be measured. The thesis will also expose the streamers’ perspective on these two audience participation games through semi-structured interviews (N=3). Finally, we will try to draw conclusions in term of game design, pinpoint the knowledge contribution of this thesis in terms of live streaming integration in games, but also about the method used to analyze audience engagement or general observations about audience participation games. New research issues in the area of education with audience participation games for classrooms will also be presented for future works.
Conference Paper
Full-text available
With the rise of live streaming and esports in recent years, it becomes increasingly important for the HCI community to understand this phenomenon. The organizers encourage people to submit papers with novel interfaces they wish to explore in a live streaming context. In this workshop, participants will discuss different facets of the live streaming experience and obtain a greater understanding of the culture that exists on streaming platforms like Twitch. They will then participate in a design exercise, forming groups and iterating on a design together. Discussion/design topics will include ideas encouraging audience participation, moderation of toxicity, and other such topics. Participants will leave the workshop with ideas about how they can better design games and other experiences taking the live streaming ecosystem into consideration.
Article
Full-text available
Audio description (AD) is one of the main methods that people who are blind or low vision (B/LV) use to access film, television, and theatre content. AD is a second audio track inserted into the space(s) where speech is absent, which tends to be only a few seconds. Contained in that second track is an audio description of the important visual information contained within a specific scene. However, as there is insufficient time to describe all visual information, decisions about what is important to describe and how to present that information (style) to optimize a B/LV viewer’s entertainment experience are required. Most research to date has considered only short-term, single-episode experiences to gauge viewers’ reactions to the AD content. In addition, this research typically has used a monotone, single style of audio description, which is defined as “the conventional style” in this paper. We use an integrative style instead, that is defined as ‘AD designed to fit a specific show”, and differed between shows. We carried out a within-subjects longitudinal study with eight episodes of a dark comedy, using different description styles and describers in order to assess viewer engagement and preferences for AD describer style, language use, timing, and fit to the show. Twenty-four blind participants viewed and rated all eight episodes. Major findings included that most participants found the integrative style entertaining, a fit with the specific episodes, and enjoyable. Some participants, however, preferred the conventional style and struggled with the language and topic of a dark comedy and its associated descriptions.
Article
Full-text available
This article presents a study aimed at assessing an application offering audio description for mobile devices. By means of questionnaires, three features of the application were evaluated: usability, utility and quality. Fifteen blind and visually impaired volunteers participated in the study, which took place in real-life conditions—during a film festival screening. The results indicated positive ratings of all three assessed features, but also pointed to specific elements that could be subject to improvement. Overall, the application was considered as having good potential. Modifications were introduced, and the application is currently fully operational. The application can be used as a tool for providing improved access to cinema content, making cinema accessible for all.
Article
Full-text available
This article presents a study that was aimed at assessing presence and emotional experience of blind and visually impaired users when exposed to emotive and non-emotive content with Polish audio description (AD) delivered with two voices: human and synthetic. The results show that AD narrated by a human prompted significantly higher levels of presence for drama and was assessed by participants as significantly more interesting and less confusing. Presence rates for documentary were similar, with no statistically significant differences in relation to AD voice type. Levels of interest and confusion were comparable for both voices, with a slightly higher result for interest for the clip with AD delivered by the synthetic voice. Overall, the respondents preferred human-narrated AD for drama, but did not show a preference between synthesized and human-narrated AD for documentary. The findings are of direct practical relevance both to the industry and to academia, as they provide important information on how to select voices for audiovisual productions in practice, and suggest that AD reproduced with a synthetic voice is an effective and viable solution for voicing documentaries.
Conference Paper
Full-text available
Accessible games have been researched and developed for many years, however, blind people still have very limited access and knowledge of them. This can pose a serious limitation , especially for blind children, since in recent years electronic games have become one of the most common and wide spread means of entertainment and socialization. For our implementation we use binaural technology which allows the player to hear and navigate the game space by adding localization information to the game sounds. With our implementation and user studies we provide insight on what constitutes an accessible game for blind people as well as a functional game engine for such games. The game engine developed allows the quick development of games for the visually impaired. Our work provides a good starting point for future developments on the field and, as the user studies show, was very well perceived by the visually impaired children that tried it.
Article
Audio Description for film and television is a pre-recorded track that uses verbal descriptions to provide information on visual aspects of a film or TV programme. In the UK it is currently the only accessibility strategy available for visually impaired audiences and although it provides access to a large number of people, its shortcomings also fail to engage others in audiovisual experiences. The Enhancing Audio Description project explores how digital audio technologies can be applied to the creation of alternatives to Audio Description with the aim of personalising access strategies. Such personalisation would allow users to select the method utilised to access audiovisual experiences, by having choices that include traditional forms of accessibility as well as sound design based methods. The present article analyses the results of a survey and focus groups in which visually impaired participants discussed the advantages and disadvantages of AD and it demonstrates not only the diversity of experiences and needs of visually impaired groups but also their eagerness for change.
Conference Paper
Augmented reality applications have focused on visually integrating virtual objects into real environments. In this paper, we propose an auditory augmented reality, where we integrate acoustic virtual objects into the real world. We sonify objects that do not intrinsically produce sound, with the purpose of revealing additional information about them. Using spatialized (3D) audio synthesis, acoustic virtual objects are placed at specific real-world coordinates, obviating the need to explicitly tell the user where they are. Thus, by leveraging the innate human capacity for 3D sound source localization and source separation, we create an audio natural user interface. In contrast with previous work, we do not create acoustic scenes by transducing low-level (for instance, pixel-based) visual information. Instead, we use computer vision methods to identify high-level features of interest in an RGB-D stream, which are then sonified as virtual objects at their respective real-world coordinates. Since our visual and auditory senses are inherently spatial, this technique naturally maps between these two modalities, creating intuitive representations. We evaluate this concept with a head-mounted device, featuring modes that sonify flat surfaces, navigable paths and human faces.
Conference Paper
User research has become an important aspect in game development. Especially, user telemetry, i.e., the remote tracking of in-game behavioral data and its subsequent analysis has attracted increased attention in game development and research in the last years. Unfortunately, case studies of game industry-academic relationships concerning the application of user telemetry methods in practice are still sparse. In this case study we report our experiences from employing gameplay visualizations during the development of a commercial mobile game. In particular, we describe the process of the collaboration and discuss developer feedback regarding the visualization itself as well as issues regarding user telemetry integration into the development process.