Accessible interactive digital signage for visually impaired

Abstract

In this workshop we discuss the potential of cross-modal haptic-auditory feedback for empowering visually impaired people to experience Interactive Digital Signage.

Full text

Accessible interactive digital signage
for visually impaired
Balandino Di Donato
b.didonato@gold.ac.uk
Goldsmiths, University of London
London, United Kingdom
Tychonas Michailidis
tychonas.michailidis@solent.ac.uk
Solent University
Southampton, United Kingdom
ABSTRACT
In this workshop we discuss the potential of cross-modal haptic-auditory feedback for empowering
visually impaired people to experience Interactive Digital Signage.
CCS CONCEPTS
•Human-centered computing →Haptic devices
;
Accessibility technologies
;Sound-based input
/ output.
KEYWORDS
Haptic feedback, auditory feedback, vision impairment, interactive audio
ACM Reference Format:
Balandino Di Donato and Tychonas Michailidis. 2019. Accessible interactive digital signage for visually impaired.
In CHI ’19: ACM CHI Conference on Human Factors in Computing Systems, June 03–05, 2019, Glasgow, UK. ACM,
New York, NY, USA, 2 pages. hps://doi.org/-
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored.
Abstracting with credit is permied. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from permissions@acm.org.
CHI ’19, May 04–09, 2019, Glasgow, UK
©2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN -. . . $-
hps://doi.org/-

Accessible interactive digital signage for visually impaired CHI ’19, May 04–09, 2019, Glasgow, UK
INTRODUCTION
Interactive Digital Signage (IDS) is an emergent and on-going field in marketing and consumer-
business interaction. However, visually impaired people are deprived of experiencing them. On extend-
ing GraVVITAS capabilities, a system for presenting 2D graphics to people with vision impairment,
[
1
] uses Ultrahaptics to deliver mid-air haptic feedback. Similarly, [
3
] applied mid-air haptic feedback
to AUMI
1
, a camera-based Digital Musical Instrument (DMI), to enhance the spatial awareness of
1hp://aumapp.com
movement and control of the virtual instrument. We believe that coupling haptic with auditory
feedback has the potential of improving IDS experience for visually impaired.
AIM
In this workshop, we will discuss the potential of enhancing the experience of interactive digital
signage for people with vision impairment. Specifically, we propose the adoption of cross-modal
haptic-auditory feedback to imbue pleasant experiences to visually impaired of the IDS content.
Figure 1: User grabbing the crumpled pa-
per
Figure 2: User squeezing the crumpled pa-
per
CASE STUDY
A prototype was developed in Unity
2
using the Ultrahaptics STRATOS device
3
. Our system prototype
2hps://www.unity3d.com
3
hps://www.ultrahaptics.
com/products-programs/
stratos-explore-development-kit/
is based on earlier work focused interaction with a virtual piece of paper through mid-air hand
metaphoric hand gestures captured using electrocardiography (EMG) based technology
4
. A crumpling
4hps://vimeo.com/128395577
paper sound was generated through the interaction with the virtual paper through a crumpling
paper gesture. Work by [
2
] demonstrates the use of mid-air tactile interaction with a virtual piano
by emulating naturally opposing forces of the real instrument. Similarly, here we explore ways of
interacting with the crumpled paper through auditory-mid-air haptic feedback. Our prototype renders
a crumpled paper as a sphere, with diameter and density changing when grabbed and squeezed
through gestural interaction (Fig. 1 and 2), contemporary, the sound of the crumpled paper is produced.
In addition, we aim to discuss the potential benefit of using cross-modal audio-haptics interaction
for digital signage in real-world scenarios. For example, in advertising, users can unscrew the cup of a
jar and interact with the content on screen through auditory and haptic feedback.
REFERENCES
[1]
Cagatay Goncu and Armin Kroll. 2018. Accessible Haptic Objects for People with Vision Impairment. In Extended Abstracts
of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18). Montreal QC, Canada.
[2]
I. Hwang, H. Son, and J. R. Kim. 2017. AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic
feedback. In 2017 IEEE World Haptics Conference (WHC). Förstenfeldbruck, Germany, 213–218.
[3]
John Sullivan, Aditya Tirumala Bukkapatnam, and Marcelo M. Wanderley. 2018. Mid-Air Haptics for Digital Musical
Instruments. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18).
Montreal QC, Canada.