Accessible interactive digital signage
for visually impaired
Balandino Di Donato
Goldsmiths, University of London
London, United Kingdom
Southampton, United Kingdom
In this workshop we discuss the potential of cross-modal haptic-auditory feedback for empowering
visually impaired people to experience Interactive Digital Signage.
•Human-centered computing →Haptic devices
Haptic feedback, auditory feedback, vision impairment, interactive audio
ACM Reference Format:
Balandino Di Donato and Tychonas Michailidis. 2019. Accessible interactive digital signage for visually impaired.
In CHI ’19: ACM CHI Conference on Human Factors in Computing Systems, June 03–05, 2019, Glasgow, UK. ACM,
New York, NY, USA, 2 pages. hps://doi.org/-
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored.
Abstracting with credit is permied. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from email@example.com.
CHI ’19, May 04–09, 2019, Glasgow, UK
©2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN -. . . $-
Accessible interactive digital signage for visually impaired CHI ’19, May 04–09, 2019, Glasgow, UK
Interactive Digital Signage (IDS) is an emergent and on-going field in marketing and consumer-
business interaction. However, visually impaired people are deprived of experiencing them. On extend-
ing GraVVITAS capabilities, a system for presenting 2D graphics to people with vision impairment,
] uses Ultrahaptics to deliver mid-air haptic feedback. Similarly, [
] applied mid-air haptic feedback
, a camera-based Digital Musical Instrument (DMI), to enhance the spatial awareness of
movement and control of the virtual instrument. We believe that coupling haptic with auditory
feedback has the potential of improving IDS experience for visually impaired.
In this workshop, we will discuss the potential of enhancing the experience of interactive digital
signage for people with vision impairment. Specifically, we propose the adoption of cross-modal
haptic-auditory feedback to imbue pleasant experiences to visually impaired of the IDS content.
Figure 1: User grabbing the crumpled pa-
Figure 2: User squeezing the crumpled pa-
A prototype was developed in Unity
using the Ultrahaptics STRATOS device
. Our system prototype
is based on earlier work focused interaction with a virtual piece of paper through mid-air hand
metaphoric hand gestures captured using electrocardiography (EMG) based technology
. A crumpling
paper sound was generated through the interaction with the virtual paper through a crumpling
paper gesture. Work by [
] demonstrates the use of mid-air tactile interaction with a virtual piano
by emulating naturally opposing forces of the real instrument. Similarly, here we explore ways of
interacting with the crumpled paper through auditory-mid-air haptic feedback. Our prototype renders
a crumpled paper as a sphere, with diameter and density changing when grabbed and squeezed
through gestural interaction (Fig. 1 and 2), contemporary, the sound of the crumpled paper is produced.
In addition, we aim to discuss the potential benefit of using cross-modal audio-haptics interaction
for digital signage in real-world scenarios. For example, in advertising, users can unscrew the cup of a
jar and interact with the content on screen through auditory and haptic feedback.
Cagatay Goncu and Armin Kroll. 2018. Accessible Haptic Objects for People with Vision Impairment. In Extended Abstracts
of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18). Montreal QC, Canada.
I. Hwang, H. Son, and J. R. Kim. 2017. AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic
feedback. In 2017 IEEE World Haptics Conference (WHC). Förstenfeldbruck, Germany, 213–218.
John Sullivan, Aditya Tirumala Bukkapatnam, and Marcelo M. Wanderley. 2018. Mid-Air Haptics for Digital Musical
Instruments. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18).
Montreal QC, Canada.