Conference PaperPDF Available

Accessibility of Different Natural User Interfaces for People with Intellectual Disabilities



Digital technologies have many advantages for users, such as virtually unlimited access to information, entertainment , and communication. Most modern human-computer interfaces are developed under the assumption that they will be used by a person with typical physical, intellectual, and perceptual abilities. Although some operating systems already include accessibility features, in most cases the effective use of the respective interface can be severely restricted if a person's abilities deviate from this norm. To what extent the class of 'natural' user interfaces-including touch, voice and touchless-are accessible to people with intellectual and possibly motor disabilities is an important but not yet investigated question. Therefore, this paper investigates the current accessibility of these three interface types. First, we conducted a field study to figure out how the target group interacts with these types of interfaces in general. Second, quantitative data on cognitive and motor skills was collected using parts of the Questionnaire for Observing Communicative Skills-Revision (OCS-R) which is widely used in institutions for people with disabilities in Germany. Finally, the accessibility of each interface type was analyzed with the help of the data obtained from the questionnaire and an expert survey, which determined the important and unimportant skills required for each interface. These findings show how usable different types of natural user interfaces are for this target group.
Accessibility of Different Natural User Interfaces for
People with Intellectual Disabilities
Melinda Braun, Matthias W¨
olfel, Gregor Renner, Christian Menschik
Karlsruhe University of Applied Sciences, Germany, {melinda.braun,matthias.woelfel}
Catholic University of Applied Sciences, Freiburg, Germany,
Furtwangen University, Germany,
Abstract—Digital technologies have many advantages for
users, such as virtually unlimited access to information, enter-
tainment, and communication. Most modern human-computer
interfaces are developed under the assumption that they will
be used by a person with typical physical, intellectual, and
perceptual abilities. Although some operating systems already
include accessibility features, in most cases the effective use
of the respective interface can be severely restricted if a
person’s abilities deviate from this norm. To what extent
the class of ‘natural‘ user interfaces—including touch, voice
and touchless—are accessible to people with intellectual and
possibly motor disabilities is an important but not yet investi-
gated question. Therefore, this paper investigates the current
accessibility of these three interface types. First, we conducted
a field study to figure out how the target group interacts
with these types of interfaces in general. Second, quantitative
data on cognitive and motor skills was collected using parts
of the Questionnaire for Observing Communicative Skills -
Revision (OCS-R) which is widely used in institutions for
people with disabilities in Germany. Finally, the accessibility
of each interface type was analyzed with the help of the data
obtained from the questionnaire and an expert survey, which
determined the important and unimportant skills required for
each interface. These findings show how usable different types
of natural user interfaces are for this target group.
Keywords-touch user interface; voice user interface; touch-
less user interface; accessibility; cognitive disabilities; intellec-
tual disabilities; digital divide
Digital technologies have many advantages for users, such
as virtually unlimited access to information, entertainment,
and communication. Although these opportunities would
also be relevant for people with intellectual disabilities (ID),
for this target group it is particularly difficult to use infor-
mation and communication technology (ICT). This results
in a digital divide—a separation between users and those
who are excluded from usage due to various constraints—
between people with (primarily intellectual) disabilities and
the ”networked ordinary citizen” [1]. We refer to persons
with ID, following the World Health Organization (WHO),
as persons with “significantly reduced ability to understand
new or complex information and to learn and apply new
skills (impaired intelligence)” [2]. This includes a spectrum
from persons with mild learning disabilities and reading and
writing skills or people using simple language, to persons
with profound intellectual disabilities, e.g. without apparent
language comprehension or intentionality. This may imply
access barriers to use digital devices, as they are mainly
developed under the assumption that they will be used by
a person with typical physical, intellectual, and perceptual
abilities. Although some operating systems already include
accessibility features, in most cases the effective use of the
provided interface can be severely restricted if a person’s
abilities deviate from this norm. It is interesting to note that
this effect is not exclusive to people with disabilities, but
now also includes your heritage or the regions where you
grew up, as, for example, the dialect you speak is not well
understood by automatic speech recognition [3].
Although there are a lot of configuration options available
today, it has not yet been investigated to what extent new
types of interfaces and their configuration options can be
used by people with ID. While traditional interfaces required
simple algorithms that could be easily written in code, new
interfaces rely on pattern recognition which requires training
data. The problem within this training data in particular lies
in its sparsity and bias as it covers only the general public.
For people with disabilities (PWD), access to information
may be limited by barriers, such as access to spoken infor-
mation for deaf people, written information for blind people,
or writing with a pen or typewriter for people with motor
disabilities. On the one hand, ICT has created new barriers
in its development, on the other hand, assistive technologies
have opened up new possibilities, such as screen readers,
braille lines and braille keyboards for blind people. Special
keyboards and/or special input systems with a single sensor
in combination with a scanning process via the alphabet, as
used by Stephen Hawking, enabled people with severe motor
disabilities to participate via the computer in a completely
new way.
In addition, more and more interfaces types have been de-
veloped over the last decades—from keyboard and mouse to
speech or gesture and now even brain-computer-interfaces—
this variety can make it difficult for many PWD to use
certain technologies and to select which of those technolo-
gies can be usable or adapted to fit their particular needs.
Therefore, more research is needed to study the use of
different interfaces by people with ID and different cognitive
2020 International Conference on Cyberworlds (CW)
2642-3596/20/$31.00 ©2020 IEEE
DOI 10.1109/CW49994.2020.00041
or motor abilities needed to use those interfaces today.
This can help to find out what interfaces are suitable for
individuals with different ID or what needs to be adapted in
those interfaces. This paper provides a first glance at what
abilities are needed to use different types of interfaces.
Current research on technological equipment of people
with ID indicates that the target group rarely uses smart-
phones or other digital devices. Research on structural barri-
ers has shown that people with ID often do not have access
to technological devices and often live in institutions that
don’t provide internet access [4], [5]. Individual problems
in computer-aided thinking and digital competence make
it difficult for people with ID to use structurally complex
digital devices. As research also shows, ICT can have many
benefits for people with ID in terms of participation in the
physical world. ICT can be used to increase participation in
social interaction [6], in indoor pathfinding [7], in access to
leisure activities [8], in teaching skills for daily living [9],
and even as a replacement for augmentative and alternative
communication (AAC) devices [10].
The universal design approach (aka design for all)was
developed to make it easier for PWD to access standard
commercial technologies, since these technologies are often
cheaper to purchase and are updated more regularly than
special assistive technology. This approach aims to design
and develop systems that can be used by everyone, no
matter the physical or cognitive abilities. Since the abilities
of users are very diverse, it is almost impossible to take
everything into account when designing technologies, but
some mistakes that make access to technology difficult for
certain groups of people can be avoided [11].
Education and skills training are common areas where
digital devices are used by people with ID. In order to
develop an e-training platform for the target group on which
the use of common applications such as Facebook, YouTube
or WhatsApp can be trained, Ferreras et al. found that the
most common problem preventing the target group from
using ICTs is the difficulty in finding suitable apps and that
they lack knowledge on how to use them [12].
Due to a large number of different technologies, inter-
faces, and applications available today it is often difficult
to find suitable and accessible interfaces or devices for
people with ID. Current research refers mainly to special
applications and functions and special types of disabilities,
not to the type of interface in relation to people with ID
in general. To limit the digital divide, the use of natural
interfaces, as discussed below, and their possible adaptation
for people with ID is needed.
A. Touch User Interfaces
We refer to touch user interfaces to any type of computer-
pointing input technology that requires to touch a surface
with or without a display. In the last decade, touch interfaces
have replaced other forms of interfaces and are now widely
used on smartphones, tablets and are also getting more
attention as an alternative input device on PCs [13].
While smartphones and touch interfaces are ubiquitous in
society, people with ID still face problems in their use, such
as small screen, text or button sizes, difficult error handling,
not enough provided feedback and a large number of in-
teraction methods (tap, flick, pinch etc.) [14], [15]. People
with additional visual impairment in particular experience
problems interacting with touch interfaces, although there
are some touch interfaces with accessibility features [16].
Saenz de Urturi Breton et al. offer a set of guidelines that
developers can consider when designing accessible touch
interfaces to improve usability [15].
B. Voice User Interfaces
We refer to voice user interfaces—which are also known
as conversational user interfaces and include voice assis-
tants—to any type of interfaces that lets the user interact
with a machine and perform tasks with voice input and can
also include voice output [17]. Voice user interfaces can run
on devices such as PCs and smartphones but are now more
commonly used on specially designed devices such as Echo
from Amazon (with Alexa) or Google Home (with Google
According to Pradhan et al., voice assistants can un-
intentionally be accessible to people with disabilities and
increase efficiency and independence when using a digital
device. At the same time, people with speech impairments
can face accessibility issues due to speech recognition of the
device [18]. Even for people with ID, voice assistants can
be a suitable interface for operating a device. Balasuriya
et al. [19] observed people with ID using voice assistants
to perform specific tasks and showed that most of the
participants could easily use their voice to activate the
interface at the first attempt. Some participants had difficul-
ties pronouncing “Siri” or “Google”, others needed several
attempts to activate the interface, but then had problems
consistently maintaining correct pronunciation. Participants
appreciated voice assistants, as they can avoid spelling and
typing problems. For this reason, Balasuriya et al. propose
adjustable input settings to make voice interfaces accessible
to speech-impaired users [19].
C. Touchless User Interfaces
We refer to touchless user interfaces to any type of
interface which allows us to command the computer via
body motion and gestures without physically touching a
keyboard, mouse, or screen and also exclude the class
of voice user interfaces. Touchless user interfaces have
not yet entered the mainstream, but are widely used in
special applications such as gaming (e.g. using the Microsoft
Kinect), as an input modality in virtual reality (e.g. using
a Leap Motion Controller), and head- or eye-tracking (i.e.
Camera Mouse, Tobii Dynavox). In the field of assistive
technologies, touchless approaches exist that enable the
control of wheelchairs or other assistive devices. They can
recognize specific movements of the hands, face or other
parts of the body and thus are especially interesting for
people who are restricted in motor control and cannot use
buttons, joysticks or touch [20].
Currently there is not much research on touchless user
interfaces and people with (intellectual) disabilities. Saenz-
de-Urturi and Garcia-Zapirain Soto developed and tested a
Kinect-based game to correct poor posture of elderly people
that took into account cognitive and physical disabilities
of their target group. They stated that those interfaces
have much potential for specialized forms of (elderly) care
applications that are low-cost and enjoyable [21].
D. Adjustability of User Interfaces
Current research indicates that commercial technologies
have the potential to contribute to assistive or educational
settings for people with ID. It also shows, however, that
these technologies either need to be adapted or users with
ID need to be guided by non-disabled persons in order to
exploit the full potential of the respective device [19], [22],
[23]. While simpler interfaces can be adjusted without much
effort—e.g. extending a knob by clay or replacing one button
with another one—interfaces that heavily rely on pattern
recognition, such as the investigated interfaces, are much
harder to be adapted.
In order to find out how interfaces need to be adapted
in the future, this study must first determine the current
accessibility-status of various user interfaces when used
by people with ID. For this first overview, touch, voice,
and touchless interfaces have been considered. Since there
is no standardized method or questionnaire to date that
examines the use of different user interfaces by people with
ID, an experimental approach using an observational study,
a questionnaire and an accessibility analysis was chosen.
Of course, such a study can never be comprehensive and
depends on the use case. In our observation, we decided on
a generic application for navigating and playing music that
can be well designed for different interface types.
A. Questionnaire for Observing Communicative Skills
To find out what basic cognitive and motor skills the
people with ID have, a questionnaire using modules of
the Questionnaire for Observing Communicative Skills -
Revision (OCS-R) was distributed to the participating in-
stitutions and filled out by carers or managers for the
individuals involved. The OCS-R is a structured diagnostic
observation instrument for assessing communicative abilities
and forms of expression of children, adolescents, and adults
with limitations in their communicative abilities or their
communicative development [24]. The OCS-R is initially
not designed for interfaces-related decisions and of course
cannot take into account all parameters that would be
required to define the fit to a particular interface, but the
importance of this questionnaire lies in its availability as
it is already present (filled in) in many institutions. This
fact can speed up and simplify the process of selecting
suitable interfaces enormously since the carers of people
with ID do not have to fill out an additional questionnaire.
In addition, institutions often employ people without a
technical background. A questionnaire that is too interface-
specific could be quite difficult to be filled out by people
without technical background knowledge. This makes the
OCS-R a potentially suitable tool to find out which interface
types might be suitable for an individual or which interface
may be used when adapted correctly.
To find out which categories or parameters of the OCS-R
are relevant for the three interface types mentioned above,
all items were later evaluated by experts in the field of
human-machine interaction or interaction/interface design.
This evaluation resulted in an adapted version of the OCS-
R, which contains only the relevant items about interface-
specific abilities.
B. Target Group
This study included individuals with a range of differ-
ent forms and degrees of ID, some with additional motor
impairments. All participants were recruited from three
different institutions in southern Germany. The majority of
the participants (76.9%) live in residential groups in these
facilities. Individual, less restricted participants (23.1%) live
in outpatient residential groups, which also belong to the re-
spective facilities. Intellectual disabilities include a spectrum
from persons with mild learning disabilities to persons with
profound ID, e.g. without apparent intentionality. Since this
study uses oral and visual tasks related to different interface
technologies, the study includes an observation to find out
if the participants are able to complete these tasks or not.
Not all participants who took part in the observation filled
out the questionnaire afterwards, for this reason the number
of participants in the observation is higher than in the rest
of the study. The aim of this study is to include “all” people
with intellectual disabilities as far as possible, thus obtaining
a cross-section of the target group.
Access to this target group is made particularly difficult
by the fact that some individuals cannot give their consent
to studies themselves. For people without this possibility,
for example, parents or legal advisors must give their con-
sent before the individuals participate in scientific studies.
This process can be quite difficult and time-consuming. In
order to represent the best possible interests of all parties
involved, we have had an ethical application approved by
the German Society for Educational Science (DGfE) and
only use anonymized data.
In this section, we describe our user study which took
place from February to March 2020. A total of 91 par-
ticipants were observed using one of the three interfaces.
Of those 91 people, 23.1% had prior technical experi-
ence (i.e. experience with a smartphone or tablet). 14.3%
additionally—besides their ID—had some sort of visual
impairment, 48.4% had language restrictions (such as un-
clear pronunciation, communication only through sounds or
complete lack of speech), 4.4% had hearing impairments and
18.7% some other kind of physical limitation (like using a
wheelchair). The age of the participants ranges from 31 to
79 years, with the average of 55 years.
A. Observation of Interface Use
In order to find out how the target group interacts with the
different types of interfaces, the participants were observed
and observations noted. The allocation of the respective
interface was randomized. The task for the
touch user interface was playing music on an iPad
via touch gestures and tried out by 48.4% of the
voice user interface was playing music on an iPad
via their voice (Siri) and tried out by 32.8% of the
touchless user interface was playing music on a laptop
with gestures of their hands and tried out by 18.7% of
the participants. We used an application with the Leap
Motion controller where you have to stretch your index
finger and swipe to trigger an action.
B. Questionnaire (OCS-R)
In our adapted questionnaire we used the OCS-R modules
“basic communication skills” containing questions about
signal production, signal perception, and interaction, “per-
ception” containing questions about general perception and
specific perceptual competencies and “motor skills”. In
Signal production questions the abilities in vocaliza-
tions and spoken language as well as gestures and
manual signs. It consists of 17 items, e.g.: ”can speak
single words intelligibly”, ”uses conventional terms”,
”can speak simple sentences intelligibly”, can point
to objects purposefully”, or ”uses conventional ges-
tures/manual signs” [24].
Signal perception includes questions about the abili-
ties in speech comprehension and ways of perceiving
information. It consists of 11 items, e.g.: ”can gather
information from pictorial symbols”, ”can gather infor-
mation from spoken language”, or ”understands simple
prompts” [24].
Interaction includes 14 items that determine whether
the individual has difficulties in conversation or differ-
ent situations when interacting with people, e.g.: ”can
focus their attention on a person or object”, ”can start
a conversation with someone of their own initiative”,
or ”can maintain a conversation” [24].
Perception first asks about general perception (is the
individual able to see/hear correctly) and then ques-
tions specific perceptual competencies. It consists of
11 items, e.g.: ”can hear with no impairments”, ”can
see with no impairments”, ”can recognize shapes”, or
”can recognize pictorial symbols” [24].
Motor skills asks about the individuals’ motor abilities.
It consists of 14 items, e.g.: ”can press a switch”, ”can
purposefully look in a direction”, ”can purposefully
grasp an object”, ”can hold an object”, or ”can point
accurately [24].
The answers range from 1 = never, 2 = rarely, 3 =
frequently and 4 = always [24]. The questionnaire was
filled out by caregivers for 51 of the 91 persons that also
participated in the observations.
C. Interface Accessibility
In order to find out whether the examined interfaces
can currently be used by the participants, we asked seven
experts working in the field of human-machine interaction
or interaction/interface design to give their opinion on the
importance of the statements of the OCS-R for the three
interface types surveyed. The expert survey used a scale
from 4 = very important to 1 = not important for each
statement. With these results, the existing skills of the 51
participants—collected from the questionnaire (OCS-R)—
can be compared with the relevant skills for each interface
to find out how many of the participants can currently use
the respective interfaces.
In this section, we analyze the collected data and present
our findings.
A. Observation of Interface Use
Table I gives a summary of the interface usage, task
understanding, and task completion. In discussions with the
participants and their carers it became clear that many of
the participants would like to use more technology in their
daily life to help them with various everyday tasks. Most
participants were very interested and had fun using the
devices. However, difficulties currently exist in this regard,
for instance in the procurement and financing of technology
such as smartphones or tablets and the availability of Internet
access in German institutions. It is especially difficult to find
out what kind of technology or interfaces and applications
are available for an individual person and to teach the
person how to use this technology. Furthermore, some care
Table I
User Interface In Total Task Understood Task Completed
understood partially understood not understood completed partially completed not completed
Touch 44 37 (84.1%) 6 (13.6%) 1 (2.3%) 25 (56.8%) 12 (27.3%) 7 (15.9%)
Voice 30 15 (50.0%) 6 (20.0%) 9 (30.0%) 8 (26.7%) 2 (6.7%) 20 (66.7%)
Touchless 17 12 (70.6%) 3 (17.7%) 2 (11.8%) 9 (52.9%) 2 (11.8%) 6 (35.3%)
workers expressed concerns regarding data privacy and se-
curity, when people with ID would use such devices without
knowing privacy settings.
Touch User Interface: The task of playing music on a
touch interface was understood by 37 out of 44 (84.1%)
participants and completed by 25 (56.8%) participants. In
this study group, only 4 participants (9.1%) had previous
experience with technology. Of these 4 people, 3 could com-
plete the task without problems. Due to impaired vision, one
of the four people could not complete the task but claimed
to have experience with voice commands on an iPhone. It
should be noted that of the remaining 40 participants with
no previous technical experience, 22 (55.0%) were able to
complete the task. A lot of people who did not complete
the task fully had problems with small play button size,
letting go of the button (pressing too long and using it as
a physical button), keeping their whole hand on the screen
and not being able to only use one finger of their hand to
Voice User Interface: In the observation of voice interface
use, 15 out of 30 (50.0%) participants understood the task
and only 8 (26.7%) did complete the task. A lot of partic-
ipants had some sort of speech restriction and background
noises caused additional errors. This study group consisted
of only 6 people (20.0%) with previous technical experience.
Of these 6 people, 4 were able to complete the task. One
person did not want to use voice and used touch instead and
another was not able to stay concentrated during the task.
Touchless User Interface: Trying out the touchless user
interface, 12 out of 17 (70.6%) participants understood what
they had to do and 9 (52.9%) completed the task. The
task was particularly difficult for those with problems in
the fine motor skills of their hands, as they were unable to
stretch a finger in isolation. In addition, many participants
did not understand the concept of gesture recognition, which
was shown by the fact that they touched the controller
and wanted to use it as a button. In this study group, 11
participants (64.7%) had previous technical experiences. 7
of those people could complete the task without problems,
one did not want to try. Although the other three understood
the task and tried to complete it, they could not operate the
interface properly due to motor limitations of their hands.
Among the remaining participants with no previous technical
knowledge, only two were able to complete the task.
B. Questionnaire (OCS-R)
Here we present the results of the respective categories of
the OCS-R.
Signal Production: Looking at all 51 participants, the
mean value in this category is 2.5 (rarely to frequently).
Looking at the distribution in Table II, it is becoming clear
that 47.1% of all participants have scores below 2.4, meaning
they rarely to never have these abilities. Only 25.5% of
all participants always have these abilities. This shows that,
overall, the target group shows large gaps in skills in this
Signal Perception: The mean value in this category is 2.9
(frequently). The majority of participants have these abilities
frequently (52.9%) or always (21.6%). 25.5% rarely or never
have these abilities (Table II).
Interaction: The mean value is 2.3 (rarely). Looking at the
percentage values in Table II, it is obvious that the majority
of the participants have problems in this category. 54.9%
rarely or never have these abilities.
Perception: The mean value in this category 2.8 (fre-
quently). 31.4% rarely or never have the abilities in this
category, but the majority (68.6%) of participants have
scored over 2.4 (frequently or always).
Motor skills: The participants have an average score of
3.4 (frequently) in this category. Only a few have larger
problems here, with scores of 2.4 or lower (7.8%). The
majority frequently or always has all motor skills queried
(Table II). As our target group is people with ID, these
results were predictable. Not all people with an ID have
additional problems with motor skills, but there are still
participants who lack some of these skills.
C. Interface Accessibility
This section examines the results of the expert survey and
identifies the relevant skills of the OCS-R for each interface.
The important skills are then compared to the actual skills
the participants have. This helps to find out how many of
the participants can currently use the respective interfaces.
In the following accessibility analyses, only those items of
the OCS-R that were considered important by the experts
were looked at.
Touch User Interface: As seen in Table III, 18 differ-
ent skills from all five categories are important for touch
interface use. In the category ”signal production” 1 of 17
skills is important, in ”signal perception” 2 of 11 skills are
important, in ”interaction” 2 of 14 skills are important, in
Table II
Range Signal production Signal perception Interaction Perception Motor skills
always (>3.5)) 25.5% 21.6% 11.8% 17.6% 60.8%
frequently (2.5..3.5) 27.5% 52,9% 33.3% 51.0% 31.4%
rarely (1.5..2.5) 21.6% 19.6% 31.4% 27.5% 7.8%
never (<1.5) 25.5% 5.9% 23.5% 3.9% 0.0%
Table III
User Interface Signal Production Signal Perception Interaction Perception Motor Skills
Touch 1/17 (5.9%) 2/11 (18.2%) 2/14 (14.3%) 6/11 (54.6%) 7/14 (50.0%)
Voice 6/17 (35.3%) 8/11 (72.7%) 10/14 (71.4%) 1/11 (9.1%) 0/14 (0.0%)
Touchless 3/17 (17.7%) 3/11 (27.3%) 2/14 (14.3%) 1/11 (9.1%) 2/14 (14.3%)
”perception” 6 of 11 are important and in ”motor skills” 7
of 14 skills are important for touch interface use. The mean
values for importance range from 4.0 (very important) to
1.0 (unimportant). Only values above 2.5 were classified as
Comparing the important skills with the actual skills of
the individuals (Table IV), 19.6% of the participants would
be able to use a touch interface without any problems. The
majority (51.0%) would still have the important abilities
most of the time, maybe facing some minor problems while
using. For 29.5% there would be major problems because
they lack the abilities that are essential for the use of touch
Voice User Interface: As seen in Table III, there have
been examined 25 relevant abilities for voice interface use
belonging to the categories “signal production” (6 of 17),
”signal perception” (8 of 17), “interaction” (10 of 14) and
“perception” (1 of 11).
Looking at the important abilities for voice interfaces,
25.5% would be able to use this interface type without any
problems, as they have a score of 3.5 or higher (Table IV).
39.2% would still be able to use the interface, possibly facing
some issues while using. 35.3% would most likely not be
able to use voice interfaces without any adjustments.
Touchless User Interface: According to the experts, 11
skills from all five categories are important for the efficient
use of touchless interfaces (Table III). In the category ”signal
production” 3 of 17 skills have been rated important, in
”signal perception” 3 of 11 skills, in ”interaction” 2 of 14,
in ”perception” 1 of 11 and in ”motor skills” 2 of 14.
19.6% of the participants would be able to use touchless
user interfaces, as they have a score of 3.5 or higher in the
important skills (Table IV). 47.1% maybe would face some
problems, most likely still being able to use the interface.
33.3% would not be able to use touchless user interfaces or
face major problems while using them.
Comparisment between the User Interfaces: Looking at
the individual participants and their scores in all three
interfaces, it is interesting to note that only 13.7% can always
use all of the three (Table V). For these people today’s
Table IV
Range Touch Voice Touchless
always (>3.5)) 19.6% 25.5% 19.6%
frequently (2.5..3.5) 51.0% 39.2% 47.1%
rarely (1.5..2.5) 27.5% 29.4% 29.4%
never (<1.5) 2.0% 5.9% 3.9%
Table V
Range can use at least ... interface(s)
three two one
always (>3.5) 13.7% 19.6% 31.4%
at least frequently (>2.5) 54.9% 64.7% 76.5%
at least rarely (>1.5) 90.2% 94.1% 98.0%
interfaces are most likely usable and accessible, and they
do not experience a digital divide when dealing with them.
19.6% can always use at least two of the interfaces and
31.4% can always use at least one of the interfaces. 54.9%
can use all three, 64.9% can use two, and 76.5% can use at
least one of the interfaces frequently. For this group it would
still be possible to use all or some of the interfaces, maybe
facing smaller problems. It is promising to note that 90.2%
can at least rarely use all three, two (94.1%) or one (98.1%)
of the interfaces. For people with scores below 2.5 (can
rarely use...), it would make sense to work out adaptations
to enable them to participate in digital technologies.
To investigate the correlation between the interface pos-
sible usage we used Spearman’s rank. Touch and voice
interfaces show a moderate positive relationship of 0.56,
touch and touchless interfaces show a strong positive re-
lationship of 0.91, and voice and touchless interfaces show
a positive relationship of 0.78. This shows, not surprisingly,
that a person who is able to operate a touch interface
could most likely also operate a touchless interface and vice
versa. It is interesting to observe that voice and touchless
interfaces seem to have more in common than voice and
touch interfaces.
This study examined the current state of accessibility and
the need for adaptation of present natural user interfaces
when used by people with intellectual disabilities. It focused
on the three interface types touch, voice and touchless and
consisted of three parts: an on-site observation, a question-
naire and an accessibility analysis. In conclusion, in the
on-site observation, most of the test persons understood
the operation of touch interfaces, but only slightly more
than half of the participants were able to complete the
task. Problems exist especially when people with ID have
additional visual difficulties or limited fine motor skills.
Moreover, a large number of test persons found it difficult
to operate a virtual button. Observing the voice interface
use, it is noticeable that many of the participants had some
sort of speech restriction, which made it difficult for them
to use the interface or complete the task they were given.
The touchless interface was particularly difficult for those
having problems with motor skills. In our observation, most
of the participants trying the touchless interface were quite
technology-conscious with rather mild ID. This could bias
the results in this regard.
The expert survey revealed important skills for the respec-
tive interface types. For touch user interfaces this resulted
in 18 important skills from all five categories of the ques-
tionnaire, for voice user interfaces 25 important skills from
the categories “signal production”, “interaction” and “signal
perception” and for touchless user interfaces 11 skills from
all five categories were identified. In the future, it might be
possible to extend the questionnaire with further skills that
are important for the respective interface types in order to
make it even more detailed and accurate.
The analysis of the accessibility status of the current
interface types touch, voice, and touchless showed that there
still exist large gaps in this area, but also that there is great
potential for improvement. People with ID currently face
major problems in accessing, selecting, or using different
types of interfaces. In our study, 29.5% would most likely
not be able to use a touch interface, 35.3% would not be able
to use a voice interface and 33.3% would not be able to use
a touchless interface. 51.0% with touch, 39.2% with voice,
and 47.1% with touchless interfaces would still face minor
problems in daily use, having mean scores between 2.5 and
3.4 (only frequently and not always having the important
abilities). The comparison showed the strongest correlation
between touch and touchless interfaces, meaning that people
who are or aren’t able to use touch interfaces are most likely
able or not able to also use touchless interfaces. For our
target group—people with intellectual disabilities—we share
the statement of Pradhan et al. that accessibility issues can
occur in voice interfaces due to speech recognition problems
This analysis and the prior observation in the participating
institutions once again highlighted the existing digital divide.
At the moment, there are major obstacles in the use of tech-
nology for people with ID. It is noticeable that the technical
infrastructure in german institutions is little or non-existent
in terms of technology. Only a few of the participants in the
observation had technical equipment available themselves
or were able to use the Internet. The caregivers are usually
not technically competent and do not know which technical
devices might be suitable for an individual. Moreover, the
acquisition of technical equipment is often a problem from a
financial point of view. Nevertheless, the participants in the
observation and the staff and managers of the institutions
were very interested in changing the current status and
integrating more technology into the everyday life of people
with ID. Some even had more or less precise ideas about
the areas of life in which technology could be of particular
help the participants.
This study had some limitations, since there is no stan-
dardized method or questionnaire to date that examines the
use of different interface types by people with ID. We
had to choose an experimental approach using parts of
the OCS-R, which is initially not designed for interface-
related decisions. In addition, fewer participants were able
to try out the touchless interface because the Leap Motion
Controller was not available for testing at first. However, the
results show that the three interfaces types are accessible
to some people with ID, but the majority faces major
problems while using. In order to make current interface
types usable for people with any level of knowledge or
ability, accessibility or universal design decisions can be
made in the development process. Because the abilities
of users are very diverse, it is almost impossible to take
everything into account when designing technologies [11].
While some mistakes can be avoided by those decisions,
people with ID have to be trained in interface use and
existing interfaces must be adapted or modified so that the
individual can use them in the best possible way. In addition,
the best fitting interface based on the abilities of the person
in question must be selected, for which this study should
provide a first indication. Further studies will be necessary
to investigate other types of interfaces and abilities and how
exactly they have to be adapted to fit the needs of people
with ID.
This study is part of a project funded by the Federal
Ministry of Education and Research (BMBF) in Germany
within the framework of the program “FH Sozial 2017”.
We would also like to thank the participating institutions
for people with disabilities. Without their help and patience,
it would have not been possible to conduct this survey.
[1] D. Lussier-Desrochers, C. L. Normand, A. Romero-Torres,
Y. Lachapelle, V. Godin-Tremblay, M.-v. Dupont, J. Roux,
L. P´
epin-Beauchesne, and P. Bilodeau, “Bridging the digital
divide for people with intellectual disability,” CP, vol. 11,
no. 1, May 2017.
[2] W. H. O. (WHO), “Definition: in-
tellectual disability.” [Online]. Available:
[3] A. Koenecke, A. Nam, E. Lake, J. Nudell, M. Quartey,
Z. Mengesha, C. Toups, J. R. Rickford, D. Jurafsky, and
S. Goel, “Racial disparities in automated speech recognition,”
Proceedings of the National Academy of Sciences, vol. 117,
no. 14, pp. 7684–7689, 2020.
[4] S. Johansson, J. Gulliksen, and C. Gustavsson, “Disability
digital divide: the use of the internet, smartphones, computers
and tablets among people with disabilities in Sweden,” Univ
Access Inf Soc, Mar. 2020.
[5] National Telecommunications and Information Administra-
tion and Economics and Statistics Administration, “Exploring
the Digital Nation: America’s Emerging Online Experience,
U.S. Department of Commerce, Tech. Rep., Jun. 2013.
[6] M. Donnelly, R. Bond, M. Mulvenna, L. Taggart, D. Hill,
P. Fitzsimons, S. Martin, and A. Hassiotis, “Facilitating social
connectedness for people with autism and intellectual dis-
ability using an interactive app,” in Proceedings of the 32nd
International BCS Human Computer Interaction Conference
32, 2018, pp. 1–4.
[7] J. C. Torrado, G. Montoro, and J. Gomez, “Easing the
integration: A feasible indoor wayfinding system for cognitive
impaired people,” Pervasive and Mobile Computing, vol. 31,
pp. 137–146, Sep. 2016.
[8] G. E. Lancioni, N. N. Singh, M. F. O’Reilly, J. Sigafoos,
G. Alberti, V. Perilli, V. Chiariello, G. Grillo, and C. Turi, “A
tablet-based program to enable people with intellectual and
other disabilities to access leisure activities and video calls,
Disability and Rehabilitation: Assistive Technology, vol. 15,
no. 1, pp. 14–20, Jan. 2020.
[9] P.-F. Wu, H. I. Cannella-Malone, J. E. Wheaton, and C. A.
Tullis, “Using Video Prompting With Different Fading Pro-
cedures to Teach Daily Living Skills: A Preliminary Exami-
nation,” Focus Autism Other Dev Disabl, vol. 31, no. 2, pp.
129–139, Jun. 2016.
[10] C. Mongeau and D. Lussier-Desrochers, “Mobile Technolo-
gies Used as Communication Support System for People with
Intellectual Disabilities: An Exploratory Study,” in Advances
in Design for Inclusion, G. Di Bucchianico and P. F. Kercher,
Eds. Cham: Springer, 2018, vol. 587, pp. 254–263.
[11] J. Abascal, “Human-computer interaction in assistive tech-
nology: from ”Patchwork” to ”Universal Design”,” in IEEE
International Conference on Systems, Man and Cybernetics,
vol. vol.3. Yasmine Hammamet, Tunisia: IEEE, 2002, p. 6.
[12] A. Ferreras, R. Poveda, M. Qu´
ılez, and N. Poll, “Improving
the Quality of Life of Persons with Intellectual Disabilities
Through ICTs,” Stud Health Technol Inform, vol. 242, pp.
257–264, 2017.
[13] R. G ¨
undogdu, A. Bejan, C. Kunze, and M. W¨
olfel, “Ac-
tivating people with dementia using natural user interface
interaction on a surface computer,” in Proceedings of the 11th
EAI International Conference on Pervasive Computing Tech-
nologies for Healthcare - PervasiveHealth ’17. Barcelona,
Spain: ACM Press, 2017, pp. 386–394.
[14] P. Williams and S. Shekhar, “People with Learning Disabili-
ties and Smartphones: Testing the Usability of a Touch-Screen
Interface,” Education Sciences, vol. 9, no. 4, p. 263, Oct.
[15] Z. Saenz de Urturi Breton, F. Jorge Hernandez,
A. Mendez Zorrilla, and B. Garcia Zapirain, “Mobile
communication for Intellectually Challenged people: A
proposed set of requirements for interface design on touch
screen devices,Commun Mob Comput, vol. 1, no. 1, p. 1,
[16] M. Rodriguez-Sanchez, M. Moreno-Alvarez, E. Martin,
S. Borromeo, and J. Hernandez-Tamames, “Accessible smart-
phones for blind users: A case study for a wayfinding system,”
Expert Systems with Applications, vol. 41, no. 16, pp. 7210–
7222, Nov. 2014.
[17] R. Dasgupta, Voice User Interface Design: Moving from GUI
to Mixed Modal Interaction. Berkeley, CA: Apress, 2018.
[18] A. Pradhan, K. Mehta, and L. Findlater, “”Accessibility Came
by Accident”: Use of Voice-Controlled Intelligent Personal
Assistants by People with Disabilities,” in Proceedings of
the 2018 CHI Conference on Human Factors in Computing
Systems - CHI ’18. Montreal QC, Canada: ACM Press, 2018,
pp. 1–13.
[19] S. S. Balasuriya, L. Sitbon, A. A. Bayor, M. Hoogstrate, and
M. Brereton, “Use of voice activated interfaces by people with
intellectual disability,” in Proceedings of the 30th Australian
Conference on Computer-Human Interaction. Melbourne
Australia: ACM, Dec. 2018, pp. 102–112.
[20] G. Kouroupetroglou and P. Das, Eds., Assistive Technologies
and Computer Access for Motor Disabilities:, ser. Advances
in Medical Technologies and Clinical Practice. IGI Global,
[21] Z. Saenz-de Urturi and B. Garcia-Zapirain Soto, “Kinect-
Based Virtual Game for the Elderly that Detects Incorrect
Body Postures in Real Time,Sensors, vol. 16, no. 5, p. 704,
May 2016.
[22] T. Barlott, T. Aplin, E. Catchpole, R. Kranz, D. Le Goullon,
A. Toivanen, and S. Hutchens, “Connectedness and ICT:
Opening the door to possibilities for people with intellectual
disabilities,” Journal of Intellectual Disabilities, pp. 1–19,
Feb. 2019.
[23] B. A. Jimenez and K. Alamer, “Using Graduated Guidance
to Teach iPad Accessibility Skills to High School Students
With Severe Intellectual Disabilities,J Spec Educ Technol,
vol. 33, no. 4, pp. 237–246, Dec. 2018.
[24] M. Scholz, M. Wagner, and J. M. Stegkemper, “OCS-
R Manual (Version 1.06),” 2019. [Online]. Available:
... An analysis of the current accessibility status of the natural user interface types "touch, voice, and touchless" showed a lot of existing problems when used by people with ID, but also a great potential for improvement. People with ID currently have difficulties in "accessing, selecting, or using different types of interfaces" [4]. This is the reason for the so-called digital divide that has formed between people with ID and the "regular" user [2]. ...
... There is currently not much research that specifically addresses people with ID and the use of user interfaces; studies available are predominantly related to specific applications or features, or specific types of disabilities. Research suggests that, to limit the digital divide and to "exploit the full potential of the respective device" [4], current digital technologies have to be made usable for people with ID through adaptation of their user interfaces or through guidance by non-disabled persons [4,[6][7][8]. While simple or analog interfaces can be modified easily, e.g., adapting a door knob with clay or replacing a button with a bigger one, most natural user interfaces rely on pattern recognition, so they aren't as easy to be adapted [4]. ...
... There is currently not much research that specifically addresses people with ID and the use of user interfaces; studies available are predominantly related to specific applications or features, or specific types of disabilities. Research suggests that, to limit the digital divide and to "exploit the full potential of the respective device" [4], current digital technologies have to be made usable for people with ID through adaptation of their user interfaces or through guidance by non-disabled persons [4,[6][7][8]. While simple or analog interfaces can be modified easily, e.g., adapting a door knob with clay or replacing a button with a bigger one, most natural user interfaces rely on pattern recognition, so they aren't as easy to be adapted [4]. ...
Conference Paper
Information and communication technologies are ubiquitous in today's society. They have the potential to enhance the life of its users in various areas, especially the life of people with intellectual disabilities (ID). Unfortunately, natural user interfaces are often too complicated to use and not adapted to the varying needs of every user group. A possible improvement can be achieved by adapting the respective user interface to the abilities and skills of the respective user(s). Therefore, this study evaluates currently available interface types and their adaptation possibilities and requirements for people with ID. 116 individual solutions and prototypes were tested with 41 participants. We found that interfaces with pointing gestures are currently the preferred interface type for most people with ID, as this input type is used in most technologies today and provides the most accessibility features and possible adaptations. Other input types, such as voice or object interaction, offer great potential for people with disabilities and ID, but are currently more difficult to adapt to the individual needs of users with ID.
... People who lack adequate access to useful and credible information which they can evaluate and make sense of may miss crucial news and events, fall prey to misinformation, and be less able to pursue activities that enrich their lives. The internet is now the dominant source of information, yet some demographic groups are especially at risk of missing out on appropriate online resources, due to factors like costs of devices and internet access, and differences in ability and confidence in navigating information and communication technologies (ICTs) [11,76]. Older adults, who represent a rapidly growing portion of the worldwide population [106] are considered to be at risk of digital exclusion [91], including due to ageist stereotyping [53]. ...
... In Braun et al. (2020), the authors analyze the usability of digital technologies for people with cognitive and motor disabilities. In particular, they identified three types of user interfaces-including touch, voice, and touchless. ...
Full-text available
In recent times, Digital Humanities (DH), together with the discoveries of Information and Communications Technology, have enabled the rediscovery and usability of cultural content with the support of various technologies. However, it was found that not everyone is able to access web platforms or visit cultural sites easily. In particular, the epidemiological Covid-19 crisis has highlighted the retrograde state of culture in terms of accessibility and usability, conditioned by the physical and web browsing limitations that for years weighed on people with disabilities. Therefore, it was decided to investigate how DH might support the cultural accessibility of people with disabilities. In particular, it was decided to carry out a systematic review of the cultural innovations of DH together with a survey on disability and supporting technologies in order to present how to improve the quality of the cultural experience of such a target. This study proceeded with the research and selection of literature on the subject of DH and disability, with the selection, analysis, and correlation of the scientific works included. The reference time frame includes the works produced between 2018 and 2022 consulted on main databases such as Scopus and Web of Science (WoS), screened using the Preferred Reporting Items for Systematic reviews and Meta-Analyses Statement.
... Touchless technologies in all forms, provide options for better accessible interfaces for those people who are suffering with physical disabilities. Touchless technology in the form of gaze tracking and use of voice commands have been shown to allow for a higher level of accessibility [64] than other approaches. These innovations should rightly be celebrated, but it is possible that future touchless technologies could create barriers to people living with physical disabilities. ...
Full-text available
Touchless Technology is facilitating the move to Zero User Interface(UI) propelled by the COVID-19 pandemic which has accelerated the use of this technology due to hygiene requirements. Zero UI can be defined as a controlled interface that enables user interaction with technology through voice, gestures, hand interaction, eye tracking, and biometrics such as facial recognition and contactless fingerprints. Smart devices, IoT sensors, smart appliances, smart TVs, smart assistants and consumer robotics are predominant examples of devices in which Zero UI is becoming increasingly adopted. These control interfaces include natural interaction modes such as voice or gestures. Touchscreens and shared devices such as kiosks, self-service counters and interactive displays are present in our everyday lives. Each of these interactions however is a concern for consumers in a post-COVID-19 world where hygiene is of utmost importance. The one-stop solution to hygienic interactions includes touchless technology such as voice control, remote mobile screen take over, biometric, and gesture control as Zero User interfaces. With the breakthroughs in image recognition and natural language processing, powered by advanced computer vision and machine learning, “Zero UI” is becoming a new normal. This paper is focusing on the progress of the touchless interaction technology during the COVID-19 pandemic, which actually accelerated development in this concept and moved it from being a luxury to a life necessity.
... There is limited knowledge on VA use in specific groups of older adults. We identified two studies that focus on benefits and challenges for older adults with cognitive impairment (e.g., dementia; Wargnier et al., 2015;Wolters et al., 2016) and two other studies that included older people with intellectual disabilities (Braun et al., 2020;Smith et al., 2020). We also see limitations at the methodological and design level. ...
... Touchless interaction with small hand gestures could offer opportunities for people with disabilities [5]. Indeed, this kind of user interface is broadly used in gaming but also in assistive technologies, as they are able to identify movements of the body, thus valuable for people with impairments that prevent them from using touch interfaces [6]. ...
Full-text available
Technological advancement is constantly evolving, and it is also developing in the mental health field. Various applications, often based on virtual reality, have been implemented to carry out psychological assessments and interventions, using innovative human–machine interaction systems. In this context, the LEAP Motion sensing technology has raised interest, since it allows for more natural interactions with digital contents, via an optical tracking of hand and finger movements. Recent research has considered LEAP Motion features in virtual-reality-based systems, to meet specific needs of different clinical populations, varying in age and type of disorder. The present paper carried out a systematic mini-review of the available literature using Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines. The inclusion criteria were (i) publication date between 2013 and 2020, (ii) being an empirical study or project report, (iii) written in English or Italian languages, (iv) published in a scholarly peer-reviewed journal and/or conference proceedings, and (v) assessing LEAP Motion intervention for four specific psychological domains (i.e., autism spectrum disorder, attention-deficit/hyperactivity disorder, dementia, and mild cognitive impairment), objectively. Nineteen eligible empirical studies were included. Overall, results show that protocols for attention-deficit hyperactivity disorder and autism spectrum disorder can promote psychomotor and psychosocial rehabilitation in contexts that stimulate learning. Moreover, virtual reality and LEAP Motion seem promising for the assessment and screening of functional abilities in dementia and mild cognitive impairment. As evidence is, however, still limited, deeper investigations are needed to assess the full potential of the LEAP Motion technology, possibly extending its applications. This is relevant, considering the role that virtual reality could have in overcoming barriers to access assessment, therapies, and smart monitoring.
Full-text available
Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems—developed by Amazon, Apple, Google, IBM, and Microsoft—to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies—such as using more diverse training datasets that include African American Vernacular English—to reduce these performance differences and ensure speech recognition technology is inclusive.
Full-text available
Although Sweden is one of the most digitalized countries and the Swedish population’s use of the internet is among the most studied in the world, little is known about how Swedes with disabilities use internet. The purpose of this study is to describe use of and perceived difficulties in use of the internet among people with disabilities and to explore digital divides in-between and within disability groups, and in comparison with the general population. This is a cross-sectional survey targeting the same issues as other nationwide surveys but adapted for people with cognitive disabilities. Participants were recruited from May to October 2017 by adaptive snowball sampling. The survey comprised questions on access to and use of devices, and use of and perceived difficulties in use of internet. A total of 771 people responded to the survey, representing 35 diagnoses/impairments. Larger proportions of people with autism, ADHD and bipolar disorder reported using internet than other disability groups. Women with autism used the internet more than any other disability group, and women with aphasia used the internet the least. People with disabilities related to language and understanding reported more difficulties using internet than other disability groups. Larger proportions of participants than the general Swedish population reported not feeling digitally included. In many but not all disability groups, larger proportions of men than women reported not feeling digitally included. Our findings show that there are differences in digital inclusion between sub-groups of diagnoses/impairments. Thus, disability digital divides are preferably investigated by sub-grouping disabilities, rather than studied as one homogeneous group.
Full-text available
Mobile phone technology is becoming ubiquitous. However, a number of unique usability challenges are still unresolved, including small screen size, device orientation changes, and an array of interaction methods (tap, flick, pinch, etc.) These challenges may be particularly acute for people with learning disabilities. This study examined the usability of touchscreen interactions, the difficulties, and possible solutions. An app was developed in which (12) participants accessed Google Maps and manipulated it to find various London Underground station locations. Text input (a password), tap, swipe, and pinch were required, and their usage was analysed. Many participants were successful in finding the required information. However, many difficulties arose, including misunderstandings of the labelling (a live ‘Welcome’ button was not tapped, whereas a short list of instructions was erroneously seen as a menu and so erroneously tapped to access each step in the process) and an over-sensitive zoom feature. Three categories of error were formulated from the findings: affordance, user, and functionality. Recommendations are offered, such as using more appropriate ‘signage’ for link buttons (affordance); manipulating the zoom feature using + and - buttons rather than a ‘pinch’, which requires two fingered dexterity (functionality); and more formal training and familiarity (user).
Full-text available
The Questionnaire for Observing Communicative Skills-Revision (OCS-R) is an observation based tool to assess communicative or communication-relevant competencies in children, adolescents, and adults with complex communication needs. Its basic idea is a multi-perspective approach: The aim is to reveal similarities and differences in the assessment of different caregivers as basis for individual support. The OCS-R consists of a questionnaire, a manual, and a computer-assisted analysis.
Full-text available
Removing barriers to accessing Information and Communication Technologies (ICTs) by Persons with Intellectual Disabilities (IDPs) is crucial. Being excluded from ICTs implies being shut down from the information society, but also from accessing essential public services, as well as from the opportunity of living an independent life. The IdICT project has the general objective of increasing the competences of IDPs, their families and the professionals that work with them to exploit ICTs with a Quality of Life approach. To do that, a training platform and a training program has been developed and tested by IDPs, relatives and professionals in six European Countries.
This study aimed to investigate the experiences of people with intellectual disability (ID) using information and communication technologies (ICTs) and the ways these technologies foster social connectedness. In partnership with a community mental health organization, this qualitative descriptive study explored the experiences of 10 people with ID using ICTs. Participants described how ICTs Opened the Door to Possibilities in their life - ICTs provided an avenue for connecting with other people, a means to pursue personal interests and a tool for organizing everyday life. Opening the door to possibilities was further understood as movement towards digital inclusion for people with ID, conceptualized as the fit between social opportunity and personal skills. We have identified the prominent role social supports play in creating (and constraining) opportunities for digital inclusion, and that digital inclusion has the potential to enable social connectedness and the development of agency. Consideration of the complex interaction between social opportunity and personal skills, and the mediating influence of supports, will enhance the inclusion of people with ID.
Conference Paper
People with intellectual disability are keen users of information technology, but the need for spelling and typing skills often presents a barrier to information and media search and access. The paper presents a study to understand how people with intellectual disabilities can use Voice Activated Interfaces (VAIs) to access information and assist in daily activities. The study involves observations and video analysis of 18 adults with intellectual disability using VAIs and performing 4 tasks: calibrating the VAIs, using voice assistant (Siri or Google) to search images, using voice to query Youtube, and using the voice assistant to perform a daily task (managing calendar, finding directions, etc.). 72% of participants stated that this was their preferred form of input. 50% could perform all four tasks they attempted with successful outcomes, and 55% three of the tasks. We identify the main barriers and opportunities for existing VAIs and suggest future improvements mainly around audio feedback given to participants. Notably, we found that participants' mental model of the VAIs was that of a person, implications for which include the user having to speak in long polite sentences and expecting voice responses and feedback about the state of the device. We suggest ways that VAIs can be adjusted so that they are more inclusive.
Design and implement voice user interfaces. This guide to VUI helps you make decisions as you deal with the challenges of moving from a GUI world to mixed-modal interactions with GUI and VUI. The way we interact with devices is changing rapidly and this book gives you a close view across major companies via real-world applications and case studies. Voice User Interface Design provides an explanation of the principles of VUI design. The book covers the design phase, with clear explanations and demonstrations of each design principle through examples of multi-modal interactions (GUI plus VUI) and how they differ from pure VUI. The book also differentiates principles of VUI related to chat-based bot interaction models. By the end of the book you will have a vision of the future, imagining new user-oriented scenarios and new avenues, which until now were untouched. What You'll Learn: • Implement and adhere to each design principle • Understand how VUI differs from other interaction models • Work in the current VUI landscape
Purpose: This study evaluated a tablet-based program to help eight participants with moderate intellectual disability, sensory and/or motor impairments, and lack of expressive or expressive and receptive verbal skills to select and access leisure activities and video calls independently. Methods: The program relied on the use of a tablet (i.e., Samsung Galaxy Tab S2 LTE) with 8-inch screen, Android 6.0 Operating System, front camera, proximity sensor and multimedia player. The tablet was fitted with a SIM card and two specific applications, that is, WhatsApp Messenger for making video calls and MacroDroid for automating the tablet’s functioning in accordance with the program conditions. The tablet presented pictures concerning leisure activities and preferred partners for video calls. The participant could select any activity or partner by touching (or nearing his or her hand to) the tablet’s proximity sensor. Results: During the baseline (i.e., without the program), the participants failed to access leisure activities or video calls. During the post-intervention phase (i.e., with the program), they selected and accessed those activities and calls independently and spent between about 75% and 90% of the session time engaging with them. Conclusion: The tablet-based program can be highly beneficial for people like the participants of this study. • Implications for rehabilitation • A technology-aided program may enable persons with intellectual and other disabilities to independently access leisure activities and communication with distant partners. • The program may involve the use of video calls to allow communication to participants with limited or no verbal skills. • The program may be realized using a tablet (a) including Android 6.0 Operating System, proximity sensor, and multimedia player, and (b) fitted with a SIM card and applications such as WhatsApp Messenger and MacroDroid. • The program may be easily adapted to the participants’ characteristics in terms of activities available and partners to reach.