PosterPDF Available

Sensory Substitution and Spatial Navigation in Early and Late Blind Individuals Using a New SensoryFusion Application Installed on a Smartphone

Authors:
Sensory Substitution and Spatial Navigation in Early and Late Blind Individuals
Using a New SensoryFusion Application Installed on a Smartphone.
1 1,2 2,3 4 4 4 5 1,5
M.Bleau , S.Paré , I.Djerourou , D.Bernal , C.Knowledge , M.Piszczor , R.Kupers , M.Ptito
1 Chaire de recherche H.Sanders en Sciences de la Vision, École d'Optométrie, Université de Montréal, Québec, Canada ; 2 Département de Pharmacologie et Physiologie, Université de Montréal, Québec, Canada ; 3 Faculté des Sciences de la Vie,
Université de Strasbourg, France and Département des Sciences Biologiques, Université de Montréal, Québec, Canada ; 4 Signal Garden Inc., Silicon Valley, California ; 5 Institute of Neuroscience and Pharmacology, University of Copenhagen, Denmark.
Introduction
Blind individuals often report difficulties while navigating in their environment and detecting
objects beyond the peri-personal space. Sensory substitution devices (SSDs) are therefore
needed to inform them about the environment via other senses like touch and audition (Kupers
& Ptito, 2004; Maidenbaum, Abboud, & Amedi, 2014).
Previous studies on SSDs like the Tongue Display Unit (TDU) (Chebat, Schneider, Kupers, &
Ptito, 2011), the Eyecane (Maidenbaum et al., 2014) and The vOIce (Auvray, Hanneton, &
O'Regan, 2007) have shown the potential of such devices for spatial navigation. Indeed, it has
been shown that the use of SSDs activates brain areas involved in vison guided spatial
navigation in blind and sighted individuals (Kupers, Chebat, Madsen, Paulson, & Ptito, 2010).
However, SSDs are rarely used due to their high cost, the intensive training they require and
the lack of training programs (Elli, Benetti, & Collignon, 2014).
Figure 1. Brain activation patterns during route
recognition using the TDU or a visual control
paradigm. (A) Results of blind participants, showing
activation of occipital and posterior parietal cortices,
precuneus, fusiform gyrus, right parahippocampus. (B)
Blindfolded sighted control subjects activated the
posterior parietal cortex and the precuneus. (C) Sighted
control subjects performing the route recognition task
visually activated the occipital and superior parietal
cortices, the precuneus, fusiform gyrus, and right
parahippocampus.
In this study, we are testing the currently in development application : SensoryFusion, using
the Signal Garden Electrolyte Engine platform installed on a smartphone. The system uses
multiple sensors to either detect obstacles at a distance directly in front of the user (Detection
mode) or to create a 3D map of the environment (Avoidance mode) and informs the user with
an auditory feedback. In addition, it has been designed to be easy to use and accessible to all
via smartphones, which are increasingly popular in the targeted population (Due, Kupers,
Lange, & Ptito, 2017).
We tested late blind (LB), early blind (EB) individuals and sighted control (SC) for their
navigation ability in an obstacle course using the device. Our goal is to compare their
performance and learning curves to determine if the new application has potential to help
them in daily life activities.
In this study, early blind (EB, n=12), late blind (LB, n=12) and blindfolded-sighted participants (SC, n=24) were tested for their navigation ability in an obstacle course using the SensoryFusion App (Signal Garden Electrolyte Engine platform) installed on an ARCore smartphone.
Methods
Subjects characteristics. SC (n=24), LB (n=12) and EB (n=12) were studied.
# of Participant
Age &
Gender
Onset of
Blindness
Cause of Blindness
# of Participant
Age &
Gender
Onset of
Blindness
LB1
55F
24
Retinitis Pigmentosa
EB1
56M
Birth
LB2
25M
17
Retinitis Pigmentosa
EB2
49F
Birth
LB3
62M
17
Retinitis Pigmentosa
EB3
44F
Birth
LB4
63F
15
Leber’s Amaurosis
EB4
31F
Birth
LB5
70M
38
Meningitis
EB5
18M
Birth
LB6
50M
17
Accident
EB6
49M
Birth
LB7
44F
17
Glaucoma
EB7
33M
Birth
LB8
56F
20
Retina Cancer
EB8
49F
Birth
LB9
47F
22
Diabetic Retinopathy
EB9
34F
Birth
LB10
63F
59
Retinopathy of Prematurity
EB10
46M
Birth
LB11
38F
20
Glaucoma
EB11
65M
Birth
LB12
46M
40
Meningitis
EB12
28M
Birth
Experimental Walkway
The corridor (21 x 2.4m) contained 6
identical obstacles (0.4 x 0.4 x 2m
cardboard boxes) randomly placed across
trials but always distanced by 3m from
each other.
1 2 6
2.4m
21m
Data collected
ŸDetection performance
ŸDistance estimation performance
ŸTotal crossing time
Task Description
Ÿ Detect obstacles by pointing at them
Ÿ Estimate the distance between them and
the object
Ÿ Cross the corridor as quickly as possible.
Task Description
Ÿ
ŸCross the corridor as quickly as possible
Ÿ
ŸAvoid as many obstacles as possibles
Data collected
Ÿ
Ÿ·Avoidance performance
Ÿ·Total crossing time
·
DETECTION MODE
AVOIDANCE MODE
SC LB EB
Groups
0
10
20
30
40
50
60
70
80
90
100
Percentage detected
Center Wall
0
Percentage detected
Object Position
10
20
30
40
50
60
70
80
90
100
*
SC
LB
EB
SC
LB
EB
Groups
0
50
100
150
200
250
300
350
400
450
Time (s)
First Trials
Middle Trials
Last Trials
*
*
***
*
*
SC LB EB
Groups
0
10
20
30
40
50
60
70
80
90
100
%
Avoided
Time (s)
SC LB EB
Groups
0
50
100
150
200
250 *
*
*
First Trials
Middle Trials
Last Trials
Each group had a similar performance
in the detection task: SC=79,86%,
LB=72,22% and EB=80,34%. ANOVAs
did not find any significant difference
(F=0.442, p=.655).
For obstacles in center of the corridor,
performance was 82,84% for SC,
87,75% for LB and 83,26% for EB. For
obstacles to the wall, performance was
77,19% for SC, 58,33% for LB and
77,73% for EB.
Only LB had their performance
affected by the position of obstacles
(p<0.05). However, they did not differ
from SC and LB
DETECTION MODE
Effect of obstacles position.
Across trials, the average time to cross
the obstacle course decreased for all 3
gr oup s (C S= 58. 6s , LB =43 .4 s,
EB=71.6s).
Only SC and EB showed a significant
improvement in their total crossing
time.
AVOIDANCE MODE
Each group had a similar performance
S C = 8 5 , 9 5 % , L B = 8 8 , 6 1 % a n d
EB=84,03%. ANOVAs did not find any
signicant dierence ( F=0.687
p=0.539).
Average of total crossing time for 2 first trials, 2 middle trials and 2
end trials for each group in avoidance task.
Across trials, the total crossing time is higher for SC compared to
LB and EB. Furthermore, LB seem to be intermediate between
SC and EB. Total crossing time of SC always differed from LB
(p<.01) and EB individuals (p<.05), but that LB and EB never
differed from each other.
Discussion
A device has to be assessed according to 3 principles
1) The function for which it was designed
All participants were able to use the application with less than 30 minutes of
training, giving evidence of its ease of use. Furthermore, performance and
learning are similar in SC, LB and EB participant in Detection mode, but in
Avoidance mode, crossings are faster for LB and EB.
2) The benefits that the user gets
According to subjective reports, the application allows to detect obstacles at a
distance and to avoid them, then could even help finding landmarks. The
Avoidance mode is judged the most relevant in daily life: blind tell that
detecting the totality of obstacles is useless since they search to avoid
obstacles directly in their way. Furthermore, head sweeping can affect
balance and their ability to walk in a straight line.
3) The extra coverage going beyond the capabilities of the white cane
According to participants, the system is interesting mostly for obstacles above
pelvis and thus would be a good complement to the long cane which don't
allow this type of detection (Suterko, 1967). Furthermore, it increases the
detection distance up to 3 meters in front of the person. On the other hand, the
application couldn't be used alone because of the difficulty to detect variation
on ground surface. But if this aspect and the comfort is improved, blind
individual could use it alone.
Perspective
1) New obstacles course which reproduced more real situation like doors,
pole, sidewalk/stair and big obstacles under pelvis.
2) Other studies about Eyecane and TDU in the old and new obstacle course
and a review
3) Cerebral imagery to see the activated brains areas involved while using the
application.
Acknowledgments
References
-
Auvray, M., Hanneton, S., & O’Regan, J.K. (2007). Learning to perceive with a visuo-auditory substitution system:
Localisation and object recognition with ‘the Voice’. Perception, 36(3),416-430.
Chebat, D.R., Schneider, F.C., Kupers, R., & Ptito, M. (2011). Navigation with a sensory substitution device in congenitally
blind individuals. Neuroreport, 22(7), 342-347. doi: 10.1097/WNR.0b013e3283462def
Due, B. L., Kupers, R., Lange, S., & Ptito, M. (2017). Technology enhanced vision in blind and visually impaired individuals.
Working Papers on Interaction and Communication, 1-31.
Elli, G. V., Benetti, S., & Collignon, O. (2014). Is there a future for sensory substitution outside academic laboratories?.
Multisensory research, 27(5-6), 271-291.
Kupers, R., & Ptito, M. (2004). “Seeing” through the tongue: cross-modal plasticity in the congenitally blind. Paper
presented at the International Congress Series.
Maidenbaum, S., Abboud, S., & Amedi, A. (2014). Sensory substitution: closing the gap between basic research and
widespread practical visual rehabilitation. Neuroscience & Biobehavioral Reviews, 41, 3-15.
Maidenbaum, S., Hanassy, S., Abboud, S., Buchs, G., Chebat, D. R., Levy-Tzedek, S., & Amedi, A. (2014). The "EyeCane", a
new electronic travel aid for the blind: Technology, behavior & swift learning. Restor Neurol Neurosci, 32(6), 813-824.
doi:10.3233/rnn-130351
Suterko, S. (1967). Long cane training: Its advantages and problems. Paper presented at the Proceedings of the Conference
for Mobility Trainers and Technologists.
Wiener, W. R., Welsh, R. L., & Blasch, B. B. (2010a). Foundations of orientation and mobility (Vol. 1): American Foundation
for the Blind.
Wiener, W. R., Welsh, R. L., & Blasch, B. B. (2010b). Foundations of orientation and mobility (Vol. 2): American Foundation
... The GSSD conveys a simple auditory output based on the point-to-distance principle, while signaling every potential obstacle with a singular sound source that depicts the distance of closest edges from the user. By this means, the user can associate each sound source to a specific obstacle and then plan her/his route through space (Paré et al., 2019). Illustrations of the SSDs are shown in Figure 5. ...
... Without vision, LB has to adapt their strategies by transiting into egocentric point of views with only tactile and auditory cues like CB individuals do. Although LB can learn to use SSDs very efficiently (Lee et al., 2014;Chebat et al., 2015Chebat et al., , 2017Paré et al., 2019), it is clear that they do not possess the same skills as CB (Wan et al., 2010;Chebat et al., 2015Chebat et al., , 2017. This is probably due to the fact that the cross-modal changes witnessed in the late blind are limited compared to that of CB (Park et al., 2009;Reislev et al., 2017;Wen et al., 2018). ...
... Although LB can learn to use SSDs very efficiently (Lee et al., 2014;Chebat et al., 2015Chebat et al., , 2017Paré et al., 2019), it is clear that they do not possess the same skills as CB (Wan et al., 2010;Chebat et al., 2015Chebat et al., , 2017. This is probably due to the fact that the cross-modal changes witnessed in the late blind are limited compared to that of CB (Park et al., 2009;Reislev et al., 2017;Wen et al., 2018). Therefore, the visual experience of LB seems to impair their ability to use SSDs compared to CB and their visual experience seems to be detrimental to cross-modal rewiring of the brain. ...
Article
Full-text available
The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over the last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address the means that have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions.
ResearchGate has not been able to resolve any references for this publication.