Roni Arbel’s research while affiliated with Hebrew University of Jerusalem and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (9)


Rapid plasticity in the ventral visual stream elicited by a newly learnt auditory script in congenitally blind adults
  • Article

September 2023

·

24 Reads

·

3 Citations

Neuropsychologia

Roni Arbel

·

·


(A) Examples of some of the visual stimuli used, with their corresponding spectrograms after sonification via the Eye-Music visual-to-auditory SSD for trained and scrambled faces. (B) Trained faces > scrambled faces. Left: RFX-general linear model (GLM) analysis showed a bilateral cluster in fusiform gyrus (FG) (Talairach coordinates: right: 19, –68, –13 (zoom); left: –34, –65, –14). We also found bilateral clusters of activation in a region within the middle occipital gyrus, including the location of occipital face area (OFA) as described in the sighted (Talairach coordinates: right: 21, –94, –5; left: –33, –82, –8) and a cluster of activation in the left inferior frontal gyrus (IFG, Talairach coordinates: –45, 8, 31), two other cortical regions known to be involved in face processing. No preference for faces was observed in the auditory cortex (top panel). Right: The probability map obtained from the overlap of single participants’ activations for the same contrast reveals a remarkably consistent face preference in FG. (C) Trained faces > Scrambled faces. Individual peaks of activation in bilateral FG for all participants are located around the canonical location of the fusiform face area (FFA) in the sighted, lateral to the mid-fusiform sulcus. (D) Region of interest (ROI) approach: significant preference for trained faces > scrambled faces within the canonical location of the sighted right FFA. **P < 0.005.
(A) Whole-brain parametric general linear model (GLM) shows that bilateral clusters in fusiform gyrus (FG) are maximally activated by trained faces, then by face orientation and face novelty, and last by scrambled faces. A similar modulation was observed also for occipital face area (OFA) and inferior frontal gyrus (IFG), two other cortical regions known to be involved in face processing. (B) Whole-brain RFX-GLM with the contrast Faces > Words shows recruitment of the right FG (Talairach coordinates of peak 31, –57, –16), together with a cluster in the right middle occipital gyrus, including OFA, another cortical region known to prefer face-related information. (C) Human voices > natural sounds. Left: RFX-GLM analysis shows the expected recruitment of the left superior temporal gyrus (Talairach coordinates: –60, –19, 10), compatible with the known location of the TVA but crucially, no recruitment of FG. Right: The probability map obtained from the overlap of single participants’ activations for the same contrast, reveals consistent activations in the bilateral TVA but crucially, no activation in the ventral visual stream nor in FG.
Participants' demographic information.
Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience
  • Article
  • Full-text available

October 2022

·

86 Reads

·

7 Citations

Previous evidence suggests that visual experience is crucial for the emergence and tuning of the typical neural system for face recognition. To challenge this conclusion, we trained congenitally blind adults to recognize faces via visual-to-auditory sensory-substitution (SDD). Our results showed a preference for trained faces over other SSD-conveyed visual categories in the fusiform gyrus and in other known face-responsive-regions of the deprived ventral visual stream. We also observed a parametric modulation in the same cortical regions, for face orientation (upright vs. inverted) and face novelty (trained vs. untrained). Our results strengthen the conclusion that there is a predisposition for sensory-independent and computation-specific processing in specific cortical regions that can be retained in life-long sensory deprivation, independently of previous perceptual experience. They also highlight that if the right training is provided, such cortical preference maintains its tuning to what were considered visual-specific face features.

Download

Training congenitally blind adults’ auditory identification of cartoon faces. Eye-Music transformation algorithm: Each image is scanned from left-to-right using a sweep-line approach so that the x-axis is mapped to time (i.e., characters positioned more on the left of the image are heard first). After downsampling the image to the EyeMusic’s resolution (50 × 30 pixels) the y-axis is mapped to the frequency domain using the pentatonic scale, such that parts of a character which appear higher in the image will be sonified with a higher pitch. Color is mapped to musical instruments. Red, white, blue, yellow, and green are transformed into organ notes, choir, brass instruments, string instruments, and reed tones, respectively. (A) An example of a real face transformed via the EyeMusic algorithm. (B) An example of a trained (cartoon) image transformed via the EyeMusic algorithm.
Experimental paradigm. The experiment was divided into three parts. First, participants underwent six hours of training, followed by interim testing of face identification and detection of changes to facial features (1b). The strip-method was used in this first part: each face was divided into three horizontal strips representing the upper, middle, and lower parts of the face (1a). To teach participants the composition of face parts and how to tune their hearing to perceive multiple tones at the same time, they were trained to perceived details within each strip. Following completion of this stage, participants were able to integrate their knowledge and perceive the complete face soundscapes. Second, participants underwent additional 6 h of whole-face training (2a). They subsequently took part in several tasks designed to test face identification, including the identification of learned faces in the untrained, inverted orientation (2b). Third, participants underwent an additional pre-post experiment, in which they quickly learned a new cohort of six faces and were tested on face identification before and after two hours of training (3a–3c).
Auditory face identification by congenitally blind adults. Part I: Interim exploratory testing of face training progress (after 6 h of training): (A) participants were requested to answer yes/no questions on the trained faces regarding facial features (e.g., does this character have blue eyes?) to test their ability to extract meaningful information regarding facial features. Results show that participants could correctly identify facial features, especially larger features. In addition, participants were also asked to name once each trained character, and were successful in doing so. (B) To further assure participants’ were able to detect colorful features within the complicated soundscapes, we changed some features of the trained faces (e.g., eye-colors, hair color etc.). Congenitally blind participants were relatively accurate to detect the presence of a change, but were less accurate when asked to localize the change (i.e., to identify the modified face feature) and to identify the changed color. Part II: Face identification tests following 12 h of training: (C) Participants were tested on face-shape identification of the 6 trained characters. Results show that they were able to correctly identify face-shapes with high accuracy, which was significantly higher than chance level (left bar graph). In addition, they were able to provide an answer within 2 soundscape’ repetitions (right bar graph). (D) Participants were able to successfully discriminate upright (trained) from inverted (untrained) faces with an accuracy significantly above chance level (left bar graph). In addition, we also show that it took them less than 2 soundscape’s repetition to provide an answer (right bar graph). E. Participants were asked to identify face-shapes in their upright (trained) and inverted (untrained) orientation, which resulted in a completely different soundscape. Results show that participants were able to identify untrained inverted faces with an accuracy that was significantly higher than chance level, but significantly lower than their accuracy in identifying the faces when presented in the upright and trained orientation. Part III: Face identification of a new cohort of 6 characters after 2 h of additional training: (F) Participants were requested to differentiate the previously trained faces from those belonging to the new cohort, before and after a 2-h training session. Results show that participants were able to do it with an accuracy significantly above chance level, both before and after the 2-h training (left bar graph). In addition, both before and after training, they managed to provide an answer within 2 soundscape’s repetition (right bar graph). G. Following the 2-h training session, participants were successful at identifying newly-trained faces with higher accuracy than chance. They were significantly more accurate in the identification of newly compared to previously learned faces -accuracy level
taken from Experiment 1 (left). In addition, they were also significantly faster in providing an answer when identifying newly compared to previously learned faces (right). In figures (A–G), error bars represent standard deviation. In figures (C–G), **p < 0.005, while *p < 0.05.
Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

March 2022

·

198 Reads

·

10 Citations

Unlike sighted individuals, congenitally blind individuals have little to no experience with face shapes. Instead, they rely on non-shape cues, such as voices, to perform character identification. The extent to which face-shape perception can be learned in adulthood via a different sensory modality (i.e., not vision) remains poorly explored. We used a visual-to-auditory Sensory Substitution Device (SSD) that enables conversion of visual images to the auditory modality while preserving their visual characteristics. Expert SSD users were systematically taught to identify cartoon faces via audition. Following a tailored training program lasting ~ 12 h, congenitally blind participants successfully identified six trained faces with high accuracy. Furthermore, they effectively generalized their identification to the untrained, inverted orientation of the learned faces. Finally, after completing the extensive 12-h training program, participants learned six new faces within 2 additional hours of training, suggesting internalization of face-identification processes. Our results document for the first time that facial features can be processed through audition, even in the absence of visual experience across the lifespan. Overall, these findings have important implications for both non-visual object recognition and visual rehabilitation practices and prompt the study of the neural processes underlying auditory face perception in the absence of vision.


A. OVAL orthography. Visual representation of the Hebrew alphabet translated into the OVAL orthography. Trained letters are highlighted by a green square around the letter’s Hebrew name and its correspondent transcription homologue in the Latin alphabet. Note that we depict here the Color OVAL. In the Monochromatic OVAL, all letters are white. B. OVAL Algorithm. OVAL visual letters were translated into sounds following the EyeMusic transformation algorithm: each image is scanned from left-to-right using a sweep-line approach so that x-axis is mapped to time (i.e., characters positioned more on the left of the image are heard first). Y-axis is mapped to the frequency domain using the pentatonic scale such that parts of a character which appear higher in the image, will be sonified with a higher pitch. Color is mapped to musical instruments. In the Color OVAL we used three colors: white, blue and red transformed into choir, trumpet and piano respectively. In this example we wrote with the OVAL the word “Hello”.
A. Comparison in the auditory discrimination among trained OVAL characters between pre- and post-training performances. Although pre-training discrimination was already high, indicating easy discriminability of the OVAL audemes, both groups significantly improved after-training, reaching ceiling effect. The Color group outperformed the Monochromatic group in the pre-training performance. B-C. After training, we tested the discriminability of trained vs untrained OVAL letters. B. Both groups reached high-success rate in this task, but Color readers significantly outperformed Monochromatic readers. C. Color readers could also identify trained from untrained letters significantly faster than Monochromatic readers. D-E. During the letter identification task, both groups show high accuracy, but the Color group achieved higher success rates (D). In addition the Color group also showed shorter reaction times (E). F. In the majority of trials, Color OVAL readers required only one presentation of a letter in order to correctly identify it, while Monochromatic readers require an additional repetition on average. In A-F error bars represent standard deviations (SD). Asterisks represent statistically significant differences: ** p<0.005; * p<0.05. G. Pattern of errors during identification of trained characters depicted in a confusion matrix. Each cell represents the percent of errors participants committed in identifying each letter pairs, reported separately for the Monochromatic (left) and Color (right) groups.
After only 6-hours of training both groups could successfully read OVAL strings
A. Color readers were overall more accurate than Monochromatic readers. B. Interaction Group*Word-Type (words; pseudo-words): Both groups were more accurate to read words than pseudo-words but Color readers showed an advantage in reading pseudo-words compared to Monochromatic readers. C. Color readers were overall faster than Monochromatic readers in reading OVAL strings D. Both groups read words faster than pseudo-words. E. Interaction Group*Stimulus-Length. Both groups tended to read long strings quicker than short strings. However, Monochromatic readers read long words significantly slower than Color readers. Post-hoc analyses also revealed that Monochromatic readers were significantly slower when reading long strings compared to short ones, while Color readers, did not significantly differ in the reading speed of short and long strings. F. Length effect on reading speed. This is a complete representation of speed performances when reading OVAL strings of various length, plotted separately for each group, and for words and pseudo-words. Monochromatic readers drastically increased their reading speed when length of the OVAL strings increased. Color readers tended to show the same pattern but less prominent.
The sound of reading: Color-to-timbre substitution boosts reading performance via OVAL, a novel auditory orthography optimized for visual-to-auditory mapping

November 2020

·

73 Reads

·

4 Citations

Reading is a unique human cognitive skill and its acquisition was proven to extensively affect both brain organization and neuroanatomy. Differently from western sighted individuals, literacy rates via tactile reading systems, such as Braille, are declining, thus imposing an alarming threat to literacy among non-visual readers. This decline is due to many reasons including the length of training needed to master Braille, which must also include extensive tactile sensitivity exercises, the lack of proper Braille instruction and the high costs of Braille devices. The far-reaching consequences of low literacy rates, raise the need to develop alternative, cheap and easy-to-master non-visual reading systems. To this aim, we developed OVAL, a new auditory orthography based on a visual-to-auditory sensory-substitution algorithm. Here we present its efficacy for successful words-reading, and investigation of the extent to which redundant features defining characters (i.e., adding specific colors to letters conveyed into audition via different musical instruments) facilitate or impede auditory reading outcomes. Thus, we tested two groups of blindfolded sighted participants who were either exposed to a monochromatic or to a color version of OVAL. First, we showed that even before training, all participants were able to discriminate between 11 OVAL characters significantly more than chance level. Following 6 hours of specific OVAL training, participants were able to identify all the learned characters, differentiate them from untrained letters, and read short words/pseudo-words of up to 5 characters. The Color group outperformed the Monochromatic group in all tasks, suggesting that redundant characters’ features are beneficial for auditory reading. Overall, these results suggest that OVAL is a promising auditory-reading tool that can be used by blind individuals, by people with reading deficits as well as for the investigation of reading specific processing dissociated from the visual modality.


Vision through other senses: Practical use of Sensory Substitution devices as assistive technology for visual rehabilitation

November 2014

·

268 Reads

·

22 Citations

·

Roni Arbel

·

Galit Buchs

·

[...]

·

Visual-to-auditory Sensory Substitution Devices (SSDs) are non-invasive sensory aids that provide visual information to the blind via their functioning senses, such as audition. For years SSDs have been confined to laboratory settings, but we believe the time has come to use them also for their original purpose of real-world practical visual rehabilitation. Here we demonstrate this potential by presenting for the first time new features of the EyeMusic SSD, which gives the user whole-scene shape, location & color information. These features include higher resolution and attempts to overcome previous stumbling blocks by being freely available to download and run from a smartphone platform. We demonstrate with use the EyeMusic the potential of SSDs in noisy real-world scenarios for tasks such as identifying and manipulating objects. We then discuss the neural basis of using SSDs, and conclude by discussing other steps-in-progress on the path to making their practical use more widespread.


Figure 2 | Experimental design. (a) Illustration of the left target which participants had to identify (Training session) or reach for (Test session); The left
Figure 3 | Directional error on rotation and washout trials, 6standard error. An initial error of approx. 30u upon introduction of the rotation is reduced during the rotation blocks on both vision and SSD trials. Black arrows indicate the onset and offset of rotation.
(a) An illustration of the EyeMusic SSD. The user is wearing a mobile camera on a pair of glasses, which captures the colorful image in front of him. The EyeMusic algorithm translates the image into a combination of musical notes, conveyed via scalp headphones. The sounds enable the user to reconstruct the original image, and, based on this information, perform the appropriate motor action. Inset: a close up of the EyeMusic head gear. (b) An illustration of the experimental setup. The participant is seated in front of a table, controlling the joystick with his right hand. His hand and forearm are occluded from view, and he sees the target and cursor location on the computer screen (on VIS trials). Via headphones, he hears the soundscapes marking the target location (on SSD trials).
(a) Illustration of the left target which participants had to identify (Training session) or reach for (Test session); The left (long) and the center (short) vertical bars were used to cue the participants on the start and midpoint of the scan (left panel); a schematic of the anticipated rotation during the visual rotation is shown in the middle panel, and a schematic of a possible outcome with the SSD shown on the right. The green lines represent the expected hand motion of the participants. (b) The experimental protocol: 40 trials during the baseline block, followed by 4 blocks of rotation with 60 trials each, and a single washout block with 40 trials. Between each two blocks, a 40-sec break was given, along with an on-screen report of the participant's average cumulative score. (c) A schematic of the two possible outcomes of the experiment during the rotation: if there is no cross-sensory transfer (left panel), the error on the VIS trials will drop, while the error in the SSD trials will remain constant; if, however, there is cross-sensory transfer (right panel), the drop in error on VIS trials will be accompanied by a drop in error on SSD trials.
An initial error of approx. 30° upon introduction of the rotation is reduced during the rotation blocks on both vision and SSD trials. Black arrows indicate the onset and offset of rotation.
Cross-sensory transfer of sensory-motor information: Visuomotor learning affects performance on an audiomotor task, using sensory-substitution

December 2012

·

171 Reads

·

57 Citations

Visual-to-auditory sensory-substitution devices allow users to perceive a visual image using sound. Using a motor-learning task, we found that new sensory-motor information was generalized across sensory modalities. We imposed a rotation when participants reached to visual targets, and found that not only seeing, but also hearing the location of targets via a sensory-substitution device resulted in biased movements. When the rotation was removed, aftereffects occurred whether the location of targets was seen or heard. Our findings demonstrate that sensory-motor learning was not sensory-modality-specific. We conclude that novel sensory-motor information can be transferred between sensory modalities.




Virtual 3D shape and orientation discrimination using point distance information

September 2012

·

113 Reads

·

10 Citations

Distance information is critical to our understanding of our surrounding environment, especially in virtual reality settings. Unfortunately, as we gage distance mainly visually, the blind are prevented from properly utilizing this parameter to formulate 3D cognitive maps and cognitive imagery of their surroundings. We show qualitatively that with no training it is possible for blind and blindfolded subjects to easily learn a simple transformation between virtual distance and sound, based on the concept of a virtual guide cane (paralleling in a virtual environment the “EyeCane”, developed in our lab), enabling the discrimination of virtual 3D orientation and shapes using a standard mouse and audio-system.

Citations (8)


... It is possible that affective encoding of positive stimuli in our sample of hightrauma-exposed children might also rely upon this ventral visual pathway and rely less on connections with the hippocampus and amygdala. The ventral visual stream is capable of undergoing rapid plasticity (Arbel et al., 2023). A study using transcranial directcurrent stimulation (tDCS) found that tDCS to the ventral visual stream improved memory encoding in adults (Zhao & Woodman, 2021). ...

Reference:

Impacts of early life adversity on the neurocircuitry of emotional memory in children
Rapid plasticity in the ventral visual stream elicited by a newly learnt auditory script in congenitally blind adults
  • Citing Article
  • September 2023

Neuropsychologia

... The first involves a conversion of full images to audio or tactile stimulation [42,[44][45][46][47]. On the practical level, the main advantage of such methods is their ability to preserve and convey a large amount of information present in the visual scene. This way, following a learning process, the brain is able to utilize its inherent abilities or develop new ones for comprehending various dimensions of information from the scene [48][49][50][51][52][53]. The main disadvantage of these methods relates directly to their advantage-in order to reach a meaningful level of comprehension pertaining to the scene in its entirety, extremely lengthy training times are required. ...

Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience

... Eighteen congenitally blind native Polish speakers (10 females, mean age: 34.4 years, SD = 6.9) and 18 sighted adults (11 females, mean age: 35.2 years, SD = 8.5) matched in gender and age with the blind group (all ps > 0.34) participated in the study. Congenitally blind people are a hard-to-nd clinical population, and such a sample size is comparable to or even larger than reported in previous behavioral studies with blind individuals (e.g., 8 participants in [47]; 17 participants in [48]; or 12 participants in [49]). All participants reported no additional sensory or motor disabilities, neurological problems, or psychiatric disorders. ...

Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

... Specifically, we suggest that the addition of the 'color' feature to soundscapes might have provided another discrimination feature among auditory pixels, and thus might have increased discriminability among face features. Crucially, these results are in line with another recent work from our team which similarly showed that color-to-timbre mapping enhanced discrimination of auditory letters and boosts reading performance via the EyeMusic SSD compared to identical monochrome soundscapes 16 . Future studies exploring the advantages of auditory SSD-conveyed colors by directly comparing colorful to monochromatic complicated soundscapes will advance our understanding of the extent to which the visual domain compares to the auditory domain in image perception. ...

The sound of reading: Color-to-timbre substitution boosts reading performance via OVAL, a novel auditory orthography optimized for visual-to-auditory mapping

... Participants were sometimes asked to describe their subjective experience in the course of the experiment, and these free reports were collected and reported (see, for example, Grespan Several attempts have been made to directly investigate the subjective SS experience with self-reports. Most commonly, researchers applied interviews (Karam, Russo, & Fels, 2009;Nagel et al., 2005), questionnaires ( Abboud et al., 2014;Auvray et al., 2007;Froese et al., 2012;Karam et al., 2009), and free reports ( Grespan et al., 2008;Maidenbaum et al., 2012;Ortiz et al., 2011). The studies usually adopted some initial assumptions about SS phenomenology. ...

Virtual 3D shape and orientation discrimination using point distance information

... In the context of spatial navigation, tactile sensory augmentation devices that provide access to previously unknown information have been suggested as a way to boost spatial navigation. For instance, the "EyeCane" is a cane that offers blind users a novel sense of distances toward objects in space [36][37][38]. It uses an infrared signal to measure the distance toward a pointed object and produces a corresponding auditory signal. ...

The EyeCane - Distance information for the blind
  • Citing Conference Paper
  • November 2012

Journal of Molecular Neuroscience

... The vertical dimension is mapped into a musical pentatonic pitch scale. The system has been slightly modified in a recent version (Maidenbaum et al., 2014b) with an increased image resolution of 50 × 30 and a hexatonic scale. This type of sonification using horizontal scans, has proven useful for specific tasks. ...

Vision through other senses: Practical use of Sensory Substitution devices as assistive technology for visual rehabilitation
  • Citing Article
  • November 2014

... This example indicates that the acquisition of a letter or digit depends on successful integration between visual and auditory information (for a review, see Blomert & Froyen, 2010;Raij et al., 2000). As mentioned before, letters or digits are multimodally experienced also during learning to write as it involves motor programs and action plans in addition to vision and/or audition (see Fridland, 2021;Levy-Tzedek et al., 2012;Prinz, 1997). However, one outstanding question that still remains to answer is: does this kind of acquired knowledge transfer to a new sensory modality, such as tactile modality that is typically not used to encode linguistic inputs while learning? ...

Cross-sensory transfer of sensory-motor information: Visuomotor learning affects performance on an audiomotor task, using sensory-substitution