Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr Res 97, 173-183

Program in Cognitive Neuroscience, Department of Psychology, The City College of City University of New York, 138th St. and Convent Avenue, New York, New York 10031, USA.
Schizophrenia Research (Impact Factor: 4.43). 01/2008; 97(1-3):173-83. DOI: 10.1016/j.schres.2007.08.008
Source: PubMed

ABSTRACT Viewing a speaker's articulatory movements substantially improves a listener's ability to understand spoken words, especially under noisy environmental conditions. In this study we investigated the ability of patients with schizophrenia to integrate visual and auditory speech. Our objective was to determine to what extent they experience benefit from visual articulation and to detail under what listening conditions they might show the greatest impairments.
We assessed the ability to recognize auditory and audiovisual speech in different levels of noise in 18 patients with schizophrenia and compared their performance with that of 18 healthy volunteers. We used a large set of monosyllabic words as our stimuli in order to more closely approximate performance in everyday situations.
Patients with schizophrenia showed deficits in their ability to derive benefit from visual articulatory motion. This impairment was most pronounced at signal-to-noise levels where multisensory gain is known to be maximal in healthy control subjects. A surprising finding was that despite known early auditory sensory processing deficits and reports of impairments in speech processing in schizophrenia, patients' performance in unisensory auditory speech perception remained fully intact.
Thus, the results showed a specific deficit in multisensory speech processing in the absence of any measurable deficit in unisensory speech processing and suggest that sensory integration dysfunction may be an important and, to date, rather overlooked aspect of schizophrenia.

  • Source
    • "), autism (Brandwein et al. 2013), and schizophrenia (Ross et al. 2007b). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Visual speech can greatly enhance a listener's comprehension of auditory speech when they are presented simultaneously. Efforts to determine the neural underpinnings of this phenomenon have been hampered by the limited temporal resolution of hemodynamic imaging and the fact that electro- and magnetoencephalographic (EEG/MEG) data are usually analyzed in response to simple, discrete stimuli. Recent research has shown that neuronal activity in human auditory cortex tracks the envelope of natural speech. Here, we exploit this finding by estimating a linear forward mapping between the speech envelope and EEG data and show that the latency at which the envelope of natural speech is represented in cortex is shortened by >10 ms when continuous audiovisual speech is presented in comparison to audio-only speech. In addition, we use a reverse mapping approach to reconstruct an estimate of the speech stimulus from the EEG data and, by comparing the bimodal estimate to the sum of the unimodal estimates, find no evidence of any nonlinear additive effects in the audiovisual speech condition. These findings point to an underlying mechanism that could account for enhanced comprehension during audiovisual speech. Specifically, we hypothesize that low-level acoustic features that are temporally coherent with the preceding visual stream may be synthesized into a speech object at an earlier latency which may provide an extended period of low-level processing before extraction of semantic information.
    Journal of Neurophysiology 01/2014; 111(7). DOI:10.1152/jn.00690.2013 · 3.04 Impact Factor
  • Source
    • "multisensory integration is a well-known problem of schizophrenia (Ross et al., 2007; Szycik et al., 2009; Seubert et al., 2010; Williams et al., 2010), which can be explained by an attention deficit (de Jong et al., 2010). Early studies in healthy people did not find any effect of attention on multisensory integration (reviewed in De Gelder and Bertelson, 2003), but recent works indicate the specific role of attention in multisensory perception (Talsma et al., 2010; Zvyagintsev et al., 2011; Roudaia et al., 2013). "
    [Show abstract] [Hide abstract]
    ABSTRACT: The impairment of multisensory integration in schizophrenia is often explained by deficits of attentional selection. Emotion perception, however, does not always depend on attention because affective stimuli can capture attention automatically. In our study, we specify the role of attention in the multisensory perception of emotional stimuli in schizophrenia. We evaluated attention by interference between conflicting auditory and visual information in two multisensory paradigms in patients with schizophrenia and healthy participants. In the first paradigm, interference occurred between physical features of the dynamic auditory and visual stimuli. In the second paradigm, interference occurred between the emotional content of the auditory and visual stimuli, namely fearful and sad emotions. In patients with schizophrenia, the interference effect was observed in both paradigms. In contrast, in healthy participants, the interference occurred in the emotional paradigm only. These findings indicate that the information leakage between different modalities in patients with schizophrenia occurs at the perceptual level, which is intact in healthy participants. However, healthy participants can have problems with the separation of fearful and sad emotions similar to those of patients with schizophrenia.
    Frontiers in Human Neuroscience 10/2013; 7:674. DOI:10.3389/fnhum.2013.00674 · 2.90 Impact Factor
  • Source
    • "Congenital blindness is protective for Sz Sz Silverstein et al. (2006), Silverstein et al. (2012) Visual processing Sz Butler and Javitt (2005) Auditory processing Sz Micoulaud-Franchi et al. (2012, 2011), Leitman et al. (2008) Audio–visual processing Sz de Jong et al. (2009), Ross et al. (2007) interaction enhances and environmental information is increasingly important (Ayres et al., 2005; Feinberg, 2009; Ghazanfar, 2012). Interplay between visual, auditory and somatosensory information becomes more dynamical. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The aim of this review is to describe the potential relationship between multisensory disintegration and self-disorders in schizophrenia spectrum disorders. Sensory processing impairments affecting multisensory integration have been demonstrated in schizophrenia. From a developmental perspective multisensory integration is considered to be crucial for normal self-experience. An impairment of multisensory integration is called 'perceptual incoherence'. We theorize that perceptual incoherence may evoke incoherent self-experiences including depersonalization, ambivalence, diminished sense of agency, and 'loosening of associations' between thoughts, feelings and actions that lie within the framework of 'self-disorders' as described by Sass and Parnas (2003). We postulate that subconscious attempts to restore perceptual coherence may induce hallucinations and delusions. Increased insight into mechanisms underlying 'self-disorders' may enhance our understanding of schizophrenia, improve recognition of early psychosis, and extend the range of therapeutic possibilities.
    Schizophrenia Research 08/2013; 152(1). DOI:10.1016/j.schres.2013.07.027 · 4.43 Impact Factor
Show more