Article

Immediate integration of prosodic information from speech and visual information from pictures in the absence of focused attention: a mismatch negativity study.

State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, 4A Datun Road, Chaoyang District, Beijing 100101, PR China.
Neuroscience (Impact Factor: 3.33). 07/2009; 161(1):59-66. DOI: 10.1016/j.neuroscience.2009.01.070
Source: PubMed

ABSTRACT Language is often perceived together with visual information. Recent experimental evidences indicated that, during spoken language comprehension, the brain can immediately integrate visual information with semantic or syntactic information from speech. Here we used the mismatch negativity to further investigate whether prosodic information from speech could be immediately integrated into a visual scene context or not, and especially the time course and automaticity of this integration process. Sixteen Chinese native speakers participated in the study. The materials included Chinese spoken sentences and picture pairs. In the audiovisual situation, relative to the concomitant pictures, the spoken sentence was appropriately accented in the standard stimuli, but inappropriately accented in the two kinds of deviant stimuli. In the purely auditory situation, the speech sentences were presented without pictures. It was found that the deviants evoked mismatch responses in both audiovisual and purely auditory situations; the mismatch negativity in the purely auditory situation peaked at the same time as, but was weaker than that evoked by the same deviant speech sounds in the audiovisual situation. This pattern of results suggested immediate integration of prosodic information from speech and visual information from pictures in the absence of focused attention.

0 Followers
 · 
119 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The Chinese language possesses linguistic properties that are distinct from those of the most widely studied European languages. Given such uniqueness, research on the neurocognitive processing of Chinese not only contributes to our understanding of language-specific cognitive processes but also sheds light on the universality of psycholinguistic models developed on the basis of these European languages. In this Introduction, we briefly review neurocognitive studies on the processing of Chinese in the past ten years, summarizing existing findings concerning lexical, sentential, and discourse processing in Chinese.
    Language and Cognitive Processes 12/2009; 24(7/8):929-946. · 1.54 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: When being presented with consistent and repetitive sensory stimuli, the human brain creates a predictive “memory trace” against which subsequent stimuli are compared. When later stimuli do not match this predictive model, a highly localized negative shift in the brain polarity occurs. This response, known as the mismatch negativity (MMN), is believed to represent a pre-attentive deviance-detection mechanism that serves to provide direct attention toward unanticipated events. At present, there are conflicting data as to whether visually generated and auditorily generated MMNs interact, or whether they are mediated by independent sensory-specific networks. We present compelling evidence that visual and auditory MMNs are strongly correlated, and that, upon presentation of dual-sensory “audiovisual” deviants, this synergy is heavily dictated by an individual’s unique visual response. This finding is suggestive of inhibitory interaction between the visual and auditory MMN networks. The characterization of this correlation helps one to explain (and explain away) much conflicting data published to date and opens the door to many questions regarding individual perception.
    Neurophysiology 11/2013; 45(5-6). DOI:10.1007/s11062-013-9394-1 · 0.17 Impact Factor

Preview

Download
1 Download
Available from