The role of sound intensity and stop-consonant voicing on McGurk fusions and combinations

Vrije Universiteit Brussel, Bruxelles, Brussels Capital, Belgium
European Journal of Cognitive Psychology (Impact Factor: 1.09). 09/2010; October 01(4):475-491. DOI: 10.1080/09541440143000203

ABSTRACT When presented with an auditory /b/ dubbed onto a visual /g/, listeners sometimes perceive a fused phoneme like /d/ while with the reverse presentation, they experience a combination such as /bg/. This phenomenon reported by McGurk and MacDonald (1976) is here investigated in French for both voiced and voiceless stop consonants, using two levels of auditory intensity (70 dB vs 40 dB). In a first experiment, audiovisual incongruent monosyllables (A/bi/ V/gi/, A/gi/ V/bi/, A/ki/ V/pi/, A/pi/ V/ki/) uttered by a man and by a woman speaker were recorded and dubbed, using an analogical technology. In a second experiment, the same syllables articulated by the man speaker were recorded and dubbed according to digital technology. In a third experiment, the same materials as in the second experiment were used but the presentation procedure of the experimental items was changed: Audiovisual incongruent trials were mixed up with congruent ones. In the three experiments, the role of voicing and of auditory intensity were investigated. Overall, combinations were much more numerous than fusions and both types of illusions tended to increase at low intensity. Voicing had a differential effect on both types of illusions. Combinations were more numerous with voiceless consonants but fusions tended to occur more often with voiced ones. The number of illusions was affected by the dubbing technique but not by the presentation procedure.

1 Follower
  • [Show abstract] [Hide abstract]
    ABSTRACT: This research assesses how audiovisual speech integration mechanisms are modulated by sensory and cognitive variables. For this purpose, the McGurk effect (McGurk & MacDonald, 1976) was used as an experimental paradigm. This effect occurs when participants are exposed to incongruent auditory and visual speech signals. For example, when an auditory /b/ is dubbed onto a visual /g/, listeners are led to perceive a fused phoneme like /d/. With the reverse presentation, they experience a combination such as /bg/. In two experiments, auditory intensity (40 dB, 50 dB, 60 dB, and 70 dB), face size (large: 19 × 23 cm and small: 1.8 × 2 cm) and instructions (“multiple choice” and “free response”) were manipulated. Face size and instruction were between-participants variables in both experiments, whereas intensity was a within-participants variable in the first experiment and a between-participants variable in the second one. The main effect of instruction manipulation was highly significant in both experiments, the “multiple choice” condition giving rise to more illusions than the “free response” condition. Intensity was significant in the second experiment only. Illusions were more numerous at 40 dB than at the other three intensities. Finally, a small effect of face size was observed in the second experiment only, illusions being slightly more numerous with the large face. Those results indicate that the processing chain underlying audiovisual speech perception is modulated by the perceptual salience of the visual and auditory inputs as well as by cognitive variables.
    European Journal of Cognitive Psychology 08/2010; July 2005(4-Vol. 17):541-560. DOI:10.1080/09541440440000168 · 1.09 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Auditory, visual and audiovisual syllables with and without conflicting vowel cues (/i y e ø/) presented to men and women showed (1) most to perceive roundedness by eye rather than by ear, (2) a mostly male minority to be less rely- ing on vision, (3) presence of lip rounding to be noticed more easily than absence, and (4) all to perceive openness by ear rather than by eye.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: PURPOSE: The effect of the signal-to-noise ratio (SNR) on the perception of audiovisual speech in children with and without developmental language disorder (DLD) was investigated by varying the noise level and the sound intensity of acoustic speech. The main hypotheses were that the McGurk effect (where incongruent visual speech alters the auditory speech percept) would be weaker for children with DLD than for controls and that it would get stronger with decreasing SNR in both groups. METHOD: The participants were 8-year-old children with DLD and a sample of children with normal language development. In the McGurk stimuli, the consonant uttered by the voice differed from that articulated by the face. Three sound intensities (24, 36 and 48 dB) and noise levels (-12, 0 and +6 dB) were used. Perception of unisensory visual speech was also measured. RESULTS: The children with DLD experienced a weak McGurk effect, i.e. a weak influence of visual speech on audiovisual speech perception, which remained rather constant across SNR levels. The children with DLD were inaccurate at lip-reading. CONCLUSIONS: Children with DLD have problems in perceiving spoken consonants presented audiovisually and visually. The weaker McGurk effect could be accounted for by the poorer lip-reading ability of children with DLD.
    Journal of Speech Language and Hearing Research 05/2012; 56(1). DOI:10.1044/1092-4388(2012/11-0270) · 1.93 Impact Factor

Full-text (2 Sources)

Available from
May 28, 2014

Cyrille Colin