The role of sound intensity and stop-consonant voicing on McGurk fusions and combinations

Vrije Universiteit Brussel, Bruxelles, Brussels Capital, Belgium
European Journal of Cognitive Psychology (Impact Factor: 1.09). 09/2010; October 01(4):475-491. DOI: 10.1080/09541440143000203

ABSTRACT When presented with an auditory /b/ dubbed onto a visual /g/, listeners sometimes perceive a fused phoneme like /d/ while with the reverse presentation, they experience a combination such as /bg/. This phenomenon reported by McGurk and MacDonald (1976) is here investigated in French for both voiced and voiceless stop consonants, using two levels of auditory intensity (70 dB vs 40 dB). In a first experiment, audiovisual incongruent monosyllables (A/bi/ V/gi/, A/gi/ V/bi/, A/ki/ V/pi/, A/pi/ V/ki/) uttered by a man and by a woman speaker were recorded and dubbed, using an analogical technology. In a second experiment, the same syllables articulated by the man speaker were recorded and dubbed according to digital technology. In a third experiment, the same materials as in the second experiment were used but the presentation procedure of the experimental items was changed: Audiovisual incongruent trials were mixed up with congruent ones. In the three experiments, the role of voicing and of auditory intensity were investigated. Overall, combinations were much more numerous than fusions and both types of illusions tended to increase at low intensity. Voicing had a differential effect on both types of illusions. Combinations were more numerous with voiceless consonants but fusions tended to occur more often with voiced ones. The number of illusions was affected by the dubbing technique but not by the presentation procedure.

Download full-text


Available from: Paul Deltenre, Jul 02, 2015
1 Follower
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Auditory, visual and audiovisual syllables with and without conflicting vowel cues (/i y e ø/) presented to men and women showed (1) most to perceive roundedness by eye rather than by ear, (2) a mostly male minority to be less rely- ing on vision, (3) presence of lip rounding to be noticed more easily than absence, and (4) all to perceive openness by ear rather than by eye.
  • [Show abstract] [Hide abstract]
    ABSTRACT: This research assesses how audiovisual speech integration mechanisms are modulated by sensory and cognitive variables. For this purpose, the McGurk effect (McGurk & MacDonald, 1976) was used as an experimental paradigm. This effect occurs when participants are exposed to incongruent auditory and visual speech signals. For example, when an auditory /b/ is dubbed onto a visual /g/, listeners are led to perceive a fused phoneme like /d/. With the reverse presentation, they experience a combination such as /bg/. In two experiments, auditory intensity (40 dB, 50 dB, 60 dB, and 70 dB), face size (large: 19 × 23 cm and small: 1.8 × 2 cm) and instructions (“multiple choice” and “free response”) were manipulated. Face size and instruction were between-participants variables in both experiments, whereas intensity was a within-participants variable in the first experiment and a between-participants variable in the second one. The main effect of instruction manipulation was highly significant in both experiments, the “multiple choice” condition giving rise to more illusions than the “free response” condition. Intensity was significant in the second experiment only. Illusions were more numerous at 40 dB than at the other three intensities. Finally, a small effect of face size was observed in the second experiment only, illusions being slightly more numerous with the large face. Those results indicate that the processing chain underlying audiovisual speech perception is modulated by the perceptual salience of the visual and auditory inputs as well as by cognitive variables.
    European Journal of Cognitive Psychology 08/2010; July 2005(4-Vol. 17):541-560. DOI:10.1080/09541440440000168 · 1.09 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We are constantly exposed to our own face and voice, and we identify our own faces and voices as familiar. However, the influence of self-identity upon self-speech perception is still uncertain. Speech perception is a synthesis of both auditory and visual inputs; although we hear our own voice when we speak, we rarely see the dynamic movements of our own face. If visual speech and identity are processed independently, no processing advantage would obtain in viewing one's own highly familiar face. In the present experiment, the relative contributions of facial and vocal inputs to speech perception were evaluated with an audiovisual illusion. Our results indicate that auditory self-speech conveys a processing advantage, whereas visual self-speech does not. The data thereby support a model of visual speech as dynamic movement processed separately from speaker recognition.
    Psychonomic Bulletin & Review 02/2012; 19(1):66-72. DOI:10.3758/s13423-011-0176-8 · 2.99 Impact Factor