Multisensory cortical processing of object shape and its relation to mental imagery

Department of Neurology, Emory University School of Medicine, Atlanta, Georgia 30322, USA.
Cognitive Affective & Behavioral Neuroscience (Impact Factor: 3.29). 07/2004; 4(2):251-9. DOI: 10.3758/CABN.4.2.251
Source: PubMed


Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.

Download full-text


Available from: Minming Zhang, Mar 13, 2014
62 Reads
    • "In addition, prior visual experience also recruited right LOC for haptic perceptual space, whereas only left LOC was activated for the haptic group. One potential explanation for this dissociation may be that right LOC is associated more with visual imagery than left LOC (Zhang et al. 2004). Interestingly, Snow et al. (2014) "
    [Show abstract] [Hide abstract]
    ABSTRACT: Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing. © The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail:
    Cerebral Cortex 07/2015; DOI:10.1093/cercor/bhv170 · 8.67 Impact Factor
  • Source
    • "The majority of research on visuo-haptic processing of object shape has concentrated on higher-level visual areas, in particular the LOC, an object-selective region in the ventral visual pathway (Malach et al., 1995), a sub-region of which also responds selectively to objects in both vision and touch (Amedi et al., 2001, 2002; Stilla and Sathian, 2008). The LOC responds to both haptic 3-D (Amedi et al., 2001; Zhang et al., 2004; Stilla and Sathian, 2008) and tactile 2-D stimuli (Stoesz et al., 2003; Prather et al., 2004) but does not respond during auditory object recognition cued by object-specific sounds (Amedi et al., 2002). However, when participants listened to the impact sounds made by rods and balls made of either metal or wood and categorized these sounds by the shape of the object that made them, the material of the object, or by using all the acoustic information available, the LOC was more activated when these sounds were categorized by shape than by material (James et al., 2011). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Visual and haptic unisensory object processing show many similarities in terms of categorization, recognition, and representation. In this review, we discuss how these similarities contribute to multisensory object processing. In particular, we show that similar unisensory visual and haptic representations lead to a shared multisensory representation underlying both cross-modal object recognition and view-independence. This shared representation suggests a common neural substrate and we review several candidate brain regions, previously thought to be specialized for aspects of visual processing, that are now known also to be involved in analogous haptic tasks. Finally, we lay out the evidence for a model of multisensory object recognition in which top-down and bottom-up pathways to the object-selective lateral occipital complex are modulated by object familiarity and individual differences in object and spatial imagery.
    Frontiers in Psychology 07/2014; 5:730. DOI:10.3389/fpsyg.2014.00730 · 2.80 Impact Factor
  • Source
    • "Behavioral results converged nicely with the activation pattern: D.F. performed well on texture discrimination and at chance on the shape task, whereas M.S. showed the reverse pattern. There is accumulating evidence from fMRI that laterally-located 'visual' shape-selective areas are also invoked during haptic shape processing (Allen and Humphreys, 2009; Amedi et al., 2001, 2002, 2005; Bonda et al., 1996; Deibert et al., 1999; Grefkes et al., 2002; James et al., 2002; Naumer et al., 2010; Peltier et al., 2007; Pietrini et al., 2004; Prather et al., 2004; Reed et al., 2004; Snow et al., in press; Stilla and Sathian, 2008; Stoesz et al., 2003; Tal and Amedi, 2009; Zhang et al., 2004). Most notably, Amedi et al. (2001, 2002) "
    [Show abstract] [Hide abstract]
    ABSTRACT: Shape and texture provide cues to object identity, both when objects are explored using vision and via touch (haptics). Visual shape information is processed within the lateral occipital complex (LOC), while texture is processed in medial regions of the collateral sulcus (CoS). Evidence indicates that the LOC is recruited during both visual and haptic shape processing. Here we used functional magnetic resonance imaging (fMRI) to examine whether 'visual' texture-selective areas are similarly recruited when observers discriminate texture via touch. We used a blocked design in which participants discriminated either the texture or shape of unfamiliar 3-dimensional (3D) objects, via vision or touch. We observed significant haptic texture-selective fMRI responses in medial occipitotemporal cortex within areas adjacent to, but not overlapping, those recruited during visual texture discrimination. Although areas of ventromedial temporal cortex are recruited during visual and haptic texture perception, these areas appear to be spatially distinct and modality-specific.
    NeuroImage 03/2014; DOI:10.1016/j.neuroimage.2014.03.013 · 6.36 Impact Factor
Show more