-
[show abstract]
[hide abstract]
ABSTRACT: In everyday life our brain often receives information about events and objects in the real world via several sensory modalities, because natural objects often stimulate more than one sense. These different types of information are processed in our brain along different sensory-specific pathways, but are finally integrated into a unified percept. During the last years, studies provided compelling evidence that the neural basis of multisensory integration is not restricted to higher association areas of the cortex, but can already occur at low-level stages of sensory cortical processing and even in subcortical structures. In this article we will review the potential role of several thalamic structures in multisensory interplay and discuss their extensive anatomical connections with sensory-specific and multisensory cortical structures. We conclude that sensory-specific thalamic structures may act as a crucial processing node of multisensory interplay in addition to their traditional role as sensory relaying structure.
Communicative & integrative biology 07/2011; 4(4):378-81.
-
[show abstract]
[hide abstract]
ABSTRACT: Dopamine release in cortical and subcortical structures plays a central role in reward-related neural processes. Within this context, dopaminergic inputs are commonly assumed to play an activating role, facilitating behavioral and cognitive operations necessary to obtain a prospective reward. Here, we provide evidence from human fMRI that this activating role can also be mediated by task-demand-related processes and thus extends beyond situations that only entail extrinsic motivating factors. Using a visual discrimination task in which varying levels of task demands were precued, we found enhanced hemodynamic activity in the substantia nigra (SN) for high task demands in the absence of reward or similar extrinsic motivating factors. This observation thus indicates that the SN can also be activated in an endogenous fashion. In parallel to its role in reward-related processes, reward-independent activation likely serves to recruit the processing resources needed to meet enhanced task demands. Simultaneously, activity in a wide network of cortical and subcortical control regions was enhanced in response to high task demands, whereas areas of the default-mode network were deactivated more strongly. The present observations suggest that the SN represents a core node within a broader neural network that adjusts the amount of available neural and behavioral resources to changing situational opportunities and task requirements, which is often driven by extrinsic factors but can also be controlled endogenously.
Journal of Neuroscience 03/2011; 31(13):4955-61. · 7.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Attending to the spatial location or to nonspatial features of a stimulus modulates neural activity in cortical areas that process its perceptual attributes. The feature-based attentional selection of the direction of a moving stimulus is associated with increased firing of individual neurons tuned to the direction of the movement in area V5/MT, while responses of neurons tuned to opposite directions are suppressed. However, it is not known how these multiplicatively scaled responses of individual neurons tuned to different motion-directions are integrated at the population level, in order to facilitate the processing of stimuli that match the perceptual goals. Using functional magnetic resonance imaging (fMRI) the present study revealed that attending to the movement direction of a dot field enhances the response in a number of areas including the human MT region (hMT) as a function of the coherence of the stimulus. Attending the opposite direction, however, lead to a suppressed response in hMT that was inversely correlated with stimulus-coherence. These findings demonstrate that the multiplicative scaling of single-neuron responses by feature-based attention results in an enhanced direction-selective population response within those cortical modules that processes the physical attributes of the attended stimuli. Our results provide strong support for the validity of the "feature similarity gain model" on the integrated population response as quantified by parametric fMRI in humans.
Human Brain Mapping 02/2011; 32(12):2183-92. · 5.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Combining information across modalities can affect sensory performance. We studied how co-occurring sounds modulate behavioral visual detection sensitivity (d'), and neural responses, for visual stimuli of higher or lower intensity. Co-occurrence of a sound enhanced human detection sensitivity for lower- but not higher-intensity visual targets. Functional magnetic resonance imaging (fMRI) linked this to boosts in activity-levels for sensory-specific visual and auditory cortex, plus multisensory superior temporal sulcus (STS), specifically for a lower-intensity visual event when paired with a sound. Thalamic structures in visual and auditory pathways, the lateral and medial geniculate bodies, respectively (LGB, MGB), showed a similar pattern. Subject-by-subject psychophysical benefits correlated with corresponding fMRI signals in visual, auditory, and multisensory regions. We also analyzed differential "coupling" patterns of LGB and MGB with other regions in the different experimental conditions. Effective-connectivity analyses showed enhanced coupling of sensory-specific thalamic bodies with the affected cortical sites during enhanced detection of lower-intensity visual events paired with sounds. Coupling strength between visual and auditory thalamus with cortical regions, including STS, covaried parametrically with the psychophysical benefit for this specific multisensory context. Our results indicate that multisensory enhancement of detection sensitivity for low-contrast visual stimuli by co-occurring sounds reflects a brain network involving not only established multisensory STS and sensory-specific cortex but also visual and auditory thalamus.
Journal of Neuroscience 10/2010; 30(41):13609-23. · 7.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Effective adaptation to the demands of a changing environment requires flexible cognitive control. The medial and the lateral frontal cortices are involved in such control processes, putatively in close interplay with the BG. In particular, dopaminergic projections from the midbrain (i.e., from the substantia nigra [SN] and the ventral tegmental area) have been proposed to play a pivotal role in modulating the activity in these areas for cognitive control purposes. In that dopaminergic involvement has been strongly implicated in reinforcement learning, these ideas suggest functional links between reinforcement learning, where the outcome of actions shapes behavior over time, and cognitive control in a more general context, where no direct reward is involved. Here, we provide evidence from functional MRI in humans that activity in the SN predicts systematic subsequent trial-to-trial RT prolongations that are thought to reflect cognitive control in a stop-signal paradigm. In particular, variations in the activity level of the SN in one trial predicted the degree of RT prolongation on the subsequent trial, consistent with a modulating output signal from the SN being involved in enhancing cognitive control. This link between SN activity and subsequent behavioral adjustments lends support to theoretical accounts that propose dopaminergic control signals that shape behavior both in the presence and in the absence of direct reward. This SN-based modulatory mechanism is presumably mediated via a wider network that determines response speed in this task, including frontal and parietal control regions, along with the BG and the associated subthalamic nucleus.
Journal of Cognitive Neuroscience 05/2010; 23(2):362-73. · 5.18 Impact Factor
-
Ruth M Krebs,
Marty G Woldorff,
Claus Tempelmann,
Nils Bodammer, Toemme Noesselt,
Carsten N Boehler,
Henning Scheich,
Jens-Max Hopf,
Emrah Duzel,
Hans-Jochen Heinze,
Mircea A Schoenfeld
[show abstract]
[hide abstract]
ABSTRACT: The superior colliculus (SC) has been shown to play a crucial role in the initiation and coordination of eye- and head-movements. The knowledge about the function of this structure is mainly based on single-unit recordings in animals with relatively few neuroimaging studies investigating eye-movement related brain activity in humans.
The present study employed high-field (7 Tesla) functional magnetic resonance imaging (fMRI) to investigate SC responses during endogenously cued saccades in humans. In response to centrally presented instructional cues, subjects either performed saccades away from (centrifugal) or towards (centripetal) the center of straight gaze or maintained fixation at the center position. Compared to central fixation, the execution of saccades elicited hemodynamic activity within a network of cortical and subcortical areas that included the SC, lateral geniculate nucleus (LGN), occipital cortex, striatum, and the pulvinar.
Activity in the SC was enhanced contralateral to the direction of the saccade (i.e., greater activity in the right as compared to left SC during leftward saccades and vice versa) during both centrifugal and centripetal saccades, thereby demonstrating that the contralateral predominance for saccade execution that has been shown to exist in animals is also present in the human SC. In addition, centrifugal saccades elicited greater activity in the SC than did centripetal saccades, while also being accompanied by an enhanced deactivation within the prefrontal default-mode network. This pattern of brain activity might reflect the reduced processing effort required to move the eyes toward as compared to away from the center of straight gaze, a position that might serve as a spatial baseline in which the retinotopic and craniotopic reference frames are aligned.
PLoS ONE 01/2010; 5(1):e8691. · 4.09 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The selection of one of two concurrent speech messages for comprehension was investigated in healthy young adults in two event-related potential experiments. The stories were presented from virtual locations located 30 degrees to the left and right azimuth by convolving the speech message by the appropriate head-related transfer function determined for each individual participant. In addition, task irrelevant probe stimuli were presented in rapid sequence from the same virtual locations. In experiment 1, phoneme probes (/da/ voiced by the same talkers as attended and unattended messages) and band-pass filtered noise probes were presented. Phoneme probes coinciding with the attended message gave rise to a fronto-central negativity similar to the Nd-attention effect relative to the phoneme probes coinciding with the unattended speech message, whereas noise probes from the attended message's location showed a more positive frontal ERP response compared to probes from the unattended location resembling the so-called rejection positivity. In experiment 2, phoneme probes (as in exp. 1) and frequency-shifted (+400 Hz) were compared. The latter were characterized by a succession of negative and positive components that were modulated by location. The results suggest that at least two different neural mechanisms contribute to stream segregation in a cocktail-party setting: enhanced neural processing of stimuli matching the attended message closely (indexed by the Nd-effect) and rejection of stimuli that do not match the attended message at the attended location only (indexed by the rejection positivity).
Brain research 10/2009; 1307:78-88. · 2.46 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The present study investigated the neural basis of attention in the somato-sensory system. Subjects directed their attention towards their left or right hand while functional MRI data was collected during tactile stimulation of the fingers. Activations evoked by tactile stimuli when a stimulated hand was attended vs. unattended were contrasted. The tactile stimuli elicited hemodynamic responses in the contralateral primary and secondary somatosensory cortex. No attentional modulations of the BOLD-response could be observed in these regions. However, attention-related modulations were observed at more anterior locations in the ipsi- and contralateral primary motor cortex and in the supplementary motor area. This pattern of results suggests, that attention to somato-sensory events is directly linked to the motor system and the preparation for action. This mechanism appears to be in stark contrast to visual or auditory attention, which primarily serve to separate relevant from irrelevant information.
Journal of the neurological sciences 02/2009; 279(1-2):93-8. · 2.32 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In our daily life we look at many scenes. Some are rapidly forgotten, but others we recognize later. We accurately predicted recognition success with natural scene photographs using single trial magnetoencephalography (MEG) measures of brain activation. Specifically, we demonstrate that MEG responses in the initial 600 ms following the onset of scene photographs allow for prediction accuracy rates up to 84.1% using linear Support-Vector-Machine classification (lSVM). A permutation test confirmed that all lSVM based prediction rates were significantly better than "guessing". More generally, we present four approaches to analyzing brain function using lSVMs. (1) We show that lSVMs can be used to extract spatio-temporal patterns of brain activation from MEG-data. (2) We show lSVM classification can demonstrate significant correlations between comparatively early and late processes predictive of scene recognition, indicating dependencies between these processes over time. (3) We use lSVM classification to compare the information content of oscillatory and event-related MEG-activations and show they contain a similar amount of and largely overlapping information. (4) A more detailed analysis of single-trial predictiveness of different frequency bands revealed that theta band activity around 5 Hz allowed for highest prediction rates, and these rates are indistinguishable from those obtained with a full dataset. In sum our results clearly demonstrate that lSVMs can reliably predict natural scene recognition from single trial MEG-activation measures and can be a useful tool for analyzing predictive brain function.
NeuroImage 07/2008; 42(3):1056-68. · 5.89 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We show that concurrent auditory stimuli can enhance the visual system's ability to detect brief visual events. Participants indicated which of two visual stimuli was briefly blinked off. A spatially non-aligned auditory cue - simultaneous with the blink - significantly enhanced subjects' detection ability, while a visual cue decreased detection ability relative to a no-cue condition. Control experiments indicate that the auditory-driven enhancement was not attributable to a warning effect. Also, the enhancement did not depend on an exact temporal alignment of cue-target onsets or offsets. In combination, our results provide evidence that the sound-induced enhancement is not due to a sharpening of visual temporal responses or apparent prolongation of the visual event. Rather, this enhancement seems to reflect an increase in phenomenal visual saliency.
Brain Research 02/2008; 1220:157-63. · 2.73 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Although much traditional sensory research has studied each sensory modality in isolation, there has been a recent explosion of interest in causal interplay between different senses. Various techniques have now identified numerous multisensory convergence zones in the brain. Some convergence may arise surprisingly close to low-level sensory-specific cortex, and some direct connections may exist even between primary sensory cortices. A variety of multisensory phenomena have now been reported in which sensory-specific brain responses and perceptual judgments concerning one sense can be affected by relations with other senses. We survey recent progress in this multisensory field, foregrounding human studies against the background of invasive animal work and highlighting possible underlying mechanisms. These include rapid feedforward integration, possible thalamic influences, and/or feedback from multisensory regions to sensory-specific brain areas. Multisensory interplay is more prevalent than classic modular approaches assumed, and new methods are now available to determine the underlying circuits.
Neuron 02/2008; 57(1):11-23. · 14.74 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In a study that combined event related potential (ERP) and magnetic field (ERMF) data, we analyzed the timing and direction of information flow between striate (S) and extrastriate (ES) cortex by applying a generalized mutual information measure (DIT for "directed information transfer") during a visual spatial attention task. ERP and ERMF recordings showed that selective attention to stimulus arrays in one visual field enhanced late responses (around 200 ms after the stimulus presentation) that were localized in S (ERMF) and ES (ERP) cortex. The results of the DIT analysis indicate there is a significant attention related increase in the flow of information back from ES to S cortex at around 220 ms, with an associated decrease in the flow of information forward from S cortex to ES cortex. These results support the hypothesis that a feedback mechanism guides attention-related processing in primary visual cortex and provide evidence that DIT can by used to evaluate the direction of information flow between cortical areas.
Human Brain Mapping 02/2008; 29(2):193-206. · 5.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Whenever temporally incongruent audiovisual sequences are presented, the perceived flash rate follows the physical flutter rate. Increasing the auditory flutter rate increases the perceived flicker rate (visual illusions). Likewise, decreasing the flutter rate decreases the perceived flicker rate (visual suppressions). Here, we investigated the electrophysiological correlates of this perceptual phenomenon. Two sequences of visual flashes and auditory beeps were presented either synchronously (both visual flashes (F) and auditory beeps (B) at 3 or 5 Hz, respectively) or asynchronously at different rates (3F5B or 5F3B). Event-related potentials were acquired, while subjects reported the perceived number of flashes (response options: 3, 4, and 5). During asynchronous trials, subjects' flash counts were significantly higher when the flutter rate exceeded the flicker rate (i.e. visual illusions occurred); and lower flutter rate was below the flicker rate (i.e. visual suppressions occurred). Differential brain responses for reported illusions and suppressions (incorrect flash counts) vs. no-illusions/suppressions (correct flash counts) were found over parieto-occipital sites, followed by slow modulations over frontal and occipital areas. Importantly, the modulation over occipital electrodes starting around 500 ms had an inverse polarity for illusions vs. suppressions. These results provide evidence that both sound-induced visual illusions and suppressions are mediated by an interplay of distributed brain regions, in the attempt to fuse asynchronous audiovisual stimuli into a synchronous percept.
Brain Research 11/2007; 1220:132-41. · 2.73 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The ventriloquist creates the illusion that his or her voice emerges from the visibly moving mouth of the puppet [1]. This well-known illusion exemplifies a basic principle of how auditory and visual information is integrated in the brain to form a unified multimodal percept. When auditory and visual stimuli occur simultaneously at different locations, the more spatially precise visual information dominates the perceived location of the multimodal event. Previous studies have examined neural interactions between spatially disparate auditory and visual stimuli [2-5], but none has found evidence for a visual influence on the auditory cortex that could be directly linked to the illusion of a shifted auditory percept. Here we utilized event-related brain potentials combined with event-related functional magnetic resonance imaging to demonstrate on a trial-by-trial basis that a precisely timed biasing of the left-right balance of auditory cortex activity by the discrepant visual input underlies the ventriloquist illusion. This cortical biasing may reflect a fundamental mechanism for integrating the auditory and visual components of environmental events, which ensures that the sounds are adaptively localized to the more reliable position provided by the visual input.
Current Biology 11/2007; 17(19):1697-703. · 9.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The brain should integrate related but not unrelated information from different senses. Temporal patterning of inputs to different modalities may provide critical information about whether those inputs are related or not. We studied effects of temporal correspondence between auditory and visual streams on human brain activity with functional magnetic resonance imaging (fMRI). Streams of visual flashes with irregularly jittered, arrhythmic timing could appear on right or left, with or without a stream of auditory tones that coincided perfectly when present (highly unlikely by chance), were noncoincident with vision (different erratic, arrhythmic pattern with same temporal statistics), or an auditory stream appeared alone. fMRI revealed blood oxygenation level-dependent (BOLD) increases in multisensory superior temporal sulcus (mSTS), contralateral to a visual stream when coincident with an auditory stream, and BOLD decreases for noncoincidence relative to unisensory baselines. Contralateral primary visual cortex and auditory cortex were also affected by audiovisual temporal correspondence or noncorrespondence, as confirmed in individuals. Connectivity analyses indicated enhanced influence from mSTS on primary sensory areas, rather than vice versa, during audiovisual correspondence. Temporal correspondence between auditory and visual streams affects a network of both multisensory (mSTS) and sensory-specific areas in humans, including even primary visual and auditory cortex, with stronger responses for corresponding and thus related audiovisual inputs.
Journal of Neuroscience 11/2007; 27(42):11431-41. · 7.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We investigated the effect of visual eccentricity and spatial alignment on judgments of audiovisual synchrony. Sequences of flashes at 4, 6, and 8 Hz were presented centrally, or at horizontal eccentricities of 6 degrees or 18 degrees. Concurrent sequences of clicks were presented at the same rate as the flashes, or at higher or lower rates. Subjects judged whether the flash rate was the same as (synchronous with), faster than, or slower than the click rate. With the 4- and 6-Hz flash rates, subjects' judgments of audiovisual synchrony increased with increasing eccentricity, but only when the click rate was more rapid than the flash rate. This effect remained even when the size of the peripheral visual stimuli was adjusted to compensate for cortical magnification, and was not significantly influenced by the spatial proximity of the auditory and visual signals. However, it was absent when the auditory and visual stimuli were presented serially rather than concurrently. With the 8-Hz flash rate, synchrony judgments were prevalent irrespective of eccentricity. When two serially presented flash rates were compared, visual-visual matching judgments increased with eccentricity at flash rates of 6 Hz and higher, but decreased at flash rates below 6 Hz. Finally, when two concurrent flash rates were compared, visual-visual synchrony judgments increased with eccentricity at all flash-rate combinations. Together, these results suggest that while perceptual uncertainty can play a role in synchrony judgments at rates of 6 Hz and higher, below 6 Hz eccentricity produces a widening of the window of apparent audiovisual temporal synchrony which perceptual uncertainty cannot explain.
Cognitive Brain Research 11/2005; 25(2):443-58. · 3.77 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Traditional split-field studies and patient research indicate a privileged role for the right hemisphere in emotional processing [1-7], but there has been little direct fMRI evidence for this, despite many studies on emotional-face processing [8-10](see Supplemental Background). With fMRI, we addressed differential hemispheric processing of fearful versus neutral faces by presenting subjects with faces bilaterally [11-13]and orthogonally manipulating whether each hemifield showed a fearful or neutral expression prior to presentation of a checkerboard target. Target discrimination in the left visual field was more accurate after a fearful face was presented there. Event-related fMRI showed right-lateralized brain activations for fearful minus neutral left-hemifield faces in right visual areas, as well as more activity in the right than in the left amygdala. These activations occurred regardless of the type of right-hemifield face shown concurrently, concordant with the behavioral effect. No analogous behavioral or fMRI effects were observed for fearful faces in the right visual field (left hemisphere). The amygdala showed enhanced functional coupling with right-middle and anterior-fusiform areas in the context of a left-hemifield fearful face. These data provide behavioral and fMRI evidence for right-lateralized emotional processing during bilateral stimulation involving enhanced coupling of the amygdala and right-hemispheric extrastriate cortex.
Current Biology 04/2005; 15(5):424-9. · 9.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The influence of context-dependent interactions on attention-related neural activity was studied in the human primary visual cortex (V1) with event-related fMRI. Retinotopic field-sign mapping was used to determine the localization of V1 with respect to adjacent retinotopic areas. Observers reported the orientation of a Gabor patch at pre-cued extrafoveal locations when it was salient among distractor Gabors and when it was not. Saliency was caused by local orientation contrast between Gabors-a mechanism that is thought to arise from context-dependent interactions in the V1 proper. A comparison of the attention-related BOLD response for salient and non-salient stimuli in V1 revealed that salient Gabors caused a significantly smaller BOLD response than non-salient Gabors. This differential effect was not observed in higher-order visual areas (V3/V3A, MT+/LO, IPS). When attention was not focused onto the target, the size of the BOLD response was generally reduced in all visual areas, and no difference was seen in V1 for salient and non-salient Gabors. These findings suggest that contextual interactions underlying saliency influence attentional modulations in V1 and support the view that perceptual and attentional mechanisms share neural circuits at this early stage of visual processing.
NeuroImage 07/2004; 22(2):574-82. · 5.89 Impact Factor
-
Mircea Ariel Schoenfeld MD,
Toemme Noesselt PhD,
Dorothe Poggel PhD,
Claus Tempelmann PhD,
Jens-Max Hopf MD,
Martin G. Woldorff PhD,
Hans-Jochen Heinze MD,
Steven A. Hillyard PhD,
Mircea Ariel Schoenfeld, Toemme Noesselt,
Dorothe Poggel,
Claus Tempelmann,
Jens‐Max Hopf,
Martin G. Woldorff,
Hans‐Jochen Heinze,
Steven A. Hillyard
[show abstract]
[hide abstract]
ABSTRACT: This study investigated the neural substrates of preserved visual functioning in a patient with homonymous hemianopsia and Riddoch syndrome after a posterior cerebral artery stroke affecting the primary visual cortex (area V1). The limited visual abilities of this patient included above-chance verbal reports of movement and color change as well as discrimination of movement direction in the hemianopic field. Functional magnetic resonance imaging showed that motion and color-change stimuli presented to the hemianopic field produced activation in several extrastriate areas of the lesioned hemisphere that were defined using retinotopic mapping. Magnetoencephalographic recordings indicated that evoked activity occurred earlier in the higher-tier visual areas V4/V8 and V5 than in the lower-tier areas V2/V3 adjacent to the lesion. In addition, the functional magnetic resonance imaging analysis showed an increased functional connectivity between areas V4/V8 and V5 of the lesioned hemisphere in comparison with the same areas in the intact hemisphere during the presentation of color changes. These results suggest that visual perception after the V1 lesion in Riddoch syndrome is mediated by subcortical pathways that bypass V1 and project first to higher-tier visual areas V5 and V4/V8 and subsequently to lower-tier areas V2/V3.
Annals of Neurology 11/2002; 52(6):814 - 824. · 11.09 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Recordings of event-related potentials (ERPs) and event-related magnetic fields (ERMFs) were combined with functional magnetic resonance imaging (fMRI) to study visual cortical activity in humans during spatial attention. While subjects attended selectively to stimulus arrays in one visual field, fMRI revealed stimulus-related activations in the contralateral primary visual cortex and in multiple extrastriate areas. ERP and ERMF recordings showed that attention did not affect the initial evoked response at 60-90 ms poststimulus that was localized to primary cortex, but a similarly localized late response at 140-250 ms was enhanced to attended stimuli. These findings provide evidence that the primary visual cortex participates in the selective processing of attended stimuli by means of delayed feedback from higher visual-cortical areas.
Neuron 09/2002; 35(3):575-87. · 14.74 Impact Factor
