H. Steven Scholte

University of Amsterdam | UVA · Department of Psychology

The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity t...

Although certain changes in the brain may reflect fear learning, there are no known markers that indicate whether an aversive experience will develop into fear memory. We examined the moment-to-moment dynamics of human fear learning by applying multi-voxel pattern analysis to single-trial blood oxygen level-dependent magnetic resonance imaging data...

The visual appearance of natural scenes is governed by a surprisingly simple hidden structure. The distributions of contrast values in natural images generally follow a Weibull distribution, with beta and gamma as free parameters. Beta and gamma seem to structure the space of natural images in an ecologically meaningful way, in particular with resp...

Humans recognize objects in a dynamically changing world integrating evidence across a vast variety of timescales. This ability is showcased by performance on rapid serial visual presentation (RSVP) tasks in which observers succeed at recognizing objects in rapid sequences of natural scenes, at up to 13 ms/image. To date, the computational mechanis...

Arousal levels strongly affect task performance. Yet, what arousal level is optimal for a task depends on its difficulty. Easy task performance peaks at higher arousal levels, whereas performance on difficult tasks displays an inverted U-shape relationship with arousal, peaking at medium arousal levels, an observation first made by Yerkes and Dodso...

Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. There is a clear need to incorporate recurrent processing in deep convolutional neural networks (DCNNs) but the computations underlying recurrent processing remain unclear. In...

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. Yet, the specific neural mechanisms underlying the effects of spatial attention on performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effect...

The costs of TV commercial (TVC) campaigns are exponentially increasing from concepting, to storyboard design, to producing the commercial and broadcasting it in the media. It is therefore paramount to determine the future effectiveness of the commercial as soon as is possible during this process. Unfortunately, the reliability of the tools typical...

Most of our knowledge about human emotional memory comes from animal research. Based on this work, the amygdala is often labelled the brain's "fear center", but it is unclear to what degree neural circuitries underlying fear and extinction learning are conserved across species. Neuroimaging studies in humans yield conflicting findings, with many st...

While modern convolutional neural networks achieve outstanding accuracy on many image classification tasks, they are, once trained, much more sensitive to image degradation compared to humans. Much of this sensitivity is caused by the resultant shift in data distribution. As we show, dynamically recalculating summary statistics for normalization ov...

Object and scene recognition both require mapping of incoming sensory information to existing conceptual knowledge about the world. A notable finding in brain-damaged patients is that they may show differentially impaired performance for specific categories, such as for “living exemplars”. While numerous patients with category-specific impairments...

Although feedforward activity may suffice for recognizing objects in isolation, additional visual operations that aid object recognition might be needed for real-world scenes. One such additional operation is figure-ground segmentation, extracting the relevant features and locations of the target object while ignoring irrelevant features. In this s...

An organism’s level of arousal strongly affects task performance. Yet, what level of arousal is optimal for performance depends on task difficulty. For easy tasks, performance is best at higher arousal levels, whereas arousal levels show an inverted-U-shaped relationship with performance for difficult tasks, with best performance at medium arousal...

We present the Amsterdam Open MRI Collection (AOMIC): three datasets with multimodal (3 T) MRI data including structural (T1-weighted), diffusion-weighted, and (resting-state and task-based) functional BOLD MRI data, as well as detailed demographics and psychometric variables from a large set of healthy participants (N = 928, N = 226, and N = 216)....

In common sense experience based on introspection, consciousness is singular. There is only one ‘me’ and that is the one that is conscious. This means that ‘singularity’ is a defining aspect of ‘consciousness’. However, the three main theories of consciousness, Integrated Information, Global Workspace and Recurrent Processing theory, are generally...

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. The specific mechanisms linking neural changes to changes in performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of noise suppression...

While feed-forward activity may suffice for recognizing objects in isolation, additional visual operations that aid object recognition might be needed for real-world scenes. One such additional operation is figure-ground segmentation; extracting the relevant features and locations of the target object while ignoring irrelevant features. In this stu...

People often seek out stories, videos or images that detail death, violence or harm. Considering the ubiquity of this behavior, it is surprising that we know very little about the neural circuits involved in choosing negative information. Using fMRI, the present study shows that choosing intensely negative stimuli engages similar brain regions as t...

Feedforward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to so...

A fundamental component of interacting with our environment is gathering and interpretation of sensory information. When investigating how perceptual information influences decision-making, most researchers have relied on manipulated or unnatural information as perceptual input, resulting in findings that may not generalize to real-world scenes. Un...

We present the Amsterdam Open MRI Collection (AOMIC): three datasets with multimodal (3T) MRI data including structural (T1-weighted), diffusion-weighted, and (resting-state and task-based) functional BOLD MRI data, as well as detailed demographics and psychometric variables from a large set of healthy participants (N = 928, N = 226, and N = 216)....

Competitions are part and parcel of daily life and require people to invest time and energy to gain advantage over others and to avoid (the risk of ) falling behind. Whereas the behavioral mechanisms underlying competition are well documented, its neurocognitive underpinnings remain poorly understood.We addressed this using neuroimaging and computa...

Feedforward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to so...

People often seek out stories, videos or images that detail death, violence or harm. Considering the ubiquity of this behavior, it is surprising that we know very little about the neural circuits involved in choosing negative information. Here we show that choosing intensely negative stimuli engages similar brain regions as those that support extri...

While modern convolutional neural networks achieve outstanding accuracy on many image classification tasks, they are, compared to humans, much more sensitive to image degradation. Here, we describe a variant of Batch Normalization, LocalNorm, that regularizes the normalization layer in the spirit of Dropout while dynamically adapting to the local i...

Over the past few years, Magnetic Resonance Spectroscopy (MRS) has become a popular method to non-invasively study the relationship between in-vivo concentrations of neurotransmitters such as GABA and Glutamate and cognitive functions in the human brain. However, currently, it is unclear to what extent MRS measures reflect stable trait-like neurotr...

Artificial Neural Networks (ANNs) are bio-inspired models of neural computation that have proven highly effective. Still, ANNs lack a natural notion of time, and neural units in ANNs exchange analog values in a frame-based manner, a computationally and energetically inefficient form of communication. This contrasts sharply with biological neurons t...

Smoking is a major heritable and modifiable risk factor for many diseases, including cancer, common respiratory disorders and cardiovascular diseases. Fourteen genetic loci have previously been associated with smoking behaviour-related traits. We tested up to 235,116 single nucleotide variants (SNVs) on the exome-array for association with smoking...

Feedforward deep convolutional neural networks (DCNNs) are matching and even surpassing human performance on object recognition. This performance suggests that activation of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to solve specific visual subtasks. Recent findings in...

Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these d...

Influence of non-animal image content on behavioral and EEG results from Experiment 2. A) Behavioral reaction time and accuracy as a function of complexity (LOW, MED, HIGH) and task instruction (speeded or accurate) when only including (left) or excluding (right) non-animal scenes with vehicles, humans and man-made objects (manually annotated). B-C...

Background: Smoking and alcohol use have been associated with common genetic variants in multiple loci. Rare variants within these loci hold promise in the identification of biological mechanisms in substance use. Exome arrays and genotype imputation can now efficiently genotype rare nonsynonymous and loss of function variants. Such variants are e...

Abstract Even though human fear-conditioning involves affective learning as well as expectancy learning, most studies assess only one of the two distinct processes. Commonly used read-outs of associative fear learning are the fear-potentiated startle reflex (FPS), pupil dilation and US-expectancy ratings. FPS is thought to reflect the affective asp...

Over the past decade, multivariate "decoding analyses" have become a popular alternative to traditional mass-univariate analyses in neuroimaging research. However, a fundamental limitation of using decoding analyses is that it remains ambiguous which source of information drives decoding performance, which becomes problematic when the to-be-decoded...

The human eye can provide powerful insights into the emotions and intentions of others; however, how pupillary changes influence observers’ behavior remains largely unknown. The present fMRI–pupillometry study revealed that when the pupils of interacting partners synchronously dilate, trust is promoted, which suggests that pupil mimicry affiliates...

Object recognition is thought to be mediated by rapid feed-forward activation of object-selective cortex, with limited contribution of feedback. However, disruption of visual evoked activity beyond feed-forward processing stages has been demonstrated to affect object recognition performance. Here, we unite these findings by reporting that the detec...

Over the past decade, multivariate pattern analyses and especially decoding analyses have become a popular alternative to traditional mass-univariate analyses in neuroimaging research. However, a fundamental limitation of decoding analyses is that the source of information driving the decoder is ambiguous, which becomes problematic when the to-be-d...

A fundamental component of interacting with our environment is the gathering and interpretation of sensory information. When investigating how perceptual information shapes the mechanisms of decision-making, most researchers have relied on the use of manipulated or unnatural information as perceptual input, resulting in findings that may not genera...

Blindsight refers to the observation of residual visual abilities in the hemianopic field of patients without a functional V1. Given the within- and between-subject variability in the preserved abilities and the phenomenal experience of blindsight patients, the fine-grained description of the phenomenon is still debated. Here we tested a patient wi...

Commentary on "Principles for models of neural information processing" by Kendrick Kay.

Artificial Neural Networks (ANNs) are bio-inspired models of neural computation that have proven highly effective. Still, ANNs lack a natural notion of time, and neural units in ANNs exchange analog values in a frame-based manner, a computationally and energetically inefficient form of communication. This contrasts sharply with biological neurons t...

Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural network...

Background Smoking and alcohol use behaviors in humans have been associated with common genetic variants within multiple genomic loci. Investigation of rare variation within these loci holds promise for identifying causal variants impacting biological mechanisms in the etiology of disordered behavior. Microarrays have been designed to genotype rare...

Convolutional neural networks (CNNs) have recently emerged as promising models of human vision based on their ability to predict hemodynamic brain responses to visual stimuli measured with functional magnetic resonance imaging (fMRI). However, the degree to which CNNs can predict temporal dynamics of visual object recognition reflected in neural me...

Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural network...

The present study tested whether the neural patterns that support imagining "performing an action", "feeling a bodily sensation" or "being in a situation" are directly involved in understanding other people's actions, bodily sensations and situations. Subjects imagined the content of short sentences describing emotional actions, interoceptive sensa...

Perception is inherently subjective, and individual differences in phenomenology are well illustrated by the phenomenon of synesthesia (highly specific, consistent, and automatic cross-modal experiences, in which the external stimulus corresponding to the additional sensation is absent). It is unknown why some people develop synesthesia and others...

Background Body Integrity Identity Disorder (BIID) is a condition in which individuals perceive a mismatch between their internal body scheme and physical body shape, resulting in an absolute desire to be either amputated or paralyzed. The condition is hypothesized to be of congenital nature, but evidence for a neuro-anatomical basis is sparse. Me...

Raw volumetric data. (XLSX)

Raw clinical data. (SAV)

A big challenge for cognitive neuroscience is to build and apply models based on theoretical knowledge about the human visual system. Recently computational models, inspired from machine learning/computer vision, are increasingly compared and tested against brain responses. These models are highly complex, for example, Convolutional Neural Networks...

A number of recent studies have shown that deep neural networks (DNN) map to the human visual hierarchy. However, based on a large number of subjects and accounting for the correlations between DNN layers, we show that there is no one-to-one mapping of DNN layers to the human visual system. This suggests that the depth of DNN, which is also critica...