H. Steven Scholte

University of Amsterdam | UVA · Department of Psychology

The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity t...

Humans recognize objects in a dynamically changing world integrating evidence across a vast variety of timescales. This ability is showcased by performance on rapid serial visual presentation (RSVP) tasks in which observers succeed at recognizing objects in rapid sequences of natural scenes, at up to 13 ms/image. To date, the computational mechanis...

Models are the hallmark of mature scientific inquiry. In psychology, this maturity has been reached in a pervasive question-what models best represent facial expressions of emotion? Several hypotheses propose different combinations of facial movements [action units (AUs)] as best representing the six basic emotions and four conversational signals a...

Visual perception involves binding of distinct features into a unified percept. Although traditional theories link feature binding to time-consuming recurrent processes, Holcombe and Cavanagh (2001) demonstrated ultrafast, early binding of features that belong to the same object. The task required binding of orientation and luminance within an exce...

Deep Neural Networks (DNNs) may surpass human-level performance on vision tasks such as object recognition and detection, but their model behavior still differs from human behavior in important ways. One prominent example of this difference, and the main focus of our paper, is that DNNs trained on ImageNet exhibit an object texture bias, while huma...

Deep convolutional neural networks (DCNNs) are able to partially predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of f...

Humans can quickly recognize objects in a dynamically changing world. This ability is showcased by the fact that observers succeed at recognizing objects in rapidly changing image sequences, at up to 13 ms/image. To date, the mechanisms that govern dynamic object recognition remain poorly understood. Here, we developed deep learning models for dyna...

Deep convolutional neural networks (DCNNs) are able to predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of four DCNN a...

Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. There is a clear need to incorporate recurrent processing in deep convolutional neural networks, but the computations underlying recurrent processing remain unclear. In this a...

Arousal levels strongly affect task performance. Yet, what arousal level is optimal for a task depends on its difficulty. Easy task performance peaks at higher arousal levels, whereas performance on difficult tasks displays an inverted U-shape relationship with arousal, peaking at medium arousal levels, an observation first made by Yerkes and Dodso...

Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. There is a clear need to incorporate recurrent processing in deep convolutional neural networks (DCNNs) but the computations underlying recurrent processing remain unclear. In...

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. Yet, the specific neural mechanisms underlying the effects of spatial attention on performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effect...

The costs of TV commercial (TVC) campaigns are exponentially increasing from concepting, to storyboard design, to producing the commercial and broadcasting it in the media. It is therefore paramount to determine the future effectiveness of the commercial as soon as is possible during this process. Unfortunately, the reliability of the tools typical...

Most of our knowledge about human emotional memory comes from animal research. Based on this work, the amygdala is often labeled the brain’s “fear center”, but it is unclear to what degree neural circuitries underlying fear and extinction learning are conserved across species. Neuroimaging studies in humans yield conflicting findings, with many stu...

While modern convolutional neural networks achieve outstanding accuracy on many image classification tasks, they are, once trained, much more sensitive to image degradation compared to humans. Much of this sensitivity is caused by the resultant shift in data distribution. As we show, dynamically recalculating summary statistics for normalization ov...

Object and scene recognition both require mapping of incoming sensory information to existing conceptual knowledge about the world. A notable finding in brain-damaged patients is that they may show differentially impaired performance for specific categories, such as for “living exemplars”. While numerous patients with category-specific impairments...

Although feedforward activity may suffice for recognizing objects in isolation, additional visual operations that aid object recognition might be needed for real-world scenes. One such additional operation is figure-ground segmentation, extracting the relevant features and locations of the target object while ignoring irrelevant features. In this s...

An organism’s level of arousal strongly affects task performance. Yet, what level of arousal is optimal for performance depends on task difficulty. For easy tasks, performance is best at higher arousal levels, whereas arousal levels show an inverted-U-shaped relationship with performance for difficult tasks, with best performance at medium arousal...

We present the Amsterdam Open MRI Collection (AOMIC): three datasets with multimodal (3 T) MRI data including structural (T1-weighted), diffusion-weighted, and (resting-state and task-based) functional BOLD MRI data, as well as detailed demographics and psychometric variables from a large set of healthy participants (N = 928, N = 226, and N = 216)....

In common sense experience based on introspection, consciousness is singular. There is only one ‘me’ and that is the one that is conscious. This means that ‘singularity’ is a defining aspect of ‘consciousness’. However, the three main theories of consciousness, Integrated Information, Global Workspace and Recurrent Processing theory, are generally...

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. The specific mechanisms linking neural changes to changes in performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of noise suppression...

While feed-forward activity may suffice for recognizing objects in isolation, additional visual operations that aid object recognition might be needed for real-world scenes. One such additional operation is figure-ground segmentation; extracting the relevant features and locations of the target object while ignoring irrelevant features. In this stu...

People often seek out stories, videos or images that detail death, violence or harm. Considering the ubiquity of this behavior, it is surprising that we know very little about the neural circuits involved in choosing negative information. Using fMRI, the present study shows that choosing intensely negative stimuli engages similar brain regions as t...

Feed-forward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to s...

A fundamental component of interacting with our environment is gathering and interpretation of sensory information. When investigating how perceptual information influences decision-making, most researchers have relied on manipulated or unnatural information as perceptual input, resulting in findings that may not generalize to real-world scenes. Un...

We present the Amsterdam Open MRI Collection (AOMIC): three datasets with multimodal (3T) MRI data including structural (T1-weighted), diffusion-weighted, and (resting-state and task-based) functional BOLD MRI data, as well as detailed demographics and psychometric variables from a large set of healthy participants (N = 928, N = 226, and N = 216)....

Competitions are part and parcel of daily life and require people to invest time and energy to gain advantage over others and to avoid (the risk of ) falling behind. Whereas the behavioral mechanisms underlying competition are well documented, its neurocognitive underpinnings remain poorly understood.We addressed this using neuroimaging and computa...

Feedforward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to so...

People often seek out stories, videos or images that detail death, violence or harm. Considering the ubiquity of this behavior, it is surprising that we know very little about the neural circuits involved in choosing negative information. Here we show that choosing intensely negative stimuli engages similar brain regions as those that support extri...

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl L...

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl L...

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl L...

While modern convolutional neural networks achieve outstanding accuracy on many image classification tasks, they are, compared to humans, much more sensitive to image degradation. Here, we describe a variant of Batch Normalization, LocalNorm, that regularizes the normalization layer in the spirit of Dropout while dynamically adapting to the local i...

Over the past few years, Magnetic Resonance Spectroscopy (MRS) has become a popular method to non-invasively study the relationship between in-vivo concentrations of neurotransmitters such as GABA and Glutamate and cognitive functions in the human brain. However, currently, it is unclear to what extent MRS measures reflect stable trait-like neurotr...

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl I...

The monthly quality control is a more extensive quality check of our 3T MRI system but also the other computers in the operator room. The monthly quality control consists of the following components: Virus scan on scan-computer Moving eyetracker-files from DOS to Windows desktop Calibration of scanner (Philips protocol) 32 channel SNR test

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl I...

Artificial Neural Networks (ANNs) are bio-inspired models of neural computation that have proven highly effective. Still, ANNs lack a natural notion of time, and neural units in ANNs exchange analog values in a frame-based manner, a computationally and energetically inefficient form of communication. This contrasts sharply with biological neurons t...

Smoking is a major heritable and modifiable risk factor for many diseases, including cancer, common respiratory disorders and cardiovascular diseases. Fourteen genetic loci have previously been associated with smoking behaviour-related traits. We tested up to 235,116 single nucleotide variants (SNVs) on the exome-array for association with smoking...

Feedforward deep convolutional neural networks (DCNNs) are matching and even surpassing human performance on object recognition. This performance suggests that activation of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to solve specific visual subtasks. Recent findings in...

Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these d...

Influence of non-animal image content on behavioral and EEG results from Experiment 2. A) Behavioral reaction time and accuracy as a function of complexity (LOW, MED, HIGH) and task instruction (speeded or accurate) when only including (left) or excluding (right) non-animal scenes with vehicles, humans and man-made objects (manually annotated). B-C...

Background: Smoking and alcohol use have been associated with common genetic variants in multiple loci. Rare variants within these loci hold promise in the identification of biological mechanisms in substance use. Exome arrays and genotype imputation can now efficiently genotype rare nonsynonymous and loss of function variants. Such variants are e...

Abstract Even though human fear-conditioning involves affective learning as well as expectancy learning, most studies assess only one of the two distinct processes. Commonly used read-outs of associative fear learning are the fear-potentiated startle reflex (FPS), pupil dilation and US-expectancy ratings. FPS is thought to reflect the affective asp...

Dear researcher, This protocoloutlines the steps to start an MRI project at the Spinoza Centre Roeterseiland (REC). If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl I...

Over the past decade, multivariate "decoding analyses" have become a popular alternative to traditional mass-univariate analyses in neuroimaging research. However, a fundamental limitation of using decoding analyses is that it remains ambiguous which source of information drives decoding performance, which becomes problematic when the to-be-decoded...

Significance Trusting others is central for cooperative endeavors to succeed. To decide whether to trust or not, people generally make eye contact. As pupils of interaction partners align, mimicking pupil size helps them to make well-informed trust decisions. How the brain integrates information from the partner and from their own bodily feedback t...

Dear researcher, This protocol gives you the steps to start an MRI project at the Spinoza Centre Roeterseiland. If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl If yo...

Dear researcher, This protocol gives you the steps to start an MRI project at the Spinoza Centre Roeterseiland. If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl If yo...

Dear researcher, This protocol gives you the steps to start an MRI project at the Spinoza Centre Roeterseiland. If you haven't had contact with Steven Scholte or Tinka Beemsterboer, make sure to contact them before completing this protocol. You can reach us by email: Steven Scholte: h.s.scholte@uva.nl Tinka Beemsterboer: t.beemsterboer@uva.nl If yo...

The monthly quality control is a more extensive quality check of our 3T MRI system but also the other computers in the operator room. The monthly quality control consists of the following components: Virus scan on scan-computer Moving eyetracker-files from DOS to Windows desktop Calibration of scanner (Philips protocol) 32 channel SNR test

Object recognition is thought to be mediated by rapid feed-forward activation of object-selective cortex, with limited contribution of feedback. However, disruption of visual evoked activity beyond feed-forward processing stages has been demonstrated to affect object recognition performance. Here, we unite these findings by reporting that the detec...

Over the past decade, multivariate pattern analyses and especially decoding analyses have become a popular alternative to traditional mass-univariate analyses in neuroimaging research. However, a fundamental limitation of decoding analyses is that the source of information driving the decoder is ambiguous, which becomes problematic when the to-be-d...

A fundamental component of interacting with our environment is the gathering and interpretation of sensory information. When investigating how perceptual information shapes the mechanisms of decision-making, most researchers have relied on the use of manipulated or unnatural information as perceptual input, resulting in findings that may not genera...

Blindsight refers to the observation of residual visual abilities in the hemianopic field of patients without a functional V1. Given the within- and between-subject variability in the preserved abilities and the phenomenal experience of blindsight patients, the fine-grained description of the phenomenon is still debated. Here we tested a patient wi...

Commentary on "Principles for models of neural information processing" by Kendrick Kay.

Artificial Neural Networks (ANNs) are bio-inspired models of neural computation that have proven highly effective. Still, ANNs lack a natural notion of time, and neural units in ANNs exchange analog values in a frame-based manner, a computationally and energetically inefficient form of communication. This contrasts sharply with biological neurons t...

Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural network...

Background Smoking and alcohol use behaviors in humans have been associated with common genetic variants within multiple genomic loci. Investigation of rare variation within these loci holds promise for identifying causal variants impacting biological mechanisms in the etiology of disordered behavior. Microarrays have been designed to genotype rare...

Convolutional neural networks (CNNs) have recently emerged as promising models of human vision based on their ability to predict hemodynamic brain responses to visual stimuli measured with functional magnetic resonance imaging (fMRI). However, the degree to which CNNs can predict temporal dynamics of visual object recognition reflected in neural me...

Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural network...

The present study tested whether the neural patterns that support imagining "performing an action", "feeling a bodily sensation" or "being in a situation" are directly involved in understanding other people's actions, bodily sensations and situations. Subjects imagined the content of short sentences describing emotional actions, interoceptive sensa...