Project

NextGenVis: Training the Next Generation of European Visual Neuroscientists

Goal: NextGenVis is a European team of early career research fellows collaborating within a Marie Curie Sklodowska Innovative Training Network. The fellows are drawn from health care sectors, universities and private companies and are all trained within the field of visual and computational neuroscience. The NextGenVis network brings together unique expertise and resources in brain imaging, psychology, neurology, ophthalmology, and computer science.

The project is called NextGenVis for a reason: the goal is to take vision research further by Training the Next Generation of Visual Neuroscientists. Training includes both academic research and collaboration with the health care and high-tech industry. The network of fellows are trained in quantitative knowledge on the adaptive properties of the visual brain in health and disease – with a strong focus on the neurocomputational basis – and how to apply this new knowledge to boost innovation in health care and technology.

Methods: Psychophysics, Computational Neuroscience, EEG/MEG, Deep Learning, fMRI Analysis, Eye Tracking, SSVEP, ESMI, Population Receptive Field Mapping, Visual Neuroscience

Date: 1 August 2015

Updates
0 new
7
Recommendations
0 new
2
Followers
0 new
21
Reads
0 new
244

Project log

Joana Carvalho
added a research item
The visual brain has the remarkable capacity to complete our percept of the world even when the information extracted from the visual scene is incomplete. This ability to predict missing information based on information from spatially adjacent regions is an intriguing attribute of healthy vision. Yet, it gains particular significance when it masks the perceptual consequences of a retinal lesion, leaving patients unaware of their partial loss of vision and ultimately delaying diagnosis and treatment. At present, our understanding of the neural basis of this masking process is limited which hinders both quantitative modelling as well as translational application. To overcome this, we asked the participants to view visual stimuli with and without superimposed artificial scotoma (AS). We used fMRI to record the associated cortical activity and applied model-based analyses to track changes in cortical population receptive fields and connectivity in response to the introduction of the AS. We found that throughout the visual field and cortical hierarchy, pRFs shifted their preferred position towards the AS border. Moreover, extrastriate areas biased their sampling of V1 towards sections outside the AS projection zone, thereby effectively masking the AS with signals from spared portions of the visual field. We speculate that the signals that drive these system-wide population modifications originate in extrastriate visual areas and, through feedback, also reconfigure the neural populations in the earlier visual areas.
Joana Carvalho
added a research item
Purpose: To evaluate the accuracy and reliability of functional magnetic resonance imaging (fMRI)-based techniques to assess the integrity of the visual field (VF). Methods: We combined 3T fMRI and neurocomputational models, that is, conventional population receptive field (pRF) mapping and a new advanced pRF framework "microprobing" (MP), to reconstruct the VF representations of different cortical areas. To demonstrate their scope, both approaches were applied in healthy participants with simulated scotomas and participants with glaucoma. For the latter group we compared the VFs obtained with standard automated perimetry (SAP) and via fMRI. Results: Using SS, we found that the fMRI-based techniques can detect absolute defects in VFs that are larger than 3°, in single participants, based on 12 minutes of fMRI scan time. Moreover, we found that the MP approach results in a less biased estimation of the preserved VF. In participants with glaucoma, we found that fMRI-based VF reconstruction detected VF defects with a correspondence to SAP that was decent, reflected by the positive correlation between fMRI-based sampling density and SAP-based contrast sensitivity loss (SAP) r2 = 0.44, P = 0.0002. This correlation was higher for MP compared to that for the conventional pRF analysis. Conclusions: The fMRI-based reconstruction of the VF enables the evaluation of vision loss and provides useful details on the properties of the visual cortex. Translational relevance: The fMRI-based VF reconstruction provides an objective alternative to detect VF defects. It may either complement SAP or could provide VF information in patients unable to perform SAP.
Marc M. Himmelberg
added a research item
Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single “motion-in-depth” pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.
Marc M. Himmelberg
added 3 research items
The excitotoxic theory of Parkinson's disease (PD) hypothesises that a pathophysiological degeneration of dopaminergic neurons stems from neural hyperactivity at early stages of disease, leading to mitochondrial stress and cell death. Recent research has harnessed the visual system of Drosophila PD models to probe this hypothesis. Here, we investigate whether abnormal visual sensitivity and excitotoxicity occur in early-onset PD Drosophila models DJ-1Δ72, DJ1-Δ93, and PINK15. We used an electroretinogram to record steady state visually evoked potentials driven by temporal contrast stimuli. At 1 day of age, all early-onset PD mutants had a twofold increase in response amplitudes when compared to w- controls. Further, we found that excitotoxicity occurs in older early-onset PD models after increased neural demand is applied via visual stimulation. In an additional analysis, we used a linear discriminant analysis to test whether there were subtle variations in neural gain control that could be used to classify Drosophila into their correct age and genotype. The discriminant analysis was highly accurate, classifying Drosophila into their correct genotypic class at all age groups at 50-70% accuracy (20% chance baseline). Differences in cellular processes link to subtle alterations in neural network operation in young flies - all of which lead to the same pathogenic outcome. Our data are the first to demonstrate abnormal gain control and excitotoxicity in early-onset PD Drosophila mutants. We conclude that early-onset PD mutations may be linked to more sensitive neuronal signalling in prodromal animals that may cause the expression of PD symptomologies later in life.
Alex Hernandez-Garcia
added a research item
Current computational models of visual salience accurately predict the distribution of fixations on isolated visual stimuli. It is not known, however, whether the global salience of a stimulus, that is, its effectiveness in the competition for attention with other stimuli, is a function of the local salience or an independent measure. Further, do task and familiarity with the competing images influence eye movements? Here, we investigated the direction of the first saccade to characterize and analyze the global visual salience of competing stimuli. Participants freely observed pairs of images while eye movements were recorded. The pairs balanced the combinations of new and already seen images, as well as task and task-free trials. Then, we trained a logistic regression model that accurately predicted the location-left or right image-of the first fixation for each stimulus pair, accounting too for the influence of task, familiarity, and lateral bias. The coefficients of the model provided a reliable measure of global salience, which we contrasted with two distinct local salience models, GBVS and Deep Gaze. The lack of correlation of the behavioral data with the former and the small correlation with the latter indicate that global salience cannot be explained by the feature-driven local salience of images. Further, the influence of task and familiarity was rather small, and we reproduced the previously reported left-sided bias. Summarized, we showed that natural stimuli have an intrinsic global salience related to the human initial gaze direction, independent of the local salience and little influenced by task and familiarity.
Joana Carvalho
added 2 research items
The characterization of receptive field (RF) properties is fundamental to understanding the neural basis of sensory and cognitive behaviour. The combination of non-invasive imaging, such as fMRI, with biologically inspired neural modelling has enabled the estimation of population RFs directly in humans. However, current approaches require making numerous a priori assumptions, so these cannot reveal unpredicted properties, such as fragmented RFs or subpopulations. This is a critical limitation in studies on adaptation, pathology or reorganization. Here, we introduce micro-probing (MP), a technique for fine-grained and largely assumption free characterization of multiple pRFs within a voxel. It overcomes many limitations of current approaches by enabling detection of unexpected RF shapes, properties and subpopulations, by enhancing the spatial detail with which we analyze the data. MP is based on tiny, fixed-size, Gaussian models that efficiently sample the entire visual space and create fine-grained probe maps. Subsequently, we derived population receptive fields (pRFs) from these maps. We demonstrate the scope of our method through simulations and by mapping the visual fields of healthy participants and of a patient group with highly abnormal RFs due to a congenital pathway disorder. Without using specific stimuli or adapted models, MP mapped the bilateral pRFs characteristic of observers with albinism. In healthy observers, MP revealed that voxels may capture the activity of multiple subpopulations RFs that sample distinct regions of the visual field. Thus, MP provides a versatile framework to visualize, analyze and model, without restrictions, the diverse RFs of cortical subpopulations in health and disease.
Purpose To evaluate the accuracy and reliability of functional magnetic resonance imaging (fMRI)-based techniques to assess the integrity of the visual field (VF). Methods We combined fMRI and neurocomputational models, i.e conventional population receptive field (pRF) mapping and a new advanced pRF framework “micro-probing” (MP), to reconstruct the visual field representations of different cortical areas. To demonstrate their scope, both approaches were applied in healthy participants with simulated scotomas (SS) and participants with glaucoma. For the latter group we compared the VFs obtained with standard automated perimetry (SAP) and via fMRI. Results Using SS, we found that the fMRI-based techniques can detect absolute defects in VFs that are larger than 3 deg, in single participants, and based on 12 minutes of fMRI scan time. Moreover, we found that MP results in a less biased estimation of the preserved VF. In participants with glaucoma, we found that fMRI-based VF reconstruction detected VF defects with a correspondence to SAP that was decent, reflected by the positive correlation between fMRI-based sampling density and SAP-based contrast sensitivity loss (SAP) r ² =0.44, p=0.0002.This correlation was higher for our new approach (MP) compared to that for the conventional pRF analysis. Conclusions fMRI-based reconstruction of the VF enables the evaluation of vision loss and provides useful details on the properties of the visual cortex. Translational Relevance fMRI-based VF reconstruction provides an objective alternative to detect VF defects. It may either complement SAP, or could provide VF information in patients unable to perform SAP.
Marc M. Himmelberg
added a research item
Biomarkers suitable for early diagnosis and monitoring disease progression are the cornerstone of developing disease-modifying treatments for neurodegenerative diseases such as Parkinson’s disease (PD). Besides motor complications, PD is also characterized by deficits in visual processing. Here, we investigate how virally-mediated overexpression of α-synuclein in the substantia nigra pars compacta impacts visual processing in a well established rodent model of PD. After a unilateral injection of vector, human α-synuclein was detected in the striatum and superior colliculus (SC). In parallel, there was a significant delay in the latency of the transient VEPs from the affected side of the SC in late stages of the disease. Inhibition of leucine-rich repeat kinase using PFE360 failed to rescue the VEP delay and instead increased the latency of the VEP waveform. A support vector machine classifier accurately classified rats according to their `disease state’ using frequency-domain data from steady state visual evoked potentials (SSVEP). Overall, these findings indicate that the latency of the rodent VEP is sensitive to changes mediated by the increased expression of α-synuclein and especially when full overexpression is obtained, whereas the SSVEP facilitated detection of α-synuclein across reflects all stages of PD model progression.
Jelle A. van Dijk
added a research item
A fundamental assumption of nearly all functional magnetic resonance imaging (fMRI) analyses is that the relationship between local neuronal activity and the blood oxygenation level dependent (BOLD) signal can be described as following linear systems theory. With the advent of ultra-high field (7T and higher) MRI scanners, it has become possible to perform sub-millimeter resolution fMRI in humans. A novel and promising application of sub-millimeter fMRI is measuring responses across cortical depth, i.e. laminar imaging. However, the cortical vasculature and associated directional blood pooling towards the pial surface strongly influence the cortical depth-dependent BOLD signal, particularly for gradient-echo BOLD. This directional pooling may potentially affect BOLD linearity across cortical depth. Here we assess whether the amplitude scaling assumption for linear systems theory holds across cortical depth. For this, we use stimuli with different luminance contrasts to elicit different BOLD response amplitudes. We find that BOLD amplitude across cortical depth scales with luminance contrast, and that this scaling is identical across cortical depth. Although nonlinearities may be present for different stimulus configurations and acquisition protocols, our results suggest that the amplitude scaling assumption for linear systems theory across cortical depth holds for luminance contrast manipulations in sub-millimeter laminar BOLD fMRI.
Joana Carvalho
added a research item
Unsolved questions in computational visual neuroscience research are whether and how neurons and their connecting cortical networks can adapt when normal vision is compromised by a neurodevelopmental disorder or damage to the visual system. This question on neuroplasticity is particularly relevant in the context of rehabilitation therapies that attempt to overcome limitations or damage, through either perceptual training or retinal and cortical implants. Studies on cortical neuroplasticity have generally made the assumption that neuronal population properties and the resulting visual field maps are stable in healthy observers. Consequently, differences in the estimates of these properties between patients and healthy observers have been taken as a straightforward indication for neuroplasticity. However, recent studies imply that the modeled neuronal properties and the cortical visual maps vary substantially within healthy participants, e.g., in response to specific stimuli or under the influence of cognitive factors such as attention. Although notable advances have been made to improve the reliability of stimulus-driven approaches, the reliance on the visual input remains a challenge for the interpretability of the obtained results. Therefore, we argue that there is an important role in the study of cortical neuroplasticity for approaches that assess intracortical signal processing and circuitry models that can link visual cortex anatomy, function, and dynamics.
Alex Hernandez-Garcia
added a research item
Current computational models of visual salience accurately predict the distribution of fixations on isolated visual stimuli. These predictions relate to the relative distribution of fixations within a single image, that is they form a local salience map. It is not known, however, whether the global salience of a visual stimulus, that is its effectiveness in the competition for attention with other stimuli, is a function of the local salience or an independent measure. Further, do task and familiarity with the images influence eye movements in situations of competing stimuli? Here we investigated the global salience of images and its properties. Participants freely observed pairs of images while eye movements were recorded. In half of the experiment we presented pairs of new images and pairs of already seen images. In the other half, new and familiar images were mixed and participants indicated either new or previously shown images respectively. Then, we trained a logistic regression model to predict the location---left or right image---of the first fixations for each stimulus pair, accounting as well as for the influence of task and familiarity. The model accurately predicted the distribution of first fixations, providing an accurate measure of global salience, as indicated by the coefficients assigned to each image. Importantly, the global salience cannot be explained by the local salience of the images. Further, we reproduced the previously reported left-sided bias for the first fixation. Finally, the influence of task and familiarity was rather small. Summarized, we showed that the global salience of images well predicts human eye behavior in terms of the first fixations, is independent of the local salience and little influenced by task and familiarity of stimuli.
Marc M. Himmelberg
added an update
From 2015-2018, the NextGenVis Research Training program funded fifteen PhD students across Europe to work on important issues related to human vision. The students investigated topics ranging from basic clinical questions about eye and brain disease through to computational models of vision that inform the rapidly expanding field of machine learning.
Here we provide a short summary of at least one major output from each student and describe its potential future relevance to healthcare and industry.
 
Marc M. Himmelberg
added an update
We are very excited to show you the NextGenVis movie! We trained 15 enthusiastic and intelligent young researchers to become the next generation of visual neuroscientists. The result: a great network of amazing scientists that have shown not only to connect on a scientific but also on a social level.
 
Marc M. Himmelberg
added an update
The consortium has produced three of publications across 2018, including:
Binda, P., Kurzawski, J.W., Lunghi, C., Biagi, L., Tosetti, M., & Morrone, M.C. (2018). Response to short-term deprivation of the human adult visual cortex measured with 7T BOLD. eLife, 2018(7). DOI:10.7554/eLife.40014
Himmelberg, M.M. & Wade, A.R. (2019). Eccentricity-dependent temporal contrast tuning measured in human visual cortex with fMRI. Neuroimage, 184(1). DOI: 10.1016/j.neuroimage.2018.09.049
Grillini, A., Omblet, D., Soans, R.S., & Cornelissen, F.W. (2018). Towards using the patio-temporal properties of eye movements to classify visual field deficits, ETRA, 38. DOI: 10.1145/3204493.3204590
de Best, P.B., Raz, N., Dumoulin, S. O., & Levin, N. (2018). How Ocular Dominance and Binocularity Are Reflected by the Population Receptive Field Properties. Investigative ophthalmology & visual science, 59(13), 5301-5311.
 
Alex Hernandez-Garcia
added a research item
Data augmentation is a popular technique largely used to enhance the training of convolutional neural networks. Although many of its benefits are well known by deep learning researchers and practitioners, its implicit regularization effects, as compared to popular explicit regularization techniques, such as weight decay and dropout, remain largely unstudied. As a matter of fact, convolutional neural networks for image object classification are typically trained with both data augmentation and explicit regularization, assuming the benefits of all techniques are complementary. In this paper, we systematically analyze these techniques through ablation studies of different network architectures trained with different amounts of training data. Our results unveil a largely ignored advantage of data augmentation: networks trained with just data augmentation more easily adapt to different architectures and amount of training data, as opposed to weight decay and dropout, which require specific fine-tuning of their hyperparameters.
Marc M. Himmelberg
added an update
We are pleased to announce the NextGenVis Dissemination Conference: Stability and plasticity of human visual cortex, will be held in Pisa on Wednesday 27.2.2019.
The conference is open to all and is free of charge, with lunch provided. Those wishing to attend can register via https://goo.gl/forms/6Sm4eCCutazWhZPH2
Please find the conference programme attached.
Location: Residence Le Benedettine, Lungarno Sidney Sonnino, 18, 56125, Pisa, Italy
Time: Wednesday 27th Feburary, 9:00 - 18:00
Speakers: Mark Greenlee, Peter König, Serge Dumoulin, Michael Hoffman, Rainer Goebel, Erhart Barth, Netta Levin, Kenneth Christensen, Concetta Morrone, Tony Morland, and Frans W. Cornelissen. Posters will be presented by the NGV ESRs.
 
Alex Hernandez-Garcia
added a research item
Modern deep artificial neural networks have achieved impressive results through models with very large capacity---compared to the number of training examples---that control overfitting with the help of different forms of regularization. Regularization can be implicit, as is the case of stochastic gradient descent and parameter sharing in convolutional layers, or explicit. Most common explicit regularization techniques, such as weight decay and dropout, reduce the effective capacity of the model and typically require the use of deeper and wider architectures to compensate for the reduced capacity. Although these techniques have been proven successful in terms of improved generalization, they seem to waste capacity. In contrast, data augmentation techniques do not reduce the effective capacity and improve generalization by increasing the number of training examples. In this paper we systematically analyze the effect of data augmentation on some popular architectures and conclude that data augmentation alone---without any other explicit regularization techniques---can achieve the same performance or higher as regularized models, especially when training with fewer examples, and exhibits much higher adaptability to changes in the architecture.
Marc M. Himmelberg
added an update
The consortium has had a range of publications over the past year, including:
Area Prostriara in the Human Brain (2017). Mikellidou, K., Kurzawski, J.W., Frijia, F., Montanaro, D., Greco, V., Burr, D.C., & Morrone, M.C. Current Biology, 27(9). DOI:1016/j.cub.2017.08.065
Abnormal visual gain control and excitotoxicity in early-onset Parkinson's disease Drosophila models (2017). Himmelberg, M.M., West, R.J.H., Elliott, C.J.H. & Wade, A.R. Journal of Neurophysiology, 119(3). DOI:10.1152/jn.00681.2017
A perspective plus in Parkinson's disease (2018). Himmelberg, M.M., West, R.J.H., Wade, A.R. & Elliott, C.J.H. Movement Disorders, 33(2). DOI: 10.1002/mds.27240
Altered organisation of the visual cortex in FHONDA syndrome. Ahmadi, K., Fracasso, A., van Dijk, J.A., Krujit, C., Dumoulin, S.O. & Hoffmann, M.B. (2018) NeuroImage, In Press, DOI:10.1016/j.neuroimage.2018.02.053
A second-order orientation-contrast stimulus for population receptive field based retinotopic mapping (2017). Yildirim, F., Carvalho, J. & Cornelissen, F.W. NeuroImage, 164. DOI: 10.1016/j.neuroimage.2017.06.073
 
Alex Hernandez-Garcia
added a research item
The impressive success of modern deep neural networks on computer vision tasks has been achieved through models of very large capacity compared to the number of available training examples. This overparameterization is often said to be controlled with the help of different regularization techniques, mainly weight decay and dropout. However, since these techniques reduce the effective capacity of the model, typically even deeper and wider architectures are required to compensate for the reduced capacity. Therefore, there seems to be a waste of capacity in this practice. In this paper we build upon recent research that suggests that explicit regularization may not be as important as widely believed and carry out an ablation study that concludes that weight decay and dropout may not be necessary for object recognition if enough data augmentation is introduced.
Alex Hernandez-Garcia
added a research item
The impressive success of modern deep neural networks on computer vision tasks has been achieved through models of very large capacity compared to the number of available training examples. This overparameterization is often said to be controlled with the help of different regularization techniques, mainly weight decay and dropout. However, since these techniques reduce the effective capacity of the model, typically even deeper and wider architectures are required to compensate for the reduced capacity. Therefore, there seems to be a waste of capacity in this practice. In this paper we build upon recent research that suggests that explicit regularization may not be as important as widely believed and carry out an abla-tion study that concludes that weight decay and dropout may not be necessary for object recognition if enough data augmentation is introduced.
Marc M. Himmelberg
added a project goal
NextGenVis is a European team of early career research fellows collaborating within a Marie Curie Sklodowska Innovative Training Network. The fellows are drawn from health care sectors, universities and private companies and are all trained within the field of visual and computational neuroscience. The NextGenVis network brings together unique expertise and resources in brain imaging, psychology, neurology, ophthalmology, and computer science.
The project is called NextGenVis for a reason: the goal is to take vision research further by Training the Next Generation of Visual Neuroscientists. Training includes both academic research and collaboration with the health care and high-tech industry. The network of fellows are trained in quantitative knowledge on the adaptive properties of the visual brain in health and disease – with a strong focus on the neurocomputational basis – and how to apply this new knowledge to boost innovation in health care and technology.
Partners:
University Medical Centre Groningen (NL)
Pattern Recognition Company (DE)
Otto von Guericke University Magdeburg (DE)
BrainInnovation (NL)
University of York (UK)
White Matter Labs (DE)
Lundbeck (DK)
Maastricht University (NL)
Royal Dutch Visio (NL)
Fondazione Stella Maris (IT)
Phillips (NL)
Utrecht University (NL)
Nano Retina (IL)
Hadassah Medical Centre (IL)
EuroGrant (DE)