ArticlePDF Available

Feed-forward contour integration in primary visual cortex based on asynchronous spike propagation

Authors:
Rufin van Rullen
Rufin van Rullen
Rufin van Rullen
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Arnaud Delorme
Arnaud Delorme
Simon Jonathan Thorpe at Université Toulouse III - Paul Sabatier
Simon Jonathan Thorpe
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

Most current models of visual contour integration involve iterative lateral or feed-back interactions among neurons in V1 and V2. However, some forms of visual processing are too fast for such time-consuming loops. We propose a model avoiding iterative computation by using the fact that real neurons in the retina or LGN "re asynchronously, with the most activated "ring "rst. Thus, early "ring V1 neurons can in#uence processing of their neighbors which are still integrating information from LGN. By limiting the number of spikes to one per neuron, we show that contour integration can be obtained in a purely feed-forward way. 2001 Elsevier Science B.V. All rights reserved.
Figures - uploaded by Arnaud Delorme
Author content
All figure content in this area was uploaded by Arnaud Delorme
Content may be subject to copyright.
Content uploaded by Arnaud Delorme
Author content
All content in this area was uploaded by Arnaud Delorme on Jun 17, 2020
Content may be subject to copyright.
1
Feed-forward contour integration in primary visual cortex
based on asynchronous spike propagation.
Rufin VanRullen*, Arnaud Delorme & Simon J. Thorpe.
Centre de Recherche Cerveau & Cognition, CNRS-UPS, UMR 5549,
Faculté de Médecine de Rangueil, 133 Route de Narbonne, 31062 TOULOUSE Cedex. France.
Most current models of visual contour integration involve iterative
lateral or feedback interactions among neurons in V1 and V2. However,
some forms of visual processing are too fast for such time-consuming
loops. We propose a model avoiding iterative computation by using the
fact that real neurons in the retina or LGN fire asynchronously, with the
most activated firing first. Thus early firing V1 neurons can influence
processing of their neighbors which are still integrating information
from LGN. By limiting the number of spikes to one per neuron, we show
that contour integration can be obtained in a purely feed-forward way.
Introduction
Visual contour integration, a modulation of V1 neuron responses by contextual influences outside
their receptive field, responsible for the selective enhancement of smooth aligned contours (Fig.
1A, 1B), is thought to be mediated by lateral interactions among V1 neurons (Field et al, 1993;
Kapadia et al. 1995; Gilbert et al. 1996) or feed-back inputs from V2 (Von der Heydt et al. 1984).
Current models of contour integration involve iterative, recurrent lateral or feed-back projections
(e.g. Shashua and Ullman, 1988; Hummel and Zucker, 1983; Yen and Finkel, 1997; Li, 1998;
Heitger and Von der Heydt, 1993; Grossberg and Mingolla, 1985; Gove et al. 1995). The
underlying idea behind these models is that visual inputs are computed and sent all at once, in a
single step, to the primary visual cortex neurons (depending on the model, visual inputs can remain
active during the following steps). These V1 neurons then calculate their activity levels, and send
it, all at once, to their neighbours. The last step is then repeated until a satisfactory solution or an
equilibrium state is reached (Fig. 1C).
These time-consuming mechanisms are in contradiction with psychophysical and
electrophysiological studies of rapid visual categorization in man (Thorpe et al. 1996; VanRullen
and Thorpe, 2000) and monkey (Fabre-Thorpe et al, 1998; Vogels, 1999). Behavioral responses in
tasks where subjects have to detect a target (e.g. animal) in a briefly flashed (20 ms) natural image
can be as short as 180 ms in monkey and 250 ms in man, bringing evidence that at least some
forms of visual processing are so fast that it must be performed in a single feed-forward pass
through the visual system, with probably no more than one spike per neuron between two
successive processing stages. This leaves remarkably little time for recurrent loops or feedback to
occur. One possibility is that this sort of ultra-rapid scene categorization could be performed
without involving much contour integration. However, here we show that in fact, contour
integration can occur very rapidly if one makes use of the fact that real neurons fire
asynchronously, with the most activated neurons reaching their threshold first. This is a simple and
well-known property of integrate-and-fire neurons.
* Corresponding author.
Published in Neurocomputing, 38-40(1-4), 1003-1009, 2001.
2
Fig. 1. Contour integration enhances the smooth contour in the retinal image A to
yield to the activity pattern B in orientation columns. Classical models suppose
recurrent interactions among V1 neurons (C). We propose a model based on
asynchronous spike propagation where lateral interactions occur in a "feed-
forward" way (D). Patterns of afferent and lateral connectivity for a V1 neuron
selective to a horizontal orientation are shown.
Results
We design a two-stage model where the response of the first, input layer (which can be compared
to the retina or the LGN) is a wave of asynchronously generated spikes. Neurons in the second
layer, modelling the primary visual cortex V1, are selective to a particular orientation in their
receptive field. In addition, they are interacting locally with one another through a set of lateral
connections, similar to those proposed by Li (1998). The very first neurons to fire in V1 can start
influencing the processing of their neighbors when they are still integrating the incoming
information from LGN (Fig. 1D). This is compatible with electrophysiological recordings in cats
(Volgushev et al. 1995) showing that the delay between the rise of PSPs in a V1 cell and the
emission of its first spikes leaves enough time for lateral feed-forward interactions to occur.
Neighboring neurons forming a smooth contour will receive excitatory lateral input, whereas
neurons in clearly orthogonal directions will be inhibited. This lateral wave of activity modulation
in V1 can spread over large distances before the end of the orientation extraction process, i.e.
before the last spike has arrived from LGN. To make sure that our implementation did not permit
iterations or loops in any way, we limited the number of spikes per neuron to zero or one.
Orientation selectivity can still be achieved by making V1 neurons sensitive to the order in which
they receive afferent spikes (Thorpe and Gautrais, 1997, 1998). Under these conditions, we were
able to show for a variety of examples that contour integration indeed occured in a purely feed-
forward way (Fig. 2). More specifically, we compared the activations of V1 orientation colums
with and without these lateral feed-forward interactions. Activation in the retinotopic orientation
maps decreased at the locations where the visual input had little or no structure, and was recruited
at the locations where the contour was smooth. This was verified with artificial input images of
broken lines, circles, segments, embedded in a field of random oriented bars, as well as with
natural input pictures (Fig. 3).
3
Fig 2. An example of contour integration generated by our model. The input image
contains a smooth but broken contour (vertical line, oval shape) embedded in a field
of random oriented bars. V1 global activities (sum of activities for 8 different
orientation maps) with and without lateral interactions are shown (top). The
difference image (middle) shows the regions where activity is decreased (dark
spots) and the locations to where activity has moved (light spots). Activities for 4
(out of 8) orientation maps (bottom) demonstrate the selectivity of the orientation
columns. These results were obtained with no more than one spike per neuron.
Discussion
The main feature of these results is that they were obtained with no iteration or computational
loop, although using the same pattern of connectivity, hence the same functionality as was
proposed by Li (1998) for a clearly iterative model. The major difference between these two
approaches stems from the concept of information transmission and processing that we used. By
simply taking into account the properties of real neurons, i.e. the fact that integrate-and-fire
neurons will tend to fire asynchronously, with the most activated cells firing first, we were able to
implement a visual contour integration process without any iterations.
4
Fig. 3. Example of contour integration obtained with a natural input image. The temporal
course of contour integration is simulated by assuming a uniform distribution of spikes in
the retina and a 40 ms delay between the retina and V1. Lateral interactions enhance
activity at smooth contours (outline of the face, shoulder, or hat), and decrease activity at
locations without strong contour coherence (feathers on the hat). The effects of contour
integration can be observed in the first milliseconds of processing.
We believe that many other computational mechanisms traditionally thought to involve feed-back,
or recurrent lateral interactions, could be implemented in such an asynchronous feed-forward
information flow framework. For example, we have already proposed (VanRullen and Thorpe,
1999) a model of feed-forward spatial attention based on the temporal precedence of the attended
information under conditions where spikes are propagated asynchronously.
One could argue, on the other hand, that since our model V1 neurons are locally mutually
interconnected, there is indeed some kind of recurrent processing in our model. This can not be
true however, under conditions where a neuron is only allowed to generate one spike. Indeed,
5
when a neuron fires and influences its neighbors, driving some of them above threshold, the
resulting lateral interactions will have virtually no "backwards" influence on this neuron. This
raises the question of how to characterize a computational loop, which is well defined in a
sequential processing framework, but lacks a clear definition in the context of parallel
asynchronous networks. A neural circuit can have anatomical feedback even under conditions
where functionally it operates in a feed-forward mode (Treves et al, 1996). The critical issue is
whether or not the circuit is able to compute the desired function even when each neuron only fires
at most one spike. We suggest that the notions of iteration, loop and feedback should not depend
only on the respective positions of the involved neurons in the visual cortical hierarchy, but rather
on the relative time at which they respond to a visual stimulus. This is supported by recent
electrophysiological studies showing that activity can occur simultaneously accross multiple
hierarchically "successive" visual areas, rather than in a strictly sequential way (Bullier and
Nowak, 1995).
References
1. Bullier, J., & Nowak, L. G. (1995). Parallel versus serial processing: new vistas on the distributed organization of
the visual system. Curr Opin Neurobiol, 5(4), 497-503.
2. Fabre-Thorpe, M., Richard, G., & Thorpe, S. J. (1998). Rapid categorization of natural images by rhesus
monkeys. Neuroreport, 9(2), 303-8.
3. Field, D. J., Hayes, A., & Hess, R. F. (1993). Contour integration by the human visual system: evidence for a local
"association field". Vision Res, 33(2), 173-93.
4. Gilbert, C. D., Das, A., Ito, M., Kapadia, M., & Westheimer, G. (1996). Spatial integration and cortical dynamics.
Proc Natl Acad Sci U S A, 93(2), 615-22.
5. Gove, A., Grossberg, S., & Mingolla, E. (1995). Brightness perception, illusory contours, and corticogeniculate
feedback. Vis Neurosci, 12(6), 1027-52.
6. Grossberg, S., & Mingolla, E. (1985). Neural dynamics of perceptual grouping: textures, boundaries, and emergent
segmentations. Percept Psychophys, 38(2), 141-71.
7. Heitger, F., & von der Heydt, R. (1993). A computational model of neural contour processing: Figure-ground
segregation and illusory contours. Proc. of the 4th Intl. Conf. on Computer Vision, 32-40.
8. Hummel, R., & Zucker, S. W. (1983). On the foundation of relaxation labeling processes. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 5, 267-287.
9. Kapadia, M. K., Ito, M., Gilbert, C. D., & Westheimer, G. (1995). Improvement in visual sensitivity by changes in
local context: parallel studies in human observers and in V1 of alert monkeys. Neuron, 15(4), 843-56.
10. Li, Z. (1998). A neural model of contour integration in the primary visual cortex. Neural Comput, 10(4), 903-40.
11. Shashua, A., & Ullman, S. (1988). Structural saliency. Proceedings of the International Conference on Computer
Vision, 482-488.
12. Thorpe, S. J., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520-522.
13. Thorpe, S. J., & Gautrais, J. (1997). Rapid visual processing using spike asynchrony. In M. C. Mozer, M. Jordan, &
T. Petsche (Eds.), Advances in Neural Information Processing Systems (MIT Press ed., Vol. 9, pp. 901-907).
Cambridge: MIT Press.
14. Thorpe, S. J., & Gautrais, J. (1998). Rank order coding: a new coding scheme for rapid processing in neural
networks. In J. Bower (Ed.), Computational Neuroscience : Trends in Research. New York: Plenum Press.
15. Treves, A., Rolls, E. T., & Tovee, M. J. (1996). On the time required for recurrent processing in the brain. In V.
Torre & F. Conti (Eds.), Neurobiology: Proceedings of the International School of Biophysics, XXIII course, May
1995 (pp. 371-382). New York: Plenum Press.
16. VanRullen, R., & Thorpe, S. (1999). Spatial attention in asynchronous neural networks. NeuroComputing, 26-27,
911-918.
17. VanRullen, R., & Thorpe, S. (2000). Is it a bird? Is it a plane? Ultra-rapid visual categorisation of natural and
artifactual objects. Perception, submitted.
18. Vogels, R. (1999). Categorization of complex visual images by rhesus monkeys. Part 1: behavioural study. Eur J
Neurosci, 11(4), 1223-38.
19. Volgushev, M., Vidyasagar, T. R., & Pei, X. (1995). Dynamics of the orientation tuning of postsynaptic potentials
in the cat visual cortex. Vis Neurosci, 12(4), 621-8.
20. von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses.
Science, 224(4654), 1260-2.
21. Yen, S.-C., & Finkel, L. H. (1997). Salient contour extraction by temporal binding in a cortically-based network. In
M. C. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in Neural Information Processing Systems (MIT Press ed.,
Vol. 9, ). Cambridge: MIT Press.
... One such tech tour integration [156], is employed with edge extraction from the primary v (V1). Contour integration was first observed psycho-physically as the popping terns of small line segments that followed smooth trajectories in the presence o [157], as shown in Figure 28. ...
... One such technique, contour integra-tion [156], is employed with edge extraction from the primary visual cortex (V1). Contour integration was first observed psycho-physically as the popping out of patterns of small line segments that followed smooth trajectories in the presence of distractors [157], as shown in Figure 28. models and the easy control of perturbations. ...
... The first mechanism involves nonuniform spatial sampling of visual stimuli by the retina's photoreceptors, mimicking the exponential decrease in cones' density in the eccentric direction, as taken from [156]. The second mechanism is the variation in RFs with eccentricity to avoid aliasing [157,158,162,163] in V1 neurons. To realize the area of scale according to each eccentricity of V1 neurons, the authors constructed fovea receptacles with five frequency bands and estimated a set of scale-space bands in V1. ...
Robustness of Deep Learning Models for Vision Tasks
Article
Full-text available
  • Mar 2023
In recent years, artificial intelligence technologies in vision tasks have gradually begun to be applied to the physical world, proving they are vulnerable to adversarial attacks. Thus, the importance of improving robustness against adversarial attacks has emerged as an urgent issue in vision tasks. This article aims to provide a historical summary of the evolution of adversarial attacks and defense methods on CNN-based models and also introduces studies focusing on brain-inspired models that mimic the visual cortex, which is resistant to adversarial attacks. As the origination of CNN models was in the application of physiological findings related to the visual cortex of the time, new physiological studies related to the visual cortex provide an opportunity to create more robust models against adversarial attacks. The authors hope this review will promote interest and progress in artificially intelligent security by improving the robustness of deep learning models for vision tasks.
View
... [29][30][31] These findings suggest that local interactions between neighboring elements underlie contour integration. Similar to collinear facilitation, contour integration is thought to involve excitatory horizontal connections between cells with similar orientation preferences within the primary visual cortex (V1), 28,[32][33][34][35][36] supported by neurophysiological evidence. [37][38][39][40][41][42][43] Neuroimaging studies have further implicated both striate and extrastriate cortices, including V2, V4v, and the lateral occipital complex, contributing to contour integration. ...
Temporal Contour Integration Deficits in Children With Amblyopia
Article
Full-text available
  • Apr 2025
  • INVEST OPHTH VIS SCI
PURPOSE. Contour integration, the process of combining local visual fragments into coherent paths or shapes, is essential for visual perception. Although prior research on ambly-opia has focused primarily on spatial domain deficits in contour integration, this study investigates how amblyopia affects contour integration over time and examines the relationship between temporal contour integration deficits and visual functions. METHODS. Nineteen amblyopic children (mean age, 10.9 ± 2.4 years; 17 anisometropic, 2 anisometropic/strabismic mixed) and 26 visually normal children (mean age, 10.5 ± 1.8 years) participated in this study. Temporal contour integration was assessed by measuring the accuracy of detecting tilted contour paths, formed by collinear Gabor elements with similar orientations, under slit-viewing conditions. Performance was evaluated for amblyopic eyes (AEs) and fellow eyes (FEs) at two spatial frequencies (1.5 cpd and 3 cpd). The slit width, orientation jitter of contour elements, and stimulus movement speed were systematically varied across separate runs. Visual acuity and Randot stereoacuity were assessed before testing. RESULTS. AEs exhibited significant deficits in temporal contour processing compared with FEs. Specifically, AEs required larger slit widths to achieve performance levels comparable to FEs, with more severe amblyopia (i.e., worse AE visual acuity) necessitating even larger slit widths for temporal contour integration. Temporal contour integration deficits in AEs were most pronounced under conditions of complete Gabor collinearity or moderate stimulus movement speeds (6.4°/s). No significant differences were observed between FEs and control eyes. Notably, the temporal contour integration ability between the two eyes quantified as the AE/FE ratio of slit width thresholds showed no correlation with interocular acuity differences, stereoacuity, or spatial contour integration deficits. CONCLUSIONS. Amblyopic children demonstrate significant deficits in temporal contour integration in AEs, which seem to be independent of spatial contour integration deficits. The severity of these temporal deficits increases with worse AE visual acuity. These findings suggest that amblyopia is associated with temporal deficits in visual integration, in addition to the well-documented spatial deficits, highlighting the need for a more comprehensive understanding of amblyopic visual processing.
View
... A further, more exotic approach to achieve synchronous cell assemblies is to reconfigure the network structure for each new stimulus pattern [36, 38? ], though this is biologically motivated in only one case [38] to a certain degree. Finally, it is interesting to note that VanRullen et al. [41] criticized the whole category of models cited here as implausibly slow for the purpose of stimulus grouping and proposed a fast, feed-forward, spike-based grouping model, without regard to synchronisation. Here, in an attempt to find a minimal model that has the required computational properties, we provide conditions under which fixed, purely excitatory, chemically coupled spiking networks alone exhibit the required, fast stimulus-dependent synchrony response. ...
Cortical Spike Synchrony as a Measure of Input Familiarity
Preprint
  • Sep 2017
Spike synchrony, which occurs in various cortical areas in response to specific perception, action and memory tasks, has sparked a long-standing debate on the nature of temporal organization in cortex. One prominent view is that this type of synchrony facilitates the binding or grouping of separate stimulus components. We argue instead for a more general function: A measure of the prior probability of incoming stimuli, implemented by long-range, horizontal, intra-cortical connections. We show that networks of this kind -- pulse-coupled excitatory spiking networks in a noisy environment -- can provide a sufficient substrate for stimulus-dependent spike synchrony. This allows a quick (few spikes) estimate of the match between inputs and the input history as encoded in the network structure. Given the ubiquity of small, strongly excitatory subnetworks in cortex, we thus propose that many experimental observations of spike synchrony can be viewed as signs of input patterns that resemble long-term experience, i.e. patterns of high prior probability.
View
... Similarly, a model of binocular rivalry for ring displays, which is essentially a perceptual coupling display composed of multiple adjacent binocular rivalry stimuli, allowed for traveling waves of dominance using just a two-layer model with only local connectivity (Lee et al., 2005). These mechanisms could be like those that facilitate rapid contour integration in the primary visual cortex (Pack & Born, 2001;VanRullen et al., 2001). In the end, although we lean towards the selective attention hypothesis, it is up to future research to resolve this question. ...
Change not State: Perceptual coupling in multistable displays reflects transient bias induced by perceptual change
Article
Full-text available
  • Aug 2021
We investigated how changes in dynamic spatial context influence visual perception. Specifically, we reexamined the perceptual coupling phenomenon when two multistable displays viewed simultaneously tend to be in the same dominant state and switch in accord. Current models assume this interaction reflecting mutual bias produced by a dominant perceptual state. In contrast, we demonstrate that influence of spatial context is strongest when perception changes . First, we replicated earlier work using bistable kinetic-depth effect displays, then extended it by employing asynchronous presentation to show that perceptual coupling cannot be accounted for by the static context provided by perceptually dominant states . Next, we demonstrated that perceptual coupling reflects transient bias induced by perceptual change , both in ambiguous and disambiguated displays. We used a hierarchical Bayesian model to characterize its timing, demonstrating that the transient bias is induced 50–70 ms after the exogenous trigger event and decays within ~200–300 ms. Both endogenous and exogenous switches led to quantitatively and qualitatively similar perceptual consequences, activating similar perceptual reevaluation mechanisms within a spatial surround. We explain how they can be understood within a transient selective visual attention framework or using local lateral connections within sensory representations. We suggest that observed perceptual effects reflect general mechanisms of perceptual inference for dynamic visual scene perception.
View
... Similarly, a model of binocular rivalry for ring displays, which is essentially a perceptual coupling display composed of multiple adjacent binocular rivalry stimuli, allowed for traveling waves of dominance using just a two-layer model with only local connectivity (Lee et al., 2005). These mechanisms could be like those that facilitate rapid contour integration in the primary visual cortex (Pack & Born, 2001;VanRullen et al., 2001). In the end, although we lean towards the selective attention hypothesis, it is up to future research to resolve this question. ...
View
... This finding is in agreement with physiological studies showing that top-down influences from higher visual, or other cortical areas strongly support contour integration (Li et al., 2008;Mijovic et al., 2014). Initially, we did not expect to find such a large difference in the Long and Peak conditions as contour integration is typically described as a robust process occurring mainly in early visual areas (Li, 1998;VanRullen et al., 2001;Hess et al., 2003), and as such it should be predominantly driven by the current stimulus and less so by the current behavioral state. From our study we can not conclude whether attention or other top down processes are independent of contour integration, simply acting as an amplifier on top of a contour integration process performed in an early visual cortical area, or if these processes interact with contour integration, actively boosting the linking process of visual features. ...
Contour Integration in Dynamic Scenes: Impaired Detection Performance in Extended Presentations
Article
Full-text available
  • Sep 2017
Since scenes in nature are highly dynamic, perception requires an on-going and robust integration of local information into global representations. In vision, contour integration (CI) is one of these tasks, and it is performed by our brain in a seemingly effortless manner. Following the rule of good continuation, oriented line segments are linked into contour percepts, thus supporting important visual computations such as the detection of object boundaries. This process has been studied almost exclusively using static stimuli, raising the question of whether the observed robustness and “pop-out” quality of CI carries over to dynamic scenes. We investigate contour detection in dynamic stimuli where targets appear at random times by Gabor elements aligning themselves to form contours. In briefly presented displays (230 ms), a situation comparable to classical paradigms in CI, performance is about 87%. Surprisingly, we find that detection performance decreases to 67% in extended presentations (about 1.9–3.8 s) for the same target stimuli. In order to observe the same reduction with briefly presented stimuli, presentation time has to be drastically decreased to intervals as short as 50 ms. Cueing a specific contour position or shape helps in partially compensating this deterioration, and only in extended presentations combining a location and a shape cue was more efficient than providing a single cue. Our findings challenge the notion of CI as a mainly stimulus-driven process leading to pop-out percepts, indicating that top-down processes play a much larger role in supporting fundamental integration processes in dynamic scenes than previously thought.
View
... target vs non-target; VanRullen &Thorpe, 2001b). It seems likely that the mechanisms involved in ultra-rapid categorization are purely feedforward and encapsulated (Fabre-VanRullen, Delorme & Thorpe, 2001;VanRullen & Koch, 2003): the performance of subjects in such tasks does not increase with training even for a period as extensive as 14 days (Fabre-. Furthermore, a purely feedforward biologicallyinspired neural network was shown to be sufficient to duplicate ultra-rapid categorization ...
The Time Course of Object, Scene, and Face Categorization
Chapter
Full-text available
  • Jun 2017
We first describe Strategy Length & Internal Practicability (SLIP), a formal model for thinking about categorization, in particular about the time course of categorization. We then discuss an early application of this model to basic-levelness. We then turn to aspects of the time course of categorization that have been neglected in the categorization literature: our limited processing capacities; the necessity of having a flexible categorization apparatus; and the paradox that this inexorably brings about. We propose a twofold resolution of this paradox, attempting, in the process, to bridge work done on categorization in vision, neuropsychology, and physiology.
View
The Cerebral Cortex: A Delay-Coupled Recurrent Oscillator Network?
Chapter
  • Aug 2021
The refinement of machine learning strategies and deep convolutional networks led to the development of artificial systems whose functions resemble those of natural brains, suggesting that the two systems share the same computational principles. In this chapter, evidence is reviewed which indicates that the computational operations of natural systems differ in some important aspects from those implemented in artificial systems. Natural processing architectures are characterized by recurrence and therefore exhibit high-dimensional, non-linear dynamics. Moreover, they use learning mechanisms that support self-organization. It is proposed that these properties allow for computations that are notoriously difficult to realize in artificial systems. Experimental evidence on the organization and function of the cerebral cortex is reviewed that supports this proposal.
View
The cognitive neural mechanism of contour processing
Article
  • Feb 2019
  • Acta Physiol Sin
The core of visual processing is the identification and recognition of the objects relevant to cognitive behaviors. In natural environment, visual input is often comprised of highly complex 3-dimensional signals involving multiple visual objects. One critical determinant of object recognition is visual contour. Despite substantial insights on visual contour processing gained from previous findings, these studies have focused on limited aspects or particular stages of contour processing. So far, a systematic perspective of contour processing that comprehensively incorporates previous evidence is still missing. We therefore propose an integrated framework of the cognitive and neural mechanisms of contour processing, which involves three mutually interacting cognitive stages: contour detection, border ownership assignment and contour integration. For each stage, we provide an elaborated discussion of the neural properties, processing mechanism, and its functional interaction with the other stages by summarizing the relevant electrophysiological and human cognitive neuroscience evidence. Finally, we present the major challenges for further unraveling the mechanisms of visual contour processing.
View
Neuronal oscillations: Unavoidable and useful?
Article
  • Dec 2017
Neuronal systems have a high propensity to engage in oscillatory activity because both the properties of individual neurons as well as canonical circuit motifs favour rhythmic activity. In addition, coupled oscillators can engage in a large variety of dynamical regimes, ranging from synchronization with different phase offsets to chaotic behaviour. Which regime prevails depends on differences between preferred oscillation frequencies, coupling strength and coupling delays. The ability of delay coupled oscillator networks to generate a rich repertoire of temporally structured activation sequences is exploited by central pattern generator networks for the control of movements. However, it less clear whether temporal patterning of neuronal discharges also plays a role in cognitive processes. Here it will be argued that the temporal patterning of neuronal discharges emerging from delay coupled oscillator networks plays a pivotal role in all instances in which selective relations have to be established between the responses of distributed assemblies of neurons. Examples are the dynamic formation of functional networks, the selective routing of activity in densely interconnected networks, the attention dependent selection of sensory signals, the fast and context dependent binding of responses for further joint processing in pattern recognition and the formation of associations by learning. Special consideration is given to arguments that challenge a functional role of oscillations and synchrony in cognition because of the volatile nature of these phenomena and recent evidence will be reviewed suggesting that this volatility is functionally advantageous.
View
Rapid Visual Processing using Spike Asynchrony.
Conference Paper
Full-text available
  • Jun 1997
  • Adv Neural Inform Process Syst
We have investigated the possibility that rapid processing in the visual system could be achieved by using the order of firing in different neurones as a code, rather than more conventional firing rate schemes. Using SPIKENET, a neural net simulator based on integrate-and-fire neurones and in which neurones in the input layer function as analog- to-delay converters, we have modeled the initial stages of visual processing. Initial results are extremely promising. Even with activity in retinal output cells limited to one spike per neuron per image (effectively ruling out any form of rate coding), sophisticated processing based on asynchronous activation was nonetheless possible.
View
A Neural Model of Contour Integration in the Primary Visual Cortex
Article
Full-text available
Experimental observations suggest that contour integration may take place in V1. However, there has yet to be a model of contour integration that uses only known V1 elements, operations, and connection patterns.This article introduces such a model, using orient ation selective cells, local cortical circuits, and horizontal intracortical connections. The model is composed of recurrently connected excitatory neurons and inhibitory interneurons, receiving visual input via oriented receptive fields resembling those found in primary visual cortex. Intracortical interactions modify initial activity patterns from input, selectively amplifying the activities of edges that form smooth contours in the image. The neural activities produced by such interactions are oscillatory and edge segments within a contour oscillate in synchrony. It is shown analytically and empirically that the extent of contour enhancement and neural synchrony increases with the smoothness, length, and closure of contours, as observed in experiments on some of these phenomena. In addition, the model incorporates a feedback mechanism that allows higher visual centers selectively to enhance or suppress sensitivities to given contours, effectively segmenting one from another. The model makes the testable prediction that the horizontal cortical connections are more likely to target excitatory (or inhibitory) cells when the two linked cells have their preferred orientation aligned with (or orthogonal to) their relative receptive field center displacements.
View
View
On the Time Required for Recurrent Processing in the Brain
Article
  • Jan 1995
Can one tell which connections are involved in processing information of a given sort in the cortex, by looking at the temporal course of the responses of individual neurons? Thorpe and Imbert32 have argued that the speed with which neurons in the early visual system appear to be able to produce selective responses, e.g. when tested with different orientations of a stimulus, indicates that the processing is of a feedforward nature. They have recorded from orientation selective neurons in the primary visual cortex of the monkey, and orientation tuning curves appear very similar if computed from the firing rate over a 300 ms period, or only over a 50 ms period after response onset. These results have been interpreted as ruling out the involvement of feedback loops, i.e. that a given neuron may alter its response over time, due to the firing of other cells which it itself influences (whether directly or polysynaptically). Examples of some classes of network in which feedback effects may be important include networks with recurrent excitatory connections, such as are found between cortical pyramidal cells, and networks with recurrent lateral inhibition (see Fig. 1). Very similar interpretations to those of Thorpe and Imbert have been drawn by Oram and Perrett,24 from an analysis of the responses of “face” cells in the temporal lobe cortex.
View
Neural Dynamics of Perceptual Grouping: Textures, Boundaries, And Emergent Segmentations
Article
  • Mar 1985
A real-time visual processing theory is used to analyze and explain a wide variety of perceptual grouping and segmentation phenomena, including the grouping of textured images, randomly defined images, and images built up from periodic scenic elements. The theory explains how “local” feature processing and “emergent” features work together to segment a scene, how segmentations may arise across image regions that do not contain any luminance differences, how segmentations may override local image properties in favor of global statistical factors, and why segmentations that powerfully influence object recognition may be barely visible or totally invisible. Network interactions within a Boundary Contour (BC) System, a Feature Contour (FC) System, and an Object Recognition (OR) System are used to explain these phenomena. The BC System is defined by a hierarchy of orientationally tuned interactions, which can be divided into two successive subsystems called the OC filter and the CC loop. The OC filter contains two successive stages of oriented receptive fields which are sensitive to different properties of image contrasts. The OC filter generates inputs to the CC loop, which contains successive stages of spatially short-range competitive interactions and spatially long-range cooperative interactions. Feedback between the competitive and cooperative stages synthesizes a global context-sensitive segmentation from among the many possible groupings of local featural elements. The properties of the BC System provide a unified explanation of several ostensibly different Gestalt rules. The BC System also suggests explanations and predictions concerning the architecture of the striate and prestriate visual cortices. The BC System embodies new ideas concerning the founda-tions of geometry, on-line statistical decision theory, and the resolution of uncertainty in quan-tum measurement systems. Computer simulations establish the formal competence of the BC System as a perceptual grouping system. The properties of the BC System are compared with probabilistic and artificial intelligence models of segmentation. The total network suggests a new approach to the design of computer vision systems, and promises to provide a universal set of rules for perceptual grouping of scenic edges, textures, and smoothly shaded regions.
View
On the Foundations of Relaxation Labeling Processes
Article
  • Jun 1983
A large class of problems can be formulated in terms of the assignment of labels to objects. Frequently, processes are needed which reduce ambiguity and noise, and select the best label among several possible choices. Relaxation labeling processes are just such a class of algorithms. They are based on the parallel use of local constraints between labels. This paper develops a theory to characterize the goal of relaxation labeling. The theory is founded on a definition of con-sistency in labelings, extending the notion of constraint satisfaction. In certain restricted circumstances, an explicit functional exists that can be maximized to guide the search for consistent labelings. This functional is used to derive a new relaxation labeling operator. When the restrictions are not satisfied, the theory relies on variational cal-culus. It is shown that the problem of finding consistent labelings is equivalent to solving a variational inequality. A procedure nearly identical to the relaxation operator derived under restricted circum-stances serves in the more general setting. Further, a local convergence result is established for this operator. The standard relaxation labeling formulas are shown to approximate our new operator, which leads us to conjecture that successful applications of the standard methods are explainable by the theory developed here. Observations about con-vergence and generalizations to higher order compatibility relations are described.
View
Spatial attention in asynchronous neural networks
Article
  • Jun 1999
  • NEUROCOMPUTING
We propose a simple mechanism for spatial visual attention that involves selectively lowering the thresholds of neurons with receptive "elds in the attended region. Whereas such a mecha- nism is of no use in classical arti"cial neural networks, where all activities for each position in the visual "eld are computed simultaneously, it can be of great interest in an asynchronous neural network, where the relative order of "ring in a population of neurons constitutes the code. Since neurons in the attended region will tend to reach threshold and "re earlier, they will tend to dominate later stages of processing. We illustrate this hypothesis with simulations based on SpikeNET. ( 1999 Elsevier Science B.V. All rights reserved.
View
Rapid categorization of natural images by rhesus monkeys
Article
  • Jan 1998
  • NEUROREPORT
Two rhesus macaques were tested on a categorization task in which they had to classify previously unseen photographs flashed for only 80 ms. One monkey was trained to respond to the presence of an animal, the second to the presence of food. Although the monkeys were not quite as accurate as humans tested on the same material, they nevertheless performed this very challenging visual task remarkably well. Furthermore, their reaction times were considerably shorter than even the fastest human subject. Such data, combined with the detailed knowledge of the monkey's visual system, provide a severe challenge to current theories of visual processing. They also argue that this form of rapid visual categorization is fundamentally similar in both monkeys and humans.
View
Illusory Contours and Cortical Neuron Responses
Article
  • Jul 1984
Figures in which human observers perceive "illusory contours" were found to evoke responses in cells of area 18 in the visual cortex of alert monkeys. The cells responded as if the contours were formed by real lines or edges. Modifications that weakened the perception of contours also reduced the neuronal responses. In contrast, cells in area 17 were apparently unable to "see" these contours.
View
Parallel versus serial processing: new vistas on the distributed organization of the visual system
Article
  • Sep 1995
  • CURR OPIN NEUROBIOL
Recent functional studies question the validity of the hierarchical model of organization for processing visual information in cortical areas. The results of these studies suggest that beyond the primary visual cortex (V1), information is not serially processed through successive cortical areas, but that it is simultaneously processed in several areas. The idea that visual information is functionally segregated into different, parallel channels as it circulates through V1 and V2 towards V4 and the middle temporal visual area is also challenged by recent studies that report a smaller degree of functional specialization within the visual areas than previously thought.
View