Christoph von der Malsburg

Frankfurt Institute for Advanced Studies | FIAS

To this day there is no satisfactory answer to the question how mental patterns correspond to physical states of our brain. For more than six decades, progress has been held up by the logjam between two traditions, one inspired by neuroscience, the other by digital computing. This logjam is well illuminated by Fodor and Pylyshyn’s article of 1988,...

A programmatically described solution to the segmentation problem is taken as opportunity to dicuss the neural architecture problem of vision. At the center of this problem is the formation of holistic entities (the Gestalt phenomenon) out of masses of neurons (the binding problem). As formulated in the Dynamic Net Architecture (DNA), neurons can b...

We present a novel intelligent-system architecture called "Dynamic Net Architecture" (DNA) that relies on recurrence-stabilized networks and discuss it in application to vision. Our architecture models a (cerebral cortical) area wherein elementary feature neurons encode details of visual structures, and coherent nets of such neurons model holistic...

To this day there is no satisfactory answer to the question how mental patterns correspond to physical states of our brain. For more than six decades, progress has been held up by the logjam between two traditions, one inspired by neuroscience, the other by digital computing. This logjam is well illuminated by Fodor and Pylyshyn's article of 1988,...

The neural basis of cognition is unclear to this day. We here present a conceptual framework resolving the conflict Fodor & Pylyshyn (1988); Dever (2006) between symbolic and neural approaches. In our scheme, the cortical carriers of meaning are not individual neurons but sets of neurons supporting each other by mutual excitation. These sets and th...

Introduction: In contrast to current AI technology, natural intelligence -- the kind of autonomous intelligence that is realized in the brains of animals and humans to attain in their natural environment goals defined by a repertoire of innate behavioral schemata -- is far superior in terms of learning speed, generalization capabilities, autonomy a...

More and more, the neurosciences and the sciences concerned with mind and cognition are burying fundamental questions under layers of professional methodology. I therefore welcome Biological Cybernetics’ invitation to comment on two of my papers, (von der Malsburg 1973) and (von der Malsburg and Schneider 1986) (henceforth referred to as (I) and (I...

Um menschliche Kognition als emergentes Phänomen verstehen zu können, müssen zwei Fragen beantwortet werden: Was ist das gemeinsame Datenformat aller kognitiven Strukturen und durch welchen Mechanismus wird dieses Datenformat geformt? Für die Beantwortung dieser Fragen dürfte es nützlich sein, sich von Wissen über Strukturen und Vorgänge im Gehirn,...

On September 2, 1666, a fire broke out in a bakery in Pudding Lane in London, and within three days the City of London inside the old Roman City burned down. There had been fire prevention laws, but they had not been observed properly, there had been fire pumps, but they couldn’t reach the fire in time, there had been plans of action, but they were...

The central problem with understanding brain and mind is the neural code issue: understanding the matter of our brain as basis for the phenomena of our mind. The richness with which our mind represents our environment, the parsimony of genetic data, the tremendous efficiency with which the brain learns from scant sensory input and the creativity wi...

The central problem with understanding brain and mind is the neural code issue: understanding the matter of our brain as basis for the phenomena of our mind. The richness with which our mind represents our environment, the parsimony of genetic data, the tremendous efficiency with which the brain learns from scant sensory input and the creativity wi...

A simple model of MNIST handwritten digit recognition is presented here. The model is an adaptation of a previous theory of face recognition. It realizes translation and rotation invariance in a principled way instead of being based on extensive learning from large masses of sample data. The presented recognition rates fall short of other publicati...

A simple model of MNIST handwritten digit recognition is presented here. The model is an adaptation of a previous theory of face recognition. It realizes translation and rotation invariance in a principled way instead of being based on extensive learning from large masses of sample data. The presented recognition rates fall short of other publicati...

It is widely accepted that after the first cortical visual area, V1, a series of stages achieves a representation of complex shapes, such as faces and objects, so that they can be understood and recognized. A major challenge for the study of complex shape perception has been the lack of a principled basis for scaling of the physical differences bet...

Assuming that patterns in memory are represented as two-dimensional arrays of local features, just as they are in primary visual cortices, pattern recognition can take the form of elastic graph matching (Lades et al., 1993). Neural implementation of this may be based on preorganized fiber projections that can be activated rapidly with the help of c...

We are offering a particular interpretation (well within the range of experimentally and theoretically accepted notions) of neural connectivity and dynamics and discuss it as the data-and-process architecture of the visual system. In this interpretation the permanent connectivity of cortex is an overlay of well-structured networks, nets, which are...

This discussion is an attempt to reconcile our ideas of physical time with those of psychological time. Based on accepted arguments from relativity and on a much less accepted interpretation of quantum phenomena I am adopting a picture of physical time which accords equal and full reality status to all moments in time. This seems to be in sharp con...

We here focus on constructing a hierarchical neural system for position-invariant recognition, which is one of the most fundamental invariant recognition achieved in visual processing [1,2]. The invariant recognition have been hypothesized to be done by matching a sensory image of a particular object stimulated on the retina to the most suitable re...

Shape representation is accomplished by a series of cortical stages in which cells in the first stage (V1) have local receptive fields tuned to contrast at a particular scale and orientation, each well modeled as a Gabor filter. In succeeding stages, the representation becomes largely invariant to Gabor coding (Kobatake & Tanaka, 1994). Because of...

We present a model for the emergence of ordered fiber projections that may serve as a basis for invariant recognition. After invariance transformations are self-organized, so-called control units competitively activate fiber projections for different transformation parameters. The model builds on a well-known ontogenetic mechanism, activity-based d...

We introduce visual object detection architecture, making full use of technical merits of so-called multi-scale feature correspondence in the neurally inspired Gabor pyramid. The remarkable property of the multi-scale Gabor feature correspondence is found with scale-space approaches, which an original image Gabor-filtered with the individual freque...

In the visual cortex, memory traces for complex objects are embedded into a scaffold of feed-forward and recurrent connectivity of the hierarchically organized visual pathway. Strong evidence suggests that consolidation of the memory traces in such a memory network depends on an off-line reprocessing done in the sleep state or during restful waking...

We address the problem of integrating information about multiple objects and their positions on a visual scene. A primate visual system has fewer difficulties in rapidly achieving integration, given even when presented with several objects. Here, we propose a neurally plausible mechanism for simultaneously coordinating the local decision-making pro...

We here propose a simple but highly potential algorithm to detect a model object’s position on an input image by determining the initially unknown transformational states of the model object, in particular, size and 2D-rotation. In this algorithm, a single feature is extracted around or at the center of the input image through 2D-Gabor wavelet tran...

Object recognition invariant to translation or scale is effortless for humans, but computationally very difficult. Many algorithms have been developed, most of which do not work well in real environments, lacking the ability to handle situations not programmed explicitly. In contrast to this algorithmic schema, a better way to understanding brain a...

One way to handle the perception of images that change in position (or size, orientation or deformation) is to invoke rapidly changing fiber projections to project images into a fixed format in a higher cortical area. We propose here a model for the ontogenesis of the necessary control structures. For simplicity we limit ourselves to fiber projecti...

Visual recognition faces the difficult problem of recognizing objects despite the multitude of their appearances. Ample neuroscientific evidence shows that the cortex uses a topographic code to represent visual stimuli. We therefore develop a bilinear probabilistic model that learns transformations to build an invariant topographic code in an unsup...

Recently, experience-driven unsupervised learning was shown to create combinatorial parts-based representations in a model of hierarchical visual memory. Examining the memory's ability to recognize persons from a database of natural face images, we show that an off-line, sleep-like operating regime of the memory domain results in a significant impr...

This chapter reviews the concept of dynamic coordination, its mechanistic implementation in brain circuits, and the extent to which dynamic coordination, and specific manifestations of it, have the power to account for functions performed by interacting brain systems. In our discussions, we addressed how on-the- y changes in coupling between neural...

This chapter focuses on dynamic coordination in the brain and mind and its major implications for the cognitive neurosciences. Coordinating interactions give rise to coherent and relevant patterns of activity without disrupting the essential individual identities and functions of the activities being coordinated. Dynamic coordination deals effectiv...

An examination of how widely distributed and specialized activities of the brain are flexibly and effectively coordinated. A fundamental shift is occurring in neuroscience and related disciplines. In the past, researchers focused on functional specialization of the brain, discovering complex processing strategies based on convergence and divergence...

An examination of how widely distributed and specialized activities of the brain are flexibly and effectively coordinated. A fundamental shift is occurring in neuroscience and related disciplines. In the past, researchers focused on functional specialization of the brain, discovering complex processing strategies based on convergence and divergence...

This book presents a first generation of artificial brains, using vision as sample application. An object recognition system is built, using neurons and synapses as exclusive building elements. The system contains a feature pyramid with 8 orientations and 5 resolution levels for 1000 objects and networks for binding of features into objects. This v...

The preceding chapters presented fundamentals of pulsed neural networks for image processing. The outcome was a modular architecture containing dedicated layers for each subtask. Finally, it was shown how dedicated VLSI circuits supporting these subtasks can be implemented. The integration of particular components (CMOS imager, feature cascade, obj...

The membrane potential ai of the IAF model (section 11.3) is realised as a voltage produced by the currents accumulating on the capacitor formed by the membrane of a neuron. A threshold switch with hysteresis determines the state variable with two values xi ∈ {0,1} of neuron i by comparing the accumulator potential with a reference voltage. At the...

The experimental observation of the presence and duration of partitions and pulse patterns, respectively, is one alternative to achieve the characterisation of a stationary frequency of distribution. For larger networks, however, the simulation time exceeds our human patience, hence a theoretic coverage is necessary. As the constant synapses form a...

As we had seen from the histograms for the ISI between two neurons (Figures 2.9 - 2.10, chapter 2) a net with constant synapses is unable to remain active in a particular subset of partitions only. Instead the net is carried by all partitions, this being the reason for the absence of patterns when operating in the cold range. Furthermore, it can re...

In order to generalise about the available head-specific results for a representative number of heads, two ways seem to be promising: the one determining a minimal number of zones valid for all heads, or the other individually adapting the distances between zones for all heads. The first way is attractive, as the architecture already exists and doe...

We now want to determine the number of different modules which are required to build an artificial vision system which achieves the resolution of the retina, possesses 8 orientations per resolution plane and 5 resolution planes and recognises distant-invariantly up to 1000 objects each featuring 10 zones. The neural architecture of this vision syst...

By far the most successful version of information technology is exhibited by the brain. Animal and human nervous systems are flexible in dealing with the unexpected. They adapt, learn, evolve and integrate well into social networks, have an innate tendency to organise, are energy-efficient and massively parallel, and above all they act in total aut...

In order to find suitable microelectronic representations of neural systems that provide the necessary functionality at the lowest possible costs, the behaviour of the microelectronic system must be simulated. In the analysis of microelectronic systems of even low complexity, however, it is often the case that the behaviour of the system cannot be...

For constant synapses it is sufficient to consider the partial differential equation $${{\partial J_{\varepsilon i}(M_{\varepsilon i}, i_{\varepsilon i}(t))}\over{\partial t}} + h_{\varepsilon i} (M_{\varepsilon i}, W_{\varepsilon i}, i_{\varepsilon i}(t))= 0, (4.1)$$ with the signal energy given by $$ h_{\varepsilon i} (W_{\varepsilon i}, i_{\vare...

The simulation and calculation of the activity diagrams in Figures 8.2 a-n required about 10 days and nights. A statistical testing or significant augmentation of the architecture by means of simulation on a PC is therefore not a passable route in the long run. An essential advancement is offered by these two paths: 1. the creation of equivalent re...

Let us consider an image with 1024 ×768 pixels and let it be covered by a grid of 69 ×85 Gabor wavelet detectors, such that the pulse rate of N7a and N7b, resp., is obtained at each node of the grid. Let each detector possess 128 sensor pairs. Moreover, 6 complex, ie. 12 real Gabor wavelet detectors are applied. At each grid node, the vector of the...

The analytical model describing the macrodynamic behavior of the pulse coupled neurons was implemented in a C/C++ simulator. The major focus for the implementation was an easy-to-understand and maintain source code. There are certainly several options to optimize run time and memory consumption. However, as the theory to describe the behavior of th...

The system concept of our vision cube is based on the 3D integration technology the realisation of which is described in chapter 16. The basic concept behind the application of this technology is illustrated in Figure 14.1. Several chips are stacked on to a base chip. The stack technology allows the placement of vertical interconnections in high de...

In the last chapter, we have raised the following questions: 1 How must the present architecture be extended if distance and scale in-variance, respectively, has to be supported? 1 How can it be possible that we perceive the world as time-continuous images, although we are using a pulse presentation for the whole recognition chain and the retinal i...

This book aims at a better understanding of the animal or human brain’s function. We are using a constructive methodology, by modelling paradigmatic visual functions with the help of concrete neural models. At the same time we would like to lay the foundation for a new form of information technology according to the biological model. As the functio...

This chapter describes the system architecture and chip design of the feature detector. The design aims at a widely flexible configuration of the connection topology to realise arbitrary feature detectors, at a faithful reproduction of the simulated individual or group behaviour of neurons and synapses, and at low power dissipation as well as the i...

Feature detectors can be constructed by using the basic computation rules of pulses presented in the last chapter. For instance, for a straight edge some number of receptors can be arranged to the right and left of the edge (Figure 6.1).

In this book, we attempt to construct an artificial vision system similar to the human brain which can perform complete image recognition tasks, ie. those which comprise feature detection, binding of features to objects, and object recognition. Similarity we see implemented by the usage of neurons and synapses as simply as possible, plus the restri...

or> Meeting abstracts - A single PDF containing all abstracts in this Supplement is available here. http://www. biomedcentral.co m/content/pdf/14 71-2202 -10-S1-info.pdf< /url>

Growing neuropsychological and neurophysiological evidence suggests that the visual cortex uses parts-based representations to encode, store and retrieve relevant objects. In such a scheme, objects are represented as a set of spatially distributed local features, or parts, arranged in stereotypical fashion. To encode the local appearance and to rep...

We here address the problem of scale and rotation invariant object ecognition, making use of a correspondence-based mechanism, in which the identity of an object represented by sensory signals is determined by matching it to a representation stored in memory. The sensory representation is in general affected by various transformations, notably scal...

Die mit Abstand erfolgreichste Version von Informationstechnologie weist das Gehirn auf. Tierische und menschliche Nervensysteme sind flexibel im Umgang mit Unerwartetem, passen sich an, lernen, evolvieren, integrieren sich leicht in soziale Netzwerke, haben eine inhärente Tendenz, Ordnung zu bilden, sind extrem Energie-genügsam und massiv parallel...

A well established method to analyze dynamical systems described by coupled nonlinear differential equations is to determine their normal modes and reduce the dynamics, by adiabatic elimination of stable modes, to a much smaller system for the amplitudes of unstable modes and their nonlinear interactions. So far, this analysis is possible only for...

Classically, programs are written with specific applications in mind. Organic computing will be based on a general architecture, which apart from libraries of standard algorithms will consist of generic mechanisms of organization. Users can then create specific applications by defining goal hierarchies, by instruction and the pointing out of exampl...

We here address the problem of scale and orientation invariant object recognition, making use of a correspondence-based mechanism, in which the identity of an object represented by sensory signals is determined by matching it to a representation stored in memory. The sensory representation is in general affected by various transformations, notably...

This letter presents an improved cue integration approach to reliably separate coherent moving objects from their background scene in video sequences. The proposed method uses a probabilistic framework to unify bottom-up and top-down cues in a parallel, "democratic" fashion. The algorithm makes use of a modified Bayes rule where each pixel's poster...

We describe a neural network able to rapidly establish correspondence between neural feature layers. Each of the network's two layers consists of interconnected cortical columns, and each column consists of inhibitorily coupled subpopulations of excitatory neurons. The dynamics of the system builds on a dynamic model of a single column, which is co...

Our aim here is to create a fully neural, functionally competitive, and correspondence-based model for invariant face recognition. By recurrently integrating information about feature similarities, spatial feature relations, and facial structure stored in memory, the system evaluates face identity ("what"-information) and face position ("where"-inf...

In higher mammals, environmentally driven patterns of neural activity do not play a role in the establishment of orientation specificity and maps. It has been proposed that specific long-range interactions provide the scaffold for developing orientation maps. Our model aims at explaining how such a scaffold could develop in the first place. Broad s...

Two main theories exist with respect to face encoding and representation in the human visual system (HVS). The first one refers to the dense (holistic) representation of the face, where faces have “holon”-like appearance. The second one claims that a more appropriate face representation is given by a sparse code, where only a small fraction of the...

,We describe a neural network for invariant object recognition. The network is generative in the sense that it

Analyzing the design of networks for visual information routing is an underconstrained problem due to insufficient anatomical and physiological data. We propose here optimality criteria for the design of routing networks. For a very general architecture, we derive the number of routing layers and the fanout that minimize the required neural circuit...

Object recognition in the presence of changing scale and orientation requires mechanisms to deal with the corresponding feature transformations. Using Gabor wavelets as example, we approach this problem in a correspondence-based setting. We present a mechanism for finding feature-to-feature matches between corresponding points in pairs of images ta...

We present a model for the ontogenesis of information routing archi- tectures in the brain based on chemical markers guiding axon growth. The model produces all-to-all connectivity between given populations of input and output nodes using a minimum of cortical resources (links and intermediate nodes). The resulting structures are similar to archite...

We propose a general Bayesian framework to simultaneously solve the problems of visual tracking and structure-from-motion using a sequential Monte Carlo method. We demonstrate the functionality of the system on some simple examples.

The principle of associative memory is extended to a system with dynamical links capable of retrieval of superimposed connection patterns. The system consists of formalized neurons. Its dynamics is described by two separate Hamiltonians, one for spins and one for links. The spin part is treated in analogy to the Ising system on a 2D grid. Several s...

The European Physical Society (EPS) is a not for profit association whose members include 41 National Physical Societies in Europe, individuals from all fields of physics, and European research institutions. As a learned society, the EPS engages in activities that strengthen ties among the physicists in Europe. As a federation of National Physical...

An important requirement for the expression of cognitive structures is the ability to form mental objects by rapidly binding together constituent parts. In this sense, one may conceive the brain's data structure to have the form of graphs whose nodes are labeled with elementary features. These provide a versatile data format with the ability to ren...

We investigate a possible functional role of glial cells as information routing devices of the cerebral cortex. On the one hand, functionally motivated models of neural information processing were lately suggested which rely on short-term changes of connections between neural modules to dynamically route neural activity. Although successful in prac...

We here are pointing out a basically well-known pathway to the analysis of self-organizing systems that is now well in reach of nu- merical methods. Systems of coupled nonlinear differential equations are decomposed into normal modes, are reduced by adiabatic elimination of stable modes to a much smaller system of unstable modes and their nonlin- e...

The goal of this research program was to develop novel algorithms, architectures, and hardware for a truly smart camera, with inherent capability for semi-autonomous object recognition as well as optimal image capture. In this research, we combined striking advances in the understanding of the mechanisms of biological vision systems with similar ad...

We describe a neural network able to rapidly establish correspondence between neural fields. The network is based on a cortical columnar model described earlier. It realizes dynamic links with the help of specialized columns that evaluate similarities between the activity distributions of local feature cell populations, are subject to a topology co...