PreprintPDF Available

A Neural Dynamic Model for the Perceptual Grounding of Combinatorial Concepts

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract

The ability to combine concepts to model the world in intricate ways underlies most of the higher cognitive competences like thought and language. It has motivated the stance that cognition is purely syntax-driven computation on systems of amodal symbols - a view widely referred to as the Classical Computational Theory of Mind (CCTM). Grounded Cognition (GC) rejects this view and emphasizes the importance of being able to ground concepts in perception, i.e., to establish a connection between concepts and objects in the perceptual input. This master thesis is part of a research program to provide a neural processing account for GC based on neural principles formalized in Dynamic Field Theory (DFT) . It introduces a neural architecture for the grounding of combinatorial concepts, i.e., concepts that are built by combining other concepts. The architecture receives an arbitrary input image or video and an arbitrary combinatorial concept, which describes an object in terms of its attributes and relationships to other objects – e.g., “a red triangle which is to the right of a red circle that is below a green diagonal rectangle and above a blue object”. Its task is to ground the concept in the perceptual input, i.e., to bring the described object into the attentional foreground. The components of a combinatorial concept are grounded in a sequence of grounding steps, while the output of each grounding step is passed on to the next grounding step through self-sustained neural fields. This way, semantic compositionality is an emergent property of the neural dynamics and does not require any form of amodal symbolic computation. The capabilities of the architecture are demonstrated in a set of 6 qualitatively different simulations that vary with the complexity of the combinatorial concept and the perceptual input. The architecture is able to successfully ground the given combinatorial concept in all test cases. Another contribution of this thesis is a clear interface between the grounding system and language, and an embedding in the literature of psychological theories of concepts.
A preview of the PDF is not available
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
How does the human brain link relational concepts to perceptual experience? For example, a speaker may say "the cup to the left of the computer" to direct the listener's attention to one of two cups on a desk. We provide a neural dynamic account for both perceptual grounding, in which relational concepts enable the attentional selection of objects in the visual array, and for the generation of descriptions of the visual array using relational concepts. In the model, activation in neural populations evolves dynamically under the influence of both inputs and strong interaction as formalized in dynamic field theory. Relational concepts are modeled as patterns of connectivity to perceptual representations. These generalize across the visual array through active coordinate transforms that center the representation of target objects in potential reference objects. How the model perceptually grounds or generates relational descriptions is probed in 104 simulations that systematically vary the spatial and movement relations employed, the number of feature dimensions used, and the number of matching and nonmatching objects. We explain how sequences of decisions emerge from the time- and state-continuous neural dynamics, how relational hypotheses are generated and either accepted or rejected, followed by the selection of new objects or the generation of new relational hypotheses. Its neural realism distinguishes the model from information processing accounts, its capacity to autonomously generate sequences of processing steps distinguishes it from deep neural network accounts. The model points toward a neural dynamic theory of higher cognition.
Chapter
Full-text available
If a theory of concept composition aspires to psychological plausibility, it may first need to address several preliminary issues associated with naturally occurring human concepts: content variability, multiple representational forms, and pragmatic constraints. Not only do these issues constitute a significant challenge for explaining individual concepts, they pose an even more formidable challenge for explaining concept compositions. How do concepts combine as their content changes, as different representational forms become active, and as pragmatic constraints shape processing? Arguably, concepts are most ubiquitous and important in compositions, relative to when they occur in isolation. Furthermore, entering into compositions may play central roles in producing the changes in content, form, and pragmatic relevance observed for individual concepts. Developing a theory of concept composition that embraces and illuminates these issues would not only constitute a significant contribution to the study of concepts, it would provide insight into the nature of human cognition.
Article
Full-text available
Describing actions entails that relations between objects are discovered. A pervasively neural account of this process requires that fundamental problems are solved: the neural pointer problem, the binding problem, and the problem of generating discrete processing steps from time-continuous neural processes. We present a prototypical solution to these problems in a neural dynamic model that comprises dynamic neural fields holding representations close to sensorimotor surfaces as well as dynamic neural nodes holding discrete, language-like representations. Making the connection between these two types of representations enables the model to describe actions as well as to perceptually ground movement phrases-all based on real visual input. We demonstrate how the dynamic neural processes autonomously generate the processing steps required to describe or ground object-oriented actions. By solving the fundamental problems of neural pointing, binding, and emergent discrete processing, the model may be a first but critical step toward a systematic neural processing account of higher cognition.