Malte R. Henningsen-Schomers’s research while affiliated with Freie Universität Berlin and other places

The power of words Compared to our closest living relatives, who typically use fewer than 100 words, humans can build vocabularies of tens of hundreds of thousands of words. The ERC- funded Advanced Grant project ‘Material Constraints enabling Human Cognition’, or ‘MatCo’, will find out why. It will use novel insights from human neurobiology. These will be translated into mathematically exact computational models to find new answers to long-standing questions in cognitive science, linguistics and philosophy. The project will also explore how semantic meaning is implemented for gestures and words and, more specifically, for referential and categorical terms. To identify human cognitive capacities, MatCo will develop models replicating structural differences between human and non-human primate brains. The results will shed light on the biologically constrained networks.

Concrete symbols (e.g., sun, run) can be learned in the context of objects and actions, thereby grounding their meaning in the world. However, it is controversial whether a comparable avenue to semantic learning exists for abstract symbols (e.g., democracy). When we simulated the putative brain mechanisms of conceptual/semantic grounding using brain‐constrained deep neural networks, the learning of instances of concrete concepts outside of language contexts led to robust neural circuits generating substantial and prolonged activations. In contrast, the learning of instances of abstract concepts yielded much reduced and only short‐lived activity. Crucially, when conceptual instances were learned in the context of wordforms, circuit activations became robust and long‐lasting for both concrete and abstract meanings. These results indicate that, although the neural correlates of concrete conceptual representations can be built from grounding experiences alone, abstract concept formation at the neurobiological level is enabled by and requires the correlated presence of linguistic forms.

Language influences cognitive and conceptual processing, but the mechanisms through which such causal effects are realized in the human brain remain unknown. Here, we use a brain-constrained deep neural network model of category formation and symbol learning and analyze the emergent model-internal mechanisms at the neural circuit level. In one set of simulations, the network was presented with similar patterns of neural activity indexing instances of objects and actions belonging to the same categories. Biologically realistic Hebbian learning led to the formation of instance-specific neurons distributed across multiple areas of the network, and, in addition, to cell assembly circuits of ‘shared’ neurons responding to all category instances – the network correlates of conceptual categories. In two separate sets of simulations, the network learned the same patterns together with symbols for individual instances (‘ proper names ’) or symbols related to classes of instances sharing common features (‘ category terms ’). Learning category terms remarkably increased the number of shared neurons in the network, thereby making category representations more robust while reducing the number of neurons of instance-specific ones. In contrast, proper-name learning prevented substantial reduction of instance-specific neurons and blocked the overgrowth of category-general cells. Representational Similarity Analysis further confirmed that the neural activity patterns of category instances became more similar to each other after category-term learning, relative to both learning with proper names and without any symbols. These network-based mechanisms for concepts, proper names and category terms explain why and how symbol learning changes object perception and memory, as revealed by experimental studies. Significance Statement How do verbal symbols for specific individuals ( Micky Mouse ) and object categories ( house mouse ) causally influence conceptual representation and processing? Category terms and proper names have been shown to respectively promote category formation and instance learning, potentially by respectively directing attention to category-critical and object-specific features. Yet the mechanisms underlying these observations at the neural circuit level remained unknown. Using a mathematically precise deep neural network model constrained by properties of the human brain, we show category-term learning strengthens and solidifies conceptual representations, whereas proper names support object-specific mechanisms. Based on network-internal mechanisms and unsupervised correlation-based learning, this work offers neurobiological explanations for causal effects of symbol learning on concept formation, category building and instance representation in the human brain.

A neurobiologically constrained model of semantic learning in the human brain was used to simulate the acquisition of concrete and abstract concepts, either with or without verbal labels. Concept acquisition and semantic learning were simulated using Hebbian learning mechanisms. We measured the network's category learning performance, defined as the extent to which it successfully (i) grouped partly overlapping perceptual instances into a single (abstract or concrete) conceptual representation, while (ii) still distinguishing representations for distinct concepts. Co-presence of linguistic labels with perceptual instances of a given concept generally improved the network's learning of categories, with a significantly larger beneficial effect for abstract than concrete concepts. These results offer a neurobiological explanation for causal effects of language structure on concept formation and on perceptuo-motor processing of instances of these concepts: supplying a verbal label during concept acquisition improves the cortical mechanisms by which experiences with objects and actions along with the learning of words lead to the formation of neuronal ensembles for specific concepts and meanings. Furthermore, the present results make a novel prediction, namely, that such ‘Whorfian’ effects should be modulated by the concreteness/abstractness of the semantic categories being acquired, with language labels supporting the learning of abstract concepts more than that of concrete ones. This article is part of the theme issue ‘Concepts in interaction: social engagement and inner experiences’.

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically ‘ground’ concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their ‘shared neurons’, thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling. Neural network models have potential for improving our understanding of brain functions. In this Perspective, Pulvermüller and colleagues examine various aspects of such models that may need to be constrained to make them more neurobiologically realistic and therefore better tools for understanding brain function.

In the field of neurobiology of language, neuroimaging studies are generally based on stimulation paradigms consisting of at least two different conditions. Designing those paradigms can be very time-consuming and this traditional approach is necessarily data-limited. In contrast, in computational and corpus linguistics, analyses are often based on large text corpora, which allow a vast variety of hypotheses to be tested by repeatedly re-evaluating the data set. Furthermore, text corpora also allow exploratory data analysis in order to generate new hypotheses. By drawing on the advantages of both fields, neuroimaging and computational corpus linguistics, we here present a unified approach combining continuous natural speech and MEG to generate a corpus of speech-evoked neuronal activity.

In the field of neurobiology of language, neuroimaging studies are generally based on stimulation paradigms consisting of at least two different conditions. Depending on the desired evaluation, these conditions, in turn, have to contain dozens of items to achieve a good signal to noise ratio. Designing those paradigms can be very time-consuming. Subsequently, a group of participants is stimulated with the new paradigm, while brain activity is assessed, e.g. with EEG/MEG. The measured data are then pre-processed and finally contrasted according to the different stimulus conditions. In this way, only a limited number of analyses and hypothesis tests can be performed, while for alternative or further analyses, completely new paradigms usually need to be designed. This traditional approach is necessarily data-limited, and the cost-benefit ratio is therefore rather poor. In contrast, in computational linguistics analyses are based on text corpora, which allow a vast variety of hypotheses to be tested by repeatedly re-evaluating the data set. Furthermore, text corpora also allow exploratory data analysis in order to generate new hypotheses. By combining the two approaches, we here present a unified approach of continuous natural speech and MEG to generate a corpus-like database of speech-evoked neuronal activity.