Matthew W Crocker

Verified

Matthew verified their affiliation via an institutional email.

Learn more

Verified

Matthew verified their affiliation via an institutional email.

Learn more

• PhD (Edin)
• Professor (Full) at Saarland University

The ERP literature on language comprehension reveals variability in observing monophasic N400 versus biphasic N400-P600 effects in response to incongruent input, with the reasons for this inconsistency remaining unclear. Two interrelated factors may contribute: spatiotemporal overlap between the N400 and P600, where a strong N400-effect can obscure...

Surprisal values from large language models (LLMs) have been used to model the amplitude of the N400. This ERP component is sensitive not only to contextual word expectancy but also to semantic association, such that unexpected but associated words do not always induce an N400 increase. While LLMs are also sensitive to association, it remains uncle...

Poster presented at the 37th Annual Conference on Human Sentence Processing, May 16-18, 2024, Ann Arbor, Michigan, United States of America.

Previous studies showed that word learning is affected by children's existing knowledge. For instance, knowledge of semantic category aids word learning, whereas a dense phonological neighbourhood impedes learning of similar‐sounding words. Here, we examined to what extent children associate similar‐sounding words (e.g., rat and cat) with objects o...

Theories of the electrophysiology of language comprehension are mostly informed by event-related potential effects observed between condition averages. We here argue that a dissociation between competing effect-level explanations of event-related potentials can be achieved by turning to predictions and analyses at the single-trial level. Specifical...

The Cambridge Handbook of Computational Cognitive Sciences is a comprehensive reference for this rapidly developing and highly interdisciplinary field. Written with both newcomers and experts in mind, it provides an accessible introduction of paradigms, methodologies, approaches, and models, with ample detail and illustrated by examples. It should...

The integration of word meaning into an unfolding utterance representation is a core operation of incremental language comprehension. There is considerable debate, however, as to which component of the ERP signal—the N400 or the P600—directly reflects integrative processes, with far reaching consequences for the temporal organization and architectu...

Behavioral studies have shown that speaker gaze to objects in a co-present scene can influence listeners' expectations about how the utterance will unfold. These findings have recently been supported by ERP studies that linked the underlying mechanisms of the integration of speaker gaze with an utterance meaning representation to multiple ERP compo...

To appear in: Ron Sun (ed.), Cambridge Handbook of Computational Cognitive Sciences. CUP: Cambridge, UK.

While the N400 and P600 are the most salient ERP indices of language processing, it remains under debate which component indexes integration processes (e.g., Brouwer et al., 2017, Rabovsky et al., 2018). A recent study (Aurnhammer et al., 2021) demonstrated their differential sensitivity to how associated a word is to the context (N400 alone) versu...

Electrophysiological studies suggest that situational event knowledge plays an important role in language processing, but often fail to distinguish whether observed effects are driven by combinatorial expectations, or simple association with the context. In two ERP experiments, participants read short discourses describing ongoing events. We manipu...

In referential communication, Grice's Maxim of Quantity is thought to imply that utterances conveying unnecessary information should incur comprehension difficulties. There is, however, considerable evidence that speakers frequently encode redundant information in their referring expressions, raising the question as to whether such overspecificatio...

Expectation-based theories of language processing, such as Surprisal theory, are supported by evidence of anticipation effects in both behavioural and neurophysiological measures. Online measures of language processing, however, are known to be influenced by factors such as lexical association that are distinct from—but often confounded with—expect...

Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation model representations originally proposed for modeling syst...

The problem of spatiotemporal overlap between event-related potential (ERP) components is generally acknowledged in language research. However, its implications for the interpretation of experimental results are often overlooked. In a previous experiment on the functional interpretation of the N400 and P600, it was argued that a P600 effect to impl...

Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or...

The results of a highly influential study that tested the predictions of the Rational Speech Act (RSA) model suggest that (a) listeners use pragmatic reasoning in one-shot web-based referential communication games despite the artificial, highly constrained, and minimally interactive nature of the task, and (b) that RSA accurately captures this beha...

Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or...

Expectation-based theories of language comprehension, in particular Surprisal Theory, go a long way in accounting for the behavioral correlates of word-by-word processing difficulty, such as reading times. An open question, however, is in which component(s) of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these elec...

The Uniform Information Density (UID) hypothesis links production strategies with comprehension processes, predicting that speakers will utilize flexibility in encoding in order to increase uniformity in the rate of information transmission, as measured by surprisal (Jaeger, 2010). Evidence in support of UID comes primarily from studies focusing on...

Recently, Ankener et al. ( Frontiers in Psychology, 9 , 2387, 2018) presented a visual world study which combined both attention and pupillary measures to demonstrate that anticipating a target results in lower effort to integrate that target (noun). However, they found no indication that the anticipatory processes themselves, i.e., the reduction o...

Event‐related potentials (ERPs) provide a multidimensional and real‐time window into neurocognitive processing. The typical Waveform‐based Component Structure (WCS) approach to ERPs assesses the modulation pattern of components—systematic, reoccurring voltage fluctuations reflecting specific computational operations—by looking at mean amplitude in...

Language is processed on a more or less word-by-word basis, and the processing difficulty induced by each word is affected by our prior linguistic experience as well as our general knowledge about the world. Surprisal and entropy reduction have been independently proposed as linking theories between word processing difficulty and probabilistic lang...

Contrary to the Gricean maxims of Quantity (Grice 1975), it has been repeatedly shown that speakers often include redundant information in their utterances (over- specifications). Previous research on referential communication has long debated whether this redundancy is the result of speaker-internal or addressee-oriented processes, while it is als...

The functional interpretation of two salient language-sensitive ERP components – the N400 and the P600 – remains a matter of debate. Prominent alternative accounts link the N400 to processes related to lexical retrieval, semantic integration, or both, while the P600 has been associated with syntactic reanalysis or, alternatively, to semantic integr...

Presentation of the paper "A Framework for Distributional Formal Semantics". https://www.researchgate.net/publication/333961011_A_Framework_for_Distributional_Formal_Semantics

Behavioral studies have shown that speaker gaze to objects in a co-present scene can influence listeners’ sentence comprehension. To gain deeper insight into the mechanisms involved in gaze processing and integration, we conducted two ERP experiments (N = 30, Age: [18, 32] and [19, 33] respectively). Participants watched a centrally positioned face...

Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at...

Results from recent work testing formalizations of Gricean maxims using one-shot web-based reference-games is mixed. Some studies indicate Bayesian (RSA) models closely predict human (pragmatic) behavior; others suggest participants rarely go beyond literal meanings in such studies. We investigated whether listeners in such tasks reason as pragmati...

Psycholinguistic research has established that event knowledge influences a word's processing difficulty. Less clear, is the precise nature of this influence as indexed by ERPs: Some studies attribute it to semantic expectancy, reflected in N400 amplitude (Metusalem et al., 2012), while others attribute it to integration difficulties reflected in P...

The processing difficulty of each word we encounter in a sentence is affected by both our prior linguistic experience and our general knowledge about the world. Computational models of incremental language processing have, however, been limited in accounting for the influence of world knowledge. We develop an incremental model of language comprehen...

Speakers tend to fixate objects they are about to mention, while listeners inspect those objects that they believe to be intended referents of the speaker. These production- and comprehension-contingent gaze behaviors may form an integral part of the signal itself, making it inherently reciprocal. Here, we present work that has investigated the int...

Contra the Gricean Maxim of Quantity (Grice 1975), recent studies indicate that speakers’ referential expressions often contain more information than strictly necessary for target identification (e.g., Koolen et al. 2011). Such overspecifications may be driven by egocentric concerns (minimise speaker’s cognitive effort, cf. Keysar et al. 1998) or a...

We present an analysis of the internal mechanism of the recurrent neural model of sentence production presented by Calvillo et al. (2016). The results show clear patterns of computation related to each layer in the network allowing to infer an algorithmic account, where the semantics activates the semantically related words, then each word generate...

The N400 and the P600 component are the most salient and well-studied Event-Related brain Potential (ERP) components for sentence processing, but their functional interpretation is subject to ongoing debate (Kutas et al., 2006; Brouwer et al., 2012). This has led to a proliferation of models, as illustrated for example by the diverse proposals that...

The study of language comprehension is ultimately about meaning: How can meaning be constructed from linguistic signal, and how can it be represented? The human language comprehension system is highly efficient and accurate at attributing meaning to linguistic input. Hence, in trying to identify computational principles and representations for mean...

When reading a text describing an everyday activity, comprehenders build a model of the situation described that includes prior knowledge of the entities, locations, and sequences of actions that typically occur within the event. Previous work has demonstrated that such knowledge guides the processing of incoming information by making event boundar...

The N400 and P600 are the two most salient language-sensitive components of the Event-Related Potential (ERP) signal. Yet, their functional interpretation is still a matter of debate. Traditionally, the N400 is taken to reflect processes of semantic integration while the P600 is linked to structural reanalysis [1,2]. These views have, however, been...

Previous ERP research has extensively studied the influence of linguistic context on language processing, demonstrating that the semantic expectedness of a word is negatively correlated with N400 amplitude (Kutas & Hillyard, 1984). Such findings have traditionally been interpreted as indexing compositional semantic integration, but can often be exp...

Previous research has shown that the semantic expectedness of a word – as established by the linguistic context – is negatively correlated with N400 amplitude. While such evidence has been used to argue that the N400 indexes semantic integration processes, findings can often be explained in terms of facilitated lexical retrieval, which, among other...

In online language comprehension, the N400 component of the Event-Related Potentials (ERP) signal is inversely proportional to semantic expectancy (Kutas & Federmeier, 2011). Among other factors, a word’s expectancy is influenced by both lexical-level (Bentin et al., 1985) as well as event-level (Metusalem et al., 2012) priming: the N400 amplitude...

Event-Related Potentials (ERPs)—stimulus-locked, scalp-recorded voltage fluctuations caused by post-synaptic neural activity—have proven invaluable to the study of language comprehension. Of interest in the ERP signal are systematic, reoccurring voltage fluctuations called components, which are taken to reflect the neural activity underlying specif...

Behavioral studies demonstrate the influence of speaker gaze in visually-situated spoken language comprehension. We present an ERP experiment examining the influence of speaker's gaze congruency on listeners' comprehension of referential expressions related to a shared visual scene. We demonstrate that listeners exploit speakers' gaze toward object...

In situated communication, reference to an entity in the shared visual context can be established using either an expression that conveys precise (minimally specified) or redundant (over-specified) information. There is, however, a long-lasting debate in psycholinguistics concerning whether the latter hinders referential processing. We present evid...

Over-specifications (OS) are expressions that provide more information than minimally required for the identification of a referent, thereby violating Grice’s 2nd Quantity Maxim. In recent years, psycholinguistic research has tried to test the empirical validity of Grice’s Maxim, resulting in conflicting findings. That is, there is evidence both th...

Appendix S1. Simulation materials. Appendix S2. Derivation of word meaning representations. Appendix S3. Details of the training procedure. Appendix S4. Training on perfect word meaning representations.

Previous behavioral and electrophysiological studies have presented evidence suggesting that coercion expressions (e.g., began the book) are more difficult to process than control expressions like read the book. While this processing cost has been attributed to a specific coercion operation for recovering an event-sense of the complement (e.g., beg...

We investigated the effect of high- and low-predictability sentence contexts on the comprehension of words and similar-looking pseudo-words. We aimed to establish whether top-down context support affords for facilitation for bottom-up activation of both words and pseudo-words. Based on previous ERP findings, we explored both early (N170) and later...

In the ERP signal, referential ambiguities most commonly elicit Nref effects relative to controls. However, it is unclear whether Nref is sensitive to the degree of ambiguity (i.e. referential entropy). Previous work has also found important individual differences -- some participants instead show a late positive component (LPC). Finally, no ERP ma...

In the ERP signal, referential ambiguities most commonly elicit Nref effects relative to controls. However, it is unclear whether Nref is sensitive to the degree of ambiguity (i.e. referential entropy). Previous work has also found important individual differences -- some participants instead show a late positive component (LPC). Finally, no ERP ma...

Psycholinguistic studies investigating grammatical and semantic (i.e. biological) gender knowledge effects on language comprehension have often manipulated the match between a linguistic context and words (e.g. pronouns) in a subsequent sentence (e.g. finding the word ’her’ after a sentence talking about a ’policeman’; Hammer et al., 2008; Kreiner...

Keynote Lecture

Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component---traditionally taken to reflect semantic integration---but instead produced a P600-effect, whic...

The study of language is ultimately about meaning: How can meaning be constructed from linguistic signal, and how can it be represented? The human language comprehension system is highly efficient and accurate at attributing meaning to linguistic input. Hence, in trying to identify computational principles and representations for meaning constructi...

A defining characteristic of human language is systematicity: “the ability to produce/understand some sentences is intrinsically connected to the ability to produce/understand certain others” (Fodor & Pylyshyn, 1988). Further, Fodor and Pylyshyn (1988) argue that connectionist models are not able to display systematicity without implementing a clas...

A novel connectionist model of sentence production is presented , which employs rich situation model representations originally proposed for modeling systematicity in comprehension (Frank, Haselager, & van Rooij, 2009). The high overall performance of our model demonstrates that such representations are not only suitable for comprehension, but also...

We present a neurocomputational model of the electrophysiology of language processing. Our model is explicit about its architecture and the computational principles and representations involved. It is effectively a recurrent neural network (of the ‘Elman’-type; [1]) that constructs a situation model of the state-of-the-affairs described by a senten...

Eye movements during linguistic-visual conflicts

Visually Situated Language Comprehension has been compiled as a state-of the-art introduction to real-time language processing in visually-situated contexts. It covers the history of this emergent field, explains key methodological developments and discusses the insights these methods have enabled into how language processing interacts with our kno...

This chapter introduces the visual world paradigm, with the aim of identifying both the opportunities and challenges researchers are presented with when using overt visual attention as an index of the cognitive processes and mechanisms involved in a variety of language processing tasks. The chapter also provides an overview of the linking hypothese...

While the influence of linguistic context on language processing has been extensively studied, less is known about the mental representation, structure and use of so-called script knowledge. Scripts are defined as a person’s knowledge about temporally and causally ordered sequences of events. They are often activated by linguistic context, but othe...

We present a neurocomputational—recurrent artificial neural network—model of language processing that integrates linguistic knowledge and world/event knowledge, and that produces word surprisal estimates that take into account both. Our model constructs a cognitively motivated situation model of the state-of-the-affairs as described by a sentence....

Visually Situated Language Comprehension has been compiled as a state-of the-art introduction to real-time language processing in visually-situated contexts. It covers the history of this emergent field, explains key methodological developments and discusses the insights these methods have enabled into how language processing interacts with our kno...

Beyond the observation that both speakers and listeners rapidly inspect the visual targets of referring expressions, it has been argued that such gaze may constitute part of the communicative signal. In this study, we investigate whether a speaker may, in principle, exploit listener gaze to improve communicative success. In the context of a virtual...

We present a neurocomputational model of the electrophysiology of language processing. Our model is explicit about its architecture and the computational principles and representations involved. It is effectively a recurrent neural network (of the ‘Elman’-type; [1]) that directly instantiates a parsimonious functional-anatomic processing network li...

We introduce IDeaL (Information Density and Linguistic Encoding), a collaborative research center that investigates the hypothesis that language use may be driven by the optimal use of the communication channel. From the point of view of linguistics, our approach promises to shed light on selected aspects of language variation that are hitherto not...

Visual attention can be directed by visual and linguistic information. It is not well understood how attention is directed when linguistic information conflicts with the visual scene. Knoeferle and Crocker (2006) established the coordinated interplay account model of sentence comprehension and linguistically mediated visual attention, but it did no...

Violations of the Maxims of Quantity occur when utterances provide more (over- specified) or less (under-specified) information than strictly required for referent identification. While behavioural data suggest that under-specified (US) expressions lead to comprehension difficulty and communicative failure, there is no consensus as to whether over-...

A variety of mechanisms contribute to word learning. Learners can track co-occurring words and referents across situations in a bottom-up manner (cross-situational word learning, CSWL). Equally, they can exploit sentential contexts, relying on top–down information such as verb–argument relations and world knowledge, offering immediate constraints o...

A model of syntactic parsing that combines elements of information and probability theory is proposed. The model assigns probability and entropy scores to parse trees: trees with higher probabilities are preferred while trees with higher entropies are penalized. This model is argued to be psycholinguistically motivated by means of rational analysis...

The ability to monitor the communicative success of its utterances and, if necessary, provide feedback and repair is useful for a dialog system. We show that in situated communication, eyetracking can be used to reliably and efficiently monitor the hearer's reference resolution process. An interactive system that draws on hearer gaze to provide pos...

When looking for the referents of novel nouns, adults and young children are sensitive to cross-situational statistics (Yu and Smith, 2007; Smith and Yu, 2008). In addition, the linguistic context that a word appears in has been shown to act as a powerful attention mechanism for guiding sentence processing and word learning (Landau and Gleitman, 19...

Word learning in adults succeeds with the help of various mechanisms and is based on multi-modal information sources. The complex interplay of these different cues, however, has rarely been studied. We present two experiments investigat-ing how cross-situational word learning (CSWL) and learning based on sentence-level constraints (SLCL) interact....

Evidence from behavioral studies demonstrates that spoken language guides attention in a related visual scene and that attended scene information can influence the comprehension process. Here we model sentence comprehension within visual contexts. A recurrent neural network is trained to associate the linguistic input with the visual scene and to p...

We present two eye-tracking experiments that investigate lexical frequency and semantic context constraints in spoken-word recognition in German. In both experiments, the pivotal words were pairs of nouns overlapping at onset but varying in lexical frequency. In Experiment 1, German listeners showed an expected frequency bias towards high-frequency...

Referential gaze during situated language production and comprehension is tightly coupled with the unfolding speech stream (Griffin, 2001; Meyer, Sleiderink, & Levelt, 1998; Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995). In a shared environment, utterance comprehension may further be facilitated when the listener can exploit the speaker's f...

It appears quite odd that L2 learners should transfer lexico-syntactic gender, because linguistically, gender is extremely arbitrary and language specific. We explore possible reasons using an integrated neural network model of monolingual and bilingual gender acquisition and processing. Examination of the network’s gender representations, similar...

Gaze is known to be an important social cue in face-to-face communication indicating interest, focus of attention, or turn-taking intentions. Further, speaker gaze can influence situated utterance comprehension by driving both interlocutor's visual attention towards the same object thereby grounding and disambiguating referring expressions (Hanna a...