Harm Brouwer

Harm Brouwer
Universität des Saarlandes | UKS · Language Science and Technology

PhD

About

52
Publications
13,416
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
989
Citations
Introduction
My research interests are cognitive neuroscience, neurocomputational modeling of language comprehension, statistical modeling, computational linguistics, and (formal) semantics and pragmatics. More information: http://www.hbrouwer.eu/
Additional affiliations
July 2016 - April 2022
Universität des Saarlandes
Position
  • Scientific Staff (Wissenschaftlicher Mitarbeiter)
July 2014 - June 2016
Universität des Saarlandes
Position
  • Marie Curie "Experienced Researcher" Fellow
September 2010 - June 2014
University of Groningen
Position
  • PhD Student
Education
September 2010 - June 2014
University of Groningen
Field of study
  • Cognitive Science
September 2008 - August 2010
University of Groningen
Field of study
  • Behavioral and Cognitive Neuroscience
September 2005 - August 2008
University of Groningen
Field of study
  • Information Science

Publications

Publications (52)
Article
In traditional theories of language comprehension, syntactic and semantic processing are inextricably linked. This assumption has been challenged by the 'semantic illusion effect' found in studies using event related brain potentials. Semantically anomalous sentences did not produce the expected increase in N400 amplitude but rather one in P600 amp...
Article
Full-text available
Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component---traditionally taken to reflect semantic integration---but instead produced a P600-effect, whic...
Article
Full-text available
Event-Related Potentials (ERPs)—stimulus-locked, scalp-recorded voltage fluctuations caused by post-synaptic neural activity—have proven invaluable to the study of language comprehension. Of interest in the ERP signal are systematic, reoccurring voltage fluctuations called components, which are taken to reflect the neural activity underlying specif...
Article
Full-text available
Event-related potentials (ERPs) provide a multidimensional and real-time window into neurocognitive processing. The typical Waveform-based Component Structure (WCS) approach to ERPs assesses the modulation pattern of components - systematic, reoccurring voltage fluctuations reflecting specific computational operations - by looking at mean amplitude...
Article
Full-text available
Expectation-based theories of language comprehension, in particular Surprisal Theory, go a long way in accounting for the behavioral correlates of word-byword processing difficulty, such as reading times. An open question, however, is in which component(s) of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these elect...
Article
Full-text available
Expectation-based theories of language processing, such as Surprisal theory, are supported by evidence of anticipation effects in both behavioural and neurophysiological measures. Online measures of language processing, however, are known to be influenced by factors such as lexical association that are distinct from—but often confounded with—expect...
Article
Full-text available
Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation model representations originally proposed for modeling syst...
Article
Full-text available
The problem of spatiotemporal overlap between event-related potential (ERP) components is generally acknowledged in language research. However, its implications for the interpretation of experimental results are often overlooked. In a previous experiment on the functional interpretation of the N400 and P600, it was argued that a P600 effect to impl...
Article
Full-text available
Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or...
Conference Paper
Full-text available
Theories and models of spoken word recognition aim to explain the process of accessing lexical knowledge given an acoustic realization of a word form. There is consensus that phonological and semantic information is crucial for this process. However, there is accumulating evidence that orthographic information could also have an impact on auditory...
Preprint
Full-text available
Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with distributional meaning representations, thereby introducing the notion of semantic similarity into formal semantics, or...
Article
Full-text available
Language is processed on a more or less word-byword basis, and the processing difficulty induced by each word is affected by our prior linguistic experience as well as our general knowledge about the world. Surprisal and entropy reduction have been independently proposed as linking theories between word processing difficulty and probabilistic langu...
Article
Full-text available
The functional interpretation of two salient language-sensitive ERP components – the N400 and the P600 – remains a matter of debate. Prominent alternative accounts link the N400 to processes related to lexical retrieval, semantic integration, or both, while the P600 has been associated with syntactic reanalysis or, alternatively, to semantic integr...
Presentation
Full-text available
Presentation of the paper "A Framework for Distributional Formal Semantics". https://www.researchgate.net/publication/333961011_A_Framework_for_Distributional_Formal_Semantics
Chapter
Full-text available
Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at...
Article
Full-text available
The processing difficulty of each word we encounter in a sentence is affected by both our prior linguistic experience and our general knowledge about the world. Computational models of incremental language processing have, however, been limited in accounting for the influence of world knowledge. We develop an incremental model of language comprehen...
Poster
Full-text available
The N400 and the P600 component are the most salient and well-studied Event-Related brain Potential (ERP) components for sentence processing, but their functional interpretation is subject to ongoing debate (Kutas et al., 2006; Brouwer et al., 2012). This has led to a proliferation of models, as illustrated for example by the diverse proposals that...
Presentation
Full-text available
The study of language comprehension is ultimately about meaning: How can meaning be constructed from linguistic signal, and how can it be represented? The human language comprehension system is highly efficient and accurate at attributing meaning to linguistic input. Hence, in trying to identify computational principles and representations for mean...
Article
Full-text available
The property of projection poses a challenge to formal semantic theories, due to its apparent non-compositional nature. Projected content is therefore typically analyzed as being different from and independent of asserted content. Recent evidence, however, suggests that these types of content in fact closely interact, thereby calling for a more int...
Article
The effect of word association on sentence processing is still a matter of debate. Some studies observe no effect while others found a dependency on sentence congruity or an independent effect. In an attempt to separate the effects of sentence congruity and word association in the spatio-temporal domain, we jointly recorded scalp- and invasive-EEG...
Poster
Full-text available
The N400 and P600 are the two most salient language-sensitive components of the Event-Related Potential (ERP) signal. Yet, their functional interpretation is still a matter of debate. Traditionally, the N400 is taken to reflect processes of semantic integration while the P600 is linked to structural reanalysis [1,2]. These views have, however, been...
Poster
Full-text available
In online language comprehension, the N400 component of the Event-Related Potentials (ERP) signal is inversely proportional to semantic expectancy (Kutas & Federmeier, 2011). Among other factors, a word’s expectancy is influenced by both lexical-level (Bentin et al., 1985) as well as event-level (Metusalem et al., 2012) priming: the N400 amplitude...
Data
Appendix S1. Simulation materials. Appendix S2. Derivation of word meaning representations. Appendix S3. Details of the training procedure. Appendix S4. Training on perfect word meaning representations.
Poster
Full-text available
In the ERP signal, referential ambiguities most commonly elicit Nref effects relative to controls. However, it is unclear whether Nref is sensitive to the degree of ambiguity (i.e. referential entropy). Previous work has also found important individual differences -- some participants instead show a late positive component (LPC). Finally, no ERP ma...
Poster
Full-text available
In the ERP signal, referential ambiguities most commonly elicit Nref effects relative to controls. However, it is unclear whether Nref is sensitive to the degree of ambiguity (i.e. referential entropy). Previous work has also found important individual differences -- some participants instead show a late positive component (LPC). Finally, no ERP ma...
Chapter
Full-text available
The study of language is ultimately about meaning: How can meaning be constructed from linguistic signal, and how can it be represented? The human language comprehension system is highly efficient and accurate at attributing meaning to linguistic input. Hence, in trying to identify computational principles and representations for meaning constructi...
Poster
Full-text available
A defining characteristic of human language is systematicity: “the ability to produce/understand some sentences is intrinsically connected to the ability to produce/understand certain others” (Fodor & Pylyshyn, 1988). Further, Fodor and Pylyshyn (1988) argue that connectionist models are not able to display systematicity without implementing a clas...
Conference Paper
Full-text available
A novel connectionist model of sentence production is presented , which employs rich situation model representations originally proposed for modeling systematicity in comprehension (Frank, Haselager, & van Rooij, 2009). The high overall performance of our model demonstrates that such representations are not only suitable for comprehension, but also...
Poster
Full-text available
We present a neurocomputational model of the electrophysiology of language processing. Our model is explicit about its architecture and the computational principles and representations involved. It is effectively a recurrent neural network (of the ‘Elman’-type; [1]) that constructs a situation model of the state-of-the-affairs described by a senten...
Poster
Full-text available
Eye movements during linguistic-visual conflicts
Presentation
Full-text available
We present a neurocomputational—recurrent artificial neural network—model of language processing that integrates linguistic knowledge and world/event knowledge, and that produces word surprisal estimates that take into account both. Our model constructs a cognitively motivated situation model of the state-of-the-affairs as described by a sentence....
Presentation
Full-text available
We present a neurocomputational model of the electrophysiology of language processing. Our model is explicit about its architecture and the computational principles and representations involved. It is effectively a recurrent neural network (of the ‘Elman’-type; [1]) that directly instantiates a parsimonious functional-anatomic processing network li...
Poster
Full-text available
Visual attention can be directed by visual and linguistic information. It is not well understood how attention is directed when linguistic information conflicts with the visual scene. Knoeferle and Crocker (2006) established the coordinated interplay account model of sentence comprehension and linguistically mediated visual attention, but it did no...
Thesis
Full-text available
One decade ago, researchers using Event-Related brain Potential (ERP) measurements stumbled upon what looked like a Semantic Illusion in language comprehension: Semantically anomalous, but otherwise well-formed sentences did not affect the meaning-related N400 component, but instead increased the amplitude of the structure-related P600 component. T...
Poster
Full-text available
A Neurocomputational Model of the N400 and the P600
Conference Paper
Full-text available
We introduce PDRT-SANDBOX, a Haskell library that implements Projective Discourse Representation Theory (PDRT) (Venhuizen et al., 2013), an extension of Discourse Representation Theory (DRT) (Kamp, 1981; Kamp and Reyle, 1993). The implementation includes a translation from PDRT to DRT and first-order logic, composition via different types of merge,...
Poster
Full-text available
Implementing Projective Discourse Representation Theory
Thesis
Full-text available
One decade ago, researchers using event-related brain potential (ERP) measurements stumbled upon what looked like a Semantic Illusion in language comprehension: Semantically anomalous, but otherwise well-formed sentences did not affect the meaning-related N400 component, but instead increased the amplitude of the structure-related P600 component. T...
Chapter
Full-text available
Research into the electrophysiology of language comprehension has essentially been “speakerless.” This has left three vital aspects of communication—it is social, pragmatic, and dynamic—severely underresearched. This chapter makes a case for the investigation of language users involved in active conversation and describes the problems and possibili...
Conference Paper
Full-text available
Conventional Implicatures (CIs; in the sense of Potts 2005) are part of a larger class of projection phenomena. These phenomena also include presuppositions and anaphora, and can be described as content that is not at-issue (see Simons, Tonhauser, Beaver & Roberts 2010). Despite the shared property of projection, CIs differ from other projection ph...
Poster
Full-text available
(See the abstract of the paper with the same title)
Article
Full-text available
We propose a new functional-anatomical mapping of the N400 and the P600 to a minimal cortical network for language comprehension. Our work is an example of a recent research strategy in cognitive neuroscience, where researchers attempt to align data regarding the nature and time-course of cognitive processing (from ERPs) with data on the cortical o...
Article
Full-text available
It sometimes happens that when someone asks a question, the addressee does not give an adequate answer, for instance by leaving out part of the required information. The person who posed the question may wonder why the information was omitted, and engage in extensive processing to find out what the partial answer actually means. The present study l...
Poster
Full-text available
People do not always behave as ideal interactants. When asked a question, they may bluntly say “No”, without apologizing or giving reasons. In an ERP experiment (measuring Event Related brain Potentials), we tested how language users process polite and less polite answers to different types of requests.
Poster
Full-text available
It sometimes happens that when someone asks a question, the addressee does not give an adequate answer, for instance by leaving out information. The person who posed the question may wonder why the information was omitted, and engage in extensive processing to find out what the partial answer means. In an ERP experiment (measuring Event Related bra...
Poster
Full-text available
In traditional theories of language comprehension, syntactic and semantic processing are inextricably linked. This assumption has been challenged by the ’semantic illusion effect’ found in studies using event related brain potentials. Semantically anomalous sentences did not produce the expected increase in N400 amplitude but rather one in P600 amp...
Conference Paper
Full-text available
This paper investigates whether surprisal theory can account for differential processing difficulty in the NP-/S-coordination ambiguity in Dutch. Surprisal is es-timated using a Probabilistic Context-Free Grammar (PCFG), which is induced from an automatically annotated corpus. We find that our lexicalized surprisal model can account for the reading...

Projects

Projects (3)
Project
Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at such a unified framework has, however, proven extremely challenging. One reason for this is that formal and distributional semantics operate on a fundamentally different 'representational currency': formal semantics defines meaning in terms of models of the world, whereas distributional semantics defines meaning in terms of linguistic co-occurrence. Here, we pursue an alternative approach by deriving a vector space model that defines meaning in a distributed manner relative to formal models of the world. The resulting Distributional Formal Semantics offers probabilistic distributed representations that are also inherently compositional, and that naturally capture quantification and entailment. Moreover, when used as part of a neural network model, these representations allow for capturing incremental meaning construction and probabilistic inferencing. This framework thus lays the groundwork for an integrated distributional and formal approach to meaning.
Archived project
The property of projection poses a challenge to formal semantic theories, due to its apparent non-compositional nature. Projected content is therefore typically analyzed as being different from and independent of asserted content. Recent evidence, however, suggests that these types of content in fact closely interact, thereby calling for a more integrated analysis that captures their similarities, while respecting their differences. In this project, we developed such a unified, compositional semantic analysis of asserted and projected content. Our analysis captures the similarities and differences between presuppositions, anaphora, conventional implicatures and assertions on the basis of their information structure, that is, on basis of how their content is contributed to the unfolding discourse context. We formalize our analysis in an extension of the dynamic semantic framework of Discourse Representation Theory (DRT)--called Projective DRT (PDRT)--that employs projection variables to capture the information-structural aspects of semantic content; different constellations of such variables capture the differences between the different types of projected and asserted content within a single dimension of meaning. To show the soundness of the PDRT formalism, we derived its formal properties for constructing and combining semantic representations, which we implemented as part of an open-source Haskell library called PDRT-SANDBOX.
Archived project
Arriving at an explicit neurocomputational model of the electrophysiology of language comprehension, focusing on the N400 and the P600 component of the Event Related brain Potential (ERP) signal.