Stefan L. Frank’s research while affiliated with Radboud University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (124)


Connectionist Models of Second Language Acquisition and Processing
  • Preprint

January 2025

·

3 Reads

Stefan L. Frank

·

Naomi Tachikawa Shapiro

Connectionist models treat language processing as the flow of activation through a network of simple units and the connections between them. Language learning is modeled as the adjustment of the strengths of these connections. Such models have led to important insights into second language acquisition and processing. This chapter introduces the fundamentals of connectionist models in cognitive science and reviews the most influential word- and sentence-level models of bilingualism.


Figure 2. Percentage of passive target responses after either an active or a passive prime, for cross-language (left panel) and within-language trials (right panel). Thick black lines visualise the priming effect across all trials. Thin grey lines show the same for each simulated participant
Fixed effects summary for the logistic mixed-effects model (N = 48, 121): Estimates with 95% bootstrapped confidence intervals and p-values for each predictor
The Bilingual Dual-path model: Simulating bilingual production, processing, and development
  • Article
  • Full-text available

October 2024

·

32 Reads

Linguistic Approaches to Bilingualism

Experimental research has yielded many important psycholinguistic findings in bi-/multilingualism. However, cognitive computational models of sentence processing were limited to the single-language case until recently. In this methodological review, we discuss cognitive modelling work that uses the Bilingual Dual-path model to simulate experimental research on bilingual sentence production, processing, and development. As a detailed example of such work, we then report on new simulations conducted with the model. Finally, we suggest directions for future cognitive modelling research using this model.

Download

The need for human data when analysing the human-likeness of syntactic representations in neural language models: The case of English wh-island constraints

July 2024

·

2 Reads

Neural language models (NLMs) have frequently been tested on their ability to model grammatical phenomena in a "human-like'' way, but their behavior is often not compared to actual human data, making it unclear whether their behavior is actually human-like. An example of this can be seen with long distance filler-gap dependencies and their island constraints: Wilcox et al. (2018) devised a method to directly test whether NLMs can model these phenomena in a human-like way, and this method is accompanied by specific predictions about how models should perform. These predictions are, however, not all supported by experimental research with human participants, making it difficult to conclude that the NLMs can model these phenomena comparably to humans. Consequently, in the current study, we investigated whether the predictions made by Wilcox et al. (2018) are supported by human data by testing both a Long Short-Term Memory language model and human participants in English with the method devised by Wilcox et al. (2018), and comparing their results. The results showed that the model might exhibit "human-like'' behavior according to the predictions of Wilcox et al. (2018), but this behavior was not fully comparable to that of humans. Therefore, this study not only shows that it is important to obtain human data to validate the predictions of this specific method, but also that predictions of "human-like'' behavior for NLMs on other grammatical phenomena should always be grounded in data from human experiments.


Simulating event-related potentials in bilingual sentence comprehension: syntactic violations and syntactic transfer

June 2024

·

4 Reads

·

1 Citation

Event-related potentials (ERPs) are used to study how lan- guage is processed in the brain, including differences between native (L1) and second-language (L2) processing. A P600 ERP effect can be measured in proficient L2 learners in response to an L2 syntactic violation, indicating native-like processing. Cross-language similarity seems to be a factor that modulates P600 effect size. This manifests in a reduced P600 effect in response to a syntactic violation in the L2 when the syntactic feature involved is expressed differently in two languages. We investigate if this reduced P600 effect can be explained by assuming that ERPs reflect learning signals that arise from mismatches in predictive processing; and in particular that the P600 reflects the error that is back-propagated through the language system (Fitz & Chang, 2019). We use a recurrent neural network model of bilingual sentence processing to simulate the P600 (as back-propagated prediction error) and have it process three types of syntactic constructions differing in cross-language similarity. Simulated English-Spanish participants displayed a P600 when encountering constructions that are similar between the two languages, but a reduced P600 for constructions that differ between languages. This difference between the two P600 responses mirrors what has been observed in human ERP studies. Unlike human participants, simulated participants showed a small P600 response to constructions unique to the L2 (i.e., grammatical gender), presum- ably because of how this grammatical feature is encoded in the model. Our modelling results shed further light on the viability of error propagation as an account of ERPs, and on the effects of syntactic transfer from L1 to L2.


Communicative Efficiency in Multimodal Language Directed at Children and Adults

June 2024

·

59 Reads

Journal of Experimental Psychology General

The ecology of human communication is face to face. In these contexts, speakers dynamically modify their communication across vocal (e.g., speaking rate) and gestural (e.g., cospeech gestures related in meaning to the content of speech) channels while speaking. What is the function of these adjustments? Here we ask whether speakers dynamically make these adjustments to increase communicative success, and decrease cognitive effort while speaking. We assess whether speakers modulate word durations and produce iconic (i.e., imagistically evoking properties of referents) gestures depending on the predictability of each word they utter. Predictability is operationalized as surprisal and computed from computational language models trained on corpora of child-directed, or adult-directed language. Using data from a novel corpus (Ecological Language Corpus) of naturalistic interactions between adult–child (aged 3–4), and adult–adult, we show that surprisal predicts speakers’ multimodal adjustments and that some of these effects are modulated by whether the comprehender is a child or an adult. Thus, communicative efficiency applies generally across vocal and gestural communicative channels not being limited to structural properties of language or vocal modality.


BLiMP-NL: A corpus of Dutch minimal pairs and grammaticality judgements for language model evaluation

April 2024

·

5 Reads

·

1 Citation

·

Zoë Prins

·

·

[...]

·

Stefan L. Frank

We present a corpus of Dutch 8400 sentence pairs intended for the grammatical evaluation of language models. Each pair has a grammatical sentence and a minimally different ungrammatical sentence. The corpus covers 84 paradigms, classified into 22 syntactic phenomena. Nine sentences of each paradigm were rated for acceptability by at least 30 participants, while self-paced reading time on each word was also recorded. We report on the grammaticality effects on acceptability ratings and reading times, as well as the extent to which language models' predictions match both the ground-truth grammaticality and human ratings.


Topographically plotted fixation-related potentials (average voltage time-locked to first fixation on each word) after ocular artifact correction. Shaded areas are 95% confidence intervals
Topographically plotted regression coefficients of surprisal, time-locked to first fixation on each word. Shaded areas are standard errors
Topographically plotted fixation-related potentials (average voltage time-locked to first fixation on each word) before (red) and after (blue) ocular artifact correction. Note the large y-axis scaling differences between channels. (Color figure online)
An eye-tracking-with-EEG coregistration corpus of narrative sentences

August 2023

·

64 Reads

·

4 Citations

Language Resources and Evaluation

We present the Radboud Coregistration Corpus of Narrative Sentences (RaCCooNS), the first freely available corpus of eye-tracking-with-EEG data collected while participants read narrative sentences in Dutch. The corpus is intended for studying human sentence comprehension and for evaluating the cognitive validity of computational language models. RaCCooNS contains data from 37 participants (3 of which eye tracking only) reading 200 Dutch sentences each. Less predictable words resulted in significantly longer reading times and larger N400 sizes, replicating well-known surprisal effects in eye tracking and EEG simultaneously. We release the raw eye-tracking data, the preprocessed eye-tracking data at the fixation, word, and trial levels, the raw EEG after merger with eye-tracking data, and the preprocessed EEG data both before and after ICA-based ocular artifact correction.


The Learnability of the Wh-Island Constraint in Dutch by a Long Short-Term Memory Network

June 2023

·

2 Citations

The current study investigates whether a Long Short-Term Memory (LSTM) network can learn the wh-island constraint in Dutch in a way comparable to human native speakers. After establishing with an acceptability judgement task that native speakers demonstrate a clear sensitivity to wh-island violations, the LSTM network was tested on the same sentences. Contrary to the results of the native speakers, the network was not able to recognize wh-islands and to block gap expectancies within them. This suggests that input and the network’s inductive biases alone might not be enough to learn about syntactic island constraints, and that built-in language knowledge or abilities might be necessary.


The Learnability of the Wh-Island Constraint in Dutch by a Long Short-Term Memory Network

May 2023

·

3 Reads

The current study investigates whether a Long Short-Term Memory (LSTM) network can learn the wh-island constraint in Dutch in a way comparable to human native speakers. After establishing with an acceptability judgement task that native speakers demonstrate a clear sensitivity to wh-island violations, the LSTM network was tested on the same sentences. Contrary to the results of the native speakers, the network was not able to recognize wh-islands and to block gap expectancies within them. This suggests that input and the network’s inductive biases alone might not be enough to learn about syntactic island constraints, and that built-in language knowledge or abilities might be necessary.


Masculine generic pronouns as a gender cue in generic statements

December 2022

·

36 Reads

·

3 Citations

Discourse Processes

An eye-tracking experiment was conducted with speakers of Dutch (N = 84, 36 male), a language that falls between grammatical and natural-gender languages. We tested whether a masculine generic pronoun causes a male bias when used in generic statements—that is, in the absence of a specific referent. We tested two types of generic statements by varying conceptual number, hypothesizing that the pronoun zijn “his” was more likely to cause a male bias with a conceptually singular than a conceptually plural antecedent (e.g., Someone (conceptually singular)/Everyone (conceptually plural) with perfect pitch can tune his instrument quickly). We found male participants to exhibit a male bias but with the conceptually singular antecedent only. Female participants showed no signs of a male bias. The results show that the generically intended masculine pronoun zijn “his” leads to a male bias in conceptually singular generic contexts but that this further depends on participant gender.


Citations (60)


... This account supports the viability of error-driven learning as an account of language ERP effects, and specifically of how such effects change over different L2 learning stages. A follow-up study (Verwijmeren et al., 2024) revealed that the size of simulated P600 effects in the L2 also depends on syntactic transfer from the L1, not unlike what has been found in humans (Tokowicz & MacWhinney, 2005). ...

Reference:

The Bilingual Dual-path model: Simulating bilingual production, processing, and development
Simulating event-related potentials in bilingual sentence comprehension: syntactic violations and syntactic transfer
  • Citing Preprint
  • June 2024

... Such studies have revealed behavioral patterns consistent with neural networks representing formal linguistic structures, in cases such as subject-verb agreement (Linzen et al., 2016;Bernardy and Lappin, 2017;Gulordava et al., 2018), filler-gap dependencies (Wilcox et al., 2018(Wilcox et al., , 2023aKobzeva et al., 2023;Suijkerbuijk et al., 2023), and recursive embedding of clauses Wilcox et al., 2019a;Hu et al., 2020), all of which involve highly nontrivial formal structures which statistical models failed to capture in previous work. Example results for subject-verb agreement from GPT-2 are shown in Figure 1: we see that grammatical verb forms are relatively more probable than matched ungrammatical verb forms in all but a few cases (human accuracy in producing the right verb forms in such sentences is 85%; Marvin and Linzen, 2018). 2 The results indicate that the model can represent the non-local structural dependency between the subject of the sentence and the matrix verb. ...

The Learnability of the Wh-Island Constraint in Dutch by a Long Short-Term Memory Network
  • Citing Conference Paper
  • June 2023

... As another example, The Zurich Cognitive Language Processing Corpora, ZuCo (Hollenstein et al., 2018) and ZuCo 2 (Hollenstein, Troendle, Zhang, & Langer, 2020), comprise 12 and 18 native English speakers respectively, who were concomitantly tracked with EEG. Other corpora include the UCL corpus (Frank, Fernandez Monsalve, Thompson, & Vigliocco, 2013) with 43 participants reading sentences from English novels; RaCCooNS (Frank & Aumeistere, 2023) with 37 subjects reading 200 narrative sentences; the CFILT sarcasm dataset (Mishra, Kanojia, & Bhattacharyya, 2016), entailing 7 non-native English speakers reading sarcastic and non-sarcastic sentences; and the CFILT sentiment complexity dataset (Joshi, Mishra, Senthamilselvan, & Bhattacharyya, 2014), where 5 participants rated the sentiment of over 1000 English sentences. Chinese singlesentence corpora involve the Hong Kong Corpus of Chinese Sentence and Passage Reading (Y. ...

An eye-tracking-with-EEG coregistration corpus of narrative sentences

Language Resources and Evaluation

... Word surprisal is the negative logprobability of said word, and it intuitively quantifies its unexpectedness. This measure has been linked to human sentence processing difficulty and is predictive of eye movements (Levy, 2008;Wilcox et al., 2023b;Aurnhammer and Frank, 2018;Boston et al., 2008;Ehrlich and Rayner, 1981;Merkx and Frank, 2021;Oh and Schuler, 2023;Demberg and Keller, 2008;Smith and Levy, 2013;Shain et al., 2020). Entropy, on the other hand, quantifies the degree of uncertainty over possible outcomes (Shannon, 1948), and it has also been shown to correlate with human sentence processing effort (Keller, 2004;Linzen and Jaeger, 2014;Wilcox et al., 2023b;Hale, 2003;Linzen and Jaeger, 2016;Roark et al., 2009). ...

Comparing Gated and Simple Recurrent Neural Network Architectures as Models of Human Sentence Processing

... Our findings are also of possible interest in light of the still-alive debate surrounding generic masculine. Indeed, some studies (e.g., Gygax et al., 2008;Redl et al., 2022) have shown that when a noun is presented in its masculine form, a representation of a male referent is automatically activated. The results of our work are far from attempting to contribute to this issue, although the finding that role nouns in their masculine form are perceived as more neutral, or certainly less marked, compared to the same role nouns in their feminine form, contributes interesting data to the current debate. ...

Masculine generic pronouns as a gender cue in generic statements
  • Citing Article
  • December 2022

Discourse Processes

... Research on the comprehension of causal discourse has provided ample evidence that contingency relations trigger immediate inferences that may also take world knowledge into account (Traxler, Bybee, and Pickering 1997;Canestrelli, Mak, and Sanders 2013;Xiang and Kuperberg 2015;Noordman et al. 2015). However, the precise nature of the inference patterns involved, their interaction with world knowledge, and how they relate to behavioural measures is still understudied. ...

Causal inferences and world knowledge
  • Citing Chapter
  • April 2015

... In this study, we used the wav2vec 2.0 model [50], a recently proposed and pre-trained, self-supervised network for speech representation learning [68] that we fine-tuned using a large dataset of speech audiometry data to perform phonetic-level recognition of words. This is akin to how a human child learns to speak (and recognize words) after listening to, and "being trained" on a vast amount of speech sounds during development (e.g., [69]). More precisely, phoneme awareness in children seems to be a good predictor of reading skills [70]. ...

Neural Network Models of Language Acquisition and Processing
  • Citing Chapter
  • October 2019

... It allows us to propose mechanisms behind language behaviors and demonstrate whether these behaviors can be learned (e.g., Gelderloos, Kamelabad, & Alishahi, 2020;Huebner, Sulem, Fisher, & Roth, 2021;Nikolaus, Alishahi, & Chrupała, 2022). Recently, various studies have successfully used self-supervised artificial neural network models to simulate infant statistical learning, 1 demonstrating autonomous bootstrapping of phonemic and lexical discrimination (Lavechin, de Seyssel, Titeux et al., 2022), syllable and word segmentation (Khorrami & Räsänen, 2021), and learning of referential word meanings (Khorrami & Räsänen, 2021;Merkx, Scholten, Frank, Ernestus, & Scharenborg, 2023) from auditory or audiovisual language exposure without a need for strong linguistic priors or other innate biases. In terms of modeling studies, PLE has been previously examined in the context of speech emotion recognition (Vogelsang, Vogelsang, Diamond, & Sinha, 2023) and phonetic learning (Poli, Schatz, Dupoux, & Lavechin, 2024). ...

Modelling Human Word Learning and Recognition Using Visually Grounded Speech

Cognitive Computation

... Moreover, concatenated visio-linguistic embeddings also showed an advantage over text-and vision-only baselines in decoding brain activity related to concrete noun representations (Davis et al., 2019). More recent studies have evaluated representations extracted from fully end-to-end models trained to map between images and corresponding text captions and found that such models show improved performance over text models on sentence representation tasks (Kiela, Conneau, Jabri, & Nickel, 2018) as well as improved alignment to human word similarity and relatedness judgments (Merkx, Frank, & Ernestus, 2022). These findings beg the question of whether such emergent cross-/multimodal representations in end-to-end models are also well-aligned with brain activity involved in human concept processing. ...

Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge
  • Citing Conference Paper
  • January 2022

... However, to the best of our knowledge, the ZuCo dataset is the largest publicly available dataset that features simultaneous eye movement and EEG data recorded in a naturalistic reading setup. One recent addition is the CoCoNUt dataset by Frank and Aumeistere (2022), which contains 200 Dutch sentences with combined EEG and eye-tracking recordings. However, the selection of sentences is not completely natural, as it is guided by sentence length and word frequency. ...

An eye-tracking-with-EEG Coregistration Corpus of Narrative Sentences
  • Citing Preprint
  • June 2022