No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Influence of Globally Ungrammatical Local Syntactic Constraints on Real‐Time Sentence Comprehension: Evidence From the Visual World Paradigm and Reading
Abstract
We investigated the influence of globally ungrammatical local syntactic constraints on sentence comprehension, as well as the corresponding activation of global and local representations. In Experiment 1, participants viewed visual scenes with objects like a carousel and motorbike while hearing sentences with noun phrase (NP) or verb phrase (VP) modifiers like “The girl who likes the man (from London/very much) will ride the carousel.” In both cases, “girl” and “ride” predicted carousel as the direct object; however, the locally coherent combination “the man from London will ride…” in NP cases alternatively predicted motorbike. During “ride,” local constraints, although ruled out by the global constraints, influenced prediction as strongly as global constraints: While motorbike was fixated less than carousel in VP cases, it was fixated as much as carousel in NP cases. In Experiment 2, these local constraints likewise slowed reading times. We discuss implications for theories of sentence processing.
... Agreement relationships also appear to inform comprehenders' predictions about upcoming sentence constituents (Brouwer et al., 2017;Dahan et al., 2000;Dussias et al., 2013;Grüter et al., 2020;Hopp, 2013Hopp, , 2012Hopp & Lemmerth, 2018;Lew-Williams & Fernald, 2010). However, it is unclear whether agreement-based predictions are always fully grammatically constrained or whether they can be affected by other grammatically illicit types of information, as has been observed for other linguistic phenomena (Kamide & Kukona, 2018;Kukona et al., 2011Kukona et al., , 2014Rommers et al., 2013Rommers et al., , 2015. The fallibility of agreement constraints is a useful issue to assess as it may yield insight into the mechanisms underlying predictive processing. ...
... Interestingly, the evidence above concerns mostly lexico-semantic and thematic constraints. Much less is known about conflicts involving syntactic constraints, although one recent visual world study by Kamide and Kukona (2018) supports the possibility that these may also be fallible. In this study, participants viewed scenes which contained several objects while hearing sentences with either a noun phrase (NP) or a verb phrase (VP) modifier: e.g. ...
... lexical retrieval, priming, visual encoding, etc.). It is also unclear whether cases of syntactically unlicensed predictions, such as those observed by Kamide and Kukona (2018), extend to morphosyntactic relationships like agreement, where an ungrammatical agreement configuration is neither locally nor globally licensed. Furthermore, while previous studies have proposed that sentence-level thematic relationships require time to override conflict and fully constrain predictions (Chow et al., 2018;Kukona et al., 2011), it is unknown whether time also affects the predictive use of agreement relationships, which are typically rapidly computed by native speakers during online processing (e.g. ...
Previous research has found that comprehenders sometimes predict information that is grammatically unlicensed by sentence constraints. An open question is why such grammatically unlicensed predictions occur. We examined the possibility that unlicensed predictions arise in situations of information conflict, for instance when comprehenders try to predict upcoming words while simultaneously building dependencies with previously encountered elements in memory. German possessive pronouns are a good testing ground for this hypothesis because they encode two grammatically distinct agreement dependencies: a retrospective one between the possessive and its previously mentioned referent, and a prospective one between the possessive and its following nominal head. In two visual world eye-tracking experiments, we estimated the onset of predictive effects in participants’ fixations. The results showed that the retrospective dependency affected resolution of the prospective dependency by shifting the onset of predictive effects. We attribute this effect to an interaction between predictive and memory retrieval processes.
... Under such processing assumptions (cf. Kurumada & Jaeger, 2015), the comprehension system is predictive due to the noisy transmission of information and unexpected input during comprehension can lead to effects such as local coherence (Tabor et al., 2004;Konieczny et al., 2009;Kamide & Kukona, 2018), misinterpretation due to non-canonical order (Ferreira, 2003) In particular, the contextual reconstruction examples shown in Table 14 seem to be compatible with the recently proposed lossy surprisal theory (Futrell & Levy, 2017;Futrell et al., 2020) that assumes a noisy representation of previously seen input string. 18 Such a noisy representation of the context is then used to make predictions and thereby such predictions could be ungrammatical. ...
... From this perspective, the fallible prediction and maintenance hypothesis can be understood as a special case of the good-enough parsing hypothesis, where the parsing system relies on selective information for prediction when there is increased processing load. Such a prediction mechanism has also been demonstrated in English using a visual-world paradigm where local-coherence was shown to guide anticipation (Kamide & Kukona, 2018). The current work therefore illustrates the overarching preference of the parser to posit simple structures (cf. ...
The role of prediction during sentence comprehension is widely acknowledged to be very critical in SOV languages. Robust clause-fi?nal verbal prediction and its maintenance have been invoked to explain eff?ects such as anti-locality and lack of structural forgetting. At the same time, there is evidence that these languages avoid increased preverbal phrase complexity due to working-memory constraints. Given the critical role of prediction in processing of SOV languages, in this work, we study verbal predictions in Hindi (an SOV language) to investigate its robustness and fallibility using a series of completion studies. Analyses of verbal completions based on grammaticality (grammatical vs ungrammatical) as well as their syntactic property (in terms of verb class) show, as expected, frequent grammatical completions based on effective use of preverbal nouns and case-markers. However, there were also high instances of ungrammatical completions. In particular, consistent errors were made in conditions with 3 animate nouns with unique/similar case-markers. These errors increased in the face of adjuncts of di?ffering complexity following the preverbal nouns. The grammatical and ungrammatical completions show that native speakers of Hindi posit structures with at most 2 verbal heads and 5 core verbal relations, thus highlighting an upper bound to verbal prediction and its maintenance in such con?figurations. A rating study con?firmed that certain errors found in completion tasks can lead to grammatical illusions. Further, a detailed analysis of the completion errors in such cases revealed that the parser ignores the complete preverbal nominal features of the input and instead selectively reconstructs the input based on their frequency in the language to form illicit parses at the expense of globally consistent parses. Together, the results show that while preverbal cues are eff?ectively employed by the parser to make clause ?final structural predictions, the parsing system breaks down when the number of predicted verbs/relations exceeds beyond a certain threshold. In effect, the results suggests that processing in SOV languages is susceptible to center-embeddings similar to that in SVO languages. This highlights the over-arching influence of working-memory constraints during sentence comprehension and thereby on the parser to posit less complex structures.
... For example, the Good Enough language processing framework proposes that nonliteral interpretations 1 result from a failure to revise an initial, shallow, and inaccurate understanding of the utterance (Christianson et al., 2001;Ferreira, 2003). Other theories that would speak to similar phenomena are memory-retrieval interference (Jäger et al., 2017;Lewis et al., 2006;Van Dyke & McElree, 2006), the competition model (Bates & MacWhinney, 1982, 1989MacWhinney, 2022;MacWhinney et al., 1984), and the self-organizing model (Kamide & Kukona, 2018;Kukona et al., 2014;Tabor & Hutchins, 2004). Whereas we believe that the field is in need of thorough theoretical work contrasting the approaches above, we will not attempt to do so in this current project: it is out of its scope. ...
Language comprehension relies on integrating the perceived utterance with prior expectations. Previous investigations of expectations about sentence structure (the structural prior) have found that comprehenders often interpret rare constructions nonliterally. However, this work has mostly relied on analytic languages like English, where word order is the main way to indicate syntactic relations in the sentence. This raises the possibility that the structural prior over word order is not a universal part of the sentence processing toolkit, but rather a tool acquired only by speakers of languages where word order has special importance as the main source of syntactic information in the sentence. Moving away from English to make conclusions about more general cognitive strategies (Blasi et al., 2022), we investigate whether the structural prior over word order is a part of language processing more universally using Hindi and Russian, synthetic languages with flexible word order. We conducted two studies in Hindi (Ns = 50, 57, the latter preregistered) and three studies with the same materials, translated, in Russian (Ns = 50, 100, 100, all preregistered), manipulating plausibility and structural frequency. Structural frequency was manipulated by comparing simple clauses with the canonical word order (subject–object–verb in Hindi, subject–verb–object in Russian) to ones with a noncanonical (low frequency) word order (object–subject–verb in Hindi, object–verb–subject in Russian). We found that noncanonical sentences were interpreted nonliterally more often than canonical sentences, even though we used flexible-word-order languages. We conclude that the structural prior over word order is always evaluated in language processing, regardless of language type.
... Kukona et al. (2011) observed predictive eye movements during "arrest" to both the crook, which was related to the verb and a predictable direct object, and policeman, which was related to the verb but an unpredictable direct object. Thus, participants' predictions were not extinguished by conflicting information (e.g., see also Kamide & Kukona, 2018). Similarly, comprehenders' predictions may not be extinguished by exposure to unexpected sentences (e.g., which conflict with their predictions), which may also account for evidence showing that comprehenders do not adapt. ...
Recent psycholinguistic findings raise fundamental questions about comprehenders’ ability to rationally adapt their predictions during sentence processing. Two mouse cursor tracking experiments (each N = 85) assessed this adaptivity by manipulating the reliability of verb-based semantic cues. In Experiment 1, predictive mouse cursor movements to targets (e.g., bike) versus distractors (e.g., kite) were measured while participants heard equal proportions of nonpredictive (e.g., “spot … the bike”), predictive (e.g., “ride … the bike”), and antipredictive (e.g., “fly … the bike”) sentences. In Experiment 2, participants heard equal proportions of nonpredictive and antipredictive sentences. Participants were observed to flexibly adapt their predictions, such that they disengaged prediction in Experiment 1 when verb-based cues were unreliable and as likely to be disconfirmed as confirmed, while they generated adapted predictions in Experiment 2 when verb-based cues were reliably disconfirmed. However, links to individual differences in cognitive control were not observed. These results are interpreted as supporting rational theoretical approaches.
... Experimental sentences were segmented into acoustic regions of interest for F0, duration, and intensity analyses to identify prosodic differences between conditions. Each stimulus was cross-spliced in the following way (e.g., Degen & Tanenhaus, 2016;Kamide & Kukona, 2018). As can be seen in Fig. 1, sentence structures in Condition A and C were subsequently spliced from the original recording to a recording of the sentence in which complement clauses were replaced by experimental elements. ...
... Rather, prediction-by-association readily accommodates effects stemming from lexical associations, without requiring the involvement of mechanisms like predictionby-production. Thus, a key issue for future research will be controlling lexical associations in order to assess the prediction of form beyond association. As an example, Kamide and Kukona (2018) investigated the preactivation of semantic information using the visual world paradigm; critically, they held critical lexical content constant and manipulated the grammatical structure of sentences. As another example, Otten, Nieuwland, and Van Berkum (2007) investigated the preactivation of syntactic information using ERP; critically, they compared predictive versus control discourse contexts with overlapping lexical primes (see also Otten & Van Berkum, 2008). ...
Two visual world experiments investigated the priming of form (e.g., phonology) during language processing. In Experiment 1, participants heard high cloze probability sentences like "In order to have a closer look, the dentist asked the man to open his . . ." while viewing visual arrays with objects like a predictable target mouth, phonological competitor mouse, and unrelated distractors. In Experiment 2, participants heard target-associated nouns like "dentist" that were isolated from the sentences in Experiment 1 while viewing the same visual arrays. In both experiments, participants fixated the target (e.g., mouth) most, but also fixated the phonological competitor (e.g., mouse) more than unrelated distractors. Taken together, these results are interpreted as supporting association-based mechanisms in prediction, such that activation spreads across both semantics and form within the mental lexicon (e.g., dentist-mouth-mouse) and likewise primes (i.e., preactivates) the form of upcoming words during sentence processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
... Further work has shown a robust effect of bigram surprisal beyond the predictions of surprisal models taking larger amounts of context into account (Demberg & Keller, 2008a;Fossum & Levy, 2012;Goodkind & Bicknell, 2018a;Mitchell, Lapata, Demberg, & Keller, 2010): these results are to be expected if distant context items are subject to more noise than local context items (an assumption which we will make concrete in Section 5). More generally, a wealth of psycholinguistic evidence has suggested that local contextual information plays a privileged role in language comprehension (Kamide & Kukona, 2018;Tabor, Galantucci, & Richardson, 2004). We leave it to future work to investigate whether these more complex local context effects can be modeled using lossy-context surprisal. ...
A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context. Models that explain and predict this difficulty can be broadly divided into two kinds, expectation‐based and memory‐based. In this work, we present a new model of incremental sentence processing difficulty that unifies and extends key features of both kinds of models. Our model, lossy‐context surprisal, holds that the processing difficulty at a word in context is proportional to the surprisal of the word given a lossy memory representation of the context—that is, a memory representation that does not contain complete information about previous words. We show that this model provides an intuitive explanation for an outstanding puzzle involving interactions of memory and expectations: language‐dependent structural forgetting, where the effects of memory on sentence processing appear to be moderated by language statistics. Furthermore, we demonstrate that dependency locality effects, a signature prediction of memory‐based theories, can be derived from lossy‐context surprisal as a special case of a novel, more general principle called information locality.
The process of reading has attracted decades of scientific research. Work in this field primarily focuses on using eye gaze patterns to reveal cognitive processes while reading. However, eye gaze patterns suffer from limited resolution, jitter noise, and cognitive biases, resulting in limited accuracy in tracking cognitive reading states. Moreover, using sequential eye gaze data alone neglects the linguistic structure of text, undermining attempts to provide semantic explanations for cognitive states during reading. Motivated by the impact of the semantic context of text on the human cognitive reading process, this work uses both the semantic context of text and visual attention during reading to more accurately predict the temporal sequence of cognitive states. To this end, we present a Cognition-Aware Smart Eyewear System (CASES), which fuses semantic context and visual attention patterns during reading. The two feature modalities are time-aligned and fed to a temporal convolutional network based multi-task classification deep model to automatically estimate and further semantically explain the reading state timeseries. CASES is implemented in eyewear and its use does not interrupt the reading process, thus reducing subjective bias. Furthermore, the real-time association between visual and semantic information enables the interactions between visual attention and semantic context to be better interpreted and explained. Ablation studies with 25 subjects demonstrate that CASES improves multi-label reading state estimation accuracy by 20.90% for sentence compared to eye tracking alone. Using CASES, we develop an interactive reading assistance system. Three and a half months of deployment with 13 in-field studies enables several observations relevant to the study of reading. In particular, observed how individual visual history interacts with the semantic context at different text granularities. Furthermore, CASES enables just-in-time intervention when readers encounter processing difficulties, thus promoting self-awareness of the cognitive process involved in reading and helping to develop more effective reading habits.
The effective use of preverbal linguistic cues to make successful clause-final verbal prediction as well as robust maintenance of such predictions has been argued to be a cross-linguistic generalisation for SOV languages such as German and Japanese. In this paper, we show that native speakers of Hindi (an SOV language) falter in forming a clause-final structure in the presence of a centre-embedded relative clause with a non-canonical word order. In particular, the fallibility of the parser is illustrated by the formation of a grammatically illicit locally coherent parse during online processing. Such a parse should not be formed if the grammatically licit matrix clause final structure was being successfully formed. The formation of a locally coherent parse is further illustrated by probing various syntactic dependencies via targeted questions. We show that the parser's susceptibility to form such structures is not driven by top-down processing, rather the effect can only be explained through a bottom-up parsing approach. Further, our investigation suggests that while plausibility is essential, presence of overt agreement features might not be necessary for forming a locally coherent parse in Hindi. These results go against top-down proposals to local coherence such as lossy surprisal and are consistent with the good-enough processing model to comprehension while only partially supporting the SOPARSE account. The work highlights how top-down processing and bottom-up information interact during sentence comprehension in SOV languages – prediction suffers with increased complexity of the preverbal linguistic environment.
Psycholinguistic research spanning a number of decades has produced diverging results with regard to the nature of constraint integration in online sentence processing. For example, evidence that language users anticipatorily fixate likely upcoming referents in advance of evidence in the speech signal supports rapid context integration. By contrast, evidence that language users activate representations that conflict with contextual constraints, or only indirectly satisfy them, supports nonintegration or late integration. Here we report on a self-organizing neural network framework that addresses 1 aspect of constraint integration: the integration of incoming lexical information (i.e., an incoming word) with sentence context information (i.e., from preceding words in an unfolding utterance). In 2 simulations, we show that the framework predicts both classic results concerned with lexical ambiguity resolution (Swinney, 1979; Tanenhaus, Leiman, & Seidenberg, 1979), which suggest late context integration, and results demonstrating anticipatory eye movements (e.g., Altmann & Kamide, 1999), which support rapid context integration. We also report 2 experiments using the visual world paradigm that confirm a new prediction of the framework. Listeners heard sentences like "The boy will eat the white …" while viewing visual displays with objects like a white cake (i.e., a predictable direct object of "eat"), white car (i.e., an object not predicted by "eat," but consistent with "white"), and distractors. In line with our simulation predictions, we found that while listeners fixated white cake most, they also fixated white car more than unrelated distractors in this highly constraining sentence (and visual) context. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
Sentence processing theories typically assume that the input to our language processing mechanisms is an error-free sequence of words. However, this assumption is an oversimplification because noise is present in typical language use (for instance, due to a noisy environment, producer errors, or perceiver errors). A complete theory of human sentence comprehension therefore needs to explain how humans understand language given imperfect input. Indeed, like many cognitive systems, language processing mechanisms may even be "well designed"-in this case for the task of recovering intended meaning from noisy utterances. In particular, comprehension mechanisms may be sensitive to the types of information that an idealized statistical comprehender would be sensitive to. Here, we evaluate four predictions about such a rational (Bayesian) noisy-channel language comprehender in a sentence comprehension task: (i) semantic cues should pull sentence interpretation towards plausible meanings, especially if the wording of the more plausible meaning is close to the observed utterance in terms of the number of edits; (ii) this process should asymmetrically treat insertions and deletions due to the Bayesian "size principle"; such nonliteral interpretation of sentences should (iii) increase with the perceived noise rate of the communicative situation and (iv) decrease if semantically anomalous meanings are more likely to be communicated. These predictions are borne out, strongly suggesting that human language relies on rational statistical inference over a noisy channel.
Linear mixed-effects models (LMEMs) have become increasingly prominent in psycholinguistics and related areas. However, many researchers do not seem to appreciate how random effects structures affect the generalizability of an analysis. Here, we argue that researchers using LMEMs for confirmatory hypothesis testing should minimally adhere to the standards that have been in place for many decades. Through theoretical arguments and Monte Carlo simulation, we show that LMEMs generalize best when they include the maximal random effects structure justified by the design. The generalization performance of LMEMs including data-driven random effects structures strongly depends upon modeling criteria and sample size, yielding reasonable results on moderately-sized samples when conservative criteria are used, but with little or no power advantage over maximal models. Finally, random-intercepts-only LMEMs used on within-subjects and/or within-items data from populations where subjects and/or items vary in their sensitivity to experimental manipulations always generalize worse than separate F1 and F2 tests, and in many cases, even worse than F1 alone. Maximal LMEMs should be the ‘gold standard’ for confirmatory hypothesis testing in psycholinguistics and beyond.
This paper addresses the question whether locally coherent word sequences in spoken sentences interfere with the global interpretation. We present a visual world experiment where three depicted events were presented simultaneously while a sentence was spoken. The results suggest that local syntactic coherences are not only processed at a syntactic level, but they are interpreted on-line.
We present a new database of lexical decision times for English words and nonwords, for which two groups of British participants each responded to 14,365 monosyllabic and disyllabic words and the same number of nonwords for a total duration of 16 h (divided over multiple sessions). This database, called the British Lexicon Project (BLP), fills an important gap between the Dutch Lexicon Project (DLP; Keuleers, Diependaele, & Brysbaert, Frontiers in Language Sciences. Psychology, 1, 174, 2010) and the English Lexicon Project (ELP; Balota et al., 2007), because it applies the repeated measures design of the DLP to the English language. The high correlation between the BLP and ELP data indicates that a high percentage of variance in lexical decision data sets is systematic variance, rather than noise, and that the results of megastudies are rather robust with respect to the selection and presentation of the stimuli. Because of its design, the BLP makes the same analyses possible as the DLP, offering researchers with a new interesting data set of word-processing times for mixed effects analyses and mathematical modeling. The BLP data are available at http://crr.ugent.be/blp and as Electronic Supplementary Materials.
Electronic supplementary material
The online version of this article (doi:10.3758/s13428-011-0118-4) contains supplementary material, which is available to authorized users.
In prevailing approaches to human sentence comprehension, the outcome of the word recognition process is assumed to be a categorical representation with no residual uncertainty. Yet perception is inevitably uncertain, and a system making optimal use of available information might retain this uncertainty and interactively recruit grammatical analysis and subsequent perceptual input to help resolve it. To test for the possibility of such an interaction, we tracked readers' eye movements as they read sentences constructed to vary in (i) whether an early word had near neighbors of a different grammatical category, and (ii) how strongly another word further downstream cohered grammatically with these potential near neighbors. Eye movements indicated that readers maintain uncertain beliefs about previously read word identities, revise these beliefs on the basis of relative grammatical consistency with subsequent input, and use these changing beliefs to guide saccadic behavior in ways consistent with principles of rational probabilistic inference.
Psycholinguists have commonly assumed that as a spoken linguistic message unfolds over time, it is initially structured by
a syntactic processing module that is encapsulated from information provided by other perceptual and cognitive systems. To
test the effects of relevant visual context on the rapid mental processes that accompany spoken language comprehension, eye
movements were recorded with a head-mounted eye-tracking system while subjects followed instructions to manipulate real objects.
Visual context influenced spoken word recognition and mediated syntactic processing, even during the earliest moments of language
processing.
Dynamical, self-organizing models of sentence processing predict "digging-in" effects: The more committed the parser becomes to a wrong syntactic choice, the harder it is to reanalyze. Experiment 1 replicates previous grammaticality judgment studies (F. Ferreira & J. M. Henderson, 1991b, 1993), revealing a deleterious effect of lengthening the ambiguous region of a garden-path sentence. The authors interpret this result as a digging-in effect. Experiment 2 finds a corresponding effect on reading times. Experiment 3 finds that making 2 wrong attachments is worse than making 1. Non-self-organizing models require multiple stipulations to predict both kinds of effects. The authors show that, under an appropriately formulated self-organizing account, both results stem from self-reinforcement of node and link activations, a feature that is needed independently. An implemented model is given.
Readers' eye movements were monitored as they read sentences in which two noun phrases or two independent clauses were connected by the word or (NP-coordination and S-coordination, respectively). The word either could be present or absent earlier in the sentence. When either was present, the material immediately following or was read more quickly, across both sentence types. In addition, there was evidence that readers misanalyzed the S-coordination structure as an NP-coordination structure only when either was absent. The authors interpret the results as indicating that the word either enabled readers to predict the arrival of a coordination structure; this predictive activation facilitated processing of this structure when it ultimately arrived, and in the case of S-coordination sentences, enabled readers to avoid the incorrect NP-coordination analysis. The authors argue that these results support parsing theories according to which the parser can build predictable syntactic structure before encountering the corresponding lexical input.
Evidence from 3 experiments reveals interference effects from structural relationships that are inconsistent with any grammatical parse of the perceived input. Processing disruption was observed when items occurring between a head and a dependent overlapped with either (or both) syntactic or semantic features of the dependent. Effects of syntactic interference occur in the earliest online measures in the region where the retrieval of a long-distance dependent occurs. Semantic interference effects occur in later online measures at the end of the sentence. Both effects endure in offline comprehension measures, suggesting that interfering items participate in incorrect interpretations that resist reanalysis. The data are discussed in terms of a cue-based retrieval account of parsing, which reconciles the fact that the parser must violate the grammar in order for these interference effects to occur. Broader implications of this research indicate a need for a precise specification of the interface between the parsing mechanism and the memory system that supports language comprehension.
Three eye-tracking experiments using the ‘visual-world’ paradigm are described that explore the basis by which thematic dependencies can be evaluated in advance of linguistic input that unambiguously signals those dependencies. Following Altmann and Kamide (1999), who found that selectional information conveyed by a verb can be used to anticipate an upcoming Theme, we attempt to draw here a more precise picture of the basis for such anticipatory processing. Our data from two studies in English and one in Japanese suggest that (a) verb-based information is not limited to anticipating the immediately following (grammatical) object, but can also anticipate later occurring objects (e.g., Goals), (b) in combination with information conveyed by the verb, a pre-verbal argument (Agent) can constrain the anticipation of a subsequent Theme, and (c) in a head-final construction such as that typically found in Japanese, both syntactic and semantic constraints extracted from pre-verbal arguments can enable the anticipation, in effect, of a further forthcoming argument in the absence of their head (the verb). We suggest that such processing is the hallmark of an incremental processor that is able to draw on different sources of information (some non-linguistic) at the earliest possible opportunity to establish the fullest possible interpretation of the input at each moment in time.
Syntactic parsing processes establish dependencies between words in a sentence. These dependencies affect how comprehenders assign meaning to sentence constituents. Classical approaches to parsing describe it entirely as a bottom-up signal analysis. More recent approaches assign the comprehender a more active role, allowing the comprehender's individual experience, knowledge, and beliefs to influence his or her interpretation. This review describes developments in three related aspects of sentence processing research: anticipatory processing, Bayesian/noisy-channel approaches to sentence processing, and the ‘good-enough’ parsing hypothesis.
Language comprehension, as with all other cases of the extraction of meaningful struc- ture from perceptual input, takes places un- der noisy conditions. If human language comprehension is a rational process in the sense of making use of all available infor- mation sources, then we might expect uncer- tainty at the level of word-level input to af- fect sentence-level comprehension. However, nearly all contemporary models of sentence comprehension assume clean input—that is, that the input to the sentence-level com- prehension mechanism is a perfectly-formed, completely certain sequence of input tokens (words). This article presents a simple model of rational human sentence comprehension under noisy input, and uses the model to in- vestigate some outstanding problems in the psycholinguistic literature for theories of ra- tional human sentence comprehension. We argue that by explicitly accounting for input- level noise in sentence processing, our model provides solutions for these outstanding prob- lems and broadens the scope of theories of hu- man sentence comprehension as rational prob- abilistic inference.
Ferreira and colleagues argued that the language comprehension system creates syntactic and semantic representations that are merely ‘good enough’ (GE) given the task that the comprehender needs to perform. GE representations contrast with ones that are detailed, complete, and accurate with respect to the input. In this article, we review the original argument for GE processing, and we present new evidence that supports the concept: first, local interpretations are computed, which can interfere with global ones; second, new findings based on the recording of event-related potentials show the use of simple heuristics rather than compositional algorithms for constructing sentence meaning; and recent studies show that the comprehension system has mechanisms for handling disfluencies, but they work imperfectly. We argue that the GE approach to language comprehension is similar to the use of fast and frugal heuristics for decision-making, and that future research should explore this connection more thoroughly.
This paper investigates how people resolve syntactic category ambiguities when comprehending sentences. It is proposed that people combine: (a) context-dependent syntactic expectations (top–down statistical information) and (b) context-independent lexical-category frequencies of words (bottom–up statistical information) in order to resolve ambiguities in the lexical categories of words. Three self-paced reading experiments were conducted involving the ambiguous word “that” in different syntactic environments in order to test these and other hypotheses. The data support the top–down/bottom–up approach in which the relative frequencies of lexical entries for a word are tabulated independent of context. Data from other experiments from the literature are discussed with respect to the model proposed here.
Participants’ eye movements were recorded as they inspected a semi-realistic visual scene showing a boy, a cake, and various distractor objects. Whilst viewing this scene, they heard sentences such as ‘the boy will move the cake’ or ‘the boy will eat the cake’. The cake was the only edible object portrayed in the scene. In each of two experiments, the onset of saccadic eye movements to the target object (the cake) was significantly later in the move condition than in the eat condition; saccades to the target were launched after the onset of the spoken word cake in the move condition, but before its onset in the eat condition. The results suggest that information at the verb can be used to restrict the domain within the context to which subsequent reference will be made by the (as yet unencountered) post-verbal grammatical object. The data support a hypothesis in which sentence processing is driven by the predictive relationships between verbs, their syntactic arguments, and the real-world contexts in which they occur.
A central question for psycholinguistics concerns the role of grammatical constraints in online sentence processing. Many current theories maintain that the language processing mechanism constructs a parse or parses that are grammatically consistent with the whole of the perceived input each time it processes a word. Several bottom-up, dynamical models make a contrasting prediction: partial parses which are syntactically compatible with only a proper subpart of the input are sometimes constructed, at least temporarily. Three self-paced reading experiments probed for interference from such locally coherent structures. The first tested for a distracting effect of irrelevant Subject–Predicate interpretations of Noun Phrase–Verb Phrase sequences (e.g., The coach smiled at) on reading times. The second addressed the question of whether the interference effects can be treated as lexical interference, instead of involving the formation of locally coherent syntactic structures. The third replicated the reading time effects of the first two experiments with grammaticality judgments. We evaluate the dynamical account, comparing it to other approaches that also predict effects of local coherence, and arguing against accounts which rule out the formation of merely locally coherent structures.
Three experiments addressed the question whether semantic content or pragmatic context can direct the initial syntactic analysis assigned to sentences. Each experiment determined whether syntactic processing biases that have been observed in sentences presented in isolation can be overcome. In two experiments that measured eye movements, we found that the syntactic processing biases remained even when they resulted in thematically based anomaly or when they conflicted with discourse biases. In a third experiment, we used a self-paced reading task to replicate some of the results obtained using eye movement measures. We argue that the data support the existence of a syntactic processing module.
Human sentence processing involves integrat- ing probabilistic knowledge from a variety of sources in order to incrementally determine the hierarchical structure for the serial input stream. While a large number of sentence pro- cessing effects have been explained in terms of comprehenders' rational use of probabilistic information, effects of local coherences have not. We present here a new model of local coherences, viewing them as resulting from a belief-update process, and show that the rele- vant probabilities in our model are calculable from a probabilistic Earley parser. Finally, we demonstrate empirically that an implemented version of the model makes the correct predic- tions for the materials from the original exper- iment demonstrating local coherence effects.
Several studies have demonstrated that as listeners hear sentences describing events in a scene, their eye movements anticipate upcoming linguistic items predicted by the unfolding relationship between scene and sentence. While this may reflect active prediction based on structural or contextual expectations, the influence of local thematic priming between words has not been fully examined. In Experiment 1, we presented verbs (e.g., arrest) in active (Subject-Verb-Object) sentences with displays containing verb-related patients (e.g., crook) and agents (e.g., policeman). We examined patient and agent fixations following the verb, after the agent role had been filled by another entity, but prior to bottom-up specification of the object. Participants were nearly as likely to fixate agents "anticipatorily" as patients, even though the agent role was already filled. However, the patient advantage suggested simultaneous influences of both local priming and active prediction. In Experiment 2, using passive sentences (Object-Verb-Subject), we found stronger, but still graded influences of role prediction when more time elapsed between verb and target, and more syntactic cues were available. We interpret anticipatory fixations as emerging from constraint-based processes that involve both non-predictive thematic priming and active prediction.
Resolving links between subsequent referents (e.g., the car) and open discourse roles (as in Keith drove to London yesterday. The car kept overheating) is crucial for discourse understanding. This article investigates the contribution of lexical semantic factors (e.g., that drive implies using a vehicle) as compared to more general contextual factors in the on-line resolution of such links. We report an eye-tracking experiment that measures immediate and delayed effects of both kinds of information as readers resolve the reference. The results indicate that lexical information dominates the initial linking process with more general contextual influences emerging later. They are discussed in terms of the distinction between early bonding and subsequent resolution processes that has been proposed for other kinds of anaphoric interpretation (Sanford, Garrod, Lucas, & Henderson, 1983).
Recent studies of eye movements in reading and other information processing tasks, such as music reading, typing, visual search, and scene perception, are reviewed. The major emphasis of the review is on reading as a specific example of cognitive processing. Basic topics discussed with respect to reading are (a) the characteristics of eye movements, (b) the perceptual span, (c) integration of information across saccades, (d) eye movement control, and (e) individual differences (including dyslexia). Similar topics are discussed with respect to the other tasks examined. The basic theme of the review is that eye movement data reflect moment-to-moment cognitive processes in the various tasks examined. Theoretical and practical considerations concerning the use of eye movement data are also discussed.
This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple information-theoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale [Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL (Vol. 2, pp. 159-166)], in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.
In human sentence processing, cognitive load can be defined many ways. This report considers a definition of cognitive load in terms of the total probability of structural options that have been disconfirmed at some point in a sentence: the surprisal of word w i given its prefix w 0...i-1 on a phrase-structural language model. These loads can be e#ciently calculated using a probabilistic Earley parser (Stolcke, 1995) which is interpreted as generating predictions about reading time on a word-by-word basis. Under grammatical assumptions supported by corpusfrequency data, the operation of Stolcke's probabilistic Earley parser correctly predicts processing phenomena associated with garden path structural ambiguity and with the subject/object relative asymmetry.
Proceedings of the 10th Annual Meeting of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL‐HLT) Conference
- K. Bicknell
- R Levy
Attention and performance XII: The psychology of reading
- L Frazier
Frazier, L. (1987). Sentence processing: A tutorial review. In M. Coltheart (Ed.), Attention and performance
XII: The psychology of reading (pp. 559-586). Hillsdale, NJ: Erlbaum.
Proceedings of the 13th Conference on Empirical Methods in Natural Language Processing
- R Levy
Proceedings of the 31th Annual Conference of the Cognitive Science Society
- L. Konieczny
- D. Müller
- W. Hachmann
- S. Schwarzkopf
- S. Wolfer
Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies
- J Hale
Proceedings of the 32nd Annual Conference of the Cognitive Science Society
- L. Konieczny
- H. Weldle
- S. Wolfer
- D. Müller
- P. Baumann
Proceedings of the 35th Annual Meeting of the Berkeley Linguistics Society
- K. Bicknell
- R. Levy
- V Demberg
- Y Kamide
- A Kukona
Y. Kamide, A. Kukona / Cognitive Science 42 (2018)