No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Abstract
Predictive processing in the parietal, temporal, frontal, and sensory cortex allows us to anticipate future meanings to maximize the efficiency of language comprehension, with the temporoparietal junction (TPJ) and inferior frontal gyrus (IFG) thought to be situated toward the top of a predictive hierarchy. Although the regions underpinning this fundamental brain function are well-documented, it remains unclear how they interact to achieve efficient comprehension. To this end, we recorded functional magnetic resonance imaging (fMRI) in 22 participants (11 males) while they comprehended sentences presented part by part, in which we manipulated the constraint provided by sentential contexts on upcoming semantic information. Using this paradigm, we examined the connectivity patterns of bilateral TPJ and IFG during anticipatory phases (i.e., before the onset of targets) and integration phases (i.e., after the onset of targets). When upcoming semantic content was highly predictable in strong constraint contexts, both the left TPJ and bilateral IFG showed stronger visual coupling, while the right TPJ showed stronger connectivity with regions within control, default mode, and visual networks, including the IFG, parahippocampal gyrus, posterior cingulate, and fusiform gyrus. These connectivity patterns were weaker when predicted semantic content appeared, in line with predictive coding theory. Conversely, for less-predictable content, these connectivity patterns were stronger during the integration phase. Overall, these results suggest that both top-down semantic prediction and bottom-up integration during predictive processing are supported by flexible coupling of frontoparietal regions with control, memory, and sensory systems.
ResearchGate has not been able to resolve any citations for this publication.
Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition.
Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.
Control processes allow us to constrain the retrieval of semantic information from long-term memory so that it is appropriate for the task or context. Control demands are influenced by the strength of the target information itself and by the circumstances in which it is retrieved, with more control needed when relatively weak aspects of knowledge are required and after the sustained retrieval of related concepts. To investigate the neurocognitive basis of individual differences in these aspects of semantic control, we used resting-state fMRI to characterise the intrinsic connectivity of left ventrolateral prefrontal cortex (VLPFC), implicated in controlled retrieval, and examined associations on a paced serial semantic task, in which participants were asked to detect category members amongst distractors. This task manipulated both the strength of target associations and the requirement to sustain retrieval within a narrow semantic category over time. We found that individuals with stronger connectivity between VLPFC and medial prefrontal cortex within the default mode network (DMN) showed better retrieval of strong associations (which are thought to be recalled more automatically). Stronger connectivity between the same VLPFC seed and another DMN region in medial parietal cortex was associated with larger declines in retrieval over the course of the category. In contrast, participants with stronger connectivity between VLPFC and cognitive control regions within the ventral attention network (VAN) had better controlled retrieval of weak associations and were better able to sustain their comprehension throughout the category. These effects overlapped in left insular cortex within the VAN, indicating that a common pattern of connectivity is associated with different aspects of controlled semantic retrieval induced by both the structure of long-term knowledge and the sustained retrieval of related information.
Although prediction plays an important role in language comprehension, its precise neural basis remains unclear. This fMRI study investigated whether and how semantic-category-specific and common cerebral areas are recruited in predictive semantic processing during sentence comprehension. We manipulated the semantic constraint of sentence contexts, upon which a tool-related, a building-related, or no specific category of noun is highly predictable. This noun-predictability effect was measured not only over the target nouns but also over their preceding transitive verbs. Both before and after the appearance of target nouns, left anterior supramarginal gyrus was specifically activated for tool-related nouns and left parahippocampal place area was activated specifically for building-related nouns. The semantic-category common areas included a subset of left inferior frontal gyrus during the anticipation of incoming target nouns (activity enhancement for high predictability) and included a wide spread of areas (bilateral inferior frontal gyrus, left superior/middle temporal gyrus, left medial pFC, and left TPJ) during the integration of actually perceived nouns (activity reduction for high predictability). These results indicated that the human brain recruits fine divisions of cortical areas to distinguish different semantic categories of predicted words, and anticipatory semantic processing relies, at least partially, on top–down prediction conducted in higher-level cortical areas.
Coordinating among the demands of the external environment and internal plans requires cognitive control supported by a fronto-parietal control network (FPCN). Evidence suggests that multiple control systems span the FPCN whose operations are poorly understood. Previously (Nee and D'Esposito, 2016; 2017), we detailed frontal dynamics that support control processing, but left open their role in broader cortical function. Here, I show that the FPCN consists of an external/present-oriented to internal/future-oriented cortical gradient extending outwardly from sensory-motor cortices. Areas at the ends of this gradient act in a segregative manner, exciting areas at the same level, but suppressing areas at different levels. By contrast, areas in the middle of the gradient excite areas at all levels, promoting integration of control processing. Individual differences in integrative dynamics predict higher-level cognitive ability and amenability to neuromodulation. These data suggest that an intermediary zone within the FPCN underlies integrative processing that supports cognitive control.
Semantic control, the ability to selectively access and manipulate meaningful information on the basis of context demands, is a critical component of semantic cognition. The precise neural correlates of semantic control are disputed, with particular debate surrounding parietal involvement, the spatial extent of the posterior temporal contribution and network lateralisation. Here semantic control is revisited, utilising improved analysis techniques and a decade of additional data to refine our understanding of the network. A meta-analysis of 925 peaks over 126 contrasts illuminated a left-focused network consisting of inferior frontal gyrus, posterior middle temporal gyrus, posterior inferior temporal gyrus and dorsomedial prefrontal cortex. This extended the temporal region implicated, and found no parietal involvement. Although left-lateralised overall, relative lateralisation varied across the implicated regions. Supporting analyses confirmed the multimodal nature of the semantic control network and situated it within the wider set of regions implicated in semantic cognition.
Semantic retrieval is flexible, allowing us to focus on subsets of features and associations that are relevant to the current task or context: for example, we use taxonomic relations to locate items in the supermarket (carrots are a vegetable), but thematic associations to decide which tools we need when cooking (carrot goes with peeler). We used fMRI to investigate the neural basis of this form of semantic flexibility; in particular, we asked how retrieval unfolds differently when participants have advanced knowledge of the type of link to retrieve between concepts (taxonomic or thematic). Participants performed a semantic relatedness judgement task: on half the trials, they were cued to search for a taxonomic or thematic link, while on the remaining trials, they judged relatedness without knowing which type of semantic relationship would be relevant. Left inferior frontal gyrus showed greater activation when participants knew the trial type in advance. An overlapping region showed a stronger response when the semantic relationship between the items was weaker, suggesting this structure supports both top-down and bottom-up forms of semantic control. Multivariate pattern analysis further revealed that the neural response in left inferior frontal gyrus reflects goal information related to different conceptual relationships. Top-down control specifically modulated the response in visual cortex: when the goal was unknown, there was greater deactivation to the first word, and greater activation to the second word. We conclude that top-down control of semantic retrieval is primarily achieved through the gating of task-relevant ‘spoke’ regions.
Recent accounts of large-scale cortical organisation suggest that the default mode network (DMN) is positioned at the top of a principal gradient, reflecting the separation between heteromodal and unimodal sensory-motor regions in patterns of connectivity and in geodesic distance along the cortical surface (Margulies et al., 2016). This isolation of DMN from external inputs might allow the integration of disparate sources of information that can constrain subsequent cognition. We tested this hypothesis by manipulating the degree to which semantic decisions for ambiguous words (e.g. jam) were constrained by preceding visual cues depicting relevant spatial contexts (e.g. supermarket or road) and/or facial emotions (e.g. happy vs. frustrated). We contrasted (i) the effects of a single preceding cue with a no-cue condition employing scrambled images, and (ii) convergent spatial and emotion cues with single cues. Single cues elicited stronger activation in the multiple demand network relative to no cues, consistent with the requirement to maintain information in working memory. The availability of two convergent cues elicited stronger activation within DMN regions (bilateral angular gyrus, middle temporal gyrus, medial prefrontal cortex, and posterior cingulate), even though behavioural performance was unchanged by cueing – consequently task difficulty is unlikely to account for the observed differences in brain activation. A regions-of-interest analysis along the unimodal-to-heteromodal principal gradient revealed maximal activation for the convergent cue condition at the heteromodal end, corresponding to the DMN. Our findings are consistent with the view that regions of DMN support states of information integration that constrain ongoing cognition and provide a framework for understanding the location of these effects at the heteromodal end of the principal gradient.
Significance
Preterm babies are cared for in neonatal intensive care units (NICU), which are busy places with a lot of mechanical noise increasingly recognized to disrupt normal brain development. NICUs therefore invest in developmental care procedures, with music for example, but neurobiological evidence for these interventions is missing. We present results from a clinical trial to study the effects of a music intervention on preterm infants’ brain development. Based on resting-state fMRI, we provide evidence that music enhanced connectivity in a brain circuitry involving the salience network with regions implicated in sensory and higher-order cognitive functions, previously found to be altered in preterm infants. To our knowledge, this study is unique in observing an impact of music on brain development in preterm newborns.
Humans have a striking capacity to combine words into sentences that express new meanings. Previous research has identified key brain regions involved in this capacity, but little is known about the time course of activity in these regions, as hemodynamic methods such as fMRI provide little insight into temporal dynamics of neural activation. We performed an MEG experiment to elucidate the temporal dynamics of structure and content processing within four brain regions implicated by fMRI data from the same experiment: the temporo‐parietal junction (TPJ), the posterior temporal lobe (PTL), the anterior temporal lobe (ATL), and the anterior inferior frontal gyrus (IFG). The TPJ showed increased activity for both structure and content near the end of the sentence, consistent with a role in incremental interpretation of event semantics. The PTL, a region not often associated with core aspects of syntax, showed a strong early effect of structure, consistent with predictive parsing models, and both structural and semantic context effects on function words. These results provide converging evidence that the PTL plays an important role in lexicalized syntactic processing. The ATL and IFG, regions traditionally associated with syntax, showed minimal effects of sentence structure. The ATL, PTL and IFG all showed effects of semantic content: increased activation for real words relative to nonwords. Our fMRI‐guided MEG investigation therefore helps identify syntactic and semantic aspects of sentence comprehension in the brain in both spatial and temporal dimensions.
Built upon a wealth of neuroimaging, neurostimulation, and neuropsychology data, a recent proposal set forth a framework termed controlled semantic cognition (CSC) to account for how the brain underpins the ability to flexibly use semantic knowledge (Lambon Ralph et al., 2017; Nature Reviews Neuroscience). In CSC, the 'semantic control' system, underpinned predominantly by the prefrontal cortex, dynamically monitors and modulates the 'semantic representation' system that consists of a 'hub' (anterior temporal lobe, ATL) and multiple 'spokes' (modality-specific areas). CSC predicts that unfamiliar and exacting semantic tasks should intensify communication between the 'control' and 'representation' systems, relative to familiar and less taxing tasks. In the present study, we used functional magnetic resonance imaging (fMRI) to test this hypothesis. Participants paired unrelated concepts by canonical colours (a less accustomed task - e.g., pairing ketchup with fire-extinguishers due to both being red) or paired well-related concepts by semantic relationship (a typical task - e.g., ketchup is related to mustard). We found the 'control' system was more engaged by atypical than typical pairing. While both tasks activated the ATL 'hub', colour pairing additionally involved occipitotemporal 'spoke' regions abutting areas of hue perception. Furthermore, we uncovered a gradient along the ventral temporal cortex, transitioning from the caudal 'spoke' zones preferring canonical colour processing to the rostral 'hub' zones preferring semantic relationship. Functional connectivity also differed between the tasks: Compared with semantic pairing, colour pairing relied more upon the inferior frontal gyrus, a key node of the control system, driving enhanced connectivity with occipitotemporal 'spoke'. Together, our findings characterise the interaction within the neural architecture of semantic cognition - the control system dynamically heightens its connectivity with relevant components of the representation system, in response to different semantic contents and difficulty levels.
Research suggests that perception and imagination engage neuronal representations in the same visual areas. However, the underlying mechanisms that differentiate sensory perception from imagination remain unclear. Here, we examine the directed coupling (effective connectivity) between fronto-parietal and visual areas during perception and imagery. We found an increase in bottom-up coupling during perception relative to baseline and an increase in top-down coupling during both perception and imagery, with a much stronger increase during imagery. Modulation of the coupling from frontal to early visual areas was common to both perception and imagery. Furthermore, we show that the experienced vividness during imagery was selectively associated with increases in top-down connectivity to early visual cortex. These results highlight the importance of top-down processing in internally as well as externally driven visual experience.
This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference—and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models. Crucially, these models can entertain both discrete and continuous states, leading to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to elucidate the requisite message passing in terms of its form and scheduling. To accommodate mixed generative models (of discrete and continuous states), one also has to consider link nodes or factors that enable discrete and continuous representations to talk to each other. When mapping the implicit computational architecture onto neuronal connectivity, several interesting features emerge. For example, Bayesian model averaging and comparison, which link discrete and continuous states, may be implemented in thalamocortical loops. These and other considerations speak to a computational connectome that is inherently state dependent and self-organizing in ways that yield to a principled (variational) account. We conclude with simulations of reading that illustrate the implicit neuronal message passing, with a special focus on how discrete (semantic) representations inform, and are informed by, continuous (visual) sampling of the sensorium.
Semantic cognition refers to our ability to use, manipulate and generalize knowledge that is acquired over the lifespan to support innumerable verbal and non-verbal behaviours. This Review summarizes key findings and issues arising from a decade of research into the neurocognitive and neurocomputational underpinnings of this ability, leading to a new framework that we term controlled semantic cognition (CSC). CSC offers solutions to long-standing queries in philosophy and cognitive science, and yields a convergent framework for understanding the neural and computational bases of healthy semantic cognition and its dysfunction in brain disorders.
Making sense of the world around us depends upon selectively retrieving information relevant to our current goal or context. However, it is unclear whether selective semantic retrieval relies exclusively on general control mechanisms recruited in demanding non-semantic tasks, or instead on systems specialised for the control of meaning. One hypothesis is that the left posterior middle temporal gyrus (pMTG) is important in the controlled retrieval of semantic (not non-semantic) information; however this view remains controversial since a parallel literature links this site to event and relational semantics. In a functional neuroimaging study, we demonstrated that an area of pMTG implicated in semantic control by a recent meta-analysis was activated in a conjunction of (i) semantic association over size judgements and (ii) action over colour feature matching. Under these circumstances the same region showed functional coupling with the inferior frontal gyrus – another crucial site for semantic control. Structural and functional connectivity analysis demonstrated that this site is at the nexus of networks recruited in automatic semantic processing (the default mode network) and executively demanding tasks (the multiple-demand network). Moreover, in both task and task-free contexts, pMTG exhibited functional properties that were more similar to ventral parts of inferior frontal cortex, implicated in controlled semantic retrieval, than more dorsal inferior frontal sulcus, implicated in domain-general control. Finally, the pMTG region was functionally correlated at rest with other regions implicated in control-demanding semantic tasks, including inferior frontal gyrus and intraparietal sulcus. We suggest that pMTG may play a crucial role within a large-scale network that allows the integration of automatic retrieval in the default mode network with executively-demanding goal-oriented cognition, and that this could support our ability to understand actions and non-dominant semantic associations, allowing semantic retrieval to be ‘shaped’ to suit a task or context.
Probabilistic prediction plays a crucial role in language comprehension. When predictions are fulfilled, the resulting facilitation allows for fast, efficient processing of ambiguous, rapidly-unfolding input; when predictions are not fulfilled, the resulting error signal allows us to adapt to broader statistical changes in this input. We used functional Magnetic Resonance Imaging to examine the neuroanatomical networks engaged in semantic predictive processing and adaptation. We used a relatedness proportion semantic priming paradigm, in which we manipulated the probability of predictions while holding local semantic context constant. Under conditions of higher (versus lower) predictive validity, we replicate previous observations of reduced activity to semantically predictable words in the left anterior superior/middle temporal cortex, reflecting facilitated processing of targets that are consistent with prior semantic predictions. In addition, under conditions of higher (versus lower) predictive validity we observed significant differences in the effects of semantic relatedness within the left inferior frontal gyrus and the posterior portion of the left superior/middle temporal gyrus. We suggest that together these two regions mediated the suppression of unfulfilled semantic predictions and lexico-semantic processing of unrelated targets that were inconsistent with these predictions. Moreover, under conditions of higher (versus lower) predictive validity, a functional connectivity analysis showed that the left inferior frontal and left posterior superior/middle temporal gyrus were more tightly interconnected with one another, as well as with the left anterior cingulate cortex. The left anterior cingulate cortex was, in turn, more tightly connected to superior lateral frontal cortices and subcortical regions-a network that mediates rapid learning and adaptation and that may have played a role in switching to a more predictive mode of processing in response to the statistical structure of the wider environmental context. Together, these findings highlight close links between the networks mediating semantic prediction, executive function and learning, giving new insights into how our brains are able to flexibly adapt to our environment.
We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher-level representations to predictively pre-activate lower level representations, and whether we “commit” in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioural and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher-level inferences to predictively pre-activate information at multiple lower representational levels. We suggest that the degree and level of predictive pre-activation might be a function of its expected utility, which, in turn, may depend on comprehenders’ goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.
Despite mental imagery's ubiquitous role in human perception, cognition and behavior, one standout question remains unanswered: Why does imagery vary so much from one individual to the next? Here, we used a behavioral paradigm that measures the functional impact of a mental image on subsequent conscious perception and related these measures to the anatomy of the early visual cortex estimated by fMRI retinotopic mapping. We observed a negative relationship between primary visual cortex (V1) surface area and sensory imagery strength, but found positive relationships between V1 and imagery precision (spatial location and orientation). Hence, individuals with a smaller V1 tended to have stronger, but less precise imagery. In addition, subjective vividness of imagery was positively related to prefrontal cortex volume, but unrelated to V1 anatomy. Our findings present the first evidence for the importance of the V1 layout in shaping the strength of human imagination.
© The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
Semantic priming is a crucial phenomenon to study the organization of semantic memory. A novel type of priming effect, integrative priming, has been identified behaviorally, whereby a prime word facilitates recognition of a target word when the 2 concepts can be combined to form a unitary representation. We used both functional and anatomical imaging approaches to investigate the neural substrates supporting such integrative priming, and compare them with those in semantic priming. Similar behavioral priming effects for both semantic (Bread-Cake) and integrative conditions (Cherry-Cake) were observed when compared with an unrelated condition. However, a clearly dissociated brain response was observed between these 2 types of priming. The semantic-priming effect was localized to the posterior superior temporal and middle temporal gyrus. In contrast, the integrative-priming effect localized to the left anterior inferior frontal gyrus and left anterior temporal cortices. Furthermore, fiber tractography showed that the integrative-priming regions were connected via uncinate fasciculus fiber bundle forming an integrative circuit, whereas the semantic-priming regions connected to the posterior frontal cortex via separated pathways. The results point to dissociable neural pathways underlying the 2 distinct types of priming, illuminating the neural circuitry organization of semantic representation and integration.
© The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
The notion of prediction is studied in cognitive neuroscience with increasing intensity. We investigated the neural basis of 2 distinct aspects of word prediction, derived from information theory, during story comprehension. We assessed the effect of entropy of next-word probability distributions as well as surprisal. A computational model determined entropy and surprisal for each word in 3 literary stories. Twenty-four healthy participants listened to the same 3 stories while their brain activation was measured using fMRI. Reversed speech fragments were presented as a control condition. Brain areas sensitive to entropy were left ventral premotor cortex, left middle frontal gyrus, right inferior frontal gyrus, left inferior parietal lobule, and left supplementary motor area. Areas sensitive to surprisal were left inferior temporal sulcus ("visual word form area"), bilateral superior temporal gyrus, right amygdala, bilateral anterior temporal poles, and right inferior frontal sulcus. We conclude that prediction during language comprehension can occur at several levels of processing, including at the level of word form. Our study exemplifies the power of combining computational linguistics with cognitive neuroscience, and additionally underlines the feasibility of studying continuous spoken language materials with fMRI.
© The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
Human thought and language rely on the brain's ability to combine conceptual information. This fundamental process supports the construction of complex concepts from basic constituents. For example, both "jacket" and "plaid" can be represented as individual concepts, but they can also be integrated to form the more complex representation "plaid jacket." Although this process is central to the expression and comprehension of language, little is known about its neural basis. Here we present evidence for a neuroanatomic model of conceptual combination from three experiments. We predicted that the highly integrative region of heteromodal association cortex in the angular gyrus would be critical for conceptual combination, given its anatomic connectivity and its strong association with semantic memory in functional neuroimaging studies. Consistent with this hypothesis, we found that the process of combining concepts to form meaningful representations specifically modulates neural activity in the angular gyrus of healthy adults, independent of the modality of the semantic content integrated. We also found that individual differences in the structure of the angular gyrus in healthy adults are related to variability in behavioral performance on the conceptual combination task. Finally, in a group of patients with neurodegenerative disease, we found that the degree of atrophy in the angular gyrus is specifically related to impaired performance on combinatorial processing. These converging anatomic findings are consistent with a critical role for the angular gyrus in conceptual combination.
Copyright © 2015 the authors 0270-6474/15/353276-09$15.00/0.
The pFC enables the essential human capacities for predicting future events and preadapting to them. These capacities rest on both the structure and dynamics of the human pFC. Structurally, pFC, together with posterior association cortex, is at the highest hierarchical level of cortical organization, harboring neural networks that represent complex goal-directed actions. Dynamically, pFC is at the highest level of the perception–action cycle, the circular processing loop through the cortex that interfaces the organism with the environment in the pursuit of goals. In its predictive and preadaptive roles, pFC supports cognitive functions that are critical for the temporal organization of future behavior, including planning, attentional set, working memory, decision-making, and error monitoring. These functions have a common future perspective and are dynamically intertwined in goal-directed action. They all utilize the same neural infrastructure: a vast array of widely distributed, overlapping, and interactive cortical networks of personal memory and semantic knowledge, named cognits, which are formed by synaptic reinforcement in learning and memory acquisition. From this cortex-wide reservoir of memory and knowledge, pFC generates purposeful, goal-directed actions that are preadapted to predicted future events.
The right temporo-parietal junction (TPJ) is widely considered as part of a network that reorients attention to task-relevant, but currently unattended stimuli (Corbetta and Shulman, 2002). Despite the prevalence of this theory in cognitive neuroscience, there is little direct evidence for the principal hypothesis that TPJ sends an early reorientation signal that "circuit breaks" attentional processing in regions of the dorsal attentional network (e.g., the frontal eye fields) or is completely right lateralized during attentional processing. In this review, we examine both functional neuroimaging work on TPJ in the attentional literature as well as anatomical findings. We first critically evaluate the idea that TPJ reorients attention and is right lateralized; we then suggest that TPJ signals might rather reflect post-perceptual processes involved in contextual updating and adjustments of top-down expectations; and then finally discuss how these ideas relate to the electrophysiological (P300) literature, and to TPJ findings in other cognitive and social domains. We conclude that while much work is needed to define the computational functions of regions encapsulated as TPJ, there is now substantial evidence that it is not specialized for stimulus-driven attentional reorienting.
A neurobiological model of language is discussed that overcomes the shortcomings of the classical Wernicke-Lichtheim-Geschwind model. It is based on a subdivision of language processing into three components: Memory, Unification, and Control. The functional components as well as the neurobiological underpinnings of the model are discussed. In addition, the need for extension of the model beyond the classical core regions for language is shown. The attention network and the network for inferential processing are crucial to realize language comprehension beyond single word processing and beyond decoding propositional content. It is shown that this requires the dynamic interaction between multiple brain regions.
Semantic cognition requires a combination of semantic representations and executive control processes to direct activation in a task- and time-appropriate fashion [Jefferies, E., & Lambon Ralph, M. A. Semantic impairment in stroke aphasia versus semantic dementia: A case-series comparison. Brain, 129, 2132–2147, 2006]. We undertook a formal meta-analysis to investigate which regions within the large-scale semantic network are specifically associated with the executive component of semantic cognition. Previous studies have described in detail the role of left ventral pFC in semantic regulation. We examined 53 studies that contrasted semantic tasks with high > low executive requirements to determine whether cortical regions beyond the left pFC show the same response profile to executive semantic demands. Our findings revealed that right pFC, posterior middle temporal gyrus (pMTG) and dorsal angular gyrus (bordering intraparietal sulcus) were also consistently recruited by executively demanding semantic tasks, demonstrating patterns of activation that were highly similar to the left ventral pFC. These regions overlap with the lesions in aphasic patients who exhibit multimodal semantic impairment because of impaired regulatory control (semantic aphasia)—providing important convergence between functional neuroimaging and neuropsychological studies of semantic cognition. Activation in dorsal angular gyrus and left ventral pFC was consistent across all types of executive semantic manipulation, regardless of whether the task was receptive or expressive, whereas pMTG activation was only observed for manipulation of control demands within receptive tasks. Second, we contrasted executively demanding tasks tapping semantics and phonology. Our findings revealed substantial overlap between the two sets of contrasts within left ventral pFC, suggesting this region underpins domain-general control mechanisms. In contrast, we observed relative specialization for semantic control within pMTG as well as the most ventral aspects of left pFC (BA 47), consistent with our proposal of a distributed network underpinning semantic control.
Interaction with everyday objects requires the representation of conceptual object properties, such as where and how an object is used. What are the neural mechanisms that support this knowledge? While research on semantic dementia has provided evidence for a critical role of the anterior temporal lobes (ATLs) in object knowledge, fMRI studies using univariate analysis have primarily implicated regions outside the ATL. In the present human fMRI study we used multivoxel pattern analysis to test whether activity patterns in ATLs carry information about conceptual object properties. Participants viewed objects that differed on two dimensions: where the object is typically found (in the kitchen or the garage) and how the object is commonly used (with a rotate or a squeeze movement). Anatomical region-of-interest analyses covering the ventral visual stream revealed that information about the location and action dimensions increased from posterior to anterior ventral temporal cortex, peaking in the temporal pole. Whole-brain multivoxel searchlight analysis confirmed these results, revealing highly significant and regionally specific information about the location and action dimensions in the anterior temporal lobes bilaterally. In contrast to conceptual object properties, perceptual and low-level visual properties of the objects were reflected in activity patterns in posterior lateral occipitotemporal cortex and occipital cortex, respectively. These results provide fMRI evidence that object representations in the anterior temporal lobes are abstracted away from perceptual properties, categorizing objects in semantically meaningful groups to support conceptual object knowledge.
The rapid growth of the literature on neuroimaging in humans has led to major advances in our understanding of human brain function but has also made it increasingly difficult to aggregate and synthesize neuroimaging findings. Here we describe and validate an automated brain-mapping framework that uses text-mining, meta-analysis and machine-learning techniques to generate a large database of mappings between neural and cognitive states. We show that our approach can be used to automatically conduct large-scale, high-quality neuroimaging meta-analyses, address long-standing inferential problems in the neuroimaging literature and support accurate 'decoding' of broad cognitive states from brain activity in both entire studies and individual human subjects. Collectively, our results have validated a powerful and generative framework for synthesizing human neuroimaging data on an unprecedented scale.
Information processing in the cerebral cortex involves interactions among distributed areas. Anatomical connectivity suggests that certain areas form local hierarchical relations such as within the visual system. Other connectivity patterns, particularly among association areas, suggest the presence of large-scale circuits without clear hierarchical relations. In this study the organization of networks in the human cerebrum was explored using resting-state functional connectivity MRI. Data from 1,000 subjects were registered using surface-based alignment. A clustering approach was employed to identify and replicate networks of functionally coupled regions across the cerebral cortex. The results revealed local networks confined to sensory and motor cortices as well as distributed networks of association regions. Within the sensory and motor cortices, functional connectivity followed topographic representations across adjacent areas. In association cortex, the connectivity patterns often showed abrupt transitions between network boundaries. Focused analyses were performed to better understand properties of network connectivity. A canonical sensory-motor pathway involving primary visual area, putative middle temporal area complex (MT+), lateral intraparietal area, and frontal eye field was analyzed to explore how interactions might arise within and between networks. Results showed that adjacent regions of the MT+ complex demonstrate differential connectivity consistent with a hierarchical pathway that spans networks. The functional connectivity of parietal and prefrontal association cortices was next explored. Distinct connectivity profiles of neighboring regions suggest they participate in distributed networks that, while showing evidence for interactions, are embedded within largely parallel, interdigitated circuits. We conclude by discussing the organization of these large-scale cerebral networks in relation to monkey anatomy and their potential evolutionary expansion in humans to support cognition.
One of the most consistent observations in human functional imaging is that a network of brain regions referred to as the "default network" increases its activity during passive states. Here we explored the anatomy and function of the default network across three studies to resolve divergent hypotheses about its contributions to spontaneous cognition and active forms of decision making. Analysis of intrinsic activity revealed the network comprises multiple, dissociated components. A midline core (posterior cingulate and anterior medial prefrontal cortex) is active when people make self-relevant, affective decisions. In contrast, a medial temporal lobe subsystem becomes engaged when decisions involve constructing a mental scene based on memory. During certain experimentally directed and spontaneous acts of future-oriented thought, these dissociated components are simultaneously engaged, presumably to facilitate construction of mental models of personally significant events.
The right temporoparietal junction (rTPJ) is a brain area that plays a critical role in a variety of cognitive functions. Although different theoretical proposals tried to explain the ubiquitous role of rTPJ, recent evidence suggests that rTPJ may be a fundamental cortical region involved in different kinds of predictions.
This systematic review aims to better investigate the potential role of rTPJ under a predictive processing perspective, providing an overview of cognitive impairments in neurological patients as the consequence of structural or functional disconnections or damage of rTPJ.
Results
confirm the involvement of rTPJ across several tasks and neurological pathologies. RTPJ, via its connections with other brain networks, would integrate diverse information and update internal models of the world. Against traditional views, which tend to focus on distinct domains, we argue that the role of rTPJ can be parsimoniously interpreted as a key hub involved in domain-general predictions.
This alternative account of rTPJ role in aberrant predictive processing opens different perspectives, stimulating new hypotheses in basic research and clinical contexts.
Context-based predictions facilitate speech processing. However, details of predictive processing mechanisms and how factors like language experience shape facilitative processing remain debated. This electroencephalograph study aimed to shed light on these issues by investigating the effect of dialectal experience on lexical prediction. Stimulus sentences were produced in three Mandarin Chinese dialects (home dialect, familiar regional dialect, and unfamiliar regional dialect). Critical nouns varied between strong and weak predictability. Only when listening to the home dialect was an enhanced ERP deflection observed at the transitive verb before a strongly predictable object noun (compared to the weakly predictable condition); the ERP amplitude at the verb correlated significantly with the following noun’s predictability. The predictable object nouns elicited a reduced N400 in all three dialects but only an additional reduction in N1 in the home dialect condition. Conjointly, our results suggest that effortful meaning computation may partially underlie anticipatory lexical processing and that language experience modulates the way lexical prediction facilitates the early stage of acoustic/phonological processing (in addition to the later stage of meaning integration).
Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.
Introduction:
The ventral attention network (VAN) is an important mediator of stimulus-driven attention. Multiple cortical areas, such as the middle and inferior frontal gyri, anterior insula, inferior parietal lobule, and temporo-parietal junction have been linked in this processing. However, knowledge of network connectivity has been devoid of structural specificity.
Methods:
Using relevant task-based fMRI studies, an activation likelihood estimation (ALE) of the VAN was generated Regions of interest corresponding to the HCP cortical parcellation scheme were co-registered onto this ALE in MNI coordinate space and visually assessed for inclusion in the network. DSI-based fiber tractography was performed to determine the structural connections between cortical areas comprising the VAN.
Results:
Fourteen regions within the right cerebral hemisphere were found to overlap the ALE of the VAN: 6a, 6r, 7AM, 7PM, 8C, AVI, FOP4, MIP, p9-46v, PCV, PFm, PGi, TPOJ1, and TPOJ2. Regions demonstrated consistent U-shaped interconnections between adjacent parcellations, and the SLF was found to connect frontal and parietal areas of the network.
Conclusions:
We present a tractographic model of the VAN. This model comprises parcellations within the frontal and parietal cortices that are linked via the SLF. Future studies may refine this model with the ultimate goal of clinical application.
Predictive coding is an increasingly influential and ambitious concept in neuroscience viewing the brain as a 'hypothesis testing machine' that constantly strives to minimize prediction error, the gap between its predictions and the actual sensory input. Despite the invaluable contribution of this framework to the formulation of brain function, its neuroanatomical foundations have not been fully defined. To address this gap, we conducted activation likelihood estimation (ALE) meta-analysis of 39 neuroimaging studies of three functional domains (action perception, language and music) inherently involving prediction. The ALE analysis revealed a widely distributed brain network encompassing regions within the inferior and middle frontal gyri, anterior insula, premotor cortex, pre-supplementary motor area, temporoparietal junction, striatum, thalamus/subthalamus and the cerebellum. This network is proposed to subserve domain-general prediction and its relevance to motor control, attention, implicit learning and social cognition is discussed in light of the predictive coding scheme. Better understanding of the presented network may help advance treatments of neuropsychiatric conditions related to aberrant prediction processing and promote cognitive enhancement in healthy individuals.
The ability to make predictions is thought to facilitate language processing. During language comprehension such predictions appear to occur at multiple levels of linguistic representations (i.e. semantic, syntactic and lexical). The neural mechanisms that define the network sensitive to linguistic predictability have yet to be adequately defined. The purpose of the present study was to explore the neural network underlying predictability during the normal reading of connected text. Predictability values for different linguistic information were obtained from a pre-existing text corpus. Forty-one subjects underwent simultaneous eye-tracking and fMRI scans while reading these select paragraphs. Lexical, semantic, and syntactic predictability measures were then correlated with functional activation associated with fixation onset on the individual words. Activation patterns showed both positive and negative correlations to lexical, semantic, and syntactic predictabilities. Conjunction analysis revealed regions specific to or shared between each type of predictability. The regions associated with the different predictability measures were largely separate. Results suggest that most linguistic predictions are graded in nature, activating components of the existing language system. A number of regions were also found to be uniquely associated with full lexical predictability, most notably the anterior temporal lobe and the inferior posterior temporal cortex.
Neuroimaging studies have found that theory of mind (ToM) and discourse comprehension involve similar brain regions. These brain regions may be associated with three cognitive components that are necessarily or frequently involved in ToM and discourse comprehension, including social concept representation and retrieval, domain-general semantic integration, and domain-specific integration of social semantic contents. Using fMRI, we investigated the neural correlates of these three cognitive components by exploring how discourse topic (social/nonsocial) and discourse processing period (ending/beginning) modulate brain activation in a discourse comprehension (and also ToM) task. Different sets of brain areas showed sensitivity to discourse topic, discourse processing period, and the interaction between them, respectively. The most novel finding was that the right temporoparietal junction and middle temporal gyrus showed sensitivity to discourse processing period only during social discourse comprehension, indicating that they selectively contribute to domain-specific semantic integration. Our finding indicates how different domains of semantic information are processed and integrated in the brain and provides new insights into the neural correlates of ToM and discourse comprehension.
Linguistic expressions consist of sequences of words combined together to form phrases and sentences. The neurocognitive process handling word combination is drawing increasing attention among the neuroscientific community, given that the underlying syntactic and semantic mechanisms of such basic combinations-although essential to the generation of more complex structures-still need to be consistently determined. The current experiment was conducted to disentangle the neural networks supporting syntactic and semantic processing at the level of two-word combinations. We manipulated the combinatorial load by using words of different grammatical classes within the phrase, such that determiner-noun combinations (this ship) were used to boost neural activity in syntax-related areas, while adjective-noun combinations (blue ship) were conversely used to measure neural response in semantic-related combinations. By means of functional magnetic resonance imaging (fMRI), we found that syntax-related processing mainly activates the most ventral part of the inferior frontal gyrus (IFG), along the frontal operculum (FOP) and anterior insula (aINS). Fine-grained analysis in BA44 confirmed that the most inferior-ventral portion is highly sensitive to syntactic computations driven by function words. Semantic-related processing on the contrary, rather engages the anterior dorsal part of the left IFG and the left angular gyrus (AG) that is two regions which appear to perform different functions within the semantic network. Our findings suggest that syntactic and semantic contribution to phrasal formation can be already differentiated at a very basic level, with each of these two processes comprising non-overlapping areas on the cerebral cortex. Specifically, they confirm the role of the ventral IFG for the construction of syntactically legal linguistic constructions, and the prominence of the more anterior IFG and the AG for conceptual semantics.
Although numerous studies have demonstrated that the language processing system can predict upcoming content during comprehension, there is still no clear picture of the anticipatory stage of predictive processing. This electroencephalograph study examined the cognitive and neural oscillatory mechanisms underlying anticipatory processing during language comprehension, and the consequences of this prediction for bottom-up processing of predicted/unpredicted content. Participants read Mandarin Chinese sentences that were either strongly or weakly constraining and that contained critical nouns that were congruent or incongruent with the sentence contexts. We examined the effects of semantic predictability on anticipatory processing prior to the onset of the critical nouns and on integration of the critical nouns. The results revealed that, at the integration stage, the strong-constraint condition (compared to the weak-constraint condition) elicited a reduced N400 and reduced theta activity (4–7 Hz) for the congruent nouns, but induced beta (13–18 Hz) and theta (4–7 Hz) power decreases for the incongruent nouns, indicating benefits of confirmed predictions and potential costs of disconfirmed predictions. More importantly, at the anticipatory stage, the strongly constraining context elicited an enhanced sustained anterior negativity and beta power decrease (19–25 Hz), which indicates that strong prediction places a higher processing load on the anticipatory stage of processing. The differences (in the ease of processing and the underlying neural oscillatory activities) between anticipatory and integration stages of lexical processing were discussed with regard to predictive processing models.
Significance
We describe an overarching organization of large-scale connectivity that situates the default-mode network at the opposite end of a spectrum from primary sensory and motor regions. This topography, based on the differentiation of connectivity patterns, is also embedded in the spatial distance along the cortical surface between these respective systems. In addition, this connectivity gradient accounts for the respective positions of canonical networks and captures a functional spectrum from perception and action to more abstract cognitive functions. These results suggest that the default-mode network consists of regions at the top of a representational hierarchy that describe the current cognitive landscape in the most abstract terms.
DOWNLOAD FULL-TEXT: http://authors.elsevier.com/a/1RCQd2VHXflCB
It is widely agreed upon that linguistic predictions are an integral part of language comprehension. Yet, experimental proof of their existence remains challenging. Here, we introduce a new predictive eye gaze reading task combining eye tracking and functional magnetic resonance imaging (fMRI) that allows us to infer the existence and timing of linguistic predictions via anticipatory eye-movements. Participants read different types of word sequences (i.e., regular sentences, meaningless jabberwocky sentences, non-word lists) up to the pre-final word. The final target word was displayed with a temporal delay and its screen position was dependent on the syntactic word category (nouns vs verbs). During the delay, anticipatory eye-movements into the correct target word area were indicative of linguistic predictions. For fMRI analysis, the predictive sentence conditions were contrasted to the non-word condition, with the anticipatory eye-movements specifying differences in timing across conditions. A conjunction analysis of both sentence conditions revealed the neural substrate of word category prediction, namely a distributed network of cortical and subcortical brain regions including language systems, basal ganglia, thalamus, and hippocampus. Direct contrasts between the regular sentence condition and the jabberwocky condition indicate that prediction of word category in meaningless jabberwocky sentences relies on classical left-hemispheric language systems involving Brodman's area 44/45 in the left inferior frontal gyrus, left superior temporal areas, and the dorsal caudate nucleus. Regular sentences, in contrast, allowed for the prediction of specific words. Word-specific predictions were specifically associated with more widely distributed temporal and parietal cortical systems, most prominently in the right hemisphere. Our results support the presence of linguistic predictions during sentence processing and demonstrate the validity of the predictive eye gaze paradigm for measuring syntactic and semantic aspects of linguistic predictions, as well as for investigating their neural substrates.
Copyright © 2015 Elsevier Ltd. All rights reserved.
There is a growing literature investigating the relationship between oscillatory neural dynamics measured using electroencephalography (EEG) and/or magnetoencephalography (MEG), and sentence-level language comprehension. Recent proposals have suggested a strong link between predictive coding accounts of the hierarchical flow of information in the brain, and oscillatory neural dynamics in the beta and gamma frequency ranges. We propose that findings relating beta and gamma oscillations to sentence-level language comprehension might be unified under such a predictive coding account. Our suggestion is that oscillatory activity in the beta frequency range may reflect both the active maintenance of the current network configuration responsible for representing the sentence-level meaning under construction, and the top-down propagation of predictions to hierarchically lower processing levels based on that representation. In addition, we suggest that oscillatory activity in the low and middle gamma range reflect the matching of top-down predictions with bottom-up linguistic input, while evoked high gamma might reflect the propagation of bottom-up prediction errors to higher levels of the processing hierarchy. We also discuss some of the implications of this predictive coding framework, and we outline ideas for how these might be tested experimentally.
Copyright © 2015 Elsevier Ltd. All rights reserved.
When listening to speech we not only form predictions about what is coming next, but also when something is coming. For example, metric stress may be utilized to predict the next salient speech event (i.e. the next stressed syllable) and in turn facilitate speech comprehension. However, speech comprehension can also be facilitated by semantic context, that is, which content word is likely to appear next. In the current fMRI experiment we investigated (1) the brain networks that underlie metric and semantic predictions by means of prediction errors (2) how semantic processing is influenced by a metrically regular or irregular sentence context, and (3) whether task demands influence both processes. The results are three-fold: First, while metrically incongruent sentences activated a bilateral fronto-striatal network, semantically incongruent trials led to activation of fronto-temporal areas. Second, metrically regular context facilitated speech comprehension in the left-fronto-temporal language network. Third, attention directed to metric or semantic aspects in speech engaged different subcomponents of the left inferior frontal gyrus (IFG). The current results suggest that speech comprehension relies on different forms of prediction, and extends known speech comprehension networks to subcortical sensorimotor areas.
A growing body of neuroimaging evidence has shown that Chinese character processing recruits differential activation from alphabetic languages due to its unique linguistic features. As more investigations on Chinese character processing have recently become available, we applied a meta-analytic approach to summarize previous findings and examined the neural networks for orthographic, phonological, and semantic processing of Chinese characters independently. The activation likelihood estimation (ALE) method was used to analyze eight studies in the orthographic task category, eleven in the phonological and fifteen in the semantic task categories. Converging activation among three language-processing components was found in the left middle frontal gyrus, the left superior parietal lobule and the left mid-fusiform gyrus, suggesting a common sub-network underlying the character recognition process regardless of the task nature. With increasing task demands, the left inferior parietal lobule and the right superior temporal gyrus were specialized for phonological processing, while the left middle temporal gyrus was involved in semantic processing. Functional dissociation was identified in the left inferior frontal gyrus, with the posterior dorsal part for phonological processing and the anterior ventral part for semantic processing. Moreover, bilateral involvement of the ventral occipito-temporal regions was found for both phonological and semantic processing. The results provide better understanding of the neural networks underlying Chinese orthographic, phonological, and semantic processing, and consolidate the findings of additional recruitment of the left middle frontal gyrus and the right fusiform gyrus for Chinese character processing as compared with the universal language network that has been based on alphabetic languages.
Functional MRI (fMRI) allows one to study task-related regional responses and task-dependent connectivity analysis using psychophysiological interaction (PPI) methods. The latter affords the additional opportunity to understand how brain regions interact in a task-dependent manner. The current implementation of PPI in Statistical Parametric Mapping (SPM8) is configured primarily to assess connectivity differences between two task conditions, when in practice fMRI tasks frequently employ more than two conditions. Here we evaluate how a generalized form of context-dependent PPI (gPPI; http://www.nitrc.org/projects/gppi), which is configured to automatically accommodate more than two task conditions in the same PPI model by spanning the entire experimental space, compares to the standard implementation in SPM8. These comparisons are made using both simulations and an empirical dataset. In the simulated dataset, we compare the interaction beta estimates to their expected values and model fit using the Akaike information criterion (AIC). We found that interaction beta estimates in gPPI were robust to different simulated data models, were not different from the expected beta value, and had better model fits than when using standard PPI (sPPI) methods. In the empirical dataset, we compare the model fit of the gPPI approach to sPPI. We found that the gPPI approach improved model fit compared to sPPI. There were several regions that became non-significant with gPPI. These regions all showed significantly better model fits with gPPI. Also, there were several regions where task-dependent connectivity was only detected using gPPI methods, also with improved model fit. Regions that were detected with all methods had more similar model fits. These results suggest that gPPI may have greater sensitivity and specificity than standard implementation in SPM. This notion is tempered slightly as there is no gold standard; however, data simulations with a known outcome support our conclusions about gPPI. In sum, the generalized form of context-dependent PPI approach has increased flexibility of statistical modeling, and potentially improves model fit, specificity to true negative findings, and sensitivity to true positive findings.
