No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Sparse Representation of Whole-brain FMRI Signals for Identification of Functional Networks
Abstract
There have been several recent studies that used sparse representation for fMRI signal analysis and activation detection based on the assumption that each voxel’s fMRI signal is linearly composed of sparse components. Previous studies have employed sparse coding to model functional networks in various modalities and scales. These prior contributions inspired the exploration of whether/how sparse representation can be used to identify functional networks in a voxel-wise way and on the whole brain scale. This paper presents a novel, alternative methodology of identifying multiple functional networks via sparse representation of whole-brain task-based fMRI signals. Our basic idea is that all fMRI signals within the whole brain of one subject are aggregated into a big data matrix, which is then factorized into an over-complete dictionary basis matrix and a reference weight matrix via an effective online dictionary learning algorithm. Our extensive experimental results have shown that this novel methodology can uncover multiple functional networks that can be well characterized and interpreted in spatial, temporal and frequency domains based on current brain science knowledge. Importantly, these well-characterized functional network components are quite reproducible in different brains. In general, our methods offer a novel, effective and unified solution to multiple fMRI data analysis tasks including activation detection, de-activation detection, and functional network identification.
... For fMRI feature extractor, we used a single fully connected layer to conduct a linear decomposition of fMRI data. A L1 regularization (with a penalty coefficient 5e-6) was applied to introduce the sparsity of the decomposition for the convenience of brain activity pattern visualization [18]. It is noted that we adopted a pre-trained fMRI autoencoder to initialize the fMRI feature extractor. ...
... In BI-AVAN, the fMRI feature extractor is responsible for encoding the input fMRI signal to a feature vector. After the training process, the weight matrix of fMRI feature extractor contains the patterns learned from raw fMRI signals and each pattern can be interpreted as a specific brain network [18]. The weight matrix has the size of 1024×48637, thus each row of the matrix represents a brain network. ...
... To visualize these brain networks, the values in each row are mapped back to the brain volume space and results in 1024 brain networks in total. The original fMRI data can be represented as a linear combination of these 1024 brain networks [18]. ...
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks. However, most of the visual attention studies adopted eye-tracking data rather than the direct measurement of brain activity to characterize human visual attention. In addition, the adversarial relationship between the attention-related objects and attention-neglected background in the human visual system was not fully exploited. To bridge these gaps, we propose a novel brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our BI-AVAN model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner. We use independent eye-tracking data as ground truth for validation and experimental results show that our model achieves robust and promising results when inferring meaningful human visual attention and mapping the relationship between brain activities and visual stimuli. Our BI-AVAN model contributes to the emerging field of leveraging the brain's functional architecture to inspire and guide the model design in artificial intelligence (AI), e.g., deep neural networks.
... While a large number of studies focus on characterizing the concurrent task-evoked brain regions/networks (Calhoun et al., 2001;Lv et al., 2015a;Darnai et al., 2019), the neuroscience of inherent functional differences in composition patterns of multitask fMRI signals has been rarely tapped. Investigating the differences between different task fMRI signals composition could improve better understanding for the organization of the brain's cognitive functioning, and might contribute to disease diagnosis and classification. ...
... One reason causing these disparities is that to find the key features for task classification, the number of dictionary atoms in the second stage of our method was set to 50, resulting in only 50 functional components defined. In contrast, the atom number was usually set to 400 in previous task-based activation identification studies using sparse decomposition method (Lv et al., 2015a;Zhao et al., 2018). In addition, for the classification purpose, our method concatenated and aggregated the tfMRI signals of seven tasks together for the model training, which leads to truncations of some tasks, instead of applying the sparse decomposition method to single and complete task fMRI data alone (Lv et al., 2015a). ...
... In contrast, the atom number was usually set to 400 in previous task-based activation identification studies using sparse decomposition method (Lv et al., 2015a;Zhao et al., 2018). In addition, for the classification purpose, our method concatenated and aggregated the tfMRI signals of seven tasks together for the model training, which leads to truncations of some tasks, instead of applying the sparse decomposition method to single and complete task fMRI data alone (Lv et al., 2015a). The truncation would lead to incomplete task designs (Extended Data Fig. 1-1), thus impacting the characterization of task-related functional activations. ...
Task-based functional magnetic resonance imaging (tfMRI) has been widely used to induce functional brain activities corresponding to various cognitive tasks. A relatively under-explored question is whether there exist fundamental differences in fMRI signal composition patterns that can effectively classify the task states of tfMRI data, furthermore, whether there exist key functional components in characterizing the diverse tfMRI signals. Recently, fMRI signal composition patterns of multiple tasks have been investigated via deep learning models, where relatively large populations of fMRI datasets are indispensable and the neurologic meaning of their results is elusive. Thus, the major challenges arise from the high dimensionality, low signal-to-noise ratio, interindividual variability, a small sample size of fMRI data, and the explainability of classification results. To address the above challenges, we proposed a computational framework based on group-wise hybrid temporal and spatial sparse representations (HTSSR) to identify and differentiate multitask fMRI signal composition patterns. Using relatively small cohorts of Human Connectome Project (HCP) tfMRI data as test-bed, the experimental results demonstrated that the multitask of fMRI data can be successfully classified with an average accuracy of 96.67%, where the key components in differentiating the multitask can be characterized, suggesting the effectiveness and explainability of the proposed method. Moreover, both task-related components and resting-state networks (RSNs) can be reliably detected. Therefore, our study proposed a novel framework that identifies the interpretable and discriminative fMRI composition patterns and can be potentially applied for controlling fMRI data quality and inferring biomarkers in brain disorders with small sample neuroimaging datasets.
... Furthermore, FBNs can't be selected automatically in previous methods, and the number of FBNs needs to be set manually in a heuristic or experiential manner. Another limitation is the assumption of linearity or independence in most of existing methods [17,4,40]. For example, independent component analysis (ICA) [4,43,7] regards the identification of functional brain networks as a problem of blind source separation, that is, it is assumed that the observed signals are composed of the linear superposition of independent blind source signals. ...
... At present, the commonly used methods to construct functional networks based on fMRI data include: general linear model (GLM) [17], independent component analysis (ICA) [4,43,8,7] and sparse dictionary learning (SDL) [40,31,20,21]. Because these methods require less data and are more stable than deep learning methods, they are widely used in clinical practices. ...
Using deep learning models to recognize functional brain networks (FBNs) in functional magnetic resonance imaging (fMRI) has been attracting increasing interest recently. However, most existing work focuses on detecting static FBNs from entire fMRI signals, such as correlation-based functional connectivity. Sliding-window is a widely used strategy to capture the dynamics of FBNs, but it is still limited in representing intrinsic functional interactive dynamics at each time step. And the number of FBNs usually need to be set manually. More over, due to the complexity of dynamic interactions in brain, traditional linear and shallow models are insufficient in identifying complex and spatially overlapped FBNs across each time step. In this paper, we propose a novel Spatial and Channel-wise Attention Autoencoder (SCAAE) for discovering FBNs dynamically. The core idea of SCAAE is to apply attention mechanism to FBNs construction. Specifically, we designed two attention modules: 1) spatial-wise attention (SA) module to discover FBNs in the spatial domain and 2) a channel-wise attention (CA) module to weigh the channels for selecting the FBNs automatically. We evaluated our approach on ADHD200 dataset and our results indicate that the proposed SCAAE method can effectively recover the dynamic changes of the FBNs at each fMRI time step, without using sliding windows. More importantly, our proposed hybrid attention modules (SA and CA) do not enforce assumptions of linearity and independence as previous methods, and thus provide a novel approach to better understanding dynamic functional brain networks.
... It was suggested that a cortical region could be simultaneously involved in many brain functions (Fox et al. 2005;Bullmore and Sporns 2009;Lv et al. 2014Lv et al. , 2015. We adopted the method in (Lv et al. 2014 to quantify such an involvement, which was also referred as "functional integration capacity." ...
... The reasons for the joint use of network atlas and the data-driven network decomposition result are 2-folds: 1) warping an atlas to individual spaces regards less individual variability while the size and boundary of a brain site could be variable across subjects; 2) there is no overlapping between brain sites on a network atlas due to their abrupt boundaries such that each cortical landmark is theoretically allowed to be involved in only one brain site or one network. However, it was suggested that a cortical region could be involved in many brain functions simultaneously (Fox et al. 2005;Bullmore and Sporns 2009;Lv et al. 2014Lv et al. , 2015, and the quantification of such involvement was also referred as "functional integration capacity." ...
Prior studies reported the global structure of brain networks exhibits the “small-world” and “rich-world” attributes. However, the underlying structural and functional architecture highlighted by these graph theory findings hasn’t been explicitly related to the morphology of the cortex. This could be attributed to the lower resolution of used folding patterns, such as gyro-sulcal patterns. By defining a novel gyral folding pattern, termed gyral hinge (GH), which is the conjunction of ordinary gyri from multiple directions, we found GHs possess the highest length and cost in the white matter fiber connective network, and the shortest paths in the network tend to travel through GHs in their middle part. Based on these findings, we would hypothesize GHs could reside in the centers of a network core, thereby accounting for the highest cost and the highest communication capacity in a corticocortical network. The following results further support our hypothesis: 1) GHs possess stronger functional network integration capacity. 2) Higher cost is found on the connection with GHs to hinges and GHs to GHs. 3) Moving GHs introduces higher extra network cost. Our findings and hypotheses could reveal a profound relationship among the cortical folding patterns, axonal wiring architectures, and brain functions.
... Considering the sparsity property of the human functional brain networks (Lv et al., 2015), a robust feature selection (RFS) algorithm was used for feature selection because it is based on l 2,1 -norm sparsity regularization and regression (Nie et al., 2010), regarding the training data as a whole and enforced sparsity to select the most prominent variables . Given training data X = [x 1 , x 2 , ⋯, x n ] ∈ R d×n and the corresponding labels y ∈ R n×c , the core of RFS is to solve the following optimization problem: ...
... The high complexity of the brain often leads us to only consider a few specific brain regions and their intrinsic connections when studying a particular mental process (20,21). Traditional methods such as regionwise activation t-tests can only identify active regions specific to the current process and exists a priori knowledge (22), while methods like dictionary learning (23), ICA (24), and PCA (25) may select only a few important features without fully utilizing them and fail to identify this set of fingerprints accurately. Different mental processes may activate different brain regions. ...
Mental processing delineates the functions of the human mind encompassing a wide range of motor, sensory, emotional, and cognitive processes, each of which is underlain by the neuroanatomical substrates. Identifying accurate representation of neuroanatomical substrates of mental processing could inform understanding of its neural mechanism. The challenge is that it is unclear whether a specific mental process possesses a ’neuroanatomy fingerprint’, i.e., a unique pattern of neuroanatomy that determines the mental process. We used multi-task deep learning to disentangle the neuroanatomy fingerprint of a mental process from other multiple mental processes. The neuroanatomy fingerprint is a unique set of functional activity strength of whole-brain regions with high discrimination ability (AUC = 0.99) among different mental processes. The functional activity strength distribution of neuroanatomy fingerprint unveils both common and distinct brain regions involved in different mental processes. The neuroanatomy fingerprint also addresses a long-standing hypothesis: At least in Chinese and US populations, there consistently exists a neuroanatomy fingerprint to underlie the associated mental process.
One-Sentence Summary
Each mental process has a neuroanatomy fingerprint, which is represented as a unique pattern of functional activity strength of whole-brain regions, to differentiate from other different mental processes.
... Benesty, Chen,Huang, & Cohen, 2009;Lv, Jiang, Li, Zhu, Chen, et al., 2015). Specifically, we first calculated the task paradigm curves convolved with hemodynamic response function (HRF).Next, we computed the PCC values between the convolved task paradigm curves and the atoms Here, 1 refers to the c-th component in temporal features 1 derived from DBN stage (c = 1, ⋯, 1). ...
Decoding human brain activity on various task-based functional brain imaging data is of great significance for uncovering the functioning mechanism of the human mind. Currently, most feature extraction model-based methods for brain state decoding are shallow machine learning models, which may struggle to capture complex and precise spatio-temporal patterns of brain activity from the highly noisy fMRI raw data. Moreover, although decoding models based on deep learning methods benefit from their multi-layer structure that could extract spatio-temporal features at multi-scale, the relatively large populations of fMRI datasets are indispensable and the explainability of their results is elusive. To address the above problems, we proposed a computational framework based on hybrid spatio-temporal deep belief network and sparse representations to differentiate multi-task fMRI (tfMRI) signals. Using a relatively small cohort of tfMRI data as a testbed, our framework can achieve an average classification accuracy of 97.86% and define the multi-level temporal and spatial patterns of multiple cognitive tasks. Intriguingly, our model can characterize the key components for differentiating the multi-task fMRI signals. Overall, the proposed framework can identify the interpretable and discriminative fMRI composition patterns at multiple scales, offering an effective methodology for basic neuroscience and clinical research with relatively small cohorts.
... To identify the matching FBNs across two sessions, we manually selected representative FBNs from session A and match them with responding FBNs in session B based on their highest Pearson Correlation Coefficient (PCC) values and the maximum number of overlapping voxels (overlapping rate) (Benesty et al., 2009;Lv et al., 2015a). A detailed pipeline for selection of representative FBNs can be found in the Supplementary material (1.4). ...
One of human brain's remarkable traits lies in its capacity to dynamically coordinate the activities of multiple brain regions or networks, adapting to an externally changing environment. Studying the dynamic functional brain networks (DFNs) and their role in perception, assessment, and action can significantly advance our comprehension of how the brain responds to patterns of sensory input. Movies provide a valuable tool for studying DFNs, as they offer a naturalistic paradigm that can evoke complex cognitive and emotional experiences through rich multimodal and dynamic stimuli. However, most previous research on DFNs have predominantly concentrated on the resting-state paradigm, investigating the topological structure of temporal dynamic brain networks generated via chosen templates. The dynamic spatial configurations of the functional networks elicited by naturalistic stimuli demand further exploration. In this study, we employed an unsupervised dictionary learning and sparse coding method combing with a sliding window strategy to map and quantify the dynamic spatial patterns of functional brain networks (FBNs) present in naturalistic functional magnetic resonance imaging (NfMRI) data, and further evaluated whether the temporal dynamics of distinct FBNs are aligned to the sensory, cognitive, and affective processes involved in the subjective perception of the movie. The results revealed that movie viewing can evoke complex FBNs, and these FBNs were time-varying with the movie storylines and were correlated with the movie annotations and the subjective ratings of viewing experience. The reliability of DFNs was also validated by assessing the Intra-class coefficient (ICC) among two scanning sessions under the same naturalistic paradigm with a three-month interval. Our findings offer novel insight into comprehending the dynamic properties of FBNs in response to naturalistic stimuli, which could potentially deepen our understanding of the neural mechanisms underlying the brain's dynamic changes during the processing of visual and auditory stimuli.
... To further evaluate the effectiveness of the proposed MAMSM, it is compared with SDL (Lv et al., 2015)and STAAE (Dong et al., 2020b). SDL is the traditional way to build FBNs. ...
The investigation of functional brain networks (FBNs) using task-based functional magnetic resonance imaging (tfMRI) has gained significant attention in the field of neuroimaging. Despite the availability of several methods for constructing FBNs, including traditional methods like GLM and deep learning methods such as spatiotemporal self-attention mechanism (STAAE), these methods have design and training limitations. Specifically, they do not consider the intrinsic characteristics of fMRI data, such as the possibility that the same signal value at different time points could represent different brain states and meanings. Furthermore, they overlook prior knowledge, such as task designs, during training. This study aims to overcome these limitations and develop a more efficient model by drawing inspiration from techniques in the field of natural language processing (NLP). The proposed model, called the Multi-head Attention-based Masked Sequence Model (MAMSM), uses a multi-headed attention mechanism and mask training approach to learn different states corresponding to the same voxel values. Additionally, it combines cosine similarity and task design curves to construct a novel loss function. The MAMSM was applied to seven task state datasets from the Human Connectome Project (HCP) tfMRI dataset. Experimental results showed that the features acquired by the MAMSM model exhibit a Pearson correlation coefficient with the task design curves above 0.95 on average. Moreover, the model can extract more meaningful networks beyond the known task-related brain networks. The experimental results demonstrated that MAMSM has great potential in advancing the understanding of functional brain networks.
... Gael Varoquaux et al [3] and Harrison et al [4] decompose fMRI data with multi subject dictionary learning approach and assume that the decomposed components follow a Gaussian distribution or delta Gaussian distribution. JingLei Lv et al [5] compare the two hypotheses and incorporate sparsity into the dictionary learning decomposition. Subsequently, C.F. Beckmann et al [6] studied the fMRI data with independent component analysis (ICA) to obtain the SMs and TCs. ...
... In recent years, many researchers have combined functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS) to measure physiological signals in task-evoked experimental processes to obtain the underlying neuroscientific mechanism of working memory (Ragland et al., 2002;Herff et al., 2013;Lv et al., 2014Lv et al., , 2015Yeung et al., 2021). ...
Although theoretical studies have suggested that working-memory capacity is crucial for academic achievement, few empirical studies have directly investigated the relationship between working-memory capacity and programming ability, and no direct neural evidence has been reported to support this relationship. The present study aimed to fill this gap in the literature. Using a between-subject design, 17 programming novices and 18 advanced students performed an n-back working-memory task. During the experiment, their prefrontal hemodynamic responses were measured using a 48-channel functional near-infrared spectroscopy (fNIRS) device. The results indicated that the advanced students had a higher working-memory capacity than the novice students, validating the relationship between programming ability and working memory. The analysis results also showed that the hemodynamic responses in the prefrontal cortex can be used to discriminate between novices and advanced students. Additionally, we utilized an attention-based convolutional neural network to analyze the spatial domains of the fNIRS signals and demonstrated that the left prefrontal cortex was more important than other brain regions for programming ability prediction. This result was consistent with the results of statistical analysis, which in turn improved the interpretability of neural networks.
... To explore the functional brain networks (FBNs), various analytical approaches have been proposed, each providing a different model for mapping brain functional pattern. Most of the previous methods for mapping FBNs using fMRI are based on the temporal correlation between the source and voxel signals, such as indenpendent component analysis (ICA) [3] and sparse dictionary learning (SDL) [4]. Many of these approaches can be viewed as a blind source separation problem, the fMRI signals are modeled as a linear combination of the sources and the temporal patterns [5]. ...
Using functional magnetic resonance imaging (fMRI) and deep learning to explore functional brain networks (FBNs) has attracted many researchers. However, most of these studies are still based on the temporal correlation between the sources and voxel signals, and lack of researches on the dynamics of brain function. Due to the widespread local correlations in the volumes, FBNs can be generated directly in the spatial domain in a self-supervised manner by using spatial-wise attention (SA), and the resulting FBNs has a higher spatial similarity with templates compared to the classical method. Therefore, we proposed a novel Spatial-Temporal Convolutional Attention (STCA) model to discover the dynamic FBNs by using the sliding windows. To validate the performance of the proposed method, we evaluate the approach on HCP-rest dataset. The results indicate that STCA can be used to discover FBNs in a dynamic way which provide a novel approach to better understand human brain.
... As a reference system, the fractogram template provides a new mean to aggregate features from multiple subjects and data sets for machine-learning approaches. [65][66][67] ...
Purpose:
Brain templates provide an essential standard space for statistical analysis of brain structure and function. Despite recent advances, diffusion MRI still lacks a template of fiber orientation distribution (FOD) and tractography that is unbiased for both white and gray matter. Therefore, we aim to build up a set of such templates for better white-matter analysis and joint structural and functional analysis.
Methods:
We have developed a multimodal registration method to leverage the complementary information captured by T1 -weighted, T2 -weighted, and diffusion MRI, so that a coherent transformation is generated to register FODs into a common space and average them into a template. Consequently, the anatomically constrained fiber-tracking method was applied to the FOD template to generate a tractography template. Fiber-centered functional connectivity analysis was then performed as an example of the benefits of such an unbiased template.
Results:
Our FOD template preserves fine structural details in white matter and also, importantly, clear folding patterns in the cortex and good contrast in the subcortex. Quantitatively, our templates show better individual-template agreement at the whole-brain scale and segmentation scale. The tractography template aligns well with the gray matter, which led to fiber-centered functional connectivity showing high cross-group consistency.
Conclusion:
We have proposed a novel methodology for building a tissue-unbiased FOD and anatomically constrained tractography template based on multimodal registration. Our templates provide a standard space and statistical platform for not only white-matter analysis but also joint structural and functional analysis, therefore filling an important gap in multimodal neuroimage analysis.
... Canonical polyadic decomposition (CPD), one of the 40 typical tensor decomposition methods, has been increasingly 41 applied to multi-subject fMRI data. Different from matrix 42 methods such as independent component analysis (ICA) [6], 43 [7], negative matrix decomposition and dictionary learning 44 [8], [9], [10], [11], [12], CPD can well retain the high-order 45 structure information of multi-subject fMRI data and preserve 46 the uniqueness under some mild conditions [13]. Generally 47 speaking, CPD treats multi-subject fMRI data as a three-way 48 tensor in terms of spatial, temporal and subject modes [14], 49 and decomposes fMRI data as shared spatial maps (SMs), 50 shared time courses (TCs), and subject-specific intensities 51 [15], [16], [17]. ...
Complex-valued shift-invariant canonical polyadic decomposition (CPD) under a spatial phase sparsity constraint (pcsCPD) shows excellent separation performance when applied to band-pass filtered complex-valued multi-subject fMRI data. However, some useful information may also be eliminated when using a band-pass filter to suppress unwanted noise. As such, we propose an alternating rank-
${R}$
and rank-1 least squares optimization to relax the CPD model. Based upon this optimization method, we present a novel constrained CPD algorithm with temporal shift-invariance and spatial sparsity and orthonormality constraints. More specifically, four steps are conducted until convergence for each iteration of the proposed algorithm: 1) use rank-
${R}$
least-squares fit under spatial phase sparsity constraint to update shared spatial maps after phase de-ambiguity; 2) use orthonormality constraint to minimize the cross-talk between shared spatial maps; 3) update the aggregating mixing matrix using rank-
${R}$
least-squares fit; 4) utilize shift-invariant rank-1 least-squares on a series of rank-1 matrices reconstructed by each column of the aggregating mixing matrix to update shared time courses, and subject-specific time delays and intensities. The experimental results of simulated and actual complex-valued fMRI data show that the proposed algorithm improves the estimates for task-related sensorimotor and auditory networks, compared to pcsCPD and tensorial spatial ICA. The proposed alternating rank-
${R}$
and rank-1 least squares optimization is also flexible to improve CPD-related algorithm using alternating least squares.
... In the context of MS studies, there are some [35], [36] that directly apply existing DL algorithms such as ODL [26] to obtain group-level brain networks. Whereas, authors in other publications developed DL algorithms explicitly for MS-fMRI studies [37]- [40]. ...
Considering that functional magnetic resonance imaging (fMRI) signals from multiple subjects (MS) can be represented together as a sum of common and a sum of distinct rank-1 matrices, a new MS dictionary learning (DL) algorithm named sparse group (common + distinct) bases (sgBACES) is proposed. Unlike existing MS-DL algorithms that ignore fMRI data’s prior information, it is formulated as a penalized plus constrained rank-1 matrix approximation, where
$l_{1}$
norm-based adaptive sparse penalty,
$l_{0}$
norm-based dictionary regularization, and lag-1 based autocorrelation maximization have been introduced in the minimization problem. Besides, spatial dependence among neighbouring voxels has been exploited for fine-tuning the sparsity parameters. To my best knowledge, the sgBACES algorithm is the first to effectively take temporal and spatial prior information into account for an MS-fMRI-DL framework. It also has the advantage of not requiring a separate sparse coding stage. Studies based on synthetic and experimental fMRI datasets are used to compare the performance of sgBACES with the state-of-the-art algorithms in terms of correlation strength and computation time. It emerged that the proposed sgBACES algorithm enhanced the signal-to-noise ratio (SNR) of the recovered time courses (TCs) and the precision of the recovered spatial maps (SMs). A 10.2% increase in the mean correlation value over the ShSSDL algorithm is observed for motor-task based fMRI data.
... To date, it has been proven that the task-activated brain networks (BNs) can be reconstructed from task-fMRI while the resting-state BNs can be obtained from resting-state fMRI (rs-fMRI) [3,4]. Recently, some studies have shown that the brain networks, including task-activated and restingstate BNs, can be inferred from task-fMRI simultaneously [5,6]. Independent component analysis (ICA) is one of the most popular ways to identify the resting-state brain networks. ...
How to identify and characterize functional brain networks (BN) is fundamental to gain system-level insights into the mechanisms of brain organizational architecture. Current functional magnetic resonance (fMRI) analysis highly relies on prior knowledge of specific patterns in either spatial (e.g., resting-state network) or temporal (e.g., task stimulus) domain. In addition, most approaches aim to find group-wise common functional networks, individual-specific functional networks have been rarely studied. In this work, we propose a novel Twin-Transformers framework to simultaneously infer common and individual functional networks in both spatial and temporal space, in a self-supervised manner. The first transformer takes space-divided information as input and generates spatial features, while the second transformer takes time-related information as input and outputs temporal features. The spatial and temporal features are further separated into common and individual ones via interactions (weights sharing) and constraints between the two transformers. We applied our TwinTransformers to Human Connectome Project (HCP) motor task-fMRI dataset and identified multiple common brain networks, including both task-related and resting-state networks (e.g., default mode network). Interestingly, we also successfully recovered a set of individual-specific networks that are not related to task stimulus and only exist at the individual level.
... This is different from the traditional region-region connectome, but a fibre-centred connectome. As a reference system, the fractogram template provides a new mean to aggregate features from multiple subjects and datasets for machine learning approaches (Ganesan et al., 2021;Lv et al., 2017Lv et al., , 2015. ...
A brain template provides a standard space for statistical analysis of brain structure and function. For decades, the T1- and T2-weighted brain templates have been widely used for brain grey matter anatomical and functional analysis. However, T1- and T2-weighted templates provide very limited information about the axonal organization within the white matter. Recent advances in Diffusion MRI have enabled the detailed modelling of the axonal fibre orientation distribution (FOD) in white matter. Therefore, building a FOD template is essential for more robust white matter anatomy related analysis; however, it is important that this template aligns well with the cortical and subcortical structures. From such a FOD template, a tractography template can be also generated by fibre tracking algorithms, which can be used for subsequent applications, such as to perform the joint structural and functional analysis while ensuring rigorous fibre-to-fibre correspondence. In this paper, we explore the potential of generating the FOD template based on multimodal registration, in order to constrain the tempalte unbiased to both white and grey matter. We combine the information from T1-weighted, T2-weighted and Diffusion MRI to generate a coherent transformation for FOD registration and template generation. Our FOD template preserves the structural details at the white-grey matter boundary. To illustrate the benefit of this new approach, the resulting tractography template was used for joint structural-functional connectivity analysis.
... The regularization parameter λ is known to determine the sparsity and scale of the network. If the λ value is too small, then the created network model would be too rough and involved too much noise; if the λ value was too large, then the network model would be sparser (Lv et al., 2015). Different parameter λ settings are also known to have a certain impact on the reliability of the network topology, especially modularity (Li and Wang, 2015). ...
Resting-state functional connectivity hypernetworks, in which multiple nodes can be connected, are an effective technique for diagnosing brain disease and performing classification research. Conventional functional hypernetworks can characterize the complex interactions within the human brain in a static form. However, an increasing body of evidence demonstrates that even in a resting state, neural activity in the brain still exhibits transient and subtle dynamics. These dynamic changes are essential for understanding the basic characteristics underlying brain organization and may correlate significantly with the pathological mechanisms of brain diseases. Therefore, considering the dynamic changes of functional connections in the resting state, we proposed methodology to construct resting state high-order functional hyper-networks (rs-HOFHNs) for patients with depression and normal subjects. Meanwhile, we also introduce a novel property (the shortest path) to extract local features with traditional local properties (cluster coefficients). A subgraph feature-based method was introduced to characterize information relating to global topology. Two features, local features and subgraph features that showed significant differences after feature selection were subjected to multi-kernel learning for feature fusion and classification. Compared with conventional hyper network models, the high-order hyper network obtained the best classification performance, 92.18%, which indicated that better classification performance can be achieved if we needed to consider multivariate interactions and the time-varying characteristics of neural interaction simultaneously when constructing a network.
... Deep learning using Convolutional Neural Networks (CNNs) has become the state-of-the-art for image segmentation, classification, detection and retrieval related tasks (Khan et al., 2020). Recent work (Zhao et al., 2018) has demonstrated the capability of CNNs for automatic recognition of spatial restingstate network maps, although this and other machine learning approaches for subject-specific identification of RSNs (Hacker et al., 2013;Lv et al., 2015) were not designed to work with ICA. In Vergun et al., 2016, several machine learning approaches, including a one layer neural network, were used to label ICA components in epilepsy, achieving up to 90% accuracy. ...
Manual classification of functional resting state networks (RSNs) derived from Independent Component Analysis (ICA) decomposition can be labor intensive and requires expertise, particularly in large multi-subject analyses. Hence, a fully automatic algorithm that can reliably classify these RSNs is desirable. In this paper, we present a deep learning approach based on a Siamese Network to learn a discriminative feature representation for single-subject ICA component classification. Advantages of this supervised framework are that it requires relatively few training data examples and it does not require the number of ICA components to be specified. In addition, our approach permits one-shot learning, which allows generalization to new classes not seen in the training set with only one example of each new class. The proposed method is shown to out-perform traditional convolutional neural network (CNN) and template matching methods in identifying eleven subject-specific RSNs, achieving 100% accuracy on a holdout data set and over 99% accuracy on an outside data set. We also demonstrate that the method is robust to scan-rescan variation. Finally, we show that the functional connectivity of default mode and salience networks identified by the proposed technique is altered in a group analysis of mild traumatic brain injury (TBI), severe TBI, and healthy subjects.
... Standard approaches to fMRI data analysis routinely make use of compression and dimension reduction-based approaches such as independent component analysis (ICA) (Bai et al., 2007;Calhoun et al., 2009), canonical correlation analysis (Friman et al., 2001;Hardoon et al., 2007;Lin et al., 2014), and less frequently sparse dictionary learning (Lee et al., 2011;Eavani et al., 2012;Wu et al., 2014;Lv et al., 2015), but these approaches typically rely on strong assumptions like linearity and spatial independence which are violated for fMRI data. In addition, SVM-based classification methods, known within the field under the name multi-voxel pattern analysis (MVPA) (Norman et al., 2006;Mahmoudi et al., 2012) have been widely used to localize particular kinds of task effects within the brain while making weaker statistical assumptions. ...
While functional magnetic resonance imaging (fMRI) remains one of the most widespread and important methods in basic and clinical neuroscience, the data it produces-time series of brain volumes-continue to pose daunting analysis challenges. The current standard ("mass univariate") approach involves constructing a matrix of task regressors, fitting a separate general linear model at each volume pixel ("voxel"), computing test statistics for each model, and correcting for false positives post hoc using bootstrap or other resampling methods. Despite its simplicity, this approach has enjoyed great success over the last two decades due to: 1) its ability to produce effect maps highlighting brain regions whose activity significantly correlates with a given variable of interest; and 2) its modeling of experimental effects as separable and thus easily interpretable. However, this approach suffers from several well-known drawbacks, namely: inaccurate assumptions of linearity and noise Gaussianity; a limited ability to capture individual effects and variability; and difficulties in performing proper statistical testing secondary to independently fitting voxels. In this work, we adopt a different approach, modeling entire volumes directly in a manner that increases model flexibility while preserving interpretability. Specifically, we use a generalized additive model (GAM) in which the effects of each regressor remain separable, the product of a spatial map produced by a variational autoencoder and a (potentially nonlinear) gain modeled by a covariate-specific Gaussian Process. The result is a model that yields group-level effect maps comparable or superior to the ones obtained with standard fMRI analysis software while also producing single-subject effect maps capturing individual differences. This suggests that generative models with a decomposable structure might offer a more flexible alternative for the analysis of task-based fMRI data.
... For many years, decoding the brain's activities has been one of the major topics in neuroscience. Inferring brain states consists of predicting the tasks subjects performed and identifying brain regions related to specific cognitive functions (Friston et al., 1994;Lv et al., 2015;McKeown et al., 1998;Norman, Polyn, Detre, & Haxby, 2006). Deep learning (DL) methods based on a variety of artificial neural networks have gained considerable attention in the scientific community for more than a decade, breaking benchmark records in several domains, including vision, speech, and natural language processing (Krizhevsky, Sutskever, & Hinton, 2017;LeCun, Bengio, & Hinton, 2015). ...
Decoding brain cognitive states from neuroimaging signals is an important topic in neuroscience. In recent years, deep neural networks (DNNs) have been recruited for multiple brain state decoding and achieved good performance. However, the open question of how to interpret the DNN black box remains unanswered. Capitalizing on advances in machine learning, we integrated attention modules into brain decoders to facilitate an in‐depth interpretation of DNN channels. A four‐dimensional (4D) convolution operation was also included to extract temporo‐spatial interaction within the fMRI signal. The experiments showed that the proposed model obtains a very high accuracy (97.4%) and outperforms previous researches on the seven different task benchmarks from the Human Connectome Project (HCP) dataset. The visualization analysis further illustrated the hierarchical emergence of task‐specific masks with depth. Finally, the model was retrained to regress individual traits within the HCP and to classify viewing images from the BOLD5000 dataset, respectively. Transfer learning also achieves good performance. Further visualization analysis shows that, after transfer learning, low‐level attention masks remained similar to the source domain, whereas high‐level attention masks changed adaptively. In conclusion, the proposed 4D model with attention module performed well and facilitated interpretation of DNNs, which is helpful for subsequent research. The 4DResNet with attention module obtains very high accuracy (97.4%) on the HCP dataset brain decoding. The attention module facilitates in‐depth interpretability of the fMRI decoding neural network.
... The functional brain networks (FBNs) learned from fMRI provide a powerful tool to model brain functions and disorders even in the absence of an external task [3][4][5][6]. In previous studies, various machine learning methods have been successfully applied on fMRI to exploit FBNs, such as general linear model (GLM) [7,8], independent component analysis (ICA) [9][10][11][12][13], and sparse dictionary learning (SDL) [14][15][16][17]. GLM is a model-driven method that is widely used in task-based fMRI (tfMRI) analysis. ...
It has been of great interest in the neuroimaging community to model spatiotemporal brain function and related disorders based on resting state functional magnetic resonance imaging (rfMRI). Although a variety of deep learning models have been proposed for modeling rfMRI, the dominant models are limited in capturing the long-distance dependency (LDD) due to their sequential nature. In this work, we propose a spatiotemporal attention auto-encoder (STAAE) to discover global features that address LDDs in volumetric rfMRI. The unsupervised STAAE framework can spatiotemporally model the rfMRI sequence and decompose the rfMRI into spatial and temporal patterns. The spatial patterns have been extensively explored and are also known as resting state networks (RSNs), yet the temporal patterns are underestimated in last decades. To further explore the application of temporal patterns, we developed a resting state temporal template (RSTT)-based classification framework using the STAAE model and tested it with attention-deficit hyperactivity disorder (ADHD) classification. Five datasets from ADHD-200 were used to evaluate the performance of our method. The results showed that the proposed STAAE outperformed three recent methods in deriving ten well-known RSNs. For ADHD classification, the proposed RSTT-based classification framework outperformed methods in recent studies by achieving a high accuracy of 72.5%. Besides, we found that the RSTTs derived from NYU dataset still work on the other four datasets, but the accuracy on different test datasets decreased with the increase in the age gap to NYU dataset, which likely supports the idea of that there exist age differences of brain activity among ADHD patients.
... In previous studies, a variety of conventional methods for modeling fMRI data to reconstruct and characterize FBN have been developed, such as general linear model (GLM) (Beckmann et al., 2003), independent component analysis (ICA) (Calhoun and Adali, 2006), and sparse dictionary learning (SDL) (Lv et al., 2015). Although these methods can construct meaningful FBNs, due to their superficial nature, they may not be able to detect hierarchical FBNs and temporal features, which is an important nature for FBNs (Meunier et al., 2009;Qiang et al., 2020b). ...
Naturalistic functional magnetic resonance imaging (NfMRI) has become an effective tool to study brain functional activities in real-life context, which reduces the anxiety or boredom due to difficult or repetitive tasks and avoids the problem of unreliable collection of brain activity caused by the subjects’ microsleeps during resting state. Recent studies have made efforts on characterizing the brain’s hierarchical organizations from fMRI data by various deep learning models. However, most of those models have ignored the properties of group-wise consistency and inter-subject difference in brain function under naturalistic paradigm. Another critical issue is how to determine the optimal neural architecture of deep learning models, as manual design of neural architecture is time-consuming and less reliable. To tackle these problems, we proposed a two-stage deep belief network (DBN) with neural architecture search (NAS) combined framework (two-stage NAS-DBN) to model both the group-consistent and individual-specific naturalistic functional brain networks (FBNs), which reflected the hierarchical organization of brain function and the nature of brain functional activities under naturalistic paradigm. Moreover, the test-retest reliability and spatial overlap rate of the FBNs identified by our model reveal better performance than that of widely used traditional methods. In general, our model provides a promising method for characterizing hierarchical spatiotemporal features under the natural paradigm.
Background
Resting‐state functional magnetic resonance imaging (rs‐fMRI) technology and the complex network theory can be used to elucidate the underlying mechanisms of brain diseases. The successful application of functional brain hypernetworks provides new perspectives for the diagnosis and evaluation of clinical brain diseases; however, many studies have not assessed the attribute information of hyperedges and could not retain the high‐order topology of hypergraphs. In addition, the study of multi‐scale and multi‐layered organizational properties of the human brain can provide richer and more accurate data features for classification models of depression.
Purpose
This work aims to establish a more accurate classification framework for the diagnosis of major depressive disorder (MDD) using the high‐order line graph algorithm. And accuracy, sensitivity, specificity, precision, F 1 score are used to validate its classification performance.
Methods
Based on rs‐fMRI data from 38 MDD subjects and 28 controls, we constructed a human brain hypernetwork and introduced a line graph model, followed by the construction of a high‐order line graph model. The topological properties under each order line graph were calculated to measure the classification performance of the model. Finally, intergroup features that showed significant differences under each order line graph model were fused, and a support vector machine classifier was constructed using multi‐kernel learning. The Kolmogorov–Smirnov nonparametric permutation test was used as the feature selection method and the classification performance was measured with the leave‐one‐out cross‐validation method.
Results
The high‐order line graph achieved a better classification performance compared with other traditional hypernetworks (accuracy = 92.42%, sensitivity = 92.86%, specificity = 92.11%, precision = 89.66%, F 1 = 91.23%). Furthermore, the brain regions found in the present study have been previously shown to be associated with the pathology of depression.
Conclusions
This work validated the classification model based on the high‐order line graph, which can better express the topological features of the hypernetwork by comprehensively considering the hyperedge information under different connection strengths, thereby significantly improving the classification accuracy of MDD. Therefore, this method has potential for use in the clinical diagnosis of MDD.
Gyri and sulci are 2 fundamental cortical folding patterns of the human brain. Recent studies have suggested that gyri and sulci may play different functional roles given their structural and functional heterogeneity. However, our understanding of the functional differences between gyri and sulci remains limited due to several factors. Firstly, previous studies have typically focused on either the spatial or temporal domain, neglecting the inherently spatiotemporal nature of brain functions. Secondly, analyses have often been restricted to either local or global scales, leaving the question of hierarchical functional differences unresolved. Lastly, there has been a lack of appropriate analytical tools for interpreting the hierarchical spatiotemporal features that could provide insights into these differences. To overcome these limitations, in this paper, we proposed a novel hierarchical interpretable autoencoder (HIAE) to explore the hierarchical functional difference between gyri and sulci. Central to our approach is its capability to extract hierarchical features via a deep convolutional autoencoder and then to map these features into an embedding vector using a carefully designed feature interpreter. This process transforms the features into interpretable spatiotemporal patterns, which are pivotal in investigating the functional disparities between gyri and sulci. We evaluate the proposed framework on Human Connectome Project task functional magnetic resonance imaging dataset. The experiments demonstrate that the HIAE model can effectively extract and interpret hierarchical spatiotemporal features that are neuroscientifically meaningful. The analyses based on the interpreted features suggest that gyri are more globally activated, whereas sulci are more locally activated, demonstrating a distinct transition in activation patterns as the scale shifts from local to global. Overall, our study provides novel insights into the brain’s anatomy–function relationship.
Using functional magnetic resonance imaging (fMRI) and deep learning to discover the spatial pattern of brain function, or functional brain networks (FBNs) has been attracted many reseachers. Most existing works focus on static FBNs or dynamic functional connectivity among fixed spatial network nodes, but ignore the potential dynamic/time-varying characteristics of the spatial networks themselves. And most of works based on the assumption of linearity and independence, that oversimplify the relationship between blood-oxygen level dependence (BOLD) signal changes and the heterogeneity of neuronal activity within voxels. To overcome these problems, we proposed a novel spatial-wise attention based method called Spatial and Channel-wise Attention Autoencoder (SCAAE) to discover the dynamic FBNs without the assumptions of linearity or independence. The core idea of SCAAE is to apply the spatial-wise attention to generate FBNs directly, relying solely on the spatial information present in fMRI volumes. Specifically, we trained the SCAAE in a self-supervised manner, using the autoencoder to guide the spatial-wise attention (SA) to focus on the activation regions. Experimental results show that the SA can generate multiple meaningful FBNs at each fMRI time point, which spatial similarity are close to the FBNs derived by known classical methods, such as ICA. To validate the generalization of the method, we evaluate the approach on HCP-rest, HCP-task and ADHD-200 dataset. The results demonstrate that spatial-wise attention mechanism can be used to discover time-varying FBNs, and the identified dynamic FBNs over time clearly show the process of time-varying spatial patterns fading in and out. Thus we provide a novel method to understand human brain better.
The functional activity of the default mode network (DMN) in the resting state is complex and spontaneous. Modeling spatio-temporal patterns of DMN based on four-dimensional Resting-state functional Magnetic Resonance Imaging (Rs-fMRI) provides a basis for exploring spontaneous brain functional activities. However, how to utilize spatio-temporal features to complete the multi-level description of 4D Rs-fMRI with diverse characteristics in the shallow stage of the model and accurately characterize the DMN holistic spatio-temporal patterns remains challenging in the current DMN spatio-temporal patterns modeling. To this end, we propose a Multi-scale Pyramid Attention Network (MPANet) to focus on shallow features and model the spatio-temporal patterns of resting-state personalized DMN. Specifically, in the spatial stage, we design a multi-scale pyramid block in the shallow layer to expand the receptive field and extract granular information at different levels, which realize feature enhancement and guides the model to characterize the DMN spatial pattern. In the temporal stage, parallel guidance from spatial to the temporal pattern is achieved through the fast down-sampling operation and introduction of multi-head attention blocks for a more effective fusion of spatio-temporal features. The results based on a publicly available dataset demonstrate that MPANet outperforms other state-of-the-art methods. This network presents a robust tool for modeling the spatio-temporal patterns of individuals with DMN, and its exceptional performance suggests promising potential for clinical applications.
Recently, deep learning models have achieved superior performance for mapping functional brain networks from functional magnetic resonance imaging (fMRI) data compared with traditional methods. However, due to the lack of sufficient data and the high dimensionality of brain volume, deep learning models of fMRI tend to suffer from overfitting. In addition, existing methods rarely studied fMRI data augmentation and its application. To address these issues, we developed a VAE-GAN framework that combined a VAE (variational auto-encoder) with a GAN (generative adversarial net) for functional brain network identification and fMRI augmentation. As a generative model, the VAE-GAN models the distribution of fMRI so that it enables the extraction of more generalized features, and thus relieve the overfitting issue. The VAE-GAN is easier to train on fMRI than a standard GAN since it uses latent variables from VAE to generate fake data rather than relying on random noise that is used in a GAN, and it can generate higher quality of fake data than VAE since the discriminator can promote the training of the generator. In other words, the VAE-GAN inherits the advantages of VAE and GAN and avoids their limitations in modeling of fMRI data. Extensive experiments on task fMRI datasets from HCP have proved the effectiveness and superiority of the proposed VAE-GAN framework for identifying both temporal features and functional brain networks compared with existing models, and the quality of fake data is higher than those from VAE and GAN. The results on resting state fMRI of Attention Deficit Hyperactivity Disorder (ADHD)-200 dataset further demonstrated that the fake data generated by the VAE-GAN can help improve the performance of brain network modeling and ADHD classification.
Background:
It has been recently shown that deep learning models exhibited remarkable performance of representing functional Magnetic Resonance Imaging (fMRI) data for the understanding of brain functional activities. With hierarchical structure, deep learning models can infer hierarchical functional brain networks (FBN) from fMRI. However, the applications of the hierarchical FBNs have been rarely studied.
Methods:
In this work, we proposed a hierarchical recurrent variational auto-encoder (HRVAE) to unsupervisedly model the fMRI data. The trained HRVAE encoder can predict hierarchical temporal features from its three hidden layers, and thus can be regarded as a hierarchical feature extractor. Then LASSO (least absolute shrinkage and selection operator) regression was applied to estimate the corresponding hierarchical FBNs. Based on the hierarchical FBNs from each subject, we constructed a novel classification framework for brain disorder identification and test it on the Autism Brain Imaging Data Exchange (ABIDE) dataset, a world-wide multi-site database of autism spectrum disorder (ASD). We analyzed the hierarchy organization of FBNs, and finally used the overlaps of hierarchical FBNs as features to differentiate ASD from typically developing controls (TDC).
Results:
The experimental results on 871 subjects from ABIDE dataset showed that the HRVAE model can effectively derive hierarchical FBNs including many well-known resting state networks (RSN). Moreover, the classification result improved the state-of-the-art by achieving a very high accuracy of 82.1%.
Conclusions:
This work presents a novel data-driven deep learning method using fMRI data for ASD identification, which could provide valuable reference for clinical diagnosis. The classification results suggest that the interactions of hierarchical FBNs have association with brain disorder, which promotes the understanding of FBN hierarchy and could be applied to other brain disorder analysis.
Learning an effective and compact representation of human brain function from high-dimensional fMRI data is crucial for studying the brain's functional organization. Traditional representation methods such as independent component analysis (ICA) and sparse dictionary learning (SDL) mainly rely on matrix decomposition which represents the brain function as spatial brain networks and the corresponding temporal patterns. The correspondence of those brain networks across individuals are built by viewing them as one-hot vectors and then performing the matching. However, those one-hot vectors do not encode the regularity and/or variability of different brains very well, and thus are limited in effectively representing the functional brain activities across individuals and among different time points. To address this problem, in this paper, we formulate the human brain functional representation as an embedding problem, and propose a novel embedding framework based on the Transformer model to encode the brain function in a compact, stereotyped and comparable latent space where the brain activities are represented as dense embedding vectors. We evaluate the proposed embedding framework on the publicly available Human Connectome Project (HCP) task fMRI dataset. The experiments on brain state prediction task indicate the effectiveness and generalizability of the learned embedding. We also explore the interpretability of the learned embedding from both spatial and temporal perspective. In general, our approach provides novel insights on representing the regularity and variability of human brain function in a general, comparable, and stereotyped latent space.
The extraction of neuroimaging features of migraine patients and the design of identification models are of great significance for the auxiliary diagnosis of related diseases. Compared with the commonly used image features, this study directly uses time-series signals to characterize the functional state of the brain in migraine patients and healthy controls, which can effectively utilize the temporal information and reduce the computational effort of classification model training. Firstly, Group Independent Component Analysis and Dictionary Learning were used to segment different brain areas for small-sample groups and then the regional average time-series signals were extracted. Next, the extracted time series were divided equally into multiple subseries to expand the model input sample. Finally, the time series were modeled using a bi-directional long-short term memory network to learn the pre-and-post temporal information within each time series to characterize the periodic brain state changes to improve the diagnostic accuracy of migraine. The results showed that the classification accuracy of migraine patients and healthy controls was 96.94%, the area under the curve was 0.98, and the computation time was relatively shorter. The experiments indicate that the method in this paper has strong applicability, and the combination of time-series feature extraction and bi-directional long-short term memory network model can be better used for the classification and diagnosis of migraine. This work provides a new idea for the lightweight diagnostic model based on small-sample neuroimaging data, and contributes to the exploration of the neural discrimination mechanism of related diseases.
Characterizing individualized spatio-temporal patterns of functional brain networks (FBNs) via functional magnetic resonance imaging (fMRI) provides a foundation for understanding complex brain function. Although previous studies have achieved promising performances based on either shallow or deep learning models, there is still much space to improve the accuracy of spatio-temporal pattern characterization of FBNs by optimally integrating the four-dimensional (4D) features of fMRI. In this study, we introduce a novel Spatio-Temporal Attention 4D Convolutional Neural Network (STA-4DCNN) model to characterize individualized spatio-temporal patterns of FBNs. Particularly, STA-4DCNN is composed of two subnetworks, in which the first Spatial Attention 4D CNN (SA-4DCNN) models the spatio-temporal features of 4D fMRI data and then characterizes the spatial pattern of FBNs, and the second Temporal Guided Attention Network (T-GANet) further characterizes the temporal pattern of FBNs under the guidance of the spatial pattern together with 4D fMRI data. We evaluate the proposed STA-4DCNN on seven different task fMRI and one resting state fMRI datasets from the publicly released Human Connectome Project. The experimental results demonstrate that STA-4DCNN has superior ability and generalizability in characterizing individualized spatio-temporal patterns of FBNs when compared to other state-of-the-art models. We further apply STA-4DCNN on another independent ABIDE I resting state fMRI dataset including both autism spectrum disorder (ASD) and typical developing (TD) subjects, and successfully identify abnormal spatio-temporal patterns of FBNs in ASD compared to TD. In general, STA-4DCNN provides a powerful tool for FBN characterization and for clinical applications on brain disease characterization at the individual level.
Deep learning approaches have been widely adopted in the medical image analysis field. However, a most of existing deep learning approaches focus on achieving promising performances such as classification, detection, and segmentation, and much less effort is devoted to the explanation of the designed models. Similarly, in the brain imaging field, many deep learning approaches have been designed and applied to characterize and predict human brain states. However, these models lack interpretation. In response, we propose a novel domain knowledge informed self-attention graph pooling-based (SAGPool) graph convolutional neural network to study human brain states. Specifically, the dense individualized and common connectivity-based cortical landmarks system (DICCCOL, structural brain connectivity profiles) and holistic atlases of functional networks and interactions system (HAFNI, functional brain connectivity profiles) are integrated with the SAGPool model to better characterize and interpret the brain states. Extensive experiments are designed and carried out on the large-scale human connectome project (HCP) Q1 and S1200 dataset. Promising brain state classification performances are observed (e.g., an average of 93.7% for seven-task classification and 100% for binary classification). In addition, the importance of the brain regions, which contributes most to the accurate classification, is successfully quantified and visualized. A thorough neuroscientific interpretation suggests that these extracted brain regions and their importance calculated from self-attention graph pooling layer offer substantial explainability.
Modeling and characterizing functional brain networks from task-based functional magnetic resonance imaging (fMRI) data has been a popular topic in neuroimaging community. Recently, deep belief network (DBN) has shown great advantages in modeling the hierarchical and complex task functional brain networks (FBNs). However, due to the unsupervised nature, traditional DBN algorithms may be limited in fully utilizing the prior knowledge from the task design. In addition, the FBNs extracted from different DBN layers do not have correspondences, which makes the hierarchical analysis of FBNs a challenging problem. In this paper, we propose a novel prior knowledge guided DBN (PKG-DBN) to overcome the above limitations when conducting hierarchical task FBNs analysis. Specifically, we enforce part of the time courses learnt from DBN to be task-related (in either positive or negative way) and the rest to be linear combinations of task-related components. By incorporating such constraints in the learning process, our method can simultaneously leverage the advantages of data-driven approaches and the prior knowledge of task design. Our experiment results on HCP task fMRI data showed that the proposed PKG-DBN can not only successfully identify meaningful hierarchical task FBNs with correspondence comparing to traditional DBN models, but also converge significantly faster than traditional DBN models.KeywordsHierarchical organizationSuperviseHuman connectome projectDeep neural networkFunctional brain networks
Understanding the brain's functional architecture has been an important topic in the neuroimaging field. A variety of brain network modeling methods have been proposed. Recently, deep neural network-based methods have shown a great advantage in modeling the hierarchical and complex functional brain networks (FBNs). However, most of these deep neural networks were handcrafted, making it time-consuming to find the relatively optimal architecture. To address this problem, we propose a novel unsupervised differentiable neural architecture search (NAS) algorithm, named Gumbel-Softmax based Neural Architecture Search (GS-NAS), to automate the architecture design of deep belief network (DBN) for hierarchical FBN decomposition. Specifically, we introduce the Gumbel-Softmax scheme to reframe the discrete architecture sampling procedure during NAS to be continuous. Guided by the reconstruction error minimization procedure, the architecture search can be driven by the intrinsic functional architecture of the brain, thereby revealing the possible hierarchical functional brain organization via DBN structure. The proposed GS-NAS algorithm can simultaneously optimize the number of hidden units for each layer and the network depth. Extensive experiment results on both task and resting-state functional magnetic resonance imaging data have demonstrated the effectiveness and efficiency of the proposed GS-NAS model. The identified hierarchically organized FBNs provide novel insight into understanding human brain function.
Background and objective:
To understand brain cognition and disorders, modeling the mapping between mind and brain has been of great interest to the neuroscience community. The key is the brain representation, including functional brain networks (FBN) and their corresponding temporal features. Recently, it has been proven that deep learning models have superb representation power on functional magnetic resonance imaging (fMRI) over traditional machine learning methods. However, due to the lack of high-quality data and labels, deep learning models tend to suffer from overfitting in the training process.
Methods:
In this work, we applied a recurrent Wasserstein generative adversarial net (RWGAN) to learn brain representation from volumetric fMRI data. Generative adversarial net (GAN) is widely used in natural image generation and is able to capture the distribution of the input data, which enables the extraction of generalized features from fMRI and thus relieves the overfitting issue. The recurrent layers in RWGAN are designed to better model the local temporal features of the fMRI time series. The discriminator of RWGAN works as a deep feature extractor. With LASSO regression, the RWGAN model can decompose the fMRI data into temporal features and spatial features (FBNs). Furthermore, the generator of RWGAN can generate high-quality new data for fMRI augmentation.
Results:
The experimental results on seven tasks from the HCP dataset showed that the RWGAN can learn meaningful and interpretable temporal features and FBNs, compared to HCP task designs and general linear model (GLM) derived networks. Besides, the results on different training datasets showed that the RWGAN performed better on small datasets than other deep learning models. Moreover, we used the generator of RWGAN to yield fake subjects. The result showed that the fake data can also be used to learn meaningful representation compared to those learned from real data.
Conclusions:
To our best knowledge, this work is among the earliest attempts of applying generative deep learning for modeling fMRI data. The proposed RWGAN offers a novel methodology for learning brain representation from fMRI, and it can generate high-quality fake data for the potential use of fMRI data augmentation.
Mounting evidence has demonstrated that complex brain function processes are realized by the interaction of holistic functional brain networks which are spatially distributed across specific brain regions in a temporally dynamic fashion. Therefore, modeling spatio-temporal patterns of holistic functional brain networks plays an important role in understanding brain function. Compared to traditional modeling methods such as principal component analysis, independent component analysis, and sparse coding, superior performance has been achieved by recent deep learning methodologies. However, there are still two limitations of existing deep learning approaches for functional brain network modeling. They either (1) merely modeled a single targeted network and ignored holistic ones at one time, or (2) underutilized both spatial and temporal features of fMRI during network modeling, and the spatial/temporal accuracy was thus not warranted. To address these limitations, we proposed a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) to simultaneously model both spatial and temporal patterns of holistic functional brain networks. Specifically, a spatial Multi-Head Attention Graph U-Net was first adopted to model the spatial patterns of multiple brain networks, and a temporal Multi-Head Guided Attention Network was then introduced to model the corresponding temporal patterns under the guidance of modeled spatial patterns. Based on seven task fMRI datasets from the public Human Connectome Project and resting state fMRI datasets from the public Autism Brain Imaging Data Exchange I of 1448 subjects, the proposed Multi-Head GAGNN showed superior ability and generalizability in modeling both spatial and temporal patterns of holistic functional brain networks in individual brains compared to other state-of-the-art (SOTA) models. Furthermore, the modeled spatio-temporal patterns of functional brain networks via the proposed Multi-Head GAGNN can better predict the individual cognitive behavioral measures compared to the other SOTA models. This study provided a novel and powerful tool for brain function modeling as well as for understanding the brain-cognitive behavior associations.
Objective:
Sparse representations have been utilized to identify functional connectivity (FC) of networks, while ICA employs the assumption of independence among the network sources to demonstrate FC. Here, we investigate a sparse decomposition method based on Morphological Component Analysis and K-SVD dictionary learning-MCA-KSVD-and contrast the effect of the sparsity constraint vs. the independency constraint on FC and denoising.
Methods:
Using a K-SVD algorithm, fMRI signals are decomposed into morphological components which have sparse spatial overlap. We present simulations when the independency assumption of ICA fails and MCA-KSVD recovers more accurate spatial-temporal structures. Denoising performance of both methods is investigated at various noise levels. A comprehensive experimental study was conducted on resting-state and task fMRI.
Results:
Validations show that ICA is advantageous when network components are well-separated and sparse. In such cases, the MCA-KSVD method has modest value over ICA in terms of network delineation but is significantly more effective in reducing spatial and temporal noise. Results demonstrate that the sparsity constraint yields sparser networks with higher spatial resolution while suppressing weak signals. Temporally, this localization effect yields higher contrast-to-noise ratios (CNRs) of time series.
Conclusion:
While marginally improving the spatial decomposition, MCA-KSVD denoises fMRI data much more effectively than ICA, preserving network structures and improving CNR, especially for weak networks.
Significance:
A sparsity-based decomposition approach may be useful for investigating functional connectivity in noisy cases. It may serve as an efficient decomposition method for reduced acquisition time and may prove useful for detecting weak network activations.
Compelling evidence has suggested that the human cerebellum is engaged in a wide range of cognitive tasks besides traditional opinions of motor control, and it is organized into a set of distinct functional subregions. The existing model-driven cerebellum parcellations through resting-state functional MRI (rsfMRI) and task-fMRI are relatively coarse, introducing challenges in resolving the functions of the cerebellum especially when the brain is exposed to naturalistic environments. The current study took the advantages of the naturalistic paradigm (i.e., movie viewing) fMRI (nfMRI) to derive fine parcellations via a data-driven dual-regression-like sparse representation framework. The parcellations were quantitatively evaluated by functional homogeneity, and global and local boundary confidence. In addition, the differences of cerebellum–cerebrum functional connectivities between rsfMRI and nfMRI for some exemplar parcellations were compared to provide qualitatively functional validations. Our experimental results demonstrated that the proposed study successfully identified distinct subregions of the cerebellum. This fine parcellation may serve as a complementary solution to existing cerebellum parcellations, providing an alternative template for exploring neural activities of the cerebellum in naturalistic environments.
Mild traumatic brain injury (mTBI) is usually caused by a bump, blow, or jolt to the head or penetrating head injury, and carries the risk of inducing cognitive disorders. However, identifying the biomarkers for the diagnosis of mTBI is challenging as evident abnormalities in brain anatomy are rarely found in patients with mTBI. In this study, we tested whether the alteration of functional network dynamics could be used as potential biomarkers to better diagnose mTBI. We propose a sparse dictionary learning framework to delineate spontaneous fluctuation of functional connectivity into the subject‐specific time‐varying evolution of a set of overlapping group‐level sparse connectivity components (SCCs) based on the resting‐state functional magnetic resonance imaging (fMRI) data from 31 mTBI patients in the early acute phase (<3 days postinjury) and 31 healthy controls (HCs). The identified SCCs were consistently distributed in the cohort of subjects without significant inter‐group differences in connectivity patterns. Nevertheless, subject‐specific temporal expression of these SCCs could be used to discriminate patients with mTBI from HCs with a classification accuracy of 74.2% (specificity 64.5% and sensitivity 83.9%) using leave‐one‐out cross‐validation. Taken together, our findings indicate neuroimaging biomarkers for mTBI individual diagnosis based on the temporal expression of SCCs underlying time‐resolved functional connectivity. We proposed a sparse dictionary learning framework to delineate spontaneous fluctuation of functional connectivity into the subject‐specific time‐varying evolvement of a set of overlapping group‐level sparse connectivity components (SCCs). The identified SCCs were consistently distributed in the cohort of subjects without significant inter‐group differences in connectivity patterns. Nevertheless, subject‐specific temporal expression of these SCCs could be used to effectively discriminate mTBI patients from healthy controls (HCs).
Since the complex brain functions are achieved by the interaction of functional brain networks with the specific spatial distributions and temporal dynamics, modeling the spatial and temporal patterns of functional brain networks based on 4D fMRI data offers a way to understand the brain functional mechanisms. Matrix decomposition methods and deep learning methods have been developed to provide solutions. However, the underlying nature of functional brain networks remains unclear due to underutilizing the spatio-temporal characteristics of 4D fMRI input in previous methods. To address this problem, we propose a novel Guided Attention 4D Convolutional Neural Network (GA-4DCNN) to model spatial and temporal patterns of functional brain networks simultaneously. GA-4DCNN consists of two subnetworks: the spatial 4DCNN and the temporal Guided Attention (GA) network. The 4DCNN firstly extracts spatio-temporal characteristics of fMRI input to model the spatial pattern, while the GA network further models the corresponding temporal pattern guided by the modeled spatial pattern. Based on two task fMRI datasets from the Human Connectome Project, experimental results show that the proposed GA-4DCNN has superior ability and generalizability in modeling spatial and temporal patterns compared to other state-of-the-art methods. This study provides a new useful tool for modeling and understanding brain function.
Preterm is a worldwide problem that affects infants’ lives significantly. Moreover, the early impairment is more than limited to isolated brain regions but also to global and profound negative outcomes later, such as cognitive disorder. Therefore, seeking the differences of brain connectome between preterm and term infant brains is a vital step for understanding the developmental impairment caused by preterm. Existing studies revealed that studying the relationship between brain function and structure, and further investigating their differentiable connectomes between preterm and term infant brains is a way to comprehend and unveil the differences that occur in the preterm infant brains. Therefore, in this article, we proposed a novel canonical correlation analysis (CCA) with locality preserving projection (LPP) approach to investigate the relationship between brain functional and structural connectomes and how such a relationship differs between preterm and term infant brains. CCA is proposed to study the relationship between functional and structural connections, while LPP is adopted to identify the distinguishing features from the connections which can differentiate the preterm and term brains. After investigating the whole brain connections on a fine-scale connectome approach, we successfully identified 89 functional and 97 structural connections, which mostly contributed to differentiate preterm and term infant brains from the functional MRI (fMRI) and diffusion MRI (dMRI) of the public developing Human Connectome Project (dHCP) dataset. By further exploring those identified connections, the results innovatively revealed that the identified functional connections are short-range and within the functional network. On the contrary, the identified structural connections are usually remote connections across different functional networks. In addition, these connectome-level results show the new insights that longitudinal functional changes could deviate from longitudinal structural changes in the preterm infant brains, which help us better understand the brain-behavior changes in preterm infant brains.
It has been widely demonstrated that complex brain function is mediated by the interaction of multiple concurrent brain functional networks, each of which is spatially distributed across specific brain regions in a temporally dynamic fashion. Therefore, modeling spatio-temporal patterns of those holistic brain functional networks provides a foundation for understanding the brain. Compared to conventional modeling approaches such as correlation, general linear model, and matrix decomposition methods, recent deep learning methodologies have shown a superior performance. However, the existing deep learning models either underutilized both spatial and temporal characteristics of fMRI during model training, or merely focused on modeling only one targeted brain functional network at a time while ignoring holistic ones, resulting in a significant gap in our current understanding of how the brain functions. To bridge this gap, we propose a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) to simultaneously model spatio-temporal patterns of multiple brain functional networks. In Multi-Head GAGNN, the spatial patterns of multiple brain networks are firstly modeled in a multi-head attention graph U-net, and then adopted as guidance for modeling the corresponding temporal patterns of multiple brain networks in a temporal multi-head guided attention network model. Results based on two task fMRI datasets from the public Human Connectome Project demonstrate superior ability and generalizability of Multi-Head GAGNN in simultaneously modeling spatio-temporal patterns of holistic brain functional networks compared to other state-of-the-art models. This study offers a new and powerful tool for helping understand complex brain function.
Considering the two-class classification problem in brain imaging data analysis, we propose a sparse representation-based multi-variate pattern analysis (MVPA) algorithm to localize brain activation patterns corresponding to different stimulus classes/brain states respectively. Feature selection can be modeled as a sparse representation (or sparse regression) problem. Such technique has been successfully applied to voxel selection in fMRI data analysis. However, single selection based on sparse representation or other methods is prone to obtain a subset of the most informative features rather than all. Herein, our proposed algorithm recursively eliminates informative features selected by a sparse regression method until the decoding accuracy based on the remaining features drops to a threshold close to chance level. In this way, the resultant feature set including all the identified features is expected to involve all the informative features for discrimination. According to the signs of the sparse regression weights, these selected features are separated into two sets corresponding to two stimulus classes/brain states. Next, in order to remove irrelevant/noisy features in the two selected feature sets, we perform a nonparametric permutation test at the individual subject level or the group level. In data analysis, we verified our algorithm with a toy data set and an intrinsic signal optical imaging data set. The results show that our algorithm has accurately localized two class-related patterns. As an application example, we used our algorithm on a functional magnetic resonance imaging (fMRI) data set. Two sets of informative voxels, corresponding to two semantic categories (i.e., "old people" and "young people"), respectively, are obtained in the human brain.
Common error in bibliographies: "Étude comparative de la distribution florale dans une portion des Alpes et des Jura".
We propose a novel statistical analysis method for functional magnetic resonance imaging (fMRI) to overcome the drawbacks of conventional data-driven methods such as the independent component analysis (ICA). Although ICA has been broadly applied to fMRI due to its capacity to separate spatially or temporally independent components, the assumption of independence has been challenged by recent studies showing that ICA does not guarantee independence of simultaneously occurring distinct activity patterns in the brain. Instead, sparsity of the signal has been shown to be more promising. This coincides with biological findings such as sparse coding in V1 simple cells, electrophysiological experiment results in the human medial temporal lobe, etc. The main contribution of this paper is, therefore, a new data driven fMRI analysis that is derived solely based upon the sparsity of the signals. A compressed sensing based data-driven sparse generalized linear model is proposed that enables estimation of spatially adaptive design matrix as well as sparse signal components that represent synchronous, functionally organized and integrated neural hemodynamics. Furthermore, a minimum description length (MDL)-based model order selection rule is shown to be essential in selecting unknown sparsity level for sparse dictionary learning. Using simulation and real fMRI experiments, we show that the proposed method can adapt individual variation better compared to the conventional ICA methods.
Techniques from sparse signal representation are beginning to see significant impact in computer vision, often on nontraditional applications where the goal is not just to obtain a compact high-fidelity representation of the observed signal, but also to extract semantic information. The choice of dictionary plays a key role in bridging this gap: unconventional dictionaries consisting of, or learned from, the training samples themselves provide the key to obtaining state-of-the-art results and to attaching semantic meaning to sparse signal representations. Understanding the good performance of such unconventional dictionaries in turn demands new algorithmic and analytical techniques. This review paper highlights a few representative examples of how the interaction between sparse signal representation and computer vision can enrich both fields, and raises a number of open questions for further study.
In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Search- ing for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be recon- structed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in sig- nals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representa- tion. The approach combines the discrimination power of the discriminative meth- ods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of cor- rupted signals.
Sparse coding provides a class of algorithms for finding succinct representations of stimuli; given only unlabeled input data, it discovers basis functions that cap- ture higher-level features in the data. However, finding sparse codes remains a very difficult computational problem. In this paper, we present efficient sparse coding algorithms that are based on iteratively solving two convex optimization problems: an L1-regularized least squares problem and an L2-constrained least squares problem. We propose novel algorithms to solve both of these optimiza- tion problems. Our algorithms result in a significant speedup for sparse coding, allowing us to learn larger sparse codes than possible with previously described algorithms. We apply these algorithms to natural images and demonstrate that the inferred sparse codes exhibit end-stopping and non-classical receptive field sur- round suppression and, therefore, may provide a partial explanation for these two phenomena in V1 neurons.
Sparse representation based classification has led to interesting image recognition results, while the dictionary used for sparse coding plays a key role in it. This paper presents a novel dictionary learning (DL) method to improve the pattern classification performance. Based on the Fisher discrimination criterion, a structured dictionary, whose dictionary atoms have correspondence to the class labels, is learned so that the reconstruction error after sparse coding can be used for pattern classification. Meanwhile, the Fisher discrimination criterion is imposed on the coding coefficients so that they have small within-class scatter but big between-class scatter. A new classification scheme associated with the proposed Fisher discrimination DL (FDDL) method is then presented by using both the discriminative information in the reconstruction error and sparse coding coefficients. The proposed FDDL is extensively evaluated on benchmark image databases in comparison with existing sparse representation and DL based classification methods.
In this study, we present an advanced Bayesian framework for the analysis of functional magnetic resonance imaging (fMRI) data that simultaneously employs both spatial and sparse properties. The basic building block of our method is the general linear regression model that constitutes a well-known probabilistic approach. By treating regression coefficients as random variables, we can apply an enhanced Gibbs distribution function that captures spatial constrains and at the same time allows sparse representation of fMRI time series. The proposed scheme is described as a maximum a posteriori approach, where the known expectation maximization algorithm is applied offering closed-form update equations for the model parameters. We have demonstrated that our method produces improved performance and functional activation detection capabilities in both simulated data and real applications.
Sparse coding--that is, modelling data vectors as sparse linear combinations of basis elements--is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set, adapting it to specific data. Variations of this problem include dictionary learning in signal processing, non-negative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large datasets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to state-of-the-art performance in terms of speed and optimization for both small and large datasets. Comment: revised version
In this, the 21st century, human-brain mapping celebrates 21 years of cognitive activation studies. This review looks at imaging
neuroscience and key ideas it has pursued; some ideas portend exciting developments, and others have failed gloriously. In
terms of achievements, there is much to celebrate, in the sense that it is difficult to imagine modern neuroscience without
brain imaging. I will look at recent advances from the perspectives of functional segregation and integration in the brain,
paying special attention to approaches that deal with the distributed and integrated nature of neuronal processing and the
questions they address.
A method is given for determining the time course and spatial extent of consistently and transiently task-related activations from other physiological and artifactual components that contribute to functional MRI (fMRI) recordings. Independent component analysis (ICA) was used to analyze two fMRI data sets from a subject performing 6-min trials composed of alternating 40-sec Stroop color-naming and control task blocks. Each component consisted of a fixed three-dimensional spatial distribution of brain voxel values (a “map”) and an associated time course of activation. For each trial, the algorithm detected, without a priori knowledge of their spatial or temporal structure, one consistently task-related component activated during each Stroop task block, plus several transiently task-related components activated at the onset of one or two of the Stroop task blocks only. Activation patterns occurring during only part of the fMRI trial are not observed with other techniques, because their time courses cannot easily be known in advance. Other ICA components were related to physiological pulsations, head movements, or machine noise. By using higher-order statistics to specify stricter criteria for spatial independence between component maps, ICA produced improved estimates of the temporal and spatial extent of task-related activation in our data compared with principal component analysis (PCA). ICA appears to be a promising tool for exploratory analysis of fMRI data, particularly when the time courses of activation are not known in advance.
The receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.
Although the role of cortical areas MT and MST (MT/MST) in the processing of directional motion information is well established, little is known about the way these areas contribute to the execution of complex behavioral tasks requiring the use of such information. We tested monkeys with unilateral lesions of these areas on a visual working memory task in which motion signals not only had to be encoded, but also stored for brief periods of time and then retrieved. The monkeys compared the directions of motion of two random-dot stimuli, sample and test, separated by a temporal delay. By increasing the temporal delay and spatially separating the two stimuli, placing one in the affected visual field and the other in the intact visual field, we were able to assess the contribution of MT/MST to specific components of the task: encoding (sample), retention (delay) and encoding/retrieval/comparison (test). We found that the effects of MT/MST lesions on specific components depended upon the demands of the task and the nature of the visual motion stimuli. Whenever stimuli consisted of random dots moving in a broad range of directions, MT/MST lesions appeared to affect encoding. Furthermore, when the lesions affected encoding of the sample, retention of the direction of stimulus motion was also affected. However, when the stimulus was coherent and the emphasis of the task was on the comparison of small direction differences, the absence of MT/MST had major impact on the retrieval/comparison component of the task and not on encoding or storage.
Discrimination thresholds were obtained using a delayed spatial frequency discrimination task. In Experiment 1, we found that presentation of a mask 3 s before onset of a reference Gabor patch caused a selective, spatial frequency dependent interference in a subsequent discrimination task. However, a 10 s interval abolished this masking effect. In Experiment 2, the mask was associated with a second spatial frequency discrimination task so that a representation of the mask had to be coded into short-term perceptual memory. This experiment was performed to assess whether absence of masking in the 10 s condition of Experiment 1 might be due to decay of the mask information in the perceptual or the memory representational domain. The presence of this second discrimination task now caused similar interference effects on the primary discrimination task at both the 3 s and 10 s interstimulus intervals (ISI) conditions. Finally, to test the robustness of the masking effect, the nature of the secondary masking task was changed from a spatial frequency discrimination task to an orientation discrimination task in Experiment 3. The masking effect was now abolished in both the 3 and 10 s ISI conditions. Together, the results from these experiments are consistent with the idea of a two-level perceptual memory mechanism. The results also suggest that stimulus representations during a perceptual discrimination task are shared between the perceptual and memory representation domains in a task-dependent manner.
Recent neuroimaging studies have lead to the proposal that rest is characterized by an organized, baseline level of activity, a default mode of brain function that is suspended during specific goal-oriented mental activity. Previous studies have shown that the primary function subserved by the default mode is that of an introspectively oriented, self-referential mode of mental activity. The default mode of brain function hypothesis is readdressed from the perspective of the presence of low-frequency blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) signal changes (0.012-0.1 Hz) in the resting brain. The results show that the brain during rest is not tonically active in a single mode of brain function. Rather, the findings presented here suggest that the brain recurrently toggles between an introspectively oriented mode (default mode) and a state-of-mind that tentatively might be interpreted as an extrospectively oriented mode that involves a readiness and alertness to changes in the external and internal environment.
Inferior parietal lobule (IPL) neurons were studied when monkeys performed motor acts embedded in different actions and when
they observed similar acts done by an experimenter. Most motor IPL neurons coding a specific act (e.g., grasping) showed markedly
different activations when this act was part of different actions (e.g., for eating or for placing). Many motor IPL neurons
also discharged during the observation of acts done by others. Most responded differentially when the same observed act was
embedded in a specific action. These neurons fired during the observation of an act, before the beginning of the subsequent
acts specifying the action. Thus, these neurons not only code the observed motor act but also allow the observer to understand
the agent's intentions.
During performance of attention-demanding cognitive tasks, certain regions of the brain routinely increase activity, whereas others routinely decrease activity. In this study, we investigate the extent to which this task-related dichotomy is represented intrinsically in the resting human brain through examination of spontaneous fluctuations in the functional MRI blood oxygen level-dependent signal. We identify two diametrically opposed, widely distributed brain networks on the basis of both spontaneous correlations within each network and anticorrelations between networks. One network consists of regions routinely exhibiting task-related activations and the other of regions routinely exhibiting task-related deactivations. This intrinsic organization, featuring the presence of anticorrelated networks in the absence of overt task performance, provides a critical context in which to understand brain function. We suggest that both task-driven neuronal responses and behavior are reflections of this dynamic, ongoing, functional organization of the brain.
• functional MRI
• functional connectivity
• spontaneous activity
Papers from the 2006 flagship meeting on neural computation, with contributions from physicists, neuroscientists, mathematicians, statisticians, and computer scientists.
The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation and machine learning. It draws a diverse group of attendees—physicists, neuroscientists, mathematicians, statisticians, and computer scientists—interested in theoretical and applied aspects of modeling, simulating, and building neural-like or intelligent systems. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December 2006 meeting, held in Vancouver.
Bradford Books imprint
Using spike-triggered fMRI, we sought to document regional changes in blood oxygen level-dependent (BOLD) activity associated with spontaneous generalised spike and slow-wave discharges (S&W). Five adult patients were studied who had idiopathic generalised epilepsy (IGE) and frequent S&W. EEG was recorded inside a 3T MRI, allowing acquisition of single, whole-brain fMRI images following S&W, as well as baseline images. Between 4 and 25 spike and wave complexes were captured in individuals. Four of the five individuals showed significant S&W-related BOLD signal reductions ("deactivation") in the posterior cingulate (P < 0.001 uncorrected). Significant changes were absent only in the individual with fewest spikes. Group analysis including all five subjects confirmed fMRI deactivation in the posterior cingulate. Some S&W-related BOLD signal increases were seen in the depths of the precentral sulci in individuals and on group analysis. No significant S&W-related changes in thalamic BOLD activity were observed. The posterior cingulate may have a role in the electroclinical phenomenon of S&W and "absence." Possibilities include a causative role, with reduced activity in the cingulate facilitating the onset of S&W, a secondary role, explaining the cognitive changes observed during prolonged S&W, or an epiphenomenon.
Distance or similarity measures are essential to solve many pattern recognition problems such as classification, clustering, and retrieval problems. Various distance/similarity measures that are applicable to compare two probability density functions, pdf in short, are reviewed and categorized in both syntactic and semantic relationships. A correlation coefficient and a hierarchical clustering technique are adopted to reveal similarities among numerous distance/similarity measures.
A novel group analysis tool for data-driven resting state fMRI analysis using group sparse dictionary learning and mixed model is presented along with the promising indications of Alzheimer's disease progression. Instead of using independency assumption as in popular ICA approaches, the proposed approach is based on the sparse graph assumption such that a temporal dynamics at each voxel position is a sparse combination of global brain dynamics. In estimating the unknown global dynamics and local network structures, we perform sparse dictionary learning for the concatenated temporal data across the subjects by constraining that the network structures within a group are similar. Under the homoscedasticity variance assumption across subjects and groups, we show that the mixed model group inference can be easily performed using second level GLM with summary statistics. Using extensive resting fMRI data set obtained from normal, Mild Cognitive Impairment (MCI), Clinical Dementia Rating scale (CDR) 0.5, CDR 1.0, and CDR 2.0 of Alzheimer's disease patients groups, we demonstrated that the changes of default mode network extracted by the proposed method is more closely correlated with the progression of Alzheimer's disease.
The human brain function involves complex processes with population codes of neuronal activities. Neuroscience research has demonstrated that when representing neuronal activities, sparsity is an important characterizing property. Inspired by this finding, significant amount of efforts from the scientific communities have been recently devoted to sparse representations of signals and patterns, and promising achievements have been made. However, sparse representation of fMRI signals, particularly at the population level of a group of different brains, has been rarely explored yet. In this paper, we present a novel group-wise sparse representation of task-based fMRI signals from multiple subjects via dictionary learning methods. Specifically, we extract and pool task-based fMRI signals for a set of cortical landmarks, each of which possesses intrinsic anatomical correspondence, from a group of subjects. Then an effective online dictionary learning algorithm is employed to learn an over-complete dictionary from the pooled population of fMRI signals based on optimally determined dictionary size. Our experiments have identified meaningful Atoms of Interests (AOI) in the learned dictionary, which correspond to consistent and meaningful functional responses of the brain to external stimulus. Our work demonstrated that sparse representation of group-wise fMRI signals is naturally suitable and effective in recovering population codes of neuronal signals conveyed in fMRI data.
In this paper, the problem of dictionary learning and its analogy to source separation is addressed. First, we extend the well-known method of K-SVD to incoherent K-SVD, to enforce the algorithm to achieve an incoherent dictionary. Second, a fast dictionary learning algorithm based on steepest descent method is proposed. The main advantage of this method is high speed since both coefficients and dictionary elements are updated simultaneously rather than column-by-column. Finally, we apply the proposed methods to both synthetic and real functional magnetic resonance imaging data for the detection of activated regions in the brain. The results of our experiments confirm the effectiveness of the proposed ideas. In addition, we compare the quality of results and empirically prove the superiority of the proposed dictionary learning methods over the conventional algorithms.
Functional connectomes (FCs) have been recently shown to be powerful in characterizing brain conditions. However, many previous studies assumed temporal stationarity of FCs, while their temporal dynamics are rarely explored. Here, based on the structural connectomes constructed from diffusion tensor imaging data, FCs are derived from resting-state fMRI (R-fMRI) data and are then temporally divided into quasi-stable segments via a sliding time window approach. After integrating and pooling over a large number of those temporally quasi-stable FC segments from 44 post-traumatic stress disorder (PTSD) patients and 51 healthy controls, common FC (CFC) patterns are derived via effective dictionary learning and sparse coding algorithms. It is found that there are 16 CFC patterns that are reproducible across healthy controls, and interestingly, two additional CFC patterns with altered connectivity patterns [termed signature FC (SFC) here] exist dominantly in PTSD subjects. These two SFC patterns alone can successfully differentiate 80% of PTSD subjects from healthy controls with only 2% false positive. Furthermore, the temporal transition dynamics of CFC patterns in PTSD subjects are substantially different from those in healthy controls. These results have been replicated in separate testing datasets, suggesting that dynamic functional connectomics signatures can effectively characterize and differentiate PTSD patients. Hum Brain Mapp, 2013. © 2013 Wiley Periodicals, Inc.
Functional magnetic resonance imaging (fMRI) is currently the mainstay of neuroimaging in cognitive neuroscience. Advances in scanner technology, image acquisition protocols, experimental design, and analysis methods promise to push forward fMRI from mere cartography to the true study of brain organization. However, fundamental questions concerning the interpretation of fMRI data abound, as the conclusions drawn often ignore the actual limitations of the methodology. Here I give an overview of the current state of fMRI, and draw on neuroimaging and physiological data to present the current understanding of the haemodynamic signals and the constraints they impose on neuroimaging data interpretation.
Functional magnetic resonance imaging (fMRI) time series is non-linear and composed of components at multiple temporal scales, which presents significant challenges to its analysis. In the literature, significant effort has been devoted into model-based fMRI signal analysis, while much less attention has been directed to data-driven fMRI signal analysis. In this paper, we present a novel data-driven multi-scale signal decomposition framework named Empirical Mean Curve Decomposition (EMCD). Targeted on functional brain mapping, the EMCD optimizes mean envelopes from fMRI signals and iteratively extracts coarser-to-finer scale signal components. The EMCD framework was applied to infer meaningful low-frequency information from Blood Oxygenation Level Dependent (BOLD) signals from resting state fMRI, task-based fMRI, and natural stimulus fMRI, and promising results are obtained.
Image processing strategies for functional magnetic resonance imaging (FMRI) data sets acquired using a gradient-recalled echo-planar imaging sequence are considered. The analysis is carried out using the mathematics of vector spaces. Data sets consisting of N sequential images of the same slice of brain tissue are analyzed in the time-domain and also, after Fourier transformation, in the frequency domain. A technique for thresholding is introduced that uses the shape of the response in a pixel compared with the shape of a reference waveform as the decision criterion. A method is presented to eliminate drifts in data that arise from subject movement. The methods are applied to experimental FMRI data from the motor—cortex and compared with more conventional image—subtraction methods. Several finger motion paradigms are considered in the context of the various image processing strategies. The most effective method for image processing involves thresholding by shape as characterized by the correlation coefficient of the data with respect to a reference waveform followed by formation of a cross-correlation image. Emphasis is placed not only on image formation, but also on the use of signal processing techniques to characterize the temporal response of the brain to the paradigm.
Statistical parametric maps are spatially extended statistical processes that are used to test hypotheses about regionally specific effects in neuroimaging data. The most established sorts of statistical parametric maps (e.g., Friston et al. [1991]: J Cereb Blood Flow Metab 11:690–699; Worsley et al. [1992]: J Cereb Blood Flow Metab 12:900–918) are based on linear models, for example ANCOVA, correlation coefficients and t tests. In the sense that these examples are all special cases of the general linear model it should be possible to implement them (and many others) within a unified framework. We present here a general approach that accomodates most forms of experimental layout and ensuing analysis (designed experiments with fixed effects for factors, covariates and interaction of factors). This approach brings together two well established bodies of theory (the general linear model and the theory of Gaussian fields) to provide a complete and simple framework for the analysis of imaging data.
The importance of this framework is twofold: (i) Conceptual and mathematical simplicity, in that the same small number of operational equations is used irrespective of the complexity of the experiment or nature of the statistical model and (ii) the generality of the framework provides for great latitude in experimental design and analysis.
Recently, Everitt and Bullmore [1999] proposed a mixture model for a test statistic for activation in fMRI data. The distribution of the statistic was divided into two components; one for nonactivated voxels and one for activated voxels. In this framework one can calculate a posterior probability for a voxel being activated, which provides a more natural basis for thresholding the statistic image, than that based on P-values. In this article, we extend the method of Everitt and Bullmore to account for spatial coherency of activated regions. We achieve this by formulating a model for the activation in a small region of voxels and using this spatial structure when calculating the posterior probability of a voxel being activated. We have investigated several choices of spatial models but find that they all work equally well for brain imaging data. We applied the model to synthetic data from statistical image analysis, a synthetic fMRI data set and to visual stimulation data. Our conclusion is that the method improves the estimation of the activation pattern significantly, compared to the nonspatial model and to smoothing the data with a kernel of FWHM 3 voxels. The difference between FWHM 2 smoothing and our method were more modest. Hum. Brain Mapping 11:233–248, 2000. © 2000 Wiley-Liss, Inc.
We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C
<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims.
Segregation and integration are two general principles of the brain's functional architecture; therefore brain network analysis is of significant importance in understanding brain function. Critical to brain network analysis and construction is the identification of reliable, reproducible and accurate network nodes, or Regions of Interest (ROIs). In this paper, based on functional ROIs derived from task-based fMRI, we propose a novel framework to optimize the location and size of the ROIs which ensures that the difference of structural connectivity profiles among a group of subjects is minimized. In order to facilitate the optimization procedure, we present a new approach to quantitatively describe and measure the fiber bundle similarity within and across subjects. This framework has been extensively evaluated and our experimental results suggest the promise of our approaches. This capability of accurately localizing brain network ROIs would open up many applications in brain imaging that rely on identification of functional ROIs.
Segregation and integration are two general principles of the brain's functional architecture. Therefore, brain network analysis is of significant importance in understanding brain function. Critical to brain network construction and analysis is the identification of reliable, reproducible, and accurate network nodes, or Regions of Interest (ROIs). Task-based fMRI has been widely considered as a reliable approach to identify functionally meaningful ROIs in the brain. However, recent studies have shown that factors such as spatial smoothing could considerably shift the locations of detected activation peaks. As a result, structural and functional connectivity patterns can be significantly altered. Here, we propose a novel framework by which to optimize ROI sizes and locations, ensuring that differences between the structural connectivity profiles among a group of subjects is minimized. This framework is based on functional ROIs derived from task-based fMRI and diffusion tensor imaging (DTI) data. Accordingly, we present a new approach to describe and measure the fiber bundle similarity quantitatively within and across subjects which will facilitate the optimization procedure. Experimental results demonstrated that this framework improved the localizations of fMRI-derived ROIs. Through our optimization procedure, structural and functional connectivities were more consistent across different individuals. Overall, the ability to accurately localize network ROIs could facilitate many applications in brain imaging that rely on the accurate identification of ROIs.
The working memory (WM) system is vital to performing everyday functions that require attentive, non-automatic processing of information. However, its interaction with long term memory (LTM) is highly debated. Here, we used fMRI to examine whether a popular complex WM span task, thought to force the displacement of to-be-remembered items in the focus of attention to LTM, recruited medial temporal regions typically associated with LTM functioning to a greater extent and in a different manner than traditional neuroimaging WM tasks during WM encoding and maintenance. fMRI scans were acquired while participants performed the operation span (OSPAN) task and an arithmetic task. Results indicated that performance of both tasks resulted in significant activation in regions typically associated with WM function. More importantly, significant bilateral activation was observed in the hippocampus, suggesting it is recruited during WM encoding and maintenance. Right posterior hippocampus activation was greater during OSPAN than arithmetic. Persitimulus graphs indicate a possible specialization of function for bilateral posterior hippocampus and greater involvement of the left for WM performance. Recall time-course activity within this region hints at LTM involvement during complex span.
The brain's energy economy excessively favors intrinsic, spontaneous neural activity over extrinsic, evoked activity, presumably to maintain its internal organization. Emerging hypotheses capable of explaining such an investment posit that the brain's intrinsic functional architecture encodes a blueprint for its repertoire of responses to the external world. Yet, there is little evidence directly linking intrinsic and extrinsic activity in the brain. Here we relate differences among individuals in the magnitude of task-evoked activity during performance of an Eriksen flanker task, to spontaneous oscillatory phenomena observed during rest. Specifically, we focused on the amplitude of low-frequency oscillations (LFO, 0.01-0.1 Hz) present in the BOLD signal. LFO amplitude measures obtained during rest successfully predicted the magnitude of task-evoked activity in a variety of regions that were all activated during performance of the flanker task. In these regions, higher LFO amplitude at rest predicted higher task-evoked activity. LFO amplitude measures obtained during rest were also found to have robust predictive value for behavior. In midline cingulate regions, LFO amplitudes predicted not only the speed and consistency of performance but also the magnitude of the behavioral congruency effect embedded in the flanker task. These results support the emerging hypothesis that the brain's repertoire of responses to the external world are represented and updated in the brain's intrinsic functional architecture.
The first part of this paper gives an overview of a simplified approach to the statistical analysis of PET and fMRI data, including new developments and future directions. The second part outlines a new method, based on multivariate linear models (MLM), for characterising the response in PET and fMRI data, which overcomes some of the drawbacks of current methods such as SSM, SVD, PLS and CVA.
The resting brain exhibits coherent patterns of spontaneous low-frequency BOLD fluctuations. These so-called resting-state functional connectivity (RSFC) networks are posited to reflect intrinsic representations of functional systems commonly implicated in cognitive function. Yet, the direct relationship between RSFC and the BOLD response induced by task performance remains unclear. Here we examine the relationship between a region's pattern of RSFC across participants and that same region's level of BOLD activation during an Eriksen Flanker task. To achieve this goal we employed a voxel-matched regression method, which assessed whether the magnitude of task-induced activity at each brain voxel could be predicted by measures of RSFC strength for the same voxel, across 26 healthy adults. We examined relationships between task-induced activation and RSFC strength for six different seed regions [Fox, M.D., Snyder, A.Z., Vincent, J.L., Corbetta, M., Van Essen, D.C., Raichle, M.E., 2005. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc. Natl. Acad. Sci. U. S. A. 102, 9673-9678.], as well as the "default mode" and "task-positive" resting-state networks in their entirety. Our results indicate that, for a number of brain regions, inter-individual differences in task-induced BOLD activity were predicted by one of two resting-state properties: (1) the region's positive connectivity strength with the task-positive network, or (2) its negative connectivity with the default mode network. Strikingly, most of the regions exhibiting a significant relationship between their RSFC properties and task-induced BOLD activity were located in transition zones between the default mode and task-positive networks. These results suggest that a common mechanism governs many brain regions' neural activity during rest and its neural activity during task performance.
InfoMax and FastICA are the independent component analysis algorithms most used and apparently most effective for brain fMRI. We show that this is linked to their ability to handle effectively sparse components rather than independent components as such. The mathematical design of better analysis tools for brain fMRI should thus emphasize other mathematical characteristics than independence.
Image processing strategies for functional magnetic resonance imaging (FMRI) data sets acquired using a gradient-recalled echo-planar imaging sequence are considered. The analysis is carried out using the mathematics of vector spaces. Data sets consisting of N sequential images of the same slice of brain tissue are analyzed in the time-domain and also, after Fourier transformation, in the frequency domain. A technique for thresholding is introduced that uses the shape of the response in a pixel compared with the shape of a reference waveform as the decision criterion. A method is presented to eliminate drifts in data that arise from subject movement. The methods are applied to experimental FMRI data from the motor-cortex and compared with more conventional image-subtraction methods. Several finger motion paradigms are considered in the context of the various image processing strategies. The most effective method for image processing involves thresholding by shape as characterized by the correlation coefficient of the data with respect to a reference waveform followed by formation of a cross-correlation image. Emphasis is placed not only on image formation, but also on the use of signal processing techniques to characterize the temporal response of the brain to the paradigm.
In fMRI studies, Gaussian filtering is usually applied to improve the detection of activated areas. Such lowpass filtering enhances the signal to noise ratio. However, undesirable secondary effects are a bias on the signal shape and a blurring in the spatial domain. Neighboring activated areas may be merged and the high resolution of the fMRI data compromised. In the temporal domain, activation and deactivation slopes are also blurred. We propose an alternative to Gaussian filtering by restoring the signal using a spatiotemporal Markov Random Field which preserves the shape of the transitions. We define some interaction between neighboring voxels which allows us to reduce the noise while preserving the signal characteristics. An energy function is defined as the sum of the interaction potentials and is minimized using a simulated annealing algorithm. The shape of the hemodynamic response is preserved leading to a better characterization of its properties. We demonstrate the use of this approach by applying it to simulated data and to data obtained from a typical fMRI study.
Principal component analysis (PCA) is one of several structure-seeking multivariate statistical techniques, exploratory as well as inferential, that have been proposed recently for the characterization and detection of activation in both PET and fMRI time series data. In particular, PCA is data driven and does not assume that the neural or hemodynamic response reaches some steady state, nor does it involve correlation with any pre-defined or exogenous experimental design template. In this paper, we present a generalized linear systems framework for PCA based on the singular value decomposition (SVD) model for representation of spatio-temporal fMRI data sets. Statistical inference procedures for PCA, including point and interval estimation will be introduced without the constraint of explicit hypotheses about specific task-dependent effects. The principal eigenvectors capture both the spatial and temporal aspects of fMRI data in a progressive fashion; they are inherently matched to unique and uncorrelated features and are ranked in order of the amount of variance explained. PCA also acts as a variation reduction technique, relegating most of the random noise to the trailing components while collecting systematic structure into the leading ones. Features summarizing variability may not directly be those that are the most useful. Further analysis is facilitated through linear subspace methods involving PC rotation and strategies of projection pursuit utilizing a reduced, lower-dimensional natural basis representation that retains most of the information. These properties will be illustrated in the setting of dynamic time-series response data from fMRI experiments involving pharmacological stimulation of the dopaminergic nigro-striatal system in primates.
In functional magnetic resonance imaging statistical analysis there are problems with accounting for temporal autocorrelations when assessing change within voxels. Techniques to date have utilized temporal filtering strategies to either shape these autocorrelations or remove them. Shaping, or "coloring," attempts to negate the effects of not accurately knowing the intrinsic autocorrelations by imposing known autocorrelation via temporal filtering. Removing the autocorrelation, or "prewhitening," gives the best linear unbiased estimator, assuming that the autocorrelation is accurately known. For single-event designs, the efficiency of the estimator is considerably higher for prewhitening compared with coloring. However, it has been suggested that sufficiently accurate estimates of the autocorrelation are currently not available to give prewhitening acceptable bias. To overcome this, we consider different ways to estimate the autocorrelation for use in prewhitening. After high-pass filtering is performed, a Tukey taper (set to smooth the spectral density more than would normally be used in spectral density estimation) performs best. Importantly, estimation is further improved by using nonlinear spatial filtering to smooth the estimated autocorrelation, but only within tissue type. Using this approach when prewhitening reduced bias to close to zero at probability levels as low as 1 x 10(-5).
Wavelets provide an orthonormal basis for multiresolution analysis and decorrelation or 'whitening' of nonstationary time series and spatial processes. Wavelets are particularly well suited to analysis of biological signals and images, such as human brain imaging data, which often have fractal or scale-invariant properties. We briefly define some key properties of the discrete wavelet transform (DWT) and review its applications to statistical analysis of functional magnetic resonance imaging (fMRI) data. We focus on time series resampling by 'wavestrapping' of wavelet coefficients, methods for efficient linear model estimation in the wavelet domain, and wavelet-based methods for multiple hypothesis testing, all of which are somewhat simplified by the decorrelating property of the DWT.
Using spike-triggered fMRI, we sought to document regional changes in blood oxygen level-dependent (BOLD) activity associated with spontaneous generalised spike and slow-wave discharges (S&W). Five adult patients were studied who had idiopathic generalised epilepsy (IGE) and frequent S&W. EEG was recorded inside a 3T MRI, allowing acquisition of single, whole-brain fMRI images following S&W, as well as baseline images. Between 4 and 25 spike and wave complexes were captured in individuals. Four of the five individuals showed significant S&W-related BOLD signal reductions ("deactivation") in the posterior cingulate (P<0.001 uncorrected). Significant changes were absent only in the individual with fewest spikes. Group analysis including all five subjects confirmed fMRI deactivation in the posterior cingulate. Some S&W-related BOLD signal increases were seen in the depths of the precentral sulci in individuals and on group analysis. No significant S&W-related changes in thalamic BOLD activity were observed. The posterior cingulate may have a role in the electroclinical phenomenon of S&W and "absence." Possibilities include a causative role, with reduced activity in the cingulate facilitating the onset of S&W, a secondary role, explaining the cognitive changes observed during prolonged S&W, or an epiphenomenon.
Functional magnetic resonance imaging (fMRI) time series are investigated with a multifractal method based on the Wavelet Modulus Maxima (WTMM) method to extract local singularity ("fractal") exponents. The spectrum of singularity exponents of each fMRI time series is quantified by spectral characteristics including its maximum and the corresponding dimension. We found that the range of Hölder exponents in voxels with activation is close to 1, whereas exponents are close to 0.5 in white matter voxels without activation. The maximum dimension decreases going from white matter to gray matter, and is lower still for activated time series. The full-width-at-half-maximum of the spectra is higher in activated areas. The proposed method becomes particularly effective when combining these spectral characteristics into a single parameter. Using these multifractal parameters, it is possible to identify activated areas in the human brain in both hybrid and in vivo fMRI data sets without knowledge of the stimulation paradigm applied.
Several theoretical, computational, and experimental studies suggest that neurons encode sensory information using a small number of active neurons at any given point in time. This strategy, referred to as 'sparse coding', could possibly confer several advantages. First, it allows for increased storage capacity in associative memories; second, it makes the structure in natural signals explicit; third, it represents complex data in a way that is easier to read out at subsequent levels of processing; and fourth, it saves energy. Recent physiological recordings from sensory neurons have indicated that sparse coding could be a ubiquitous strategy employed in several different modalities across different organisms.