No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Elucidating relations between fMRI, ECoG, and EEG through a common natural stimulus
Abstract
Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.
... To maximise the chances of high reliability we used 10-minute recordings and, to provide a degree of consistency in brain activity, participants watched a movie clip during the scan. We chose a movie-viewing paradigm that has been used previously in fMRI, EEG and electrocorticography (ECoG) (Haufe et al., 2018). This standard task facilitates our objective and provides a new open-source resource with direct equivalence to existing data (Haufe et al., 2018). ...
... We chose a movie-viewing paradigm that has been used previously in fMRI, EEG and electrocorticography (ECoG) (Haufe et al., 2018). This standard task facilitates our objective and provides a new open-source resource with direct equivalence to existing data (Haufe et al., 2018). We quantitatively assess consistency between separate experimental runs and provide a benchmark for the reliability of connectivity measurement using OPM-MEG. ...
... During both recordings, participants watched the same 600 s clip of the movie "Dog Day Afternoon". The clip selected, which shows the scene of a bank robbery, was identical to that used in previous papers (Haufe et al., 2018;Honey et al., 2012;Lumet, 1975). Subjects remained seated and they were asked to watch the movie; they were free to move though not explicitly encouraged to do so. ...
Magnetoencephalography with optically pumped magnetometers (OPM-MEG) offers a new way to record electrophysiological brain function, with significant advantages over conventional MEG, including adaptability to head shape/size, free movement during scanning, increased signal amplitude, and no reliance on cryogenics. However, OPM-MEG remains in its infancy, with significant questions to be answered regarding the optimal system design. Here, we present an open-source dataset acquired using a newly constructed OPM-MEG system with a triaxial sensor design, 168 channels, OPM-optimised magnetic shielding, and active background field control. We measure the test-retest reliability of the human connectome, which was computed using amplitude envelope correlation to measure whole-brain (parcellated) functional connectivity, in 10 individuals while they watch a 600 s move clip. Our results show high repeatability between experimental runs at the group level, with a correlation coefficient of 0.81 in the θ, 0.93 in α, and 0.94 in β frequency ranges. At the individual subject level, we found marked differences between individuals, but high within-subject robustness (correlations of 0.56 ± 0.25, 0.72 ± 0.15, and 0.78 ± 0.13 in α, θ, and β respectively). These results compare well to previous findings using conventional MEG and show that OPM-MEG is a viable way to robustly characterise connectivity.
... To 3 maximise the chances of high reliability we used 10-minute recordings and, to provide a 4 degree of consistency in brain activity, participants watched a movie clip during the scan. We 5 chose a movie-viewing paradigm that has been used previously to assess relationships 6 between fMRI, EEG and electrocorticography (ECoG) (Haufe et al., 2018). This standard task 7 not only facilitates our objective to measure robustness but also provides a new open-source 8 resource with direct equivalence to existing data (Haufe et al., 2018). ...
... We 5 chose a movie-viewing paradigm that has been used previously to assess relationships 6 between fMRI, EEG and electrocorticography (ECoG) (Haufe et al., 2018). This standard task 7 not only facilitates our objective to measure robustness but also provides a new open-source 8 resource with direct equivalence to existing data (Haufe et al., 2018). We quantitatively assess 9 consistency between separate experimental runs and thus provide a benchmark for the 10 reliability of connectivity measurement using OPM-MEG. ...
... During both recordings, participants watched a 600 s clip of the movie "Dog Day Afternoon" 17 (Haufe et al., 2018;Honey et al., 2012;Lumet, 1975). Subjects remained seated and 18 continued wearing the sensor helmet between scans (so that a single co-registration of sensor 19 geometry to brain anatomy could be used for both measurements, reducing co-registration 20 error). ...
Magnetoencephalography with optically pumped magnetometers (OPM-MEG) offers a new way to record electrophysiological brain function, with significant advantages over conventional MEG including adaptability to head shape/size, free movement during scanning, better spatial resolution, increased signal, and no reliance on cryogenics. However, OPM-MEG remains in its infancy, with significant questions to be answered regarding optimal system design and robustness. Here, we present an open-source dataset acquired using a newly constructed OPM-MEG system with a triaxial sensor design averaging 168 channels. Using OPM-optimised magnetic shielding and active background-field control, we measure the test-retest reliability of the human connectome. We employ amplitude envelope correlation to measure whole-brain functional connectivity in 10 individuals whilst they watch a 600 s move clip. Our results show high repeatability between experimental runs at the group level, with a correlation coefficient of 0.81 in the theta, 0.93 in alpha and 0.94 in beta frequency ranges. At the individual subject level, we found marked differences between individuals, but high within-subject robustness (correlations of 0.56 ± 0.25, 0.72 ± 0.15 and 0.78 ± 0.13 in theta, alpha and beta respectively). These results compare well to previously reported findings using conventional MEG; they show that OPM-MEG is a viable way to characterise whole brain connectivity and add significant weight to a growing argument that OPMs can overtake cryogenic sensors as the fundamental building block of MEG systems.
... The relationship between EEG signals and invasive electrophysiological recordings has been investigated before mostly in the context of seizure detection (Kokkinos et al., 2019;Meisel and Bailey, 2019) and activity localization (Yamazaki et al., 2012;Hnazaee et al., 2020). Further studies have also explored frequency-band links between the two modalities (Petroff et al., 2016;Haufe et al., 2018;Meisel and Bailey, 2019). An emerging pattern from these studies is that the analytical method used for the comparison (Ding et al., 2007;Hnazaee et al., 2020) and the signal components of EEG and ECoG selected (Yamazaki et al., 2012;Meisel and Bailey, 2019) play a key role in achieving a reliable correspondence between the two modalities. ...
... First, while EEG can provide wholebrain coverage, and thus participant-specific data are used in EEG (Karimi et al., 2022;Moon et al., 2022;Xu et al., 2022), ECoG has very limited brain coverage as electrode coverage is dictated by medical reasons for each individual; as such usually, the data are pooled across subjects from different participants to provide a wider coverage (Sellers et al., 2019;Yang et al., 2019;Ahmadipour et al., 2021). Second, single trial ECoG data can provide reliable information (Jacques et al., 2016b;Haufe et al., 2018); however, it is typical to average several trials in EEG to reduce noise and acquire higher SNR. The process of averaging across trials is a common preprocessing step in EEG studies to increase signal-to-noise ratio (Wardle et al., 2016;Guggenmos et al., 2018;Kong et al., 2020;Ashton et al., 2022), and to obtain a more reliable signal. ...
... Our findings of a spatiotemporal correspondence between patterns of category-selective responses across ECoG and fMRI/EEG are in line with previous human studies reporting on between-modality correspondence in visual areas (Puce et al., 1997;Parvizi et al., 2012;Jacques et al., 2016b;Haufe et al., 2018). However, these studies did not investigate how the correspondence changes under variations in object size and/or orientation. ...
Today, most neurocognitive studies in humans employ the non-invasive neuroimaging techniques functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG). However, how the data provided by fMRI and EEG relate exactly to the underlying neural activity remains incompletely understood. Here, we aimed to understand the relation between EEG and fMRI data at the level of neural population codes using multivariate pattern analysis. In particular, we assessed whether this relation is affected when we change stimuli or introduce identity-preserving variations to them. For this, we recorded EEG and fMRI data separately from 21 healthy participants while participants viewed everyday objects in different viewing conditions, and then related the data to electrocorticogram (ECoG) data recorded for the same stimulus set from epileptic patients. The comparison of EEG and ECoG data showed that object category signals emerge swiftly in the visual system and can be detected by both EEG and ECoG at similar temporal delays after stimulus onset. The correlation between EEG and ECoG was reduced when object representations tolerant to changes in scale and orientation were considered. The comparison of fMRI and ECoG overall revealed a tighter relationship in occipital than in temporal regions, related to differences in fMRI signal-to-noise ratio. Together, our results reveal a complex relationship between fMRI, EEG, and ECoG signals at the level of population codes that critically depends on the time point after stimulus onset, the region investigated, and the visual contents used.
... Historically, the majority of ISC research utilized functional Magnetic Resonance Imaging (fMRI). However, there has been a gradual shift towards incorporating alternative modalities such as Electroencephalography (EEG) [8], [9], [25], Electrocorticography (ECoG) [19], Magnetoencephalography (MEG) [24], functional Near-Infrared Spectroscopy (fNIRS) [26], Electrocardiography (ECG) [34], analysis of eye movements [18], and other physiological signals [6]. Notably, Haufe et al. [19] confirmed that fMRI, ECoG, and EEG exhibit comparable repeat-reliability across viewings, highlighting EEG's potential for capturing shared neural responses with its high temporal resolution and suitability for naturalistic settings due to portable equipment. ...
... However, there has been a gradual shift towards incorporating alternative modalities such as Electroencephalography (EEG) [8], [9], [25], Electrocorticography (ECoG) [19], Magnetoencephalography (MEG) [24], functional Near-Infrared Spectroscopy (fNIRS) [26], Electrocardiography (ECG) [34], analysis of eye movements [18], and other physiological signals [6]. Notably, Haufe et al. [19] confirmed that fMRI, ECoG, and EEG exhibit comparable repeat-reliability across viewings, highlighting EEG's potential for capturing shared neural responses with its high temporal resolution and suitability for naturalistic settings due to portable equipment. ...
The need for automatic and high-quality emotion annotation is paramount in applications such as continuous emotion recognition and video highlight detection, yet achieving this through manual human annotations is challenging. Inspired by inter-subject correlation (ISC) utilized in neuroscience, this study introduces a novel Electroencephalography (EEG) based ISC methodology that leverages a single-electrode and feature-based dynamic approach. Our contributions are three folds. Firstly, we reidentify two potent emotion features suitable for classifying emotions-first-order difference (FD) an differential entropy (DE). Secondly, through the use of overall correlation analysis, we demonstrate the heterogeneous synchronized performance of electrodes. This performance aligns with neural emotion patterns established in prior studies, thus validating the effectiveness of our approach. Thirdly, by employing a sliding window correlation technique, we showcase the significant consistency of dynamic ISCs across various features or key electrodes in each analyzed film clip. Our findings indicate the method's reliability in capturing consistent, dynamic shared neural synchrony among individuals, triggered by evocative film stimuli. This underscores the potential of our approach to serve as an indicator of continuous human emotion arousal. The implications of this research are significant for advancements in affective computing and the broader neuroscience field, suggesting a streamlined and effective tool for emotion analysis in real-world applications.
... The SPM t-map volumes were mapped onto the surface of the corresponding reconstructed brains (triangulated meshes). The t-values were assigned to the surface vertices by averaging the voxel intensities along 6 mm of the vertex normal directions using Gaussian weights (FWHM = 10 mm) (Hermes et al., 2012;Gaglianese et al., 2017;Haufe et al., 2018). For each electrode location, a t-value was determined by averaging the highest 5% of the t-values within a radius of 6 mm (Hermes et al., 2012;Gaglianese et al., 2017;Haufe et al., 2018). ...
... The t-values were assigned to the surface vertices by averaging the voxel intensities along 6 mm of the vertex normal directions using Gaussian weights (FWHM = 10 mm) (Hermes et al., 2012;Gaglianese et al., 2017;Haufe et al., 2018). For each electrode location, a t-value was determined by averaging the highest 5% of the t-values within a radius of 6 mm (Hermes et al., 2012;Gaglianese et al., 2017;Haufe et al., 2018). Electrodes with a t-value > 3.3 were considered as fMRI positive, which corresponds to a statistically significant activation threshold of p < 0.001 (uncorrected, DOF = 87). ...
Face recognition is impaired in patients with prosopagnosia, which may occur as a side effect of neurosurgical procedures. Face selective regions on the ventral temporal cortex have been localized with electrical cortical stimulation (ECS), electrocorticography (ECoG), and functional magnetic resonance imagining (fMRI). This is the first group study using within-patient comparisons to validate face selective regions mapping, utilizing the aforementioned modalities. Five patients underwent surgical treatment of intractable epilepsy and joined the study. Subdural grid electrodes were implanted on their ventral temporal cortices to localize seizure foci and face selective regions as part of the functional mapping protocol. Face selective regions were identified in all patients with fMRI, four patients with ECoG, and two patients with ECS. From 177 tested electrode locations in the region of interest (ROI), which is defined by the fusiform gyrus and the inferior temporal gyrus, 54 face locations were identified by at least one modality in all patients. fMRI mapping showed the highest detection rate, revealing 70.4% for face selective locations, whereas ECoG and ECS identified 64.8 and 31.5%, respectively. Thus, 28 face locations were co-localized by at least two modalities, with detection rates of 89.3% for fMRI, 85.7% for ECoG and 53.6 % for ECS. All five patients had no face recognition deficits after surgery, even though five of the face selective locations, one obtained by ECoG and the other four by fMRI, were within 10 mm to the resected volumes. Moreover, fMRI included a quite large volume artifact on the ventral temporal cortex in the ROI from the anatomical structures of the temporal base. In conclusion, ECS was not sensitive in several patients, whereas ECoG and fMRI even showed activation within 10 mm to the resected volumes. Considering the potential signal drop-out in fMRI makes ECoG the most reliable tool to identify face selective locations in this study. A multimodal approach can improve the specificity of ECoG and fMRI, while simultaneously minimizing the number of required ECS sessions. Hence, all modalities should be considered in a clinical mapping protocol entailing combined results of co-localized face selective locations.
... fMRI is also incompatible with patients with implanted electronics or metal, and is often challenging for young children (Raschle et al., 2012). Electroencephalography (EEG) offers portability and a more natural scanning environment while recording the brain's electrical activity, which has been shown to record reproducible data with naturalistic tasks (Chang et al., 2015;Desai et al., 2021;Haufe et al., 2018;Poulsen et al., 2017). While EEG provides a high temporal resolution, traditional EEG systems offer low spatial resolution, which makes it challenging to resolve different features spatially. ...
Naturalistic neuroimaging tasks, such as watching movies, are becoming increasingly popular due to being more engaging than resting state paradigms and more ecologically valid than isolated block-design tasks. As these tasks push the boundaries of naturalistic paradigms, the need for an equally naturalistic imaging device increases. Optical imaging with functional near-infrared spectroscopy (fNIRS) offers a wearable, non-invasive neuroimaging approach. Advancements in high-density diffuse optical tomography (HD-DOT) use a dense array of optical elements to provide overlapping multi-distance fNIRS light measurements for fidelity comparable to fMRI. Here, to further improve image quality, we increased the density of the imaging grid to 9.75 mm, 1st nearest neighbor spacing between sources and detectors, leading to a 4-fold increase in measurement density. This very high-density DOT (VHD-DOT) system uses 255 sources and 252 detectors to improve image quality while expanding the field of view. From simulations, the increased density led to improved image resolution across multiple metrics compared to HD-DOT. In vivo group-averaged functional localizer maps are in strong agreement with those collected in MRI on the same cohort of adult participants, indicating that VHD-DOT can be used as a surrogate for fMRI in task-based studies. For a naturalistic movie viewing task, feature regressor analysis was employed to map audiovisual features from the clip, which also revealed excellent agreement between VHD-DOT and fMRI. Template-based decoding of task and movie-viewing data demonstrates that VHD-DOT signals are repeatable and discriminable, which is necessary for more advanced naturalistic task analyses. This work builds upon previously reported HD-DOT designs to improve the image quality and resolution for whole-head optical imaging. This system is promising for future studies using complex stimuli and analysis protocols, such as decoding, and future work developing wireless VHD-DOT systems.
... Additionally, although EEG and sEMG were chosen for their ability to capture cognitive and muscular activity synchronously (due to their high temporal resolution), it should be noted that EEG has limited spatial resolution compared to other neurological methods [118][119][120][121]. This can create challenges when the focus is on exact source localization (e.g., when targeting small important areas with major evolutionary implications, such as Broca's or Wernicke's areas). ...
Despite extensive research into the biomechanical and cognitive dimensions of early hominin material culture, no study has explored these aspects together in the context of stone tool production and use. In contrast to fields like rehabilitation and sports science, where electroencephalography (EEG) and surface electromyography (sEMG) are often integrated, experimental archaeology lacks such a combined approach. This paper introduces and validates a new protocol that integrates EEG and sEMG to measure neuromechanical activity during a classic stone tool task: cutting leather with a flake. Our experimental design divides the task into three phases: Hold, Aim, and Execute. Consistent with our expectations, results show that all eight muscles are most active during task execution, with the non-dominant hand playing a key role in stabilization during both the Aim and Execute phases. In the preparatory Aim stage, we observed increased beta power in the left frontal region (linked to planning, problem-solving, and working memory) as well as heightened motor activity associated with using the non-dominant hand, which contributes to the stabilization of the target material during this stage. During the Execute phase, beta power in these cortical areas decreased, with peak muscle activation occurring alongside suspected beta desynchronization in the motor region, reflecting intensified movement activity. Overall, these findings closely align with our expectations, validating our combined EEG-sEMG protocol and highlighting the importance of segmenting tool-using tasks into distinct phases, which allows for the identification of dynamic brain-hand interactions throughout the process. The proposed step-by-step protocol offers a new methodological basis for future research into the complexities of hominin behaviors and tool use.
... Interbrain synchronization of neural activity has been studied across imaging modalities, including fMRI, EEG, ECoG, and fNIRS (Dieffenbach et al., 2021;Haufe et al., 2018). Researchers are thus able to take advantage of the strengths of each method to examine interbrain synchrony across different timescales, at different spatial resolutions and in varied environments. ...
Since 1954, The Handbook of Social Psychology has been the field's most authoritative reference work. The 6th edition of this essential resource contains 50 new chapters on a wide range of topics, written by the world's leading experts. It is available only in digital form and is free to read online and to download.
... Linguistic processing may span multiple frequency bands and rely on cross-frequency coupling (e.g., Martin, 2020;Murphy, 2024). For the sake of simplicity, we focus on high-gamma band power as an index of local, stimulus-driven neuronal activity (Haufe et al., 2018;Manning et al., 2009;Mukamel et al., 2005). We extracted the high-gamma band by applying a Butterworth band-pass infinite impulse response (IIR) filter at 70-200 Hz. ...
Naturalistic electrocorticography (ECoG) data are a rare but essential resource for studying the brain's linguistic capabilities. ECoG offers a high temporal resolution suitable for investigating processes at multiple temporal timescales and frequency bands. It also provides broad spatial coverage, often along critical language areas. Here, we share a dataset of nine ECoG participants with 1,330 electrodes listening to a 30-minute audio podcast. The richness of this naturalistic stimulus can be used for various research endeavors, from auditory perception to semantic integration. In addition to the neural data, we extract linguistic features of the stimulus ranging from phonetic information to large language model word embeddings. We use these linguistic features in encoding models that relate stimulus properties to neural activity. Finally, we provide detailed tutorials for preprocessing raw data, extracting stimulus features, and running encoding analyses that can serve as a pedagogical resource or a springboard for new research.
... Here, we use the approach of Prichard and Theiler to test for significant ρ based on the consideration that no inter-subject correlation can be present after phase-scrambling the data of each subject. Variants of this approach have previously been used to test for significant correlations [59,33,14,60] We test the validity of the three methods on simulated data in section 4. The goal is to determine under which conditions these methods determine the correct number of underlying correlated components. ...
How does one find dimensions in multivariate data that are reliably expressed across repetitions? For example, in a brain imaging study one may want to identify combinations of neural signals that are reliably expressed across multiple trials or subjects. For a behavioral assessment with multiple ratings, one may want to identify an aggregate score that is reliably reproduced across raters. Correlated Components Analysis (CorrCA) addresses this problem by identifying components that are maximally correlated between repetitions (e.g. trials, subjects, raters). Here we formalize this as the maximization of the ratio of between-repetition to within-repetition covariance. We show that this criterion maximizes repeat-reliability, defined as mean over variance across repeats, and that it leads to CorrCA or to multi-set Canonical Correlation Analysis, depending on the constraints. Surprisingly, we also find that CorrCA is equivalent to Linear Discriminant Analysis for zero-mean signals, which provides an unexpected link between classic concepts of multivariate analysis. We present an exact parametric test of statistical significance based on the F-statistic for normally distributed independent samples, and present and validate shuffle statistics for the case of dependent samples. Regularization and extension to non-linear mappings using kernels are also presented. The algorithms are demonstrated on a series of data analysis applications, and we provide all code and data required to reproduce the results.
... On the other hand, most fMRI studies report the spatial coordinates of the brain regions that become more or less active or connected on psychedelics. In general, EEG has higher temporal resolution than fMRI [29], so a future meta-analysis that focuses on the temporal, rather than spatial, characteristics of brain activity on psychedelics could consider the EEG rather than fMRI data. ...
Serotonergic psychedelics induce altered states of consciousness and have shown potential for treating a variety of neuropsychiatric disorders, including depression and addiction. Yet their modes of action are not fully understood. Here, we provide a novel, synergistic understanding of psychedelics arising from systematic reviews and meta-analyses of three hierarchical levels of analysis: (1) subjective experience (phenomenology), (2) neuroimaging and (3) molecular pharmacology. Phenomenologically, medium and high doses of LSD yield significantly higher ratings of visionary restructuralisation than psilocybin on the 5-dimensional Altered States of Consciousness Scale. Our neuroimaging results reveal that, in general, psychedelics significantly strengthen between-network functional connectivity (FC) while significantly diminishing within-network FC. Pharmacologically, LSD induces significantly more inositol phosphate formation at the 5-HT 2A receptor than DMT and psilocin, yet there are no significant between-drug differences in the selectivity of psychedelics for the 5-HT 2A , 5-HT 2C , or D 2 receptors, relative to the 5-HT 1A receptor. Our meta-analyses link DMT, LSD, and psilocybin to specific neural fingerprints at each level of analysis. The results show a highly non-linear relationship between these fingerprints. Overall, our analysis highlighted the high heterogeneity and risk of bias in the literature. This suggests an urgent need for standardising experimental procedures and analysis techniques, as well as for more research on the emergence between different levels of psychedelic effects.
... For instance, a study introduced a decision tree to classify individual finger movements using electrocorticography (ECoG), achieving classification accuracy exceeding 70% [6]. Another investigation explored the compatibility of different extraction techniques, including functional magnetic resonance imaging (fMRI), ECoG, and electroencephalography (EEG) when exposed to natural stimuli [7]. To enhance BCI performance and functionality, integrating BCI with external sensors like ultrasonic sensors, infrared sensors, and electromyography (EMG) has gained prominence. ...
This work introduces a brain–computer interface (BCI) system designed specifically for enhancing smart home control, with a primary focus on assisting elderly and disabled individuals. The system relies on electroencephalography (EEG) and head motion sensing technologies to enable users to independently control home electrical devices. The EMOTIV Insight headset serves as the EEG data and head motion acquisition device, while a combination of an Android application and an Arduino Uno microcontroller facilitates the control of home electrical devices. Wireless connectivity is established between the Android application, EEG headset, and Arduino Uno through Bluetooth, where the EEG headset uses Bluetooth low energy technology. The application processes the blink, attention level, and head motion data to enable users to seamlessly activate and deactivate desired appliances. The study also investigates the relationship between attention, blink patterns, and the extracted brainwave signals through various assessments. Furthermore, the integration of accelerometer data with EEG data enables the detection of head motion, thereby enhancing the system's control mechanism. This work aims to provide a step-by-step guide to the process required to develop an EEG-based smart home control system using an EMOTIV Insight headset. The evaluation outcomes exhibit a 90% accuracy in detecting double blinks and a 75% accuracy in identifying active attention levels. For head motion detection, upward and downward motions are recognized with 100% accuracy, while left and right motions are identified with 85% accuracy. Through successful experimentation, the developed system effectively controls four distinct home appliances, thereby validating its feasibility as a proof of concept.
... Brain signals are classified based on their recording method and the invasiveness of their collection (see: Fig. 1) [1,2,16,15]. The authors focused on fNIRS signals only [1]. ...
This paper presents a preliminary study delving into the application of machine learning-based methods for optimising parameter selection in filtering techniques. The authors focus on exploring the efficacy of two prominent filtering methods: smoothing and cascade filters, known for their profound impact on enhancing the quality of brain signals. The study specifically examines signals acquired through functional near-infrared spectroscopy (fNIRS), a non-invasive neuroimaging modality offering valuable insights into brain activity. Through meticulous analysis, the research underscores the potential of machine learning approaches in discerning optimal parameters for filtering, thereby leading to a significant enhancement in the quality and reliability of fNIRS-derived signals. The results demonstrate the effectiveness of machine learning-based methods in optimizing parameter selection for filtering techniques, particularly in the context of fNIRS signals. By leveraging these approaches, the study achieves notable improvements in the quality and reliability of brain signal data. This work sheds light on promising avenues for refining neuroimaging methodologies and advancing the field of signal processing in neuroscience. The successful application of machine learning-based techniques highlights their potential for optimizing neuroimaging data processing, ultimately contributing to a deeper understanding of brain function.
... While spatial sampling is limited by clinical considerations, icEEG has greater sensitivity and spatial specificity in relation to electrophysiological activity, compared to its scalp-based counterparts [17,18]. Finally, there is also more limited evidence of coupling between BOLD and electrophysiology in the resting state [19][20][21] although some work has been performed to evaluate average responses to natural stimulus tasks applied to each modality sequentially in humans [22][23][24]. ...
There are considerable gaps in our understanding of the relationship between human brain activity measured at different temporal and spatial scales. Here, electrocorticography (ECoG) measures were used to predict functional MRI changes in the sensorimotor cortex in two brain states: at rest and during motor performance. The specificity of this relationship to spatial co-localisation of the two signals was also investigated. We acquired simultaneous ECoG-fMRI in the sensorimotor cortex of three patients with epilepsy. During motor activity, high gamma power was the only frequency band where the electrophysiological response was co-localised with fMRI measures across all subjects. The best model of fMRI changes across states was its principal components, a parsimonious description of the entire ECoG spectrogram. This model performed much better than any others that were based either on the classical frequency bands or on summary measures of cross-spectral changes. The region-specific fMRI signal is reflected in spatially and spectrally distributed EEG activity.
... Instantaneous power of the timeseries was calculated by squaring the absolute amplitude envelope of the Hilbert transformed data. We focused on high gamma power given prior evidence of the concordance of this frequency band with BOLD signal and neuronal activity associated with cognitive processing (Hacker et al., 2017;Haufe et al., 2018), as well as evidence linking increase in HG activity following increased cognitive demand to executive function (Assem et al., 2023;Erez et al., 2021). An example of a HG timeseries is shown in Supplementary Fig. 2. ...
The extent to which tumour-infiltrated brain tissue contributes to cognitive function remains unclear. We tested the hypothesis that cortical tissue infiltrated by diffuse gliomas participates in large-scale cognitive circuits using a unique combination of intracranial electrocorticography (ECoG) and resting-state functional magnetic resonance (fMRI) imaging in four patients. We also assessed the relationship between functional connectivity with tumour-infiltrated tissue and long-term cognitive outcomes in a larger, overlapping cohort of 17 patients. We observed significant task-related high gamma (70–250 Hz) power modulations in tumour-infiltrated cortex in response to increased cognitive effort (i.e., switch counting compared to simple counting), implying preserved functionality of neoplastic tissue for complex tasks probing executive function. We found that tumour locations corresponding to task-responsive electrodes exhibited functional connectivity patterns that significantly co-localised with canonical brain networks implicated in executive function. Specifically, we discovered that tumour-infiltrated cortex with larger task-related high gamma power modulations tended to be more functionally connected to the dorsal attention network (DAN). Finally, we demonstrated that tumour-DAN connectivity is evident across a larger cohort of patients with gliomas and that it relates to long-term postsurgical outcomes in goal-directed attention. Overall, this study contributes convergent fMRI-ECoG evidence that tumour-infiltrated cortex participates in large-scale neurocognitive circuits that support executive function in health. These findings underscore the potential clinical utility of mapping large-scale connectivity of tumour-infiltrated tissue in the care of patients with diffuse gliomas.
... Magnetic resonance electroencephalography and other frameworks for integrating multiple imaging modalities should also be investigated, such as joint imaging markers from simultaneous magnetic resonance imaging (MRI) and EEG (e.g., temporal volume, cor-tical thickness) that are associated with cognitive status in healthy individuals, pathophysiological changes in neurodegenerative diseases, and after traumatic brain injury [76][77][78][79][80][81]. The panel contended that these simultaneous recordings could provide a more accurate diagnosis of pathology than either modality alone. ...
Biomarkers, ranging from molecules to behavior, can be used to identify thresholds beyond which performance of mission tasks may be compromised and could potentially trigger the activation of countermeasures. Identification of homologous brain regions and/or neural circuits related to operational performance may allow for translational studies between species. Three discussion groups were directed to use operationally relevant performance tasks as a driver when identifying biomarkers and brain regions or circuits for selected constructs. Here we summarize small-group discussions in tables of circuits and biomarkers categorized by (a) sensorimotor, (b) behavioral medicine and (c) integrated approaches (e.g., physiological responses). In total, hundreds of biomarkers have been identified and are summarized herein by the respective group leads. We hope the meeting proceedings become a rich resource for NASA’s Human Research Program (HRP) and the community of researchers.
... While other studies comparing intracranial to scalp data used a sequential recording of the two modalities (Ebrahiminia et al., 2022) or even different sets of participants (Haufe et al., 2018), we have simultaneously acquired data in the two modalities, allowing us to validate the results of the EEG source reconstruction using SEEG recordings in decoding task conditions, as well as investigate the possible synergy between invasive and non-invasive recording in decoding stimulus novelty. ...
Investigating cognitive brain functions using non-invasive electrophysiology can be challenging due to the particularities of the task-related EEG activity, the depth of the activated brain areas, and the extent of the networks involved. Stereoelectroencephalographic (SEEG) investigations in patients with drug-resistant epilepsy offer an extraordinary opportunity to validate information derived from non-invasive recordings at macro-scales. The SEEG approach can provide brain activity with high spatial specificity during tasks that target specific cognitive processes (e.g., memory). Full validation is possible only when performing simultaneous scalp SEEG recordings, which allows recording signals in the exact same brain state. This is the approach we have taken in 12 subjects performing a visual memory task that requires the recognition of previously viewed objects. The intracranial signals on 965 contact pairs have been compared to 391 simultaneously recorded scalp signals at a regional and whole-brain level, using multivariate pattern analysis. The results show that the task conditions are best captured by intracranial sensors, despite the limited spatial coverage of SEEG electrodes, compared to the whole-brain non-invasive recordings. Applying beamformer source reconstruction or independent component analysis does not result in an improvement of the multivariate task decoding performance using surface sensor data. By analyzing a joint scalp and SEEG dataset, we investigated whether the two types of signals carry complementary information that might improve the machine-learning classifier performance. This joint analysis revealed that the results are driven by the modality exhibiting best individual performance, namely SEEG.
... For BMI applications, brain signals are frequently recorded at discrete periods. In the Discrete Fourier Transform, the Fourier series is changed and applied to discretely sampled data (DFT) [88]. ...
We review
the biological and engineering concepts that underpin the
implementation of BMI applications. There are various essential
techniques that are necessary for making invasive
BMI applications a reality. We review these through providing
an analysis of (i) possible applications of invasive BMI
technology, (ii) the methods and devices for detecting and
decoding brain signals, as well as (iii) possible options for
stimulating signals into human’s brain. Finally, we discuss
the challenges and opportunities of invasive BMI for further
development in the area.
... For an instance, (Yao and Shoaran 2019) presented a decision tree ensemble to classify individual finger movements involving electrocorticography (ECoG), which is a partially invasive signal acquisition technique, with an enhanced classification accuracy of above 70%. In another effort, (Haufe et al. 2018) studied the relationship between several extraction techniques, namely functional magnetic resonance imaging, ECoG, and EEG, towards human cognitive neuroscience, by analyzing the reliability after being subjected to natural stimulus and comparing the similarity between the techniques. EEG is one of the most popular non-invasive methods developed by Berger (1929), which can be used to evaluate the electrical activity in the brain by placing electrodes on the scalp. ...
This paper presents a hybrid electroencephalography (EEG)-based brain-computer interface system combined with head motion sensing for smart home control to assist the elderly and disabled. The system mainly includes an EMOTIV Insight headset used to extract the user’s EEG data and head motion, an Android application, and an Arduino Uno that controls the appliances. The Android application is wirelessly connected via Bluetooth to the headset and the Arduino Uno through an HC-06 module. The application uses the blink, attention level, and head motion data to allow the user to turn on and off the desired appliance. Various analyses are performed to evaluate the effect of attention and blink on the extracted brain wave signals. In addition, the accelerometer’s data was used to detect the head motion and control the application in combination with the EEG data. Double blink detection achieved an accuracy of 90% whereas the active attention level detection achieved a 75% accuracy. A 100% accuracy was achieved when detecting upward and downward motion whereas an 85% accuracy was achieved for the left and right motions. Finally, as a proof of concept, the developed system was successfully used to control four different home appliances. The successful outcomes of the proposed system demonstrate that it can be easily implemented into home automation to assist disabled and elderly people due to its ease of use, portability, low cost, and expandable circuitry.
... A multiclass model and high-resolution data are needed to classify various movements accurately. Electrocorticography (ECoG) has the potential to record complex brain activity associated with dreams because of its high temporal and spatial resolution [19,23]. Deep machine learning (ML) methods facilitate easier feature extraction and can achieve high accuracy in classifying brain data [31,37,40,45,50], but can overfit when the training data set is too small [41]. ...
Dreams are often forgotten despite their impact on our emotions and memory. In our pursuit of developing an objective dream-content recording methodology (reaDream), we focused on the motor imagery (MI)-related dream component, which is reported to be present in dreams along with other sensory, perceptual, and cognitive phenomena. It has been shown that brain activation during dreamed actions corresponds to the brain activation for the same actions in a wakeful state. This allows one to decode electrocorticographic (ECoG) brain activity during sleep using a machine learning (ML) model trained on wakeful data. ECoG data is very specific to each individual and not generalized between subjects; deep ML models are prone to overfit on small amounts of data. We propose to generalize ECoG data by combining recordings from several subjects. For that, we developed a Convolutional Neural Network (CNN)-based classifier that discriminates between hand and tongue movements in different subjects. We tested a hypothesis on whether a MI classifier can be trained on motor execution (ME) data. We demonstrate that ME types are easier to distinguish compared to MI. We showed that power features are more informative than temporal features. Finally, we demonstrated how our trained models could be used to predict MI during Rapid Eye Movement (REM) sleep.
KeywordsDream researchECoGMotor imageryCNN
... Instantaneous power of the timeseries was calculated by squaring the absolute amplitude envelope of the Hilbert transformed data. For this study, we chose to focus on high gamma power given prior evidence of the concordance of this frequency band with BOLD signal and neuronal activity (Haufe et al., 2018). ...
Glioma tumours are among the most lethal brain disorders, claiming the lives of thousands of people in the United Kingdom each year. Despite the severity and prevalence of the condition, remarkably little is understood about the origins of gliomas, or the mechanisms that guide their spread within the brain. The aim of this thesis is to invoke a relatively new approach – brain network mapping – to provide insights into the origins of gliomas and their pathological spread along neural circuits. First, I provide a historical overview of both brain network mapping and glioma neurobiology, along with the recent advances and techniques popular in each field. In particular, I highlight preclinical research implying that gliomas originate from neural stem cells in the subventricular zone, as well as other work in mouse models demonstrating that gliomas infiltrate previously healthy brain networks. This thesis contributes three studies of clinical datasets which evaluate the hypothesis that glioma initiation and progression are guided by brain networks. In the first study, I describe convergent evidence from both intracranial electrocorticography recordings and resting-state functional imaging of four patients with low-grade gliomas that tumour-infiltrated cortex can participate in large-scale cognitive circuits responsive to executive function. These findings imply that gliomas integrate into neural circuits, suggesting that their development and maintenance could be sustained by functional brain networks. In support of this idea, I next demonstrate that the spatial distribution of gliomas in the brain follows the distribution of functional network hubs, as well as cellular and genomic factors related to gliomagenesis. These results suggest two possibilities regarding the origins of glioma: the predilection of gliomas to hub locations could be a result of the vulnerability of hubs to oncogenesis, or the result of tumours arriving at central network locations while spreading through brain networks. To help disambiguate between these possibilities, I developed a novel approach termed “lesion covariance network mapping” to identify networks of brain regions co-lesioned in glioma, which indicate areas along which tumours are inferred to spread. This method revealed that gliomas cluster around horns of the lateral ventricles, consistent with the hypothesis that these tumours originate from neurogenic niches within subventricular zone. The lesion covariance network method also demonstrated that glioma localisation patterns follow specific structural and functional connectivity networks disseminating from periventricular grey matter. Cumulatively, the findings of the thesis support a model wherein periventricular brain connectivity guides glioma development from the subventricular zone into distributed regions of the cortex. In the conclusion, I discuss potential clinical applications of the presented research, such as in supporting predictive modelling approaches to forecast glioma progression, for the purpose of planning pre-emptive radiation and surgical treatments of glioma.
... In the current study, we investigate the feasibility of implicit infra-low frequency EEG neurofeedback in aging people. The infra-low frequency EEG domain (f < 0.1 Hz) is thought to be linked to the dynamics of intrinsic brain networks (Hiltunen et al., 2014;Haufe et al., 2018), and infra-low frequency neurofeedback is used to enhance emotional regulation and cognitive performance (Othmer, 2011;Othmer et al., 2013;Grin-Yatsenko et al., 2018). Implicit neurofeedback does not rely on executive functioning, which allows for the potential application of the method even in patients with profound cognitive impairment. ...
Aging is associated with decreased functional connectivity in the main brain networks, which can underlie changes in cognitive and emotional processing. Neurofeedback is a promising non-pharmacological approach for the enhancement of brain connectivity. Previously, we showed that a single session of infra-low frequency neurofeedback results in increased connectivity between sensory processing networks in healthy young adults. In the current pilot study, we aimed to evaluate the possibility of enhancing brain connectivity during aging with the use of infra-low frequency neurofeedback. Nine females aged 52 ± 7 years with subclinical signs of emotional dysregulation, including anxiety, mild depression, and somatoform symptoms, underwent 15 sessions of training. A resting-state functional MRI scan was acquired before and after the training. A hypothesis-free intrinsic connectivity analysis showed increased connectivity in regions in the bilateral temporal fusiform cortex, right supplementary motor area, left amygdala, left temporal pole, and cerebellum. Next, a seed-to-voxel analysis for the revealed regions was performed using the post- vs. pre-neurofeedback contrast. Finally, to explore the whole network of neurofeedback-related connectivity changes, the regions revealed by the intrinsic connectivity and seed-to-voxel analyses were entered into a network-based statistical analysis. An extended network was revealed, including the temporal and occipital fusiform cortex, multiple areas from the visual cortex, the right posterior superior temporal sulcus, the amygdala, the temporal poles, the superior parietal lobule, and the supplementary motor cortex. Clinically, decreases in alexithymia, depression, and anxiety levels were observed. Thus, infra-low frequency neurofeedback appears to be a promising method for enhancing brain connectivity during aging, and subsequent sham-controlled studies utilizing larger samples are feasible.
... Later, slow brain dynamics received extensive attention in the context of intrinsic connectivity networks revealed by functional MRI (Buckner et al., 2013). These networks demonstrate infra-low frequency fluctuations that are correlated with infra-low EEG fluctuations (Hiltunen et al., 2014;Haufe et al., 2018). ...
Primary headaches are highly prevalent and represent a major cause of disability in young adults. Neurofeedback is increasingly used in the treatment of chronic pain; however, there are few studies investigating its efficacy in patients with headaches. We report the results of a cross-over sham-controlled study on the efficacy of neurofeedback in the prophylactic treatment of tension-type headache (TTH). Participants received ten sessions of infra-low frequency electroencephalographic neurofeedback and ten sessions of sham-neurofeedback, with the order of treatments being randomized. The study also included a basic psychotherapeutic intervention — a psychoeducational session performed before the main study phases and emotional support provided throughout the study period. The headache probability was modeled as a function of the neurofeedback and sham-neurofeedback sessions performed to date. As a result, we revealed a strong beneficial effect of neurofeedback and no influence of the sham sessions. The study supports the prophylactic use of infra-low frequency neurofeedback in patients with TTH. From a methodological point of view, we advocate for the explicit inclusion of psychotherapeutic components in neurofeedback study protocols.
... Electroencephalography (EEG) is the oldest and widely used for investigating the brain, which utilizes electrical signals recorded from the array of electrodes attached to the scalp [3]. It provides a high temporal and low spatial resolution of the brain signals [4]. On the other hand, electrocorticography (ECoG) provides high temporal and high spatial resolution signals than EEG [5]. ...
Developing technologies for understanding the functioning of the brain and treating neurological disorders is an important area of research in neuroscience. Devices that form the neural interfaces have a significant contribution in progressing this field. Technological advancements driven by microfabrication techniques and materials innovation have led to the developing of a new class of engineered microelectrode devices. These miniaturized devices provide seamless neural interfaces as demonstrated successfully in animal models. Depending on the brain region to be studied and the application involved, surface and depth micro-engineered devices have been developed for recording or stimulating electrical signals. These devices have also shown potential to be used to treat neurological disorders such as epilepsy and parkinsonian. Strategies such as nanowires as electrode materials and polymer as flexible substrates have proven to help minimize the anti-inflammatory response and maximize the density of microelectrodes. This article provides a detailed overview of the recent developments in micro-engineered surface and depth neural devices used in various animal models.
... In this framework, it is now evident that neuronal oscillations within certain frequency bands are associated with specific neural processes (Buzsaki and Draguhn 2004) and cognitive functions (Knyazev 2007). Despite these findings, low-frequency (typically defined as <0.1 Hz) oscillations of functional neuroimaging signals have gained the most considerable attention in the field of neuroscience (Demanuele et al. 2007;Fox and Raichle 2007;Scheeringa et al. 2011;Du et al. 2014;Kublbock et al. 2014;Haufe et al. 2018) since convergent evidence from functional magnetic resonance imaging (fMRI) and electroencephalography indicates that the low-frequency oscillations contain more neurophysiologically relevant information (Buzsaki and Draguhn 2004;Vanhatalo et al. 2004;Balduzzi et al. 2008). ...
Neuronal oscillations within certain frequency bands are assumed to associate with specific neural processes and cognitive functions. To examine this hypothesis, transcriptome-neuroimaging spatial correlation analysis was applied to resting-state functional magnetic resonance imaging data from 793 healthy individuals and gene expression data from the Allen Human Brain Atlas. We found that expression measures of 336 genes were correlated with fractional amplitude of low-frequency fluctuations (fALFF) in the slow-4 band (0.027-0.073 Hz), whereas there were no expression-fALFF correlations for the other frequency bands. Furthermore, functional enrichment analyses showed that these slow-4 fALFF-related genes were mainly enriched for ion channel, synaptic function, and neuronal system as well as many neuropsychiatric disorders. Specific expression analyses demonstrated that these genes were specifically expressed in brain tissue, in neurons, and during the late stage of cortical development. Concurrently, the fALFF-related genes were linked to multiple behavioral domains, including dementia, attention, and emotion. In addition, these genes could construct a protein-protein interaction network supported by 30 hub genes. Our findings of a frequency-dependent genetic modulation of spontaneous neuronal activity may support the concept that neuronal oscillations within different frequency bands capture distinct neurobiological processes from the perspective of underlying molecular mechanisms.
... In general, using a spherical RoI is a common way to demarcate regions of coverage around an electrode contact. In analyses used to correlate resting-state activity from intracranial recordings with fMRI, authors have previously used a 6 mm RoI [37] and a 5 mm RoI [38]. According to Dubey and Ray 2019, spatial spread for ECoG electrodes-sized identically to those used in our study-was minimally larger than the size of the electrode itself, at approximately a 3 mm RoI. ...
Objective. The objective of this study is to quantify the coverage of gray and white matter during intracranial electroencephalography in a cohort of epilepsy patients with surface and depth electrodes.
Methods. We included 65 patients with strip electrodes (n=12), strip and grid electrodes (n=24), strip, grid, and depth electrodes (n=7), or depth electrodes only (n=22) from the University of Utah spanning 2010-2020. Patient-specific imaging was used to generate probabilistic gray and white matter maps and atlas segmentations. The gray and white matter coverage was quantified based on spherical volumes centered on electrode centroids, with radii ranging from 1-15 mm, along with detailed finite element models of local electric fields
Results. Gray matter coverage was highly dependent on the chosen radius of influence (RoI). Using a 2.5 mm RoI, depth electrodes covered more gray matter than surface electrodes; however, surface electrodes covered more gray matter at RoI larger than 4 mm. White matter coverage was greatest for depth electrodes at all RoIs, which is noteworthy for studies involving stimulation mapping. Depth electrodes were able to record significantly more gray matter from the amygdala and hippocampus than subdural electrodes.
Significance. This study provides the first probabilistic analysis to quantify gray and white matter coverage for multiple categories of intracranial recording configurations. Depth electrodes may offer increased per contact coverage of gray matter over other recording strategies if the desired signals are local to the contact, while subdural grids and strips can sample more gray matter if the desired signals are more diffuse.
Movies captivate groups of individuals (the audience), especially if they contain themes of common motivational interest to the group. In drug addiction, a key mechanism is maladaptive motivational salience attribution whereby drug cues outcompete other reinforcers within the same environment or context. We predicted that while watching a drug-themed movie, where cues for drugs and other stimuli share a continuous narrative context, fMRI responses in individuals with heroin use disorder (iHUD) will preferentially synchronize during drug scenes.
Thirty inpatient iHUD (24 male) and 25 healthy controls (16 male) watched a drug-themed movie at baseline and at follow-up after 15 weeks.
Results revealed such drug-biased synchronization in the orbitofrontal cortex (OFC), ventromedial and ventrolateral prefrontal cortex, and insula. After 15 weeks during ongoing inpatient treatment, there was a significant reduction in this drug-biased shared response in the OFC, which correlated with a concomitant reduction in dynamically-measured craving, suggesting synchronized OFC responses to a drug-themed movie as a neural marker of craving and recovery in iHUD.
Invasive and non-invasive electrophysiological measurements during “cocktail-party”-like listening indicate that neural activity in the human auditory cortex (AC) “tracks” the envelope of relevant speech. However, due to limited coverage and/or spatial resolution, the distinct contribution of primary and non-primary areas remains unclear. Here, using 7-Tesla fMRI, we measured brain responses of participants attending to one speaker, in the presence and absence of another speaker. Through voxel-wise modeling, we observed envelope tracking in bilateral Heschl’s gyrus (HG), right middle superior temporal sulcus (mSTS) and left temporo-parietal junction (TPJ), despite the signal’s sluggish nature and slow temporal sampling. Neurovascular activity correlated positively (HG) or negatively (mSTS, TPJ) with the envelope. Further analyses comparing the similarity between spatial response patterns in the single speaker and concurrent speakers conditions and envelope decoding indicated that tracking in HG reflected both relevant and (to a lesser extent) non-relevant speech, while mSTS represented the relevant speech signal. Additionally, in mSTS, the similarity strength correlated with the comprehension of relevant speech. These results indicate that the fMRI signal tracks cortical responses and attention effects related to continuous speech and support the notion that primary and non-primary AC process ongoing speech in a push-pull of acoustic and linguistic information.
Historically, eloquent functions have been viewed as localized to focal areas of human cerebral cortex, while more recent studies suggest they are encoded by distributed networks. We examined the network properties of cortical sites defined by stimulation to be critical for speech and language, using electrocorticography from sixteen participants during word-reading. We discovered distinct network signatures for sites where stimulation caused speech arrest and language errors. Both demonstrated lower local and global connectivity, whereas sites causing language errors exhibited higher inter-community connectivity, identifying them as connectors between modules in the language network. We used machine learning to classify these site types with reasonably high accuracy, even across participants, suggesting that a site’s pattern of connections within the task-activated language network helps determine its importance to function. These findings help to bridge the gap in our understanding of how focal cortical stimulation interacts with complex brain networks to elicit language deficits.
We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video “Bang! You’re Dead” and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.
Movies captivate groups of individuals (the audience), especially if they contain themes of common motivational interest to the group. In drug addiction, a key mechanism is maladaptive motivational salience attribution whereby drug cues outcompete other reinforcers within the same environment or context. We predicted that while watching a drug-themed movie, where cues for drugs and other reinforcers share a continuous narrative context, fMRI responses in individuals with heroin use disorder (iHUD) will preferentially synchronize during drug scenes. Results revealed such drug-biased synchronization in the orbitofrontal cortex (OFC), ventromedial and ventrolateral prefrontal cortex, and insula. After 15 weeks of inpatient treatment, there was a significant reduction in this drug-biased shared response in the OFC, which correlated with a concomitant reduction in dynamically-measured craving, suggesting synchronized OFC responses to a drug-themed movie as a neural marker of craving and recovery in iHUD.
Understanding how human brains interpret and process information is important. Here, we investigated the selectivity and inter-individual differences in human brain responses to images via functional MRI. In our first experiment, we found that images predicted to achieve maximal activations using a group level encoding model evoke higher responses than images predicted to achieve average activations, and the activation gain is positively associated with the encoding model accuracy. Furthermore, anterior temporal lobe face area (aTLfaces) and fusiform body area 1 had higher activation in response to maximal synthetic images compared to maximal natural images. In our second experiment, we found that synthetic images derived using a personalized encoding model elicited higher responses compared to synthetic images from group-level or other subjects’ encoding models. The finding of aTLfaces favoring synthetic images than natural images was also replicated. Our results indicate the possibility of using data-driven and generative approaches to modulate macro-scale brain region responses and probe inter-individual differences in and functional specialization of the human visual system.
The quest to understand how the development of the brain supports the development of complex cognitive functions is fueled by advances in cognitive neuroscience methods. Intracranial EEG (iEEG) recorded directly from the developing human brain provides unprecedented spatial and temporal resolution for mapping the neurophysiological mechanisms supporting cognitive development. In this paper, we focus on episodic memory, the ability to remember detailed information about past experiences, which improves from childhood into adulthood. We review memory effects based on broadband spectral power and emphasize the importance of isolating narrowband oscillations from broadband activity to determine mechanisms of neural coordination within and between brain regions. We then review evidence of developmental variability in neural oscillations and present emerging evidence linking the development of neural oscillations to the development of memory. We conclude by proposing that the development of oscillations increases the precision of neural coordination and may be an essential factor underlying memory development. More broadly, we demonstrate how recording neural activity directly from the developing brain holds immense potential to advance our understanding of cognitive development.
The human brain extracts meaning using an extensive neural system for
semantic knowledge. Whether broadly distributed systems depend on or can
compensate after losing a highly interconnected hub is controversial. We
report intracranial recordings from two patients during a speech prediction
task, obtained minutes before and after neurosurgical treatment requiring
disconnection of the left anterior temporal lobe (ATL), a candidate semantic
knowledge hub. Informed by modern diaschisis and predictive coding fra-
meworks, we tested hypotheses ranging from solely neural network disruption
to complete compensation by the indirectly affected language-related and
speech-processing sites. Immediately after ATL disconnection, we observed
neurophysiological alterations in the recorded frontal and auditory sites,
providing direct evidence for the importance of the ATL as a semantic hub. We
also obtained evidence for rapid, albeit incomplete, attempts at neural net-
work compensation, with neural impact largely in the forms stipulated by the
predictive coding framework, in specificity, and the modern diaschisis fra-
mework, more generally. The overall results validate these frameworks and
reveal an immediate impact and capability of the human brain to adjust after
losing a brain hub.
Identifying stimulus-driven neural activity patterns is critical for studying the neural basis of cognition. This can be particularly challenging in intracranial datasets, where electrode locations typically vary across patients. This chapter first presents an overview of the major challenges to identifying stimulus-driven neural activity patterns in the general case. Next, we will review several modality-specific considerations and approaches, along with a discussion of several issues that are particular to intracranial recordings. Against this backdrop, we will consider a variety of within-subject and across-subject approaches to identifying and modeling stimulus-driven neural activity patterns in multi-patient intracranial recordings. These approaches include generalized linear models, multivariate pattern analysis, representational similarity analysis, joint stimulus-activity models, hierarchical matrix factorization models, Gaussian process models, geometric alignment models, inter-subject correlations, and inter-subject functional correlations. Examples from the recent literature serve to illustrate the major concepts and provide the conceptual intuitions for each approach.KeywordsStimulus-drivenMulti-subjectSignal processingComputational modelsDynamics
Humor plays a prominent role in our lives. Thus, understanding the cognitive and neural mechanisms of humor is particularly important. Previous studies that investigated neural substrates of humor used functional MRI and to a lesser extent EEG. In the present study, we conducted intracranial recording in human patients, enabling us to obtain the signal with high temporal precision from within specific brain locations. Our analysis focused on the temporal lobe and the surrounding areas, the temporal lobe was most densely covered in our recording. Thirteen patients watched a fragment of a Charlie Chaplin movie. An independent group of healthy participants rated the same movie fragment, helping us to identify the most funny and the least funny frames of the movie. We compared neural activity occurring during the most funny and least funny frames across frequencies in the range of 1-170 Hz. The most funny compared to least funny parts of the movie were associated with activity modulation in the broadband high-gamma (70-170 Hz; mostly activation) and to a lesser extent gamma band (40-69Hz; activation) and low frequencies (1-12 Hz, delta, theta, alpha bands; mostly deactivation). With regard to regional specificity, we found three types of brain areas: (I) temporal pole, middle and inferior temporal gyrus (both anterior and posterior) in which there was both activation in the high-gamma/gamma bands and deactivation in low frequencies; (II) ventral part of the temporal lobe such as the fusiform gyrus, in which there was mostly deactivation the low frequencies; (III) posterior temporal cortex and its environment, such as the middle occipital and the temporo-parietal junction, in which there was activation in the high-gamma/gamma band. Overall, our results suggest that humor appreciation might be achieved by neural activity across the frequency spectrum.
Technologies have always influenced human life, resulting in changes in the course of human civilization that have led to modern humanity's formation, and in which we have reached a new level of living with a wide range of facilities and comforts. Scientific approaches for treatment, manipulation, and simulation of human behavior according to the complex and multiple functions of the brain have created a new field called brain-machine interface (BMI). BMI is an ambitious and advanced technology that can be a turning point in human civilization. It is evident that artificial intelligence will play a significant role in improving this technology, and machine learning will be essential for translating recorded activities and signals.
The aim of this study is to develop a method for translating and interpreting brain signals efficiently. In the presented approach, deep neural networks are used to estimate the parameters of the distribution of observations with a generalized dynamic Bayesian networks. In addition to its flexibility in expression and structure, this model is easier to interpret when combined with the opinions of experts. Also, unlike other detection and translation methods that have a preprocessing step for artifact rejection, noise filtering, outliers, and feature extraction, our model is modeled with raw data, making it suitable for real-time and online systems. The proposed model is validated using a data set related to sleep spindle detection, and the results show that it is more efficient than other models.
Objective. A large part of the cerebral cortex is dedicated to the processing of visual stimuli and there is still much to understand about such processing modalities and hierarchies. The main aim of the present study is to investigate the differences between directional visual stimuli (DS) and non-directional visual stimuli (n-DS) processing by time-frequency analysis of brain electroencephalographic activity during a visuo-motor task. Electroencephalography (EEG) data were divided into four regions of interest (ROIs) (frontal, central, parietal, occipital). Approach. The analysis of the visual stimuli processing was based on the combination of electroencephalographic recordings and time-frequency analysis. Event related spectral perturbations (ERSPs) were computed with spectrum analysis that allow to obtain the average time course of relative changes induced by the stimulus presentation in spontaneous EEG amplitude spectrum. Main results. Visual stimuli processing enhanced the same pattern of spectral modulation in all investigated ROIs with differences in amplitudes and timing. Additionally, statistically significant differences in occipital ROI between the DS and n-DS visual stimuli processing in theta, alpha and beta bands were found. Significance. These evidences suggest that ERSPs could be a useful tool to investigate the encoding of visual information in different brain regions. Because of their simplicity and their capability in the representation of brain activity, the ERSPs might be used as biomarkers of functional recovery for example in the rehabilitation of visual dysfunction and motor impairment following a stroke, as well as diagnostic tool of anomalies in brain functions in neurological diseases tailored to personalized treatments in clinical environment.
Since the second-half of the twentieth century, intracranial electroencephalography (iEEG), including both electrocorticography (ECoG) and stereo-electroencephalography (sEEG), has provided an intimate view into the human brain. At the interface between fundamental research and the clinic, iEEG provides both high temporal resolution and high spatial specificity but comes with constraints, such as the individual's tailored sparsity of electrode sampling. Over the years, researchers in neuroscience developed their practices to make the most of the iEEG approach. Here we offer a critical review of iEEG research practices in a didactic framework for newcomers, as well addressing issues encountered by proficient researchers. The scope is threefold: (i) review common practices in iEEG research, (ii) suggest potential guidelines for working with iEEG data and answer frequently asked questions based on the most widespread practices, and (iii) based on current neurophysiological knowledge and methodologies, pave the way to good practice standards in iEEG research. The organization of this paper follows the steps of iEEG data processing. The first section contextualizes iEEG data collection. The second section focuses on localization of intracranial electrodes. The third section highlights the main pre-processing steps. The fourth section presents iEEG signal analysis methods. The fifth section discusses statistical approaches. The sixth section draws some unique perspectives on iEEG research. Finally, to ensure a consistent nomenclature throughout the manuscript and to align with other guidelines, e.g., Brain Imaging Data Structure (BIDS) and the OHBM Committee on Best Practices in Data Analysis and Sharing (COBIDAS), we provide a glossary to disambiguate terms related to iEEG research.
Background
Functional mapping of eloquent brain areas is crucial for preoperative planning in patients with brain tumors. Resting state functional MRI (rs-fMRI) allows the localization of functional brain areas without the need for task performance, making it well-suited for the pediatric population. In this study the independent component analysis (ICA) rs-fMRI functional mapping results are reported in a group of 22 pediatric patients with supratentorial brain tumors. Additionally, the functional connectivity (FC) maps of the sensori-motor network (SMN) obtained using ICA and seed-based analysis (SBA) are compared.
Results
Different resting state networks (RSNs) were extracted using ICA with varying levels of sensitivity, notably, the SMN was identified in 100% of patients, followed by the Default mode network (DMN) (91%) and Language networks (80%). Additionally, FC maps of the SMN extracted by SBA were more extensive (mean volume = 25,288.36 mm ³ , standard deviation = 13,364.36 mm ³ ) than those found on ICA (mean volume = 13,403.27 mm ³ , standard deviation = 9755.661 mm ³ ). This was confirmed by statistical analysis using a Wilcoxon signed rank t test at p < 0.01.
Conclusions
Results clearly demonstrate the successful applicability of rs-fMRI for localizing different functional brain networks in the preoperative assessment of brain areas, and thus represent a further step in the integration of computational radiology research in a clinical setting.
In this study, we quantified the coverage of gray and white matter during intracranial electroencephalography in a cohort of epilepsy patients with surface and depth electrodes. We included 65 patients with strip electrodes (n = 12), strip and grid electrodes (n = 24), strip, grid, and depth electrodes (n = 7), or depth electrodes only (n = 22). Patient-specific imaging was used to generate probabilistic gray and white matter maps and atlas segmentations. Gray and white matter coverage was quantified using spherical volumes centered on electrode centroids, with radii ranging from 1 to 15 mm, along with detailed finite element models of local electric fields. Gray matter coverage was highly dependent on the chosen radius of influence (RoI). Using a 2.5 mm RoI, depth electrodes covered more gray matter than surface electrodes; however, surface electrodes covered more gray matter at RoI larger than 4 mm. White matter coverage and amygdala and hippocampal coverage was greatest for depth electrodes at all RoIs. This study provides the first probabilistic analysis to quantify coverage for different intracranial recording configurations. Depth electrodes offer increased coverage of gray matter over other recording strategies if the desired signals are local, while subdural grids and strips sample more gray matter if the desired signals are diffuse.
Large, openly available datasets and current analytic tools promise the emergence of population neuroscience. The considerable diversity in personality traits and behaviour between individuals is reflected in the statistical variability of neural data collected in such repositories. Recent studies with functional magnetic resonance imaging (fMRI) have concluded that patterns of resting-state functional connectivity can both successfully distinguish individual participants within a cohort and predict some individual traits, yielding the notion of an individual’s neural fingerprint. Here, we aim to clarify the neurophysiological foundations of individual differentiation from features of the rich and complex dynamics of resting-state brain activity using magnetoencephalography (MEG) in 158 participants. We show that akin to fMRI approaches, neurophysiological functional connectomes enable the differentiation of individuals, with rates similar to those seen with fMRI. We also show that individual differentiation is equally successful from simpler measures of the spatial distribution of neurophysiological spectral signal power. Our data further indicate that differentiation can be achieved from brain recordings as short as 30 seconds, and that it is robust over time: the neural fingerprint is present in recordings performed weeks after their baseline reference data was collected. This work, thus, extends the notion of a neural or brain fingerprint to fast and large-scale resting-state electrophysiological dynamics. We all have the intuition that our brain makes us unique. Here, the authors show that seconds of brain activity are sufficient to differentiate an individual, even when recorded weeks or months apart.
A Brain-Computer Interface (BCI) can be considered as a technology that allows for alternative means of communication between humans and their environment using thoughts and intentions. The structure of this interface is composed of various stages, beginning with the acquisition of the brain signals, followed by several processing stages, and leading to the generation of feedback signals. The development of a BCI system involves a diverse set of expertise in order to produce a unique environment for continuous innovations. However, such diversity in technical background and expertise may lead to confusion in the terminology used by the community. As such, the IEEE P2731 WG has been tasked with the development of a functional model to facilitate the understanding of a BCI system. In this paper, we focus on the description of the functional elements that belong to the transducer stage of a BCI.
Movies, audio stories, and virtual reality are increasingly used as stimuli for functional brain imaging. Such naturalistic paradigms are in sharp contrast to the tradition of experimental reductionism in neuroscience research. Being complex, dynamic, and diverse, naturalistic stimuli set up a more ecologically relevant condition and induce highly reproducible brain responses across a wide range of spatiotemporal scales. Here, we review recent technical advances and scientific findings on imaging the brain under naturalistic stimuli. Then we elaborate on the premise of using naturalistic paradigms for multi-scale, multi-modal, and high-throughput functional characterization of the human brain. We further highlight the growing potential of using deep learning models to infer neural information processing from brain responses to naturalistic stimuli. Lastly, we advocate large-scale collaborations to combine brain imaging and recording data across experiments, subjects, and labs that use the same set of naturalistic stimuli.
Movie viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG and EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG or EEG dynamics during such naturalistic situations.
We thus compared MEG and fMRI responses to the same 15-min black-and-white movie in the same eight subjects who watched the movie twice during both MEG and fMRI recordings. We analyzed intra- and intersubject voxel-wise correlations within each imaging modality as well as the correlation of the MEG envelopes and fMRI signals. The fMRI signals showed voxel-wise within- and between-subjects correlations up to r = 0.66 and r = 0.37, respectively, whereas these correlations were clearly weaker for the envelopes of band-pass filtered (7 frequency bands below 100 Hz) MEG signals (within-subjects correlation r < 0.14 and between-subjects r < 0.05). Direct MEG–fMRI voxel-wise correlations were unreliable. Notably, applying a spatial-filtering approach to the MEG data uncovered consistent canonical variates that showed considerably stronger (up to r = 0.25) between-subjects correlations than the univariate voxel-wise analysis. Furthermore, the envelopes of the time courses of these variates up to about 10 Hz showed association with fMRI signals in a general linear model. Similarities between envelopes of MEG canonical variates and fMRI voxel time-courses were seen mostly in occipital, but also in temporal and frontal brain regions, whereas intra- and intersubject correlations for MEG and fMRI separately were strongest only in the occipital areas.
In contrast to the conventional univariate analysis, the spatial-filtering approach was able to uncover associations between the MEG envelopes and fMRI time courses, shedding light on the similarities of hemodynamic and electromagnetic brain activities during movie viewing.
The neocortex is composed of six anatomically and physiologically specialized layers. It has been proposed that integration of activity across cortical areas is mediated anatomically by associative connections terminating in superficial layers, and physiologically by slow cortical rhythms. However, the means through which neocortical anatomy and physiology interact to coordinate neural activity remains obscure. Using laminar microelectrode arrays in 19 human participants, we found that most EEG activity is below 10-Hz (delta/theta) and generated by superficial cortical layers during both wakefulness and sleep. Cortical surface grid, grid-laminar, and dual-laminar recordings demonstrate that these slow rhythms are synchronous within upper layers across broad cortical areas. The phase of this superficial slow activity is reset by infrequent stimuli and coupled to the amplitude of faster oscillations and neuronal firing across all layers. These findings support a primary role of superficial slow rhythms in generating the EEG and integrating cortical activity.
Spontaneous fluctuations in hemodynamic signals in the absence of a task or overt stimulation are used to infer neural activity. We tested this coupling by simultaneously measuring neural activity and changes in cerebral blood volume (CBV) in the somatosensory cortex of awake, head-fixed mice during periods of true rest and during whisker stimulation and volitional whisking. We found that neurovascular coupling was similar across states and that large, spontaneous CBV changes in the absence of sensory input were driven by volitional whisker and body movements. Hemodynamic signals during periods of rest were weakly correlated with neural activity. Spontaneous fluctuations in CBV and vessel diameter persisted when local neural spiking and glutamatergic input were blocked, as well as during blockade of noradrenergic receptors, suggesting a non-neuronal origin for spontaneous CBV fluctuations. Spontaneous hemodynamic signals reflect a combination of behavior, local neural activity, and putatively non-neural processes.
In neuroscience, stimulus-response relationships have traditionally been analyzed using either encoding or decoding models. Here we propose a hybrid approach that decomposes neural activity into multiple components, each representing a portion of the stimulus. The technique is implemented via canonical correlation analysis (CCA) by temporally filtering the stimulus (encoding) and spatially filtering the neural responses (decoding) such that the resulting components are maximally correlated. In contrast to existing methods, this approach recovers multiple correlated stimulus-response pairs, and thus affords a richer, multidimensional analysis of neural representations. We first validated the technique's ability to recover multiple stimulus-driven components using electroencephalographic (EEG) data simulated with a finite element model of the head. We then applied the technique to real EEG responses to auditory and audiovisual narratives experienced identically across subjects, as well as uniquely experienced video game play. During narratives, both auditory and visual stimulus-response correlations (SRC) were modulated by attention and tracked inter-subject correlations. During video game play, SRC varied with game difficulty and the presence of a dual task. Interestingly, the strongest component extracted for visual and auditory features of film clips had nearly identical spatial distributions, suggesting that the predominant encephalographic response to naturalistic stimuli is supramodal. The diversity of these findings demonstrates the utility of measuring multidimensional SRC via hybrid encoding-decoding.
Our brains integrate information across sensory modalities to generate perceptual experiences and form memories. However, it is difficult to determine the conditions under which multisensory stimulation will benefit or hinder the retrieval of everyday experiences. We hypothesized that the determining factor is the reliability of information processing during stimulus presentation, which can be measured through intersubject correlation of stimulus-evoked activity. We therefore presented biographical auditory narratives and visual animations to 72 human subjects visually, auditorily, or combined, while neural activity was recorded using electroencephalography. Memory for the narrated information, contained in the auditory stream, was tested 3 weeks later. While the visual stimulus alone led to no meaningful retrieval, this related stimulus improved memory when it was combined with the story, even when it was temporally incongruent with the audio. Further, individuals with better subsequent memory elicited neural responses during encoding that were more correlated with their peers. Surprisingly, portions of this predictive synchronized activity were present regardless of the sensory modality of the stimulus. These data suggest that the strength of sensory and supramodal activity is predictive of memory performance after 3 weeks, and that neural synchrony may explain the mnemonic benefit of the functionally uninformative visual context observed for these real-world stimuli.
Does the default mode network (DMN) reconfigure to encode information about the changing environment? This question has proven difficult, because patterns of functional connectivity reflect a mixture of stimulus-induced neural processes, intrinsic neural processes and non-neuronal noise. Here we introduce inter-subject functional correlation (ISFC), which isolates stimulus-dependent inter-regional correlations between brains exposed to the same stimulus. During fMRI, we had subjects listen to a real-life auditory narrative and to temporally scrambled versions of the narrative. We used ISFC to isolate correlation patterns within the DMN that were locked to the processing of each narrative segment and specific to its meaning within the narrative context. The momentary configurations of DMN ISFC were highly replicable across groups. Moreover, DMN coupling strength predicted memory of narrative segments. Thus, ISFC opens new avenues for linking brain network dynamics to stimulus features and behaviour.
Significance
In this study, we demonstrate that α-, β-, and γ-oscillations can be related to activity in different cortical layers in the human brain. These cortical layers are believed to instantiate feedforward and feedback information streams. Electrophysiological oscillations in α- (8–12 Hz), β- (15–30 Hz), and γ- (40–100 Hz) bands have also been associated with different roles in feedforward and feedback processes and to different cortical layers in nonhuman animals. By simultaneously recording laminar functional MRI (fMRI) and EEG, we could directly link oscillatory signals to activity in cortical layers in humans. This research provides an important neural basis for noninvasive research into the role of cortical layers in information processing using laminar fMRI alone or combined with EEG.
Attentional engagement is a major determinant of how effectively we gather information through our senses. Alongside the sheer growth in the amount and variety of information content that we are presented with through modern media, there is increased variability in the degree to which we "absorb" that information. Traditional research on attention has illuminated the basic principles of sensory selection to isolated features or locations, but it provides little insight into the neural underpinnings of our attentional engagement with modern naturalistic content. Here, we show in human subjects that the reliability of an individual's neural responses with respect to a larger group provides a highly robust index of the level of attentional engagement with a naturalistic narrative stimulus. Specifically, fast electroencephalographic evoked responses were more strongly correlated across subjects when naturally attending to auditory or audiovisual narratives than when attention was directed inward to a mental arithmetic task during stimulus presentation. This effect was strongest for audiovisual stimuli with a cohesive narrative and greatly reduced for speech stimuli lacking meaning. For compelling audiovisual narratives, the effect is remarkably strong, allowing perfect discrimination between attentional state across individuals. Control experiments rule out possible confounds related to altered eye movement trajectories or order of presentation. We conclude that reliability of evoked activity reproduced across subjects viewing the same movie is highly sensitive to the attentional state of the viewer and listener, which is aided by a cohesive narrative.
In source localization of electroencephalograpic (EEG) signals, as well as in targeted transcranial current stimulation (TCS), a volume conductor model is required to describe the flow of electric currents in the head. Boundary element models (BEM) can be readily computed to represent major tissue compartments, but cannot encode detailed anatomical information within compartments. Finite element models (FEM) can capture more tissue types and intricate anatomical structures, but with the higher precision also comes the need for semi-automated segmentation, and a higher computational cost. In either case, adjusting to the individual human anatomy requires costly magnetic resonance imaging (MRI), and thus head modeling is often based on the anatomy of an 'arbitrary' individual (e.g. Colin27). Additionally, existing reference models for the human head often do not include the cerebro-spinal fluid (CSF), and their field of view excludes portions of the head and neck - two factors that demonstrably affect current-flow patterns. Here we present a highly detailed FEM, which we call ICBM-NY. It is based on the ICBM152 anatomical template (a non-linear average of the MRI of 152 adult human brains), for which we extended the field of view to the neck, and performed a detailed segmentation of six tissue types (scalp, skull, CSF, gray matter, white matter, air cavities) at 0.5mm(3) resolution. The model was solved for 231 electrode locations. To evaluate its performance, additional FEMs and BEMs were constructed for four individual subjects. Each of the four individual FEMs (regarded as the 'ground truth') is compared to its BEM counterpart, the ICBM-NY, a BEM of the ICBM anatomy, an 'individualized' BEM of the ICBM anatomy warped to the individual head surface, and FEMs of the other individuals. Performance is measured in terms of EEG source localization and TCS targeting errors. Results show that the ICBM-NY outperforms FEMs of mismatched individual anatomies as well as the BEM of the ICBM anatomy according to both criteria. We therefore propose the ICBM-NY as a new standard head model to be used in future EEG and TCS studies whenever an individual MRI is not available. We release all model data online to facilitate broad adoption.
Functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG) research have been influential in revealing the functional characteristics of category-selective responses in human ventral temporal cortex (VTC). One important, but unanswered, question is how these two types of measurements might be related with respect to the VTC. Here we examined which components of the ECoG signal correspond to the fMRI response, by using a rare opportunity to measure both fMRI and ECoG responses from the same individuals to images of exemplars of various categories including faces, limbs, cars and houses. Our data reveal three key findings. First, we discovered that the coupling between fMRI and ECoG responses is frequency and time dependent. The strongest and most sustained correlation is observed between fMRI and high frequency broadband (HFB) ECoG responses (30-160hz). In contrast, the correlation between fMRI and ECoG signals in lower frequency bands is temporally transient, where the correlation is initially positive, but then tapers off or becomes negative. Second, we find that the strong and positive correlation between fMRI and ECoG signals in all frequency bands emerges rapidly around 100ms after stimulus onset, together with the onset of the first stimulus-driven neural signals in VTC. Third, we find that the spatial topology and representational structure of category-selectivity in VTC reflected in ECoG HFB responses mirrors the topology and structure observed with fMRI. These findings of a strong and rapid coupling between fMRI and HFB responses validate fMRI measurements of functional selectivity with recordings of direct neural activity and suggest that fMRI category-selective signals in VTC are associated with feed-forward neural processing.
Copyright © 2015. Published by Elsevier Ltd.
Multimodal data is ubiquitous in engineering, communication, robotics, vision or more generally speaking in industry and the sciences. All disciplines have developed their respective sets of analytic tools to fuse the information that is available in all measured modalities. In this paper we provide a review of classical as well as recent machine learning methods (specifically factor models) for fusing information from functional neuroimaging techniques such as LFP, EEG, MEG, fNIRS and fMRI. Early and late fusion scenarios are distinguished and appropriate factor models for the respective scenarios are presented along with example applications from selected multimodal neuroimaging studies. Further emphasis is given to the interpretability of the resulting model parameters, in particular by highlighting how factor models relate to physical models needed for source localization. The methods we discuss allow to extract information from neural data, which ultimately contributes to (a) better neuroscientific understanding, (b) enhance diagnostic performance and (c) discover neural signals of interest that correlate maximally with a given cognitive paradigm. While we clearly study the multimodal functional neuroimaging challenge, the discussed machine learning techniques have a wide applicability beyond, i.e. in general data fusion and may thus be informative to the general interested reader.
A central neuroscientific pursuit is understanding neuronal interactions that support computations underlying cognition and behavior. Although neurons interact across disparate scales, from cortical columns to whole-brain networks, research has been restricted to one scale at a time. We measured local interactions through multi-neuronal recordings while accessing global networks using scalp electroencephalography (EEG) in rhesus macaques. We measured spike count correlation, an index of functional connectivity with computational relevance, and EEG oscillations, which have been linked to various cognitive functions. We found a non-monotonic relationship between EEG oscillation amplitude and spike count correlation, contrary to the intuitive expectation of a direct relationship. With a widely used network model, we replicated these findings by incorporating a private signal targeting inhibitory neurons, a common mechanism proposed for gain modulation. Finally, we found that spike count correlation explained nonlinearities in the relationship between EEG oscillations and response time in a spatial selective attention task.
Naturalistic stimuli evoke highly reliable brain activity across viewers. Here we record neural activity from a group of naive individuals while viewing popular, previously-broadcast television content for which the broad audience response is characterized by social media activity and audience ratings. We find that the level of inter-subject correlation in the evoked encephalographic responses predicts the expressions of interest and preference among thousands. Surprisingly, ratings of the larger audience are predicted with greater accuracy than those of the individuals from whom the neural data is obtained. An additional functional magnetic resonance imaging study employing a separate sample of subjects shows that the level of neural reliability evoked by these stimuli covaries with the amount of blood-oxygenation-level-dependent (BOLD) activation in higher-order visual and auditory regions. Our findings suggest that stimuli which we judge favourably may be those to which our brains respond in a stereotypical manner shared by our peers.
According to recent functional magnetic resonance imaging (fMRI) studies, spectators of a movie may share similar spatiotemporal patterns of brain activity. We aimed to extend these findings of intersubject correlation to temporally accurate single-trial magnetoencephalography (MEG). A silent 15-min black-and-white movie was shown to eight subjects twice. We adopted a spatial filtering model and estimated its parameter values by using multi-set canonical correlation analysis (M-CCA) so that the intersubject correlation was maximized. The procedure resulted in multiple (mutually uncorrelated) time-courses with statistically significant intersubject correlations at frequencies below 10Hz; the maximum correlation was 0.28±0.075 in the≤1Hz band. Moreover, the 24-Hz frame rate elicited steady-state responses with statistically significant intersubject correlations up to 0.29±0.12. To assess the brain origin of the across-subjects correlated signals, the time-courses were correlated with minimum-norm source current estimates (MNE) projected to the cortex. The time series implied across-subjects synchronous activity in the early visual, posterior and inferior parietal, lateral temporo-occipital, and motor cortices, and in the superior temporal sulcus (STS) bilaterally. These findings demonstrate the capability of the proposed methodology to uncover cortical MEG signatures from single-trial signals that are consistent across spectators of a movie.
Ongoing neuronal activity in the CNS waxes and wanes continuously across widespread spatial and temporal scales. In the human brain, these spontaneous fluctuations are salient in blood oxygenation level-dependent (BOLD) signals and correlated within specific brain systems or "intrinsic-connectivity networks." In electrophysiological recordings, both the amplitude dynamics of fast (1-100 Hz) oscillations and the scalp potentials per se exhibit fluctuations in the same infra-slow (0.01-0.1 Hz) frequency range where the BOLD fluctuations are conspicuous. While several lines of evidence show that the BOLD fluctuations are correlated with fast-amplitude dynamics, it has remained unclear whether the infra-slow scalp potential fluctuations in full-band electroencephalography (fbEEG) are related to the resting-state BOLD signals. We used concurrent fbEEG and functional magnetic resonance imaging (fMRI) recordings to address the relationship of infra-slow fluctuations (ISFs) in scalp potentials and BOLD signals. We show here that independent components of fbEEG recordings are selectively correlated with subsets of cortical BOLD signals in specific task-positive and task-negative, fMRI-defined resting-state networks. This brain system-specific association indicates that infra-slow scalp potentials are directly associated with the endogenous fluctuations in neuronal activity levels. fbEEG thus yields a noninvasive, high-temporal resolution window into the dynamics of intrinsic connectivity networks. These results support the view that the slow potentials reflect changes in cortical excitability and shed light on neuronal substrates underlying both electrophysiological and behavioral ISFs.
Recent evidence from functional magnetic resonance imaging suggests that cortical hemodynamic responses coincide in different subjects experiencing a common naturalistic stimulus. Here we utilize neural responses in the electroencephalogram (EEG) evoked by multiple presentations of short film clips to index brain states marked by high levels of correlation within and across subjects. We formulate a novel signal decomposition method which extracts maximally correlated signal components from multiple EEG records. The resulting components capture correlations down to a one-second time resolution, thus revealing that peak correlations of neural activity across viewings can occur in remarkable correspondence with arousing moments of the film. Moreover, a significant reduction in neural correlation occurs upon a second viewing of the film or when the narrative is disrupted by presenting its scenes scrambled in time. We also probe oscillatory brain activity during periods of heightened correlation, and observe during such times a significant increase in the theta band for a frontal component and reductions in the alpha and beta frequency bands for parietal and occipital components. Low-resolution EEG tomography of these components suggests that the correlated neural activity is consistent with sources in the cingulate and orbitofrontal cortices. Put together, these results suggest that the observed synchrony reflects attention- and emotion-modulated cortical processing which may be decoded with high temporal resolution by extracting maximally correlated components of neural activity.
Neuronal activity in the brain gives rise to transmembrane currents that can be measured in the extracellular medium. Although the major contributor of the extracellular signal is the synaptic transmembrane current, other sources--including Na(+) and Ca(2+) spikes, ionic fluxes through voltage- and ligand-gated channels, and intrinsic membrane oscillations--can substantially shape the extracellular field. High-density recordings of field activity in animals and subdural grid recordings in humans, combined with recently developed data processing tools and computational modelling, can provide insight into the cooperative behaviour of neurons, their average synaptic input and their spiking output, and can increase our understanding of how these processes contribute to the extracellular signal.
The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses – the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferroni-type procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.
Evolution-driven functional changes in the primate brain are typically assessed by aligning monkey and human activation maps using cortical surface expansion models. These models use putative homologous areas as registration landmarks, assuming they are functionally correspondent. For cases in which functional changes have occurred in an area, this assumption prohibits to reveal whether other areas may have assumed lost functions. Here we describe a method to examine functional correspondences across species. Without making spatial assumptions, we assessed similarities in sensory-driven functional magnetic resonance imaging responses between monkey (Macaca mulatta) and human brain areas by temporal correlation. Using natural vision data, we revealed regions for which functional processing has shifted to topologically divergent locations during evolution. We conclude that substantial evolution-driven functional reorganizations have occurred, not always consistent with cortical expansion processes. This framework for evaluating changes in functional architecture is crucial to building more accurate evolutionary models.
There is growing evidence that several components of the mass neural activity contributing to the local field potential (LFP) can be partly separated by decomposing the LFP into nonoverlapping frequency bands. Although the blood oxygen level-dependent (BOLD) signal has been found to correlate preferentially with specific frequency bands of the LFP, it is still unclear whether the BOLD signal relates to the activity expressed by each LFP band independently of the others or if, instead, it also reflects specific relationships among different bands. We investigated these issues by recording, simultaneously and with high spatiotemporal resolution, BOLD signal and LFP during spontaneous activity in early visual cortices of anesthetized monkeys (Macaca mulatta). We used information theory to characterize the statistical dependency between BOLD and LFP. We found that the alpha (8-12 Hz), beta (18-30 Hz), and gamma (40-100 Hz) LFP bands were informative about the BOLD signal. In agreement with previous studies, gamma was the most informative band. Both increases and decreases in BOLD signal reliably followed increases and decreases in gamma power. However, both alpha and beta power signals carried information about BOLD that was largely complementary to that carried by gamma power. In particular, the relationship between alpha and gamma power was reflected in the amplitude of the BOLD signal, while the relationship between beta and gamma bands was reflected in the latency of BOLD with respect to significant changes in gamma power. These results lay the basis for identifying contributions of different neural pathways to cortical processing using fMRI.
The neurophysiological underpinnings of functional magnetic resonance imaging (fMRI) are not well understood. To understand the relationship between the fMRI blood oxygen level dependent (BOLD) signal and neurophysiology across large areas of cortex, we compared task related BOLD change during simple finger movement to brain surface electric potentials measured on a similar spatial scale using electrocorticography (ECoG). We found that spectral power increases in high frequencies (65-95 Hz), which have been related to local neuronal activity, colocalized with spatially focal BOLD peaks on primary sensorimotor areas. Independent of high frequencies, decreases in low frequency rhythms (<30 Hz), thought to reflect an aspect of cortical-subcortical interaction, colocalized with weaker BOLD signal increase. A spatial regression analysis showed that there was a direct correlation between the amplitude of the task induced BOLD change on different areas of primary sensorimotor cortex and the amplitude of the high frequency change. Low frequency change explained an additional, different part of the spatial BOLD variance. Together, these spectral power changes explained a significant 36% of the spatial variance in the BOLD signal change (R(2) = 0.36). These results suggest that BOLD signal change is largely induced by two separate neurophysiological mechanisms, one being spatially focal neuronal processing and the other spatially distributed low frequency rhythms.
We describe a set of complementary EEG data collection and processing tools recently developed at the Swartz Center for Computational Neuroscience (SCCN) that connect to and extend the EEGLAB software environment, a freely available and readily extensible processing environment running under Matlab. The new tools include (1) a new and flexible EEGLAB STUDY design facility for framing and performing statistical analyses on data from multiple subjects; (2) a neuroelectromagnetic forward head modeling toolbox (NFT) for building realistic electrical head models from available data; (3) a source information flow toolbox (SIFT) for modeling ongoing or event-related effective connectivity between cortical areas; (4) a BCILAB toolbox for building online brain-computer interface (BCI) models from available data, and (5) an experimental real-time interactive control and analysis (ERICA) environment for real-time production and coordination of interactive, multimodal experiments.
Data recorded from multiple sources sometimes exhibit non-instantaneous couplings. For simple data sets, cross-correlograms may reveal the coupling dynamics. But when dealing with high-dimensional multivariate data there is no such measure as the cross-correlogram. We propose a simple algorithm based on Kernel Canonical Correlation Analysis (kCCA) that computes a multivariate temporal filter which links one data modality to another one. The filters can be used to compute a multivariate extension of the cross-correlogram, the canonical correlogram, between data sources that have different dimensionalities and temporal resolutions. The canonical correlogram reflects the coupling dynamics between the two sources. The temporal filter reveals which features in the data give rise to these couplings and when they do so. We present results from simulations and neuroscientific experiments showing that tkCCA yields easily interpretable temporal filters and correlograms. In the experiments, we simultaneously performed electrode recordings and functional magnetic resonance imaging (fMRI) in primary visual cortex of the non-human primate. While electrode recordings reflect brain activity directly, fMRI provides only an indirect view of neural activity via the Blood Oxygen Level Dependent (BOLD) response. Thus it is crucial for our understanding and the interpretation of fMRI signals in general to relate them to direct measures of neural activity acquired with electrodes. The results computed by tkCCA confirm recent models of the hemodynamic response to neural activity and allow for a more detailed analysis of neurovascular coupling dynamics.
A fundamental question in neuroscience concerns the relation between the spiking of individual neurons and the aggregate electrical activity of neuronal ensembles as seen in local field potentials (LFPs). Because LFPs reflect both spiking activity and subthreshold events, this question is not simply one of data aggregation. Recording from 20 neurosurgical patients, we directly examined the relation between LFPs and neuronal spiking. Examining 2030 neurons in widespread brain regions, we found that firing rates were positively correlated with broadband (2-150 Hz) shifts in the LFP power spectrum. In contrast, narrowband oscillations correlated both positively and negatively with firing rates at different recording sites. Broadband power shifts were a more reliable predictor of neuronal spiking than narrowband power shifts. These findings suggest that broadband LFP power provides valuable information concerning neuronal activity beyond that contained in narrowband oscillations.
Hemodynamic activity in occipital, temporal, and parietal cortical areas were recently shown to correlate across subjects during viewing of a 30-minute movie clip. However, most of the frontal cortex lacked between-subject correlations. Here we presented 12 healthy naïve volunteers with the first 72 minutes of a movie ("Crash", 2005, Lions Gate Films) outside of the fMRI scanner to involve the subjects in the plot of the movie, followed by presentation of the last 36 minutes during fMRI scanning. We observed significant between-subjects correlation of fMRI activity in especially right hemisphere frontal cortical areas, in addition to the correlation of activity in temporal, occipital, and parietal areas. It is possible that this resulted from the subjects following the plot of the movie and being emotionally engaged in the movie during fMRI scanning. We further show that probabilistic independent component analysis (ICA) reveals meaningful activations in individual subjects during natural viewing.
The role of primary visual cortex (V1) in determining the contents of perception is controversial. Human functional magnetic resonance imaging (fMRI) studies of perceptual suppression have revealed a robust drop in V1 activity when a stimulus is subjectively invisible. In contrast, monkey single-unit recordings have failed to demonstrate such perception-locked changes in V1. To investigate the basis of this discrepancy, we measured both the blood oxygen level-dependent (BOLD) response and several electrophysiological signals in two behaving monkeys. We found that all signals were in good agreement during conventional stimulus presentation, showing strong visual modulation to presentation and removal of a stimulus. During perceptual suppression, however, only the BOLD response and the low-frequency local field potential (LFP) power showed decreases, whereas the spiking and high-frequency LFP power were unaffected. These results demonstrate that the coupling between the BOLD and electrophysiological signals in V1 is context dependent, with a marked dissociation occurring during perceptual suppression.
The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.
Systems-level organization in spontaneous infra-slow (<0.1Hz) brain activity, measured using blood oxygen signals in fMRI and optical imaging, has become a major theme in the study of neural function in both humans and animal models. Yet the neurophysiological basis of infra-slow activity (ISA) remains unresolved. In particular, is ISA a distinct physiological process, or is it a low-frequency analog of faster neural activity? Here, using whole-cortex calcium/hemoglobin imaging in mice, we show that ISA in each of these modalities travels through the cortex along stereotypical spatiotemporal trajectories that are state dependent (wake versus anesthesia) and distinct from trajectories in delta (1–4 Hz) activity. Moreover, mouse laminar electrophysiology reveals that ISA travels through specific cortical layers and is organized into unique cross-laminar temporal dynamics that are different from higher frequency local field potential activity. These findings suggest that ISA is a distinct neurophysiological process that is reflected in fMRI blood oxygen signals. Mitra et al. measure spontaneous infra-slow activity (ISA; <0.1 Hz) in mouse cortex using wide-field optical calcium/hemoglobin imaging and laminar electrophysiology. They find wide-scale cortical spatiotemporal trajectories in ISA that are state-dependent and distinct from higher frequency neural activity.
Resting-state signals in blood-oxygenation-level-dependent (BOLD) imaging are used to parcellate brain regions and define “functional connections“ between regions. Yet a physiological link between fluctuations in blood oxygenation with those in neuronal signaling pathways is missing. We present evidence from studies on mouse cortex that modulation of vasomotion, i.e., intrinsic ultra-slow (0.1 Hz) fluctuations in arteriole diameter, provides this link. First, ultra-slow fluctuations in neuronal signaling, which occur as an envelope over γ-band activity, entrains vasomotion. Second, optogenetic manipulations confirm that entrainment is unidirectional. Third, co-fluctuations in the diameter of pairs of arterioles within the same hemisphere diminish to chance for separations >1.4 mm. Yet the diameters of arterioles in distant (>5 mm), mirrored transhemispheric sites strongly co-fluctuate; these correlations are diminished in acallosal mice. Fourth, fluctuations in arteriole diameter coherently drive fluctuations in blood oxygenation. Thus, entrainment of vasomotion links neuronal pathways to functional connections. In resting-state BOLD imaging, synchronous ultra-slow (∼0.1 Hz) oscillations in blood oxygenation between brain areas are interpreted as “functional“ neuronal connections. Mateo et al. reveals a basis for this inference: neuronal activity entrains arteriole dilation that in turn drives oxygenation.
Our lives revolve around sharing experiences and memories with others. When different people recount the same events, how similar are their underlying neural representations? Participants viewed a 50-min movie, then verbally described the events during functional MRI, producing unguided detailed descriptions lasting up to 40 min. As each person spoke, event-specific spatial patterns were reinstated in default-network, medial-temporal, and high-level visual areas. Individual event patterns were both highly discriminable from one another and similar among people, suggesting consistent spatial organization. In many high-order areas, patterns were more similar between people recalling the same event than between recall and perception, indicating systematic reshaping of percept into memory. These results reveal the existence of a common spatial organization for memories in high-level cortical areas, where encoded information is largely abstracted beyond sensory constraints, and that neural patterns during perception are altered systematically across people into shared memory representations for real-life events.
Relating behavioral and neuroimaging measures is essential to understanding human brain function. Often, this is achieved by computing a correlation between behavioral measures, e.g., reaction times, and neurophysiological recordings, e.g., prestimulus EEG alpha-power, on a single-trial-basis. This approach treats individual trials as independent measurements and ignores the fact that data are acquired in a temporal order. It has already been shown that behavioral measures as well as neurophysiological recordings display power-law dynamics, which implies that trials are not in fact independent. Critically, computing the correlation coefficient between two measures exhibiting long-range temporal dependencies may introduce spurious correlations, thus leading to erroneous conclusions about the relationship between brain activity and behavioral measures. Here, we address data-analytic pitfalls which may arise when long-range temporal dependencies in neural as well as behavioral measures are ignored. We quantify the influence of temporal dependencies of neural and behavioral measures on the observed correlations through simulations. Results are further supported in analysis of real EEG data recorded in a simple reaction time task, where the aim is to predict the latency of responses on the basis of prestimulus alpha oscillations. We show that it is possible to "predict" reaction times from one subject on the basis of EEG activity recorded in another subject simply owing to the fact that both measures display power-law dynamics. The same is true when correlating EEG activity obtained from different subjects. A surrogate-data procedure is described which correctly tests for the presence of correlation while controlling for the effect of power-law dynamics. Hum Brain Mapp, 2015. © 2015 Wiley Periodicals, Inc.
© 2015 Wiley Periodicals, Inc.
Previous studies demonstrated the presence of Monochromatic Ultra-Slow Oscillations (MUSO) in human EEG. In the present study we explored the biological origin of MUSO by simultaneous recordings of EEG, Near-Infrared Spectroscopy (NIRS), arterial blood pressure, respiration and Laser Doppler flowmetry. We used a head-up tilt test in order to check whether MUSO might relate to Mayer waves in arterial blood pressure, known to be enhanced by the tilting procedure. MUSO were detected in 8 out of 10 subjects during rest and showed a striking monochromatic spectrum (0.07–0.14 Hz). The spatial topography of MUSO was complex, showing multiple foci variable across subjects. While the head-up tilt test increased the relative power of Mayer waves, it had no effect on MUSO. On the other hand, the relative spectral power of 0.1 Hz oscillations in EEG, NIRS and blood pressure signals were positively correlated across subjects in the tilted condition. Eight subjects showed a coherence between MUSO and NIRS/arterial blood pressure. Moreover, MUSO at different electrode sites demonstrated coherence not reducible to volume conduction, thus indicating that MUSO are unlikely to be generated by one source. We related our experimental findings to known biological phenomena being generated at about 0.1 Hz, i.e.: arterial blood pressure, cerebral and skin vasomotion, respiration and neuronal activity. While no definite conclusion can yet be drawn as to an exact physiological mechanism of MUSO, we suggest that these oscillations might be of a rather extraneuronal origin reflecting cerebral vasomotion.
Phase synchronization among neuronal oscillations within the same frequency band has been hypothesized to be a major mechanism for communication between different brain areas. On the other hand, cross-frequency communications are more flexible allowing interactions between oscillations with different frequencies. Among such cross-frequency interactions amplitude-to-amplitude interactions are of a special interest as they show how the strength of spatial synchronization in different neuronal populations relates to each other during a given task. While, previously, amplitude-to-amplitude correlations were studied primarily on the sensor level, we present a source separation approach using spatial filters which maximize the correlation between the envelopes of brain oscillations recorded with electro-/magnetencephalography (EEG/MEG) or intracranial multichannel recordings. Our approach, which is called canonical source power correlation analysis (cSPoC), is thereby capable of extracting genuine brain oscillations solely based on their assumed coupling behavior even when the signal-to-noise ratio of the signals is low. In addition to using cSPoC for the analysis of cross-frequency interactions in the same subject, we show that it can also be utilized for studying amplitude dynamics of neuronal oscillations across subjects. We assess the performance of cSPoC in simulations as well as in three distinctively different analysis scenarios of real EEG data, each involving several subjects. In the simulations, cSPoC outperforms unsupervised state-of-art approaches. In the analysis of real EEG recordings, we demonstrate excellent unsupervised discovery of meaningful power-to-power couplings, within as well as across subjects and frequency bands.
Background:
Activity in the living human brain can be studied using multiple methods, spanning a wide range of spatial and temporal resolutions. We investigated the relationship between electric field potentials measured with electrocorticography (ECoG) and the blood oxygen level-dependent (BOLD) response measured with functional magnetic resonance imaging (fMRI). We set out to explain the full set of measurements by modeling the underlying neural circuits.
Results:
ECoG responses in visual cortex can be separated into two visually driven components. One component is a specific temporal response that follows each stimulus contrast reversal ("stimulus locked"); the other component is an increase in the response variance ("asynchronous"). For electrodes in visual cortex (V1, V2, V3), the two measures respond to stimuli in the same region of visual space, but they have different spatial summation properties. The stimulus-locked ECoG component sums contrast approximately linearly across space; spatial summation in the asynchronous ECoG component is subadditive. Spatial summation measured using BOLD closely matches the asynchronous component. We created a neural simulation that accurately captures the main features of the ECoG time series; in the simulation, the stimulus-locked and asynchronous components arise from different neural circuits.
Conclusions:
These observations suggest that the two ECoG components arise from different neural sources within the same cortical region. The spatial summation measurements and simulations suggest that the BOLD response arises primarily from neural sources that generate the asynchronous broadband ECoG component.
Although the mammalian neocortex has a clear laminar organization, layer-specific neuronal computations remain to be uncovered. Several studies suggest that gamma band activity in primary visual cortex (V1) is produced in granular and superficial layers and is associated with the processing of visual input [1-3]. Oscillatory alpha band activity in deeper layers has been proposed to modulate neuronal excitability associated with changes in arousal and cognitive factors [4-7]. To investigate the layer-specific interplay between these two phenomena, we characterized the coupling between alpha and gamma band activity of the local field potential in V1 of the awake macaque. Using multicontact laminar electrodes to measure spontaneous signals simultaneously from all layers of V1, we found a robust coupling between alpha phase in the deeper layers and gamma amplitude in granular and superficial layers. Moreover, the power in the two frequency bands was anticorrelated. Taken together, these findings demonstrate robust interlaminar cross-frequency coupling in the visual cortex, supporting the view that neuronal activity in the alpha frequency range phasically modulates processing in the cortical microcircuit in a top-down manner [7].
Electrical brain signals are often decomposed into frequency ranges that are implicated in different functions. Using subdural electrocorticography (ECoG, intracranial EEG) and functional magnetic resonance imaging (fMRI), we measured frequency spectra and BOLD responses in primary visual cortex (V1) and intraparietal sulcus (IPS). In V1 and IPS, 30-120Hz (gamma, broadband) oscillations allowed population receptive field (pRF) reconstruction comparable to fMRI estimates. Lower frequencies, however, responded very differently in V1 and IPS. In V1, broadband activity extends down to 3Hz. In the 4-7Hz (theta) and 18-30Hz (beta) ranges broadband activity increases power during stimulation within the pRF. However, V1 9-12Hz (alpha) frequency oscillations showed a different time course. The broadband power here is exceeded by a frequency-specific power increase during stimulation of the area outside the pRF. As such, V1 alpha oscillations reflected surround suppression of the pRF, much like negative fMRI responses. They were consequently highly localized, depending on stimulus and pRF position, and independent between nearby electrodes. In IPS, all 3-25Hz oscillations were strongest during baseline recording and correlated between nearby electrodes, consistent with large-scale disengagement. These findings demonstrate V1 alpha oscillations result from locally active functional processes and relate these alpha oscillations to negative fMRI signals. They highlight that similar oscillations in different areas reflect processes with different functional roles. However, both of these roles of alpha seem to reflect suppression of spiking activity.
Making sense of the world requires us to process information over multiple timescales. We sought to identify brain regions that accumulate information over short and long timescales and to characterize the distinguishing features of their dynamics. We recorded electrocorticographic (ECoG) signals from individuals watching intact and scrambled movies. Within sensory regions, fluctuations of high-frequency (64-200 Hz) power reliably tracked instantaneous low-level properties of the intact and scrambled movies. Within higher order regions, the power fluctuations were more reliable for the intact movie than the scrambled movie, indicating that these regions accumulate information over relatively long time periods (several seconds or longer). Slow (<0.1 Hz) fluctuations of high-frequency power with time courses locked to the movies were observed throughout the cortex. Slow fluctuations were relatively larger in regions that accumulated information over longer time periods, suggesting a connection between slow neuronal population dynamics and temporally extended information processing.
We describe a statistical approach for identifying nonlinearity in time series. The method first specifies some linear process as a null hypothesis, then generates surrogate data sets which are consistent with this null hypothesis, and finally computes a discriminating statistic for the original and for each of the surrogate data sets. If the value computed for the original data is significantly different than the ensemble of values computed for the surrogate data, then the null hypothesis is rejected and nonlinearity is detected. We discuss various null hypotheses and discriminating statistics. The method is demonstrated for numerical data generated by known chaotic systems, and applied to a number of experimental time series which arise in the measurement of superfluids, brain waves, and sunspots; we evaluate the statistical significance of the evidence for nonlinear structure in each case, and illustrate aspects of the data which this approach identifies.
Simultaneous fMRI and electrophysiological recordings suggest that the BOLD contrast mechanism directly reflects the neural responses elicited by a stimulus. In a first approximation, BOLD responses and neural responses are shown to have a linear relationship for stimulus presentation of short duration. The hemodynamic response appears to be better correlated with the local field potentials, implying that activation in an area is often likely to reflect the incoming input and the local processing in a given area rather than the spiking activity. Although it is reasonable to expect that output activity will usually correlate with neurotransmitter release and presynaptic and postsynaptic currents, when input into a particular area plays what is primarily a modulatory role, fMRI experiments may reveal activation in areas in which physiological experiments find no single-unit activity.
Work on animals indicates that BOLD is preferentially sensitive to local field potentials, and that it correlates most strongly with gamma band neuronal synchronization. Here we investigate how the BOLD signal in humans performing a cognitive task is related to neuronal synchronization across different frequency bands. We simultaneously recorded EEG and BOLD while subjects engaged in a visual attention task known to induce sustained changes in neuronal synchronization across a wide range of frequencies. Trial-by-trial BOLD fluctuations correlated positively with trial-by-trial fluctuations in high-EEG gamma power (60-80 Hz) and negatively with alpha and beta power. Gamma power on the one hand, and alpha and beta power on the other hand, independently contributed to explaining BOLD variance. These results indicate that the BOLD-gamma coupling observed in animals can be extrapolated to humans performing a task and that neuronal dynamics underlying high- and low-frequency synchronization contribute independently to the BOLD signal.
We propose a novel approach to solving the electro-/magnetoencephalographic (EEG/MEG) inverse problem which is based upon a decomposition of the current density into a small number of spatial basis fields. It is designed to recover multiple sources of possibly different extent and depth, while being invariant with respect to phase angles and rotations of the coordinate system. We demonstrate the method's ability to reconstruct simulated sources of random shape and show that the accuracy of the recovered sources can be increased, when interrelated field patterns are co-localized. Technically, this leads to large-scale mathematical problems, which are solved using recent advances in convex optimization. We apply our method for localizing brain areas involved in different types of motor imagery using real data from Brain-Computer Interface (BCI) sessions. Our approach based on single-trial localization of complex Fourier coefficients yields class-specific focal sources in the sensorimotor cortices.
Spatial normalization, registration, and segmentation techniques for Magnetic Resonance Imaging (MRI) often use a target or template volume to facilitate processing, take advantage of prior information, and define a common coordinate system for analysis. In the neuroimaging literature, the MNI305 Talairach-like coordinate system is often used as a standard template. However, when studying pediatric populations, variation from the adult brain makes the MNI305 suboptimal for processing brain images of children. Morphological changes occurring during development render the use of age-appropriate templates desirable to reduce potential errors and minimize bias during processing of pediatric data. This paper presents the methods used to create unbiased, age-appropriate MRI atlas templates for pediatric studies that represent the average anatomy for the age range of 4.5-18.5 years, while maintaining a high level of anatomical detail and contrast. The creation of anatomical T1-weighted, T2-weighted, and proton density-weighted templates for specific developmentally important age-ranges, used data derived from the largest epidemiological, representative (healthy and normal) sample of the U.S. population, where each subject was carefully screened for medical and psychiatric factors and characterized using established neuropsychological and behavioral assessments. Use of these age-specific templates was evaluated by computing average tissue maps for gray matter, white matter, and cerebrospinal fluid for each specific age range, and by conducting an exemplar voxel-wise deformation-based morphometry study using 66 young (4.5-6.9 years) participants to demonstrate the benefits of using the age-appropriate templates. The public availability of these atlases/templates will facilitate analysis of pediatric MRI data and enable comparison of results between studies in a common standardized space specific to pediatric research.
Response reliability is complementary to more conventional measurements of response amplitudes, and can reveal phenomena that response amplitudes do not. Here we review studies that measured reliability of cortical activity within or between human subjects in response to naturalistic stimulation (e.g. free viewing of movies). Despite the seemingly uncontrolled nature of the task, some of these complex stimuli evoke highly reliable, selective and time-locked activity in many brain areas, including some regions that show little response modulation in most conventional experimental protocols. This activity provides an opportunity to address novel questions concerning natural vision, temporal scale of processing, memory and the neural basis of inter-group differences.
Although the electroencephalogram (EEG) is widely used in research and clinical settings, its link to the underlying neural activity during sensory processing remains poorly understood. To investigate this, we made simultaneous recordings of surface EEG, intracortical local field potential, and multiunit activity (MUA) in the alert monkey visual cortex during presentation of natural movies. Using a general linear model, we show that in single trials, EEG power in the gamma band (30-100 Hz) and phase in delta band (2-4 Hz) are significant predictors of the MUA response. Specifically, we found that the MUA response was strongest only when increases in EEG gamma power occurred during the negative-going phase of the delta wave, thus revealing a frequency-band coupling mechanism that can be exploited to infer population spiking activity. This finding may open up a new dimension in the use and interpretation of EEG in normal and pathological conditions.
As functional magnetic resonance imaging (fMRI) has become a driving force in cognitive neuroscience, it is crucial to understand the neural basis of the fMRI signal. Here, we discuss a novel neurophysiological correlate of the fMRI signal, the slow cortical potential (SCP), which also seems to modulate the power of higher-frequency activity, the more established neurophysiological correlate of the fMRI signal. We further propose a hypothesis for the involvement of the SCP in the emergence of consciousness, and review existing data that lend support to our proposal. This hypothesis, unlike several previous theories of consciousness, is firmly rooted in physiology and as such is entirely amenable to empirical testing.
PET and fMRI experiments have previously shown that several brain regions in the frontal and parietal lobe are involved in working memory maintenance. MEG and EEG experiments have shown parametric increases with load for oscillatory activity in posterior alpha and frontal theta power. In the current study we investigated whether the areas found with fMRI can be associated with these alpha and theta effects by measuring simultaneous EEG and fMRI during a modified Sternberg task This allowed us to correlate EEG at the single trial level with the fMRI BOLD signal by forming a regressor based on single trial alpha and theta power estimates. We observed a right posterior, parametric alpha power increase, which was functionally related to decreases in BOLD in the primary visual cortex and in the posterior part of the right middle temporal gyrus. We relate this finding to the inhibition of neuronal activity that may interfere with WM maintenance. An observed parametric increase in frontal theta power was correlated to a decrease in BOLD in regions that together form the default mode network. We did not observe correlations between oscillatory EEG phenomena and BOLD in the traditional WM areas. In conclusion, the study shows that simultaneous EEG-fMRI recordings can be successfully used to identify the emergence of functional networks in the brain during the execution of a cognitive task.
Although functional magnetic resonance imaging is an important tool for measuring brain activity, the hemodynamic blood oxygenation level dependent (BOLD) response is only an indirect measure of neuronal activity. Converging evidence obtained from simultaneous recording of hemodynamic and electrical measures suggest that the best correlate of the BOLD response in primary visual cortex is gamma-band oscillations ( approximately 40 Hz). Here, we examined the coupling between BOLD and gamma-band amplitudes measured with magntoencephalography (MEG) in human primary visual cortex in 10 participants. In Experiment A, participants were exposed to grating stimuli at two contrast levels and two spatial frequencies and in Experiment B square and sine wave stimuli at two spatial frequencies. The amplitudes of both gamma-band oscillations and BOLD showed tuning with stimulus contrast and stimulus type; however, gamma-band oscillations showed a 300% increase across two spatial frequencies, whereas BOLD exhibited no change. This functional decoupling demonstrates that increased amplitude of gamma-band oscillations as measured with MEG is not sufficient to drive the subsequent BOLD response.
A package of computer programs for analysis and visualization of three-dimensional human brain functional magnetic resonance imaging (FMRI) results is described. The software can color overlay neural activation maps onto higher resolution anatomical scans. Slices in each cardinal plane can be viewed simultaneously. Manual placement of markers on anatomical landmarks allows transformation of anatomical and functional scans into stereotaxic (Talairach-Tournoux) coordinates. The techniques for automatically generating transformed functional data sets from manually labeled anatomical data sets are described. Facilities are provided for several types of statistical analyses of multiple 3D functional data sets. The programs are written in ANSI C and Motif 1.2 to run on Unix workstations.