Basile Pinsard’s research while affiliated with Centre de recherche informatique de Montréal and other places

Quantitative MRI (qMRI) promises better specificity, accuracy, repeatability, and reproducibility relative to its clinically-used qualitative MRI counterpart. Longitudinal reproducibility is particularly important in qMRI. The goal is to reliably quantify tissue properties that may be assessed in longitudinal clinical studies throughout disease progression or during treatment. In this work, we present the initial data release of the quantitative MRI portion of the Courtois project on neural modelling (CNeuroMod), where the brain and cervical spinal cord of six participants were scanned at regular intervals over the course of several years. This first release includes 3 years of data collection and up to 10 sessions per participant using quantitative MRI imaging protocols (T1, magnetization transfer (MTR, MTsat), and diffusion). In the brain, T1MP2RAGE, fractional anisotropy (FA), mean diffusivity (MD), and radial diffusivity (RD) all exhibited high longitudinal reproducibility (intraclass correlation coefficient – ICC ≃ 1 and within-subject coefficient of variations – wCV < 1%). The spinal cord cross-sectional area (CSA) computed using T2w images and T1MTsat exhibited the best longitudinal reproducibility (ICC ≃ 1 and 0.7 respectively, and wCV 2.4% and 6.9%). Results from this work show the level of longitudinal reproducibility that can be expected from qMRI protocols in the brain and spinal cord in the absence of hardware and software upgrades, and could help in the design of future longitudinal clinical studies.

There is growing symbiosis between artificial and biological intelligence sciences: neural principles inspire new intelligent machines, which are in turn used to advance our theoretical understanding of the brain. To promote further collaboration between biological and artificial intelligence researchers, we introduce the 2025 edition of the Algonauts Project challenge: How the Human Brain Makes Sense of Multimodal Movies (https://algonautsproject.com/). In collaboration with the Courtois Project on Neuronal Modelling (CNeuroMod), this edition aims to bring forth a new generation of brain encoding models that are multimodal and that generalize well beyond their training distribution, by training them on the largest dataset of fMRI responses to movie watching available to date. Open to all, the 2025 challenge provides transparent, directly comparable results through a public leaderboard that is updated automatically after each submission to facilitate rapid model assessment and guide development. The challenge will end with a session at the 2025 Cognitive Computational Neuroscience (CCN) conference that will feature winning models. We welcome researchers interested in collaborating with the Algonauts Project by contributing ideas and datasets for future challenges.

Dense functional magnetic resonance imaging datasets open new avenues to create auto-regressive models of brain activity. Individual idiosyncrasies are obscured by group models, but can be captured by purely individual models given sufficient amounts of training data. In this study, we compared several deep and shallow individual models on the temporal auto-regression of BOLD time-series recorded during a natural video-watching task. The best performing models were then analyzed in terms of their data requirements and scaling, subject specificity, and the space-time structure of their predicted dynamics. We found the Chebnets, a type of graph convolutional neural network, to be best suited for temporal BOLD auto-regression, closely followed by linear models. Chebnets demonstrated an increase in performance with increasing amounts of data, with no complete saturation at 9 h of training data. Good generalization to other kinds of video stimuli and to resting-state data marked the Chebnets’ ability to capture intrinsic brain dynamics rather than only stimulus-specific autocorrelation patterns. Significant subject specificity was found at short prediction time lags. The Chebnets were found to capture lower frequencies at longer prediction time lags, and the spatial correlations in predicted dynamics were found to match traditional functional connectivity networks. Overall, these results demonstrate that large individual functional magnetic resonance imaging (fMRI) datasets can be used to efficiently train purely individual auto-regressive models of brain activity, and that massive amounts of individual data are required to do so. The excellent performance of the Chebnets likely reflects their ability to combine spatial and temporal interactions on large time scales at a low complexity cost. The non-linearities of the models did not appear as a key advantage. In fact, surprisingly, linear versions of the Chebnets appeared to outperform the original non-linear ones. Individual temporal auto-regressive models have the potential to improve the predictability of the BOLD signal. This study is based on a massive, publicly-available dataset, which can serve for future benchmarks of individual auto-regressive modeling.

Sleep benefits motor memory consolidation, which is mediated by sleep spindle activity and associated memory reactivations during non-rapid eye movement (NREM) sleep. However, the particular role of NREM2 and NREM3 sleep spindles and the mechanisms triggering this memory consolidation process remain unclear. Here, simultaneous electroencephalographic and functional magnetic resonance imaging (EEG-fMRI) recordings were collected during night-time sleep following the learning of a motor sequence task. Adopting a time-based clustering approach, we provide evidence that spindles iteratively occur within clustered and temporally organized patterns during both NREM2 and NREM3 sleep. However, the clustering of spindles in trains is related to motor memory consolidation during NREM2 sleep only. Altogether, our findings suggest that spindles' clustering and rhythmic occurrence during NREM2 sleep may serve as an intrinsic rhythmic sleep mechanism for the timed reactivation and subsequent consolidation of motor memories, through synchronized oscillatory activity within a subcortical–cortical network involved during learning.

Artificial Neural networks (ANN) trained on complex tasks are increasingly used in neuroscience to model brain dynamics, a process called brain encoding. Videogames have been extensively studied in the field of artificial intelligence, but have hardly been used yet for brain encoding. Videogames provide a promising framework to understand brain activity in a rich, engaging, and active environment. A major challenge raised by complex videogames is that individual behavior is highly variable across subjects, and we hypothesized that ANNs need to account for subject-specific behavior in order to properly capture brain dynamics. In this study, we used ANNs to model functional magnetic resonance imaging (fMRI) and behavioral gameplay data, both collected while subjects played the Shinobi III videogame. Using imitation learning, we trained an ANN to play the game while closely replicating the unique gameplay style of individual participants. We found that hidden layers of our imitation learning model successfully encoded task-relevant neural representations, and predicted individual brain dynamics with higher accuracy than models trained on other subjects' gameplay or control models. The highest correlations between layer activations and brain signals were observed in biologically plausible brain areas, i.e. somatosensory, attention, and visual networks. Our results highlight the potential of combining imitation learning, brain imaging, and videogames to uncover idiosyncratic aspects of behavior and how those relate with individual brain activity.

Videogames are emerging as a promising experimental paradigm in neuroimaging. Acquiring gameplay in a scanner remains challenging due to the lack of a scanner-compatible videogame controller that provides a similar experience to standard, commercial devices. In this paper, we introduce a videogame controller designed for use in the functional magnetic resonance imaging as well as magnetoencephalography. The controller is made exclusively of 3D-printed and commercially available parts. We evaluated the quality of our controller by comparing it to a non-MRI compatible controller that was kept outside the scanner. The comparison of response latencies showed reliable button press accuracies of adequate precision. Comparison of the subjects’ motion during fMRI recordings of various tasks showed that the use of our controller did not increase the amount of motion produced compared to a regular MR compatible button press box. Motion levels during an ecological videogame task were of moderate amplitude. In addition, we found that the controller only had marginal effect on temporal SNR in fMRI, as well as on covariance between sensors in MEG, as expected due to the use of non-magnetic building materials. Finally, the reproducibility of the controller was demonstrated by having team members who were not involved in the design build a reproduction using only the documentation. This new videogame controller opens new avenues for ecological tasks in fMRI, including challenging videogames and more generally tasks with complex responses. The detailed controller documentation and build instructions are released under an Open Source Hardware license to increase accessibility, and reproducibility and enable the neuroimaging research community to improve or modify the controller for future experiments.

In recent years, videogames have gathered interest in cognitive neuroscience for their potential to study cognition in dynamical and naturalistic contexts. Yet, inherent game complexity can push traditional modeling to its limits, and current annotation methodologies may not fully capture the depth of their content. Neuroimaging studies have resorted to manual annotations or employed modified game versions that offer greater experimental control, both approaches limiting and labor-intensive. Here, we introduce an innovative approach using the gym-retro Python library to emulate a retro action-platformer, Shinobi III: Return of the Ninja Master (Sega, 1993), which enables automated annotations from the game's memory states. We demonstrate how this method facilitates the study of the neural basis of game events, including player actions (jumping, hitting) and feedback events (killing an enemy or being hit). Four individuals played Shinobi while undergoing functional magnetic resonance magnetic imaging (fMRI). We used automated game annotation to derive event-based general linear models of the BOLD signal in each subject, and showed common and subject-specific activation patterns in visual and sensori-motor cortices. Multi-voxel pattern analysis (MVPA) conducted on session-level annotation-derived activation maps achieved highly accurate brain decoding across all annotations. Given that the gym-retro emulation platform supports over a thousand games, this study underscores the potential of automated game annotations to study complex cognitive processes in dynamic settings across a broad game spectrum.