Figure 1 - uploaded by Léa Pillette
Content may be subject to copyright.
A participant taking part in a BCI training process. Along the training PEANUT (on the left) provides the user with social presence and emotional support adapted to his performance and progression.
Citations
... In conclusion, mimicry, in the lower part of the face, may play a role in the interaction with a wide range of face representations in art. Thus, this study could serve to enrich the facial expression and mimicry research field such as contributing to the brain-machine interface project using emotional agents (Pillette et al., 2017). More specifically, as an interdisciplinary study, this work could also contribute to understanding art interactions with face representations usually explained by anthropological and art theories. ...
Facial mimicry is a reaction to facial expressions. It plays a role in social interaction. Indeed, scholars associated facial mimicry with emotional contagion and understanding others' mental states such as intentions. This is the case for facial mimicry toward human facial expressions, but we know that facial expressions are widely depicted in art through face representations (visual creations that depict facial expressions). However, despite face representation involvement in social interactions, facial reactions toward face representations in art are still unknown. The reason could be that interaction with art objects is usually analyzed within anthropology and art theories, such as conveying social agencies (a desire of action, intentions). Here, we show that facial mimicry is also observed toward face representations. This could be a means that might facilitate social interaction including emotions. Using the electromyography technique, we could show that participants mimic involuntarily face representations when these depict mouth expressions. Participant's zygomaticus and depressor were significantly activated when the pictures depict an expression including zygomaticus or depressor representation respectively. This result led us to infer that when it comes to mouth expressions, face representations in art might trigger spontaneously emotional contagion (of the expressed emotion). It might also convey information about the expressed mental states, which might help to indicate social agencies. Mimicry could participate to explain partly the social agencies of art, that might be no more just abstract concepts, but could find a real correlate in cognitive processes.
... Or, il aété reporté dans la littérature qu'un protocole de NF visantà moduler l'activité α permettait une réduction des troubles anxieux uniquement chez les personnes très anxieuses Hardt et Kamiya (1978). Ceci pourrait donc expliquer pourquoi la plupart des résultats obtenus allant dans le sens de nos hypothèses concernent ceux des données Pouréviter ces biais d'interaction social, un compagnon d'apprentissage robotisé comme celui proposé par Pillette et al. (2017) pourraitêtre utilisé en support des séances de NF pour les protocoles futurs. ...
Cette thèse porte sur la conception, l’implémentation et l’évaluation d’un système de neurofeedback EEG portable, d’aide à la gestion du stress, à destination du grand public. Un tel système permet aux utilisateurs d’apprendre à moduler leurs états mentaux par des phénomènes de plasticité cérébrale. Cependant, plusieurs facteurs peuvent compliquer cet apprentissage, comme un plus faible rapport signal sur bruit de l'EEG acquis par des électrodes sèches, la contamination par des artefacts ou encore la définition de paramètres pertinents à partir des signaux EEG. Afin d’optimiser ce retour neuronal, ma thèse propose d’abord une méthode statistique permettant de s’assurer de la qualité des signaux EEG acquis, ainsi qu’une méthode corrective d’artefacts, afin de pouvoir extraire une mesure pertinente de l’activité EEG reflétant le niveau de stress ou de relaxation de l’individu. Le développement d’un indice de neurofeedback pertinent et adapté à l’utilisateur est également proposé. A la suite de la constitution algorithmique d’un tel système, les caractéristiques d'apprentissage par neurofeedback ont pu être étudiées. En particulier, je montre qu'un apprentissage intersession semble se mettre en place et que chez les sujets stressés, des changements cérébraux s'opèrent dans la bande alpha durant les phases de repos. Finalement, par ces aspects méthodologiques, d’intégration logicielle et d’analyse longitudinale, cette thèse constitue les briques fondamentales d’un système de recommandation automatique adapté à l’utilisateur. Un tel système permettrait un suivi personnel des utilisateurs afin de leur proposer une stratégie préventive pour la gestion du stress.
... Finally, we explored the use of social and emotional feedback when creating PEANUT (i.e., Personalized Emotional Agent for Neurotechnology User Training), which is the first learning companion dedicated to BCI training [63]. Its interventions were composed of spoken sentences and displayed facial expression in between two trials (see Figure 3). ...
... The model allows the robot to adapt to a different user by associating the perceived emotion with an appropriate expression which makes the companion more socially acceptable in the environment in which it operates. Figure 3: Experimental setting where PEANUT (on the left) provides a user with social presence and emotional support adapted to his performance and progression [63]. ...
Mental Imagery based Brain-Computer Interfaces (MI-BCI) enable their users to control an interface, e.g., a prosthesis, by performing mental imagery tasks only, such as imagining a right arm movement while their brain activity is measured and processed by the system. Designing and using a BCI requires users to learn how to produce different and stable patterns of brain activity for each of the mental imagery tasks. However, current training protocols do not enable every user to acquire the skills required to use BCIs. These training protocols are most likely one of the main reasons why BCIs remain not reliable enough for wider applications outside research laboratories. Learning companions have been shown to improve training in different disciplines, but they have barely been explored for BCIs so far. This article aims at investigating the potential benefits learning companions could bring to BCI training by improving the feedback, i.e., the information provided to the user, which is primordial to the learning process and yet have proven both theoretically and practically inadequate in BCI. This paper first presents the potentials of BCI and the limitations of current training approaches. Then, it reviews both the BCI and learning companion literature regarding three main characteristics of feedback: its appearance, its social and emotional components and its cognitive component. From these considerations, this paper draws some guidelines, identify open challenges and suggests potential solutions to design and use learning companions for BCIs.
... The inconstancy of the user mood, stress, engagement, and level of attention is also a cause of the BCI performance variability and should be taken in account (Friedrich et al. 2015;Pammer-Schindler et al. 2017). Moreover, since driving a BCI is a skill that could be learned and mastered, also the protocols used to train the user could be cause of performance degradation (Jeunet, Jahanpour, and Lotte 2016;Pillette et al. 2017). ...
Many critical aspects affect the correct operation of a Brain Computer Interface. The term "BCI-illiteracy" describes the impossibility of using a BCI paradigm. At present, a universal solution does not exist and seeking innovative protocols to drive a BCI is mandatory. This work presents a meta-analytic review on recent advances in emotions recognition with the perspective of using emotions as voluntary, stimulus-independent, commands for BCIs. 60 papers, based on electroencephalography measurements, were selected to evaluate what emotions have been most recognized and what brain regions were activated by them. It was found that happiness, sadness, anger and calm were the most recognized emotions. Most discriminant locations for emotions recognition and for the particular case of discrete emotions recognition were identified in the temporal, frontal and parietal areas. The meta-analysis was mainly performed on stimulus-elicited emotions, due to the limited amount of literature about self-induced emotions. The obtained results represent a good starting point for the development of BCI driven by emotions and allow to: 1) ascertain that emotions are measurable and recognizable one from another 2) select a subset of most recognizable emotions and the corresponding active brain regions.
... However, CA is also used to study BCI users' performance and learning, i.e., how well users can modulate/self-regulate their EEG signals to control the BCI, and how much they learn to do so. For instance, CA is typically used to study how different kinds of feedback influence BCI users' training [21,31,35], or how different psychological factors influence BCI users' learning and performances [19]. ...
Objective:
While promising for many applications, Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are still scarcely used outside laboratories, due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires the use of appropriate reliability metrics to quantify both the classification algorithm and the BCI user's performances. So far, Classification Accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study BCI users' skills. Here, we propose a definition and new metrics to quantify such BCI skills for Mental Imagery (MI) BCIs, independently of any classification algorithm.
Approach: We first show in this paper that CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful self-modulation of EEG patterns by the user. We then propose a definition of MI-BCI skills that reflects how well the user can self-modulate EEG patterns, and thus how well he could control an MI-BCI. Finally, we propose new performance
metrics, classDis, restDist and classStab that specifically measure how distinct and stable the EEG patterns produced by the user are, independently of any classifier.
Main results: By re-analyzing EEG data sets with such new metrics, we indeed confirmed that CA may hide some increase in MI-BCI skills or hide the user inability to self-modulate a given EEG pattern. On the other hand, our new metrics could reveal such skill improvements as well as identify when a mental task performed by a user was no different than rest EEG.
Significance: Our results showed that when studying MI-BCI users' skills, CA should be used with care, and complemented with metrics such as the new ones proposed. Our results also stressed the need to redefine BCI user training by considering the different BCI subskills and their measures. To promote the complementary use of our new metrics, we provide the Matlab code to compute them for free and open-source.
... In this view, emotional feedback has been studied (Kübler et al., 2001) and the positive impact of multiplayer/social feedback demonstrated (Bonnet et al., 2013;Nijholt, 2015). Also, Pillette et al. (2017) showed that providing emotional support and social presence using a learning companion improved user-experience during the MI-BCI training procedure. In the same vein, studies (Barbero and Grosse-Wentrup, 2010;Kübler et al., 2001b) report that when provided with positively biased feedback, novice MI-BCI users' performance increases. ...
... For instance, several factors have been suggested to have a positive effect on clinical symptoms by influencing the neurophysiological systems underlying the symptoms, but without necessarily influencing the neurophysiological patterns targeted in NF treatments (Hammond, 2011, Gaume et al. 2016. These factors are consistent with factors highlighted in the literature for influencing BCI performance: activating the attentional networks (Blankertz et al., 2010, experiencing a flow state (Mladenovic et al., 2017, Bauer et al., 2016, a high sense of agency (Vlek et al., 2014;Jeunet et al., 2016;Braun et al., 2016) and being provided with positive social support (Pillette et al. 2017). The present framework suggests that we should not disregard these factors since they may contribute to the placebo effect (Thibault et al. 2017, Raz et Michels, 2007. ...
In their recent paper, Alkoby et al. (2017) provide the readership with an extensive and very insightful review of the factors influencing NeuroFeedback (NF) performance. These factors are drawn from both the NF literature and the Brain-Computer Interface (BCI) literature. Our short review aims to complement Alkoby et al.'s review by reporting recent additions to the BCI literature. The object is to highlight this literature and discuss its potential relevance and usefulness to better understand the processes underlying NF and further improve the design of clinical trials assessing NF efficacy. Indeed, we are convinced that while NF and BCI are fundamentally different in many ways, both the BCI and NF communities could reach compelling achievements by building upon one another. By reviewing the recent BCI literature, we identified three types of factors that influence BCI performance: task-specific, cognitive/motivational and technology-acceptance related factors. Since BCIs and neurofeedback share a common goal (i.e., learning to modulate specific neurophysiological patterns), similar cognitive and neurophysiological processes are likely to be involved during the training process. Thus, the literature on BCI training may help (1) to deepen our understanding of neurofeedback training processes and (2) to understand the variables that influence the clinical efficacy of NF. This may help to properly assess and/or control the influence of these variables during randomised controlled trials.
Brain-computer interfaces (BCIs) provide a unique technological solution to circumvent the damaged motor system. For neurorehabilitation, the BCI can be used to translate neural signals associated with movement intentions into tangible feedback for the patient, when they are unable to generate functional movement themselves. Clinical interest in BCI is growing rapidly, as it would facilitate rehabilitation to commence earlier following brain damage and provides options for patients who are unable to partake in traditional physical therapy. However, substantial challenges with existing BCI implementations have prevented its widespread adoption. Recent advances in knowledge and technology provide opportunities to facilitate a change, provided that researchers and clinicians using BCI agree on standardisation of guidelines for protocols and shared efforts to uncover mechanisms. We propose that addressing the speed and effectiveness of learning BCI control are priorities for the field, which may be improved by multimodal or multi-stage approaches harnessing more sensitive neuroimaging technologies in the early learning stages, before transitioning to more practical, mobile implementations. Clarification of the neural mechanisms that give rise to improvement in motor function is an essential next step towards justifying clinical use of BCI. In particular, quantifying the unknown contribution of non-motor mechanisms to motor recovery calls for more stringent control conditions in experimental work. Here we provide a contemporary viewpoint on the factors impeding the scalability of BCI. Further, we provide a future outlook for optimal design of the technology to best exploit its unique potential, and best practices for research and reporting of findings.
Mental-Imagery based Brain-Computer Interfaces (MI-BCIs) present new opportunities to interact with digital technologies, such as neuroprostheses or videogames, only by performing mental imagery tasks, such as imagining an object rotating. The recognition of the command for the system is based on the analysis of the brain activity of the user. The users must learn to produce brain activity patterns that are recognizable by the system in order to control BCIs. However, current training protocols do not enable 10 to 30% of persons to acquire the skills required to use BCIs. The lack of robustness of BCIs limit the development of the technology outside of research laboratories. This thesis aims at investigating how the feedback provided throughout the training can be improved and adapted to the traits and states of the users. First, we investigate the role that feedback is currently given in MI-BCI applications and training protocols. We also analyse the theories and experimental contributions discussing its role and usefulness. Then, we review the different feedback that have been used to train MI-BCI users. We focus on three main characteristics of feedback, i.e., its content, its modality of presentation and finally its timing. For each of these characteristics, we reviewed the literature to assess which types of feedback have been tested and what is their impact on the training. We also analysed which traits or states of the learners were shown to influence BCI training outcome. Based on these reviews of the literature, we hypothesised that different characteristics of feedback could be leveraged to improve the training of the learners depending on either traits or states. We reported the results of our experimental contributions for each of the characteristics of feedback. Finally, we presented different recommendations and challenges regarding each characteristic of feedback. Potential solutions were proposed to meet these recommendations in the future.
The clinical efficacy of neurofeedback is still a matter of debate. This paper analyzes the factors that should be taken into account in a transdisciplinary approach to evaluate the use of EEG NFB as a therapeutic tool in psychiatry. Neurofeedback is a neurocognitive therapy based on human–computer interaction that enables subjects to train voluntarily and modify functional biomarkers that are related to a defined mental disorder. We investigate three kinds of factors related to this definition of neurofeedback. We focus this article on EEG NFB. The first part of the paper investigates neurophysiological factors underlying the brain mechanisms driving NFB training and learning to modify a functional biomarker voluntarily. Two kinds of neuroplasticity involved in neurofeedback are analyzed: Hebbian neuroplasticity, i.e. long-term modification of neural membrane excitability and/or synaptic potentiation, and homeostatic neuroplasticity, i.e. homeostasis attempts to stabilize network activity. The second part investigates psychophysiological factors related to the targeted biomarker. It is demonstrated that neurofeedback involves clearly defining which kind of relationship between EEG biomarkers and clinical dimensions (symptoms or cognitive processes) is to be targeted. A nomenclature of accurate EEG biomarkers is proposed in the form of a short EEG encyclopedia (EEGcopia). The third part investigates human–computer interaction factors for optimizing NFB training and learning during the closed loop interaction. A model is proposed to summarize the different features that should be controlled to optimize learning. The need for accurate and reliable metrics of training and learning in line with human–computer interaction is also emphasized, including targeted biomarkers and neuroplasticity. All these factors related to neurofeedback show that it can be considered as a fertile ground for innovative research in psychiatry.
L’hallucination est une expérience subjective vécue en pleine conscience consistant en une perception impossible à distinguer d’une perception réelle, mais survenant en l’absence de tout stimulus en provenance de l’environnement externe. Les symptômes hallucinatoires, qui peuvent concerner toutes les modalités sensorielles, sont retrouvés dans divers troubles neurologiques et psychiatriques mais également chez certains sujets indemnes de toute pathologie. Dans le champ de la psychiatrie, la pathologie la plus fréquemment associée aux hallucinations reste la schizophrénie et la modalité auditive est la plus représentée, puisque 60 à 80% des patients souffrant de ce trouble sont concernés. Le retentissement fonctionnel des hallucinations auditives peut être important, altérant significativement la qualité de vie des patients.Dans ce contexte, la prise en charge de ce type de symptômes s’avère un enjeu considérable pour les personnes souffrant de schizophrénie. Pourtant, les moyens thérapeutiques actuellement disponibles (traitements médicamenteux antipsychotiques notamment) ne permettent pas toujours une rémission complète de la symptomatologie hallucinatoire et l’on considère que 25 à 30% des hallucinations auditives sont « pharmaco-résistantes ». C’est à partir de ce constat que, ces dernières années, ont émergé, pour le traitement des hallucinations auditives, des techniques de neuromodulation comme la stimulation magnétique transcrânienne répétée ou la stimulation électrique transcrânienne par courant continu. Toutefois, les résultats de ces nouvelles thérapies sur les hallucinations auditives résistantes restent modérés et le développement de stratégies alternatives demeure un enjeu de recherche majeur.Actuellement, les travaux en imagerie fonctionnelle permettent d'affiner les modèles physiopathologiques des hallucinations auditives, mais leur intérêt pourrait aller au-delà de la recherche fondamentale, avec possiblement des applications cliniques telles que l'assistance thérapeutique. Ce travail de thèse s’inscrit précisément dans le développement de l’imagerie cérébrale de « capture » des hallucinations auditives, c’est-à-dire l’identification des patterns d’activation fonctionnels associés à la survenue des hallucinations auditives.La première partie de ce travail est consacrée à la détection automatisée des hallucinations auditives en IRM fonctionnelle. L’identification des périodes hallucinatoires survenues au cours d’une session d’IRM fonctionnelle est actuellement possible par une méthode de capture semi-automatisée validée. Celle-ci permet une labellisation des données acquises au cours d’une session de repos en périodes « hallucinatoires » et « non-hallucinatoires ». Toutefois, le caractère long et fastidieux de cette méthode limite largement son emploi. Nous avons donc souhaité montrer comment les stratégies d’apprentissage machine (support vector machine ou SVM, notamment) permettent l’automatisation de cette technique par le développement de classificateurs performants, généralisables et associés à un faible coût de calcul (indispensable en vue d’une utilisation en temps réel). Nous proposons également le développement d’algorithmes de reconnaissance de la période « pré-hallucinatoire », en mettant en évidence que ce type de classificateur présente aussi des performances largement significatives. Enfin, nous avons pu montrer que l’utilisation de stratégies d’apprentissage-machine alternatives au SVM (e.g, le TV-Elastic-net), obtient des performances significativement supérieures au SVM [...]
